Zero your flash media before important shoots!

sd_card_badWith today’s recordings being written to SD, CF, SxS and SSD’s you have to know one important maintenance trick. Just formatting is not enough because it only wipes the directory, not the entire card!
Flash media can re-assign bad blocks (just like a regular hard drive), but only after you’ve tried to write to them.
Errors in movie files or pictures happen when the camera is writing to media with bad blocks but it hasn’t written there before.
-Always write zero’s (using Disk Utility) to new SD cards immediately after buying it, to re-assign bad blocks.
-Repeat writing zero’s every 6 months or before critical recordings.
Writing zeroes takes about 45 min for a 128GB (Class 10 60MB/s) card with internal Apple reader.
ALWAYS:
SHOOT>OFFLOAD>RE-FORMAT
Reformat with the camera or recorder that will be used with the card.
If something goes wrong: turn off the camera immediately, remove the SD card, and “lock” the card (making it read-only) using the physical switch on the SD card.
Immediately copy the card out.

SD-card_storageWhen corruption sneaks in:
1. is it the card? >are card or contacts dirty?
2. is it the reader? >swap it out.
3. is it the destination drive’s file system?
Run a disk utility on the destination drive and fix errors.
4. is it the software or operating system?
Try something else.

 

SD_CARD_cutKeep track of SD-cards, their age, brand and when they where zeroed last.
If zero-ing the card yields errors, it certainly is corrupt. Throw it out! Or try warranty:
Sandisk, Transcend, Crucial, Panasonic, Sony
Click here to recover damaged files or cards.
For P2 card recovery call:
Unit of Panasonic Corporation of North America Western Region, Address 3330 Cahuenga Boulevard, Suite 101 ,Los Angeles, California 90068, Phone (323) 436-3507, Fax (323) 436-3517 Contact Mr. Fast

Unit of Panasonic Corporation of North America Northeast Region Address 50 Meadowlands Parkway, Secaucus, New Jersey 07094 Phone (201) 348-7975, (201) 392-4081 Fax (201) 348-5200
Contact Mr. Feldman

 

 

 

 

Storage solution for Photographer and small indie producers.

on a MAC use Rsync

rsync -avx –progress /Volumes/Macintosh HD/Users/Joe/Documents/Data/ /Volumes/Macintosh HD/Users/Joe/Documents/DataBackup/

Another important advantage of using rsync is that it reads all files on both drives effectivly refreshing the data on a drive that is sitting on a shelf.

Resources:
Verify the integrity of a flash/SD card on Mac & Windows

GPU vs CPU how do they divide the work?

The basic components of an NLE are:

1. Media files stored on an external HD
-Media access speed is determined by: HD speed, Raid level and connection speed to workstation.
2. CPU> decode, encode and calculate non-accelerated effects.
3. GPU>accelerates:
Alpha Adjust, Basic 3D, Black & White, Brightness & Contrast
Color Balance (RGB), Color Pass (Windows only)
Color Replace, Crop Drop Shadow, Edge Feather, Eight&Four-Point Garbage Matte
Extract, Fast Color Corrector, Gamma Correction, Garbage Matte (4, 8, 16)
Gaussian Blur, Horizontal Flip, Levels, Luma Corrector, Luma Curve, Noise
Proc Amp, RGB Curves, RGB Color Corrector, Sharpen
Sixteen-Point Garbage Matte, Three-way Color Corrector, Timecode
Tint, Track Matte Key, Ultra Keyer, Video Limiter, Vertical Flip
Cross Dissolve, Dip to Black, Dip to White, Directional Blur, Fast Blur, Invert, Additive Dissolve, Film Dissolve, Warp Stabilizer

7 steps to render/playback a frame:
1. Frame is fetched from disk.
2. CPU decodes the frame (CODEC has to be properly installed)
3. Frame is modified to an intermediate format and put into memory.
4. If GPU acceleration can be used, the intermediate format is uploaded to the video card for processing.
5. The intermediate results are then downloaded from the video card back into memory.
6. The CPU starts encoding to the final delivery format.
7. The final results are written to disk or played on screen.

A video file gets decoded on the CPU. Performance is determined by how well it is optimized and if it can offload tasks to GPU or not. A lot of codecs on PC call the QT32 process which runs in a 4GB memory space.
ProRes playback is not GPU accelerated. More cores are more important than multiple GPUs, as not all instructions have been written to run outside
the CPU. It’s a common misconception that adding a GPU will
automatically take and run all the regular instructions from
the CPU. Software engineers must write applications to
send instructions to a specific processor before they can
expect speed improvements.
A typical system with eight cores needs 3 GB per core, which equals 24 GB of RAM required by After Effects to run multiprocessing properly.
Although that fits into a 32 GB computer, After Effects may still hold as
much as 6 GB of RAM for other applications, bringing the full
amount of RAM left for After Effects to 24, which is already

close to the limit.

This applies to Windows/PC only:
Below Playback the way it should be: Low CPU load and Disk I/O at the bit rate of the codec. Note the files on the timeline being read in the Disk I/O list.
1AVID_AVID-MXF_PLAY_NO_FILTER
Now look what happens when playing back a ProRes QuickTime in Premiere on a PC
1. CPU cycles taken up by Premiere
2. CPU cycles taken up by the QT32 process
3-8 The files on disk being read by the QT32 process including:
9 pagefile.sys the Page File it has to keep writing and reading off the drive to compensate for the 4GB memory limit.
PR_PRO_PLAY_NO_FILTER_pagefile

Performance Grid: Codecs, Platforms, Wrappers, Quicktime, MXF

I had an aha-moment recently after switching from a QuickTime based project on Premiere, to C300>native MXF. The system was all of a sudden snappy, no stalls, a lot more realtime performance and super fast renders and exports.
My system must have been struggling with something…
A codec that works with low CPU load on a Mac might have a high CPU load on PC.‬ On Mac QuickTime uses the 64-bit AV Foundations framework but on PC Premiere still uses the deprecated 32-bit QTkit framework. Adobe finally wrote an unofficial white paper March 2016 about performance. ProRes is not supported for export on Windows.
I try to remove QuickTime as much as possible from my PC/Windows workflows. But don’t take my word for it, do your own performance tests!
I think MXF combined with PC friendly codec is the authoring format of the future.
Please chime in using the Google form below.
mi-smile
Mike

My thoughts:
Below is my (imperfect, incomplete) performance grid that shows what platform and software I try to use with each codec. I usually work on a standalone system. On any kind of collaborative project I recommend transcoding to native Avid media and using their ISIS storage. There is still no match for Avid’s media management especially when collaborating in realtime on a deadline.

QuickTime MOV most codecs ProRes

Arri Alexa

RED DSLR AVCHD H264       H265 in MP4 container. Like GH4 Collaborate with Native Avid Media DNxHD DNxHR Native Sony XDCAM-EX XDCAM-HD XDCAM-HD422 Blackmagic Canon Native C500 C300 C100 5DM2 5DM3 7D
ICO_apple ICO_premiere ICO_resolveICO_fcpx ICO_apple ICO_premiere ICO_resolveICO_fcpx ICO_appleICO_winICO_fcpx ICO_resolve ICO_premiere ICO_win ICO_resolve ICO_premiere ICO_winICO_avid ICO_win ICO_resolve ICO_premiere ICO_appleICO_fcpx ICO_resolve ICO_premiere ICO_appleICO_winICO_fcpx ICO_resolve ICO_premiere

Most cameras record in non-Quicktime formats like MP4 or MXF, some are more PC friendly, others work better on Mac. The Alexa can record in two compression types/file formats: either you can record as a ProRes compressed Quicktime *.mov or as an MXF-wrapped DNxHD compressed file. Hopefully, the next generation of external recorders will follow suit. One MXF recorder already exists: A laptop with BM-mini Recorder running Avid Or Premiere!

Apple research:
-I love ProRes on Mac, I think it’s a great looking, very easy too use codec.
It works in 64 bit. If a production uses ProRes, I fight for editing on Mac.
-Most people seem to agree that Macs are build for Apple’s own software, not the competition’s.
QuickTime is deprecated? What does that mean in practice?
What is Apple doing with QuickTime?
-The big loser on the Mac platform is Premiere. Adobe has made strides in improving the performance of their applications, but it still doesn’t compare to what Apple has achieved with FCPX.
There is an incompatibility between Premiere and Open-GL and AMD.
-The winner is FCPX. Software and hardware are tuned to each other for maximum performance. It takes advantage of modern hardware, as it’s 64-bit (uses all your RAM), OpenCL GPU-accelerated (uses your graphics card for playback, render and export), and supports Grand Central Dispatch (uses all of your CPUs and cores).

PC/Windows research:
When I work with QuickTime and most codecs on PC, I’m constantly experiencing mini crashes (white screen lockup) and slow export times. I’m forced to restart every hour because pagefile.sys and QT32 processes running in the background bog the system down. I think the white screen lockup is the pagefile.sys being written or read by Windows from the slow internal HD! (takes about 30sec)
There is no 64-bit version of QuickTime, primarily because Apple has ceased development of QuickTime on Windows (except for security and minor updates). QuickTime has been 7.x for 10 years now, and 8.0 does not seem to be anywhere in the distance. Avid, Adobe and Resolve work great with non-Quicktime media or Avid’s own media (MXF).
The QT-32 process on PC runs in a 4 GB memory space, even when lots of memory is installed. To keep up with playback, the system has to keep swapping banks of memory (pagefile.sys) to the hard-drive instead of using all installed memory causing severe performance degradation. The other drawback is the very limited threading capability, causing a far from efficient use of all the logical cores in the system.
MXF is a much better container format, as it is just a container, unlike QuickTime, which is a full multimedia framework.
MXF requires no installation (and hence never needs updating); doesn’t launch a process when used, so 32- and 64-bit matter not; and it has no bugs or odd quirks – it just holds the video, audio and metadata.
-I avoid Quicktime on PC. Any codec! > I use native camera media, MXF, MP4 or native Avid Media Files (which are MXF).
-I don’t use ProRes on PC, with any of the NLEs, even though they say it works. It is just too much trouble, glitchy and slow.
-ProRes inside MXF on Avid is a joke. Their glossy brochures talk about compatibility, but I can’t make it work. I only use AMA to transcode to Avid media, not for editing.
-In Premiere> QuickTime for Windows cannot export H.264 on computers that have more than 16 CPU cores due to a problem in the Apple H.264 compressor component.

 

Multi-Cam:
-If possible I do Multi-Cam on Avid with transcoded DNxHD36 media if more than 4 angles for the offline. For online relink to native media.
-Next best is Premiere with the same DNxHD36 media if more than 4 angles and native media with <4 angles.

Delivery:
You can edit without QuickTime but deliver in Quicktime. When a client asks for QuickTime & certain codec, I give them what they want. This means keeping a Mac around somehow for ProRes delivery.
Saving MXF clips to MXF container in Premiere
AVID – Fast Import Apple ProRes .mov to AVID Pro Res .mxf Workflow
Arri Alexa bypassing Quicktime using MXF instead

Interoperability with DAM systems:
Dalet supports import/ingest/export in MXF
-you have to test and update your workflow company wide to make use of this.
Delivery of MXF with Closed captioning

Uninstall QuickTime to avoid all kinds of conflicts with your NLE.
It forces your NLE to use their workarounds for play back Quicktime codecs.
Don’t worry, you can always put it back, the NLE usually has it’s own process to playback QuickTime sources. Premiere works fine on Windows without QuickTime installed.
It is a great trouble shooting step to try to improve the realtime capabilities of your system.

Uninstall QuickTime on Mac:
mac-qt-library

1. Internal HD>Library>Quicktime
2. Remove all or just the components you are trouble shooting, out of this folder.
3. restart apps using QuickTime.

Uninstall Quicktime on PC/Windows:
1. Control Panel>Add/Remove Software
2. Restart Computer.
3. click here to download latest version from Apple.

Resources:
Security Alert uninstall Quicktime
-Avid’s take on Quicktime
Do your own performance tests.
How do I speed up rendering, exporting, or encoding?
PREMIERE PRO AND MULTIPLE GPUS
-Adobe’s smart rendering workflow
Good article about how video codecs work
QuickTime is deprecated? What does that mean in practice?
What is Apple doing with QuickTime?
-Adobe says it buried on a support page: ProRes is not supported on Windows!
A Guide to Common Video Formats: Containers, Compression, and Codecs
Build a balanced system
GPU vs CPU how is the work shared?
Free Resolve-12: Great basic editor, fantastic color corrector > the standard for most TV & Film. Also has one feature no other NLE’S have: Export files from timeline as individual files!
Free Red CINE-X: Transcodes any Red file into an editable format: even Quicktime ProRes on Mac and PC!