GPU vs CPU how do they divide the work?

The basic components of an NLE are:

1. Media files stored on an external HD
-Media access speed is determined by: HD speed, Raid level and connection speed to workstation.
2. CPU> decode, encode and calculate non-accelerated effects.
3. GPU>accelerates:
Alpha Adjust, Basic 3D, Black & White, Brightness & Contrast
Color Balance (RGB), Color Pass (Windows only)
Color Replace, Crop Drop Shadow, Edge Feather, Eight&Four-Point Garbage Matte
Extract, Fast Color Corrector, Gamma Correction, Garbage Matte (4, 8, 16)
Gaussian Blur, Horizontal Flip, Levels, Luma Corrector, Luma Curve, Noise
Proc Amp, RGB Curves, RGB Color Corrector, Sharpen
Sixteen-Point Garbage Matte, Three-way Color Corrector, Timecode
Tint, Track Matte Key, Ultra Keyer, Video Limiter, Vertical Flip
Cross Dissolve, Dip to Black, Dip to White, Directional Blur, Fast Blur, Invert, Additive Dissolve, Film Dissolve, Warp Stabilizer

7 steps to render/playback a frame:
1. Frame is fetched from disk.
2. CPU decodes the frame (CODEC has to be properly installed)
3. Frame is modified to an intermediate format and put into memory.
4. If GPU acceleration can be used, the intermediate format is uploaded to the video card for processing.
5. The intermediate results are then downloaded from the video card back into memory.
6. The CPU starts encoding to the final delivery format.
7. The final results are written to disk or played on screen.

A video file gets decoded on the CPU. Performance is determined by how well it is optimized and if it can offload tasks to GPU or not. A lot of codecs on PC call the QT32 process which runs in a 4GB memory space.
ProRes playback is not GPU accelerated. More cores are more important than multiple GPUs, as not all instructions have been written to run outside
the CPU. It’s a common misconception that adding a GPU will
automatically take and run all the regular instructions from
the CPU. Software engineers must write applications to
send instructions to a specific processor before they can
expect speed improvements.
A typical system with eight cores needs 3 GB per core, which equals 24 GB of RAM required by After Effects to run multiprocessing properly.
Although that fits into a 32 GB computer, After Effects may still hold as
much as 6 GB of RAM for other applications, bringing the full
amount of RAM left for After Effects to 24, which is already

close to the limit.

This applies to Windows/PC only:
Below Playback the way it should be: Low CPU load and Disk I/O at the bit rate of the codec. Note the files on the timeline being read in the Disk I/O list.
1AVID_AVID-MXF_PLAY_NO_FILTER
Now look what happens when playing back a ProRes QuickTime in Premiere on a PC
1. CPU cycles taken up by Premiere
2. CPU cycles taken up by the QT32 process
3-8 The files on disk being read by the QT32 process including:
9 pagefile.sys the Page File it has to keep writing and reading off the drive to compensate for the 4GB memory limit.
PR_PRO_PLAY_NO_FILTER_pagefile




Performance Grid: Codecs, Platforms, Wrappers, Quicktime, MXF

I had an aha-moment recently after switching from a QuickTime based project on Premiere, to C300>native MXF. The system was all of a sudden snappy, no stalls, a lot more realtime performance and super fast renders and exports.
My system must have been struggling with something…
A codec that works with low CPU load on a Mac might have a high CPU load on PC.‬ On Mac QuickTime uses the 64-bit AV Foundations framework but on PC Premiere still uses the deprecated 32-bit QTkit framework. Adobe finally wrote an unofficial white paper March 2016 about performance. ProRes is not supported for export on Windows.
I try to remove QuickTime as much as possible from my PC/Windows workflows. But don’t take my word for it, do your own performance tests!
I think MXF combined with PC friendly codec is the authoring format of the future.
Please chime in using the Google form below.
mi-smile
Mike

My thoughts:
Below is my (imperfect, incomplete) performance grid that shows what platform and software I try to use with each codec. I usually work on a standalone system. On any kind of collaborative project I recommend transcoding to native Avid media and using their ISIS storage. There is still no match for Avid’s media management especially when collaborating in realtime on a deadline.

QuickTime MOV most codecs ProRes

Arri Alexa

RED DSLR AVCHD H264       H265 in MP4 container. Like GH4 Collaborate with Native Avid Media DNxHD DNxHR Native Sony XDCAM-EX XDCAM-HD XDCAM-HD422 Blackmagic Canon Native C500 C300 C100 5DM2 5DM3 7D
ICO_apple ICO_premiere ICO_resolveICO_fcpx ICO_apple ICO_premiere ICO_resolveICO_fcpx ICO_appleICO_winICO_fcpx ICO_resolve ICO_premiere ICO_win ICO_resolve ICO_premiere ICO_winICO_avid ICO_win ICO_resolve ICO_premiere ICO_appleICO_fcpx ICO_resolve ICO_premiere ICO_appleICO_winICO_fcpx ICO_resolve ICO_premiere

Most cameras record in non-Quicktime formats like MP4 or MXF, some are more PC friendly, others work better on Mac. The Alexa can record in two compression types/file formats: either you can record as a ProRes compressed Quicktime *.mov or as an MXF-wrapped DNxHD compressed file. Hopefully, the next generation of external recorders will follow suit. One MXF recorder already exists: A laptop with BM-mini Recorder running Avid Or Premiere!

Apple research:
-I love ProRes on Mac, I think it’s a great looking, very easy too use codec.
It works in 64 bit. If a production uses ProRes, I fight for editing on Mac.
-Most people seem to agree that Macs are build for Apple’s own software, not the competition’s.
QuickTime is deprecated? What does that mean in practice?
What is Apple doing with QuickTime?
-The big loser on the Mac platform is Premiere. Adobe has made strides in improving the performance of their applications, but it still doesn’t compare to what Apple has achieved with FCPX.
There is an incompatibility between Premiere and Open-GL and AMD.
-The winner is FCPX. Software and hardware are tuned to each other for maximum performance. It takes advantage of modern hardware, as it’s 64-bit (uses all your RAM), OpenCL GPU-accelerated (uses your graphics card for playback, render and export), and supports Grand Central Dispatch (uses all of your CPUs and cores).

PC/Windows research:
When I work with QuickTime and most codecs on PC, I’m constantly experiencing mini crashes (white screen lockup) and slow export times. I’m forced to restart every hour because pagefile.sys and QT32 processes running in the background bog the system down. I think the white screen lockup is the pagefile.sys being written or read by Windows from the slow internal HD! (takes about 30sec)
There is no 64-bit version of QuickTime, primarily because Apple has ceased development of QuickTime on Windows (except for security and minor updates). QuickTime has been 7.x for 10 years now, and 8.0 does not seem to be anywhere in the distance. Avid, Adobe and Resolve work great with non-Quicktime media or Avid’s own media (MXF).
The QT-32 process on PC runs in a 4 GB memory space, even when lots of memory is installed. To keep up with playback, the system has to keep swapping banks of memory (pagefile.sys) to the hard-drive instead of using all installed memory causing severe performance degradation. The other drawback is the very limited threading capability, causing a far from efficient use of all the logical cores in the system.
MXF is a much better container format, as it is just a container, unlike QuickTime, which is a full multimedia framework.
MXF requires no installation (and hence never needs updating); doesn’t launch a process when used, so 32- and 64-bit matter not; and it has no bugs or odd quirks – it just holds the video, audio and metadata.
-I avoid Quicktime on PC. Any codec! > I use native camera media, MXF, MP4 or native Avid Media Files (which are MXF).
-I don’t use ProRes on PC, with any of the NLEs, even though they say it works. It is just too much trouble, glitchy and slow.
-ProRes inside MXF on Avid is a joke. Their glossy brochures talk about compatibility, but I can’t make it work. I only use AMA to transcode to Avid media, not for editing.
-In Premiere> QuickTime for Windows cannot export H.264 on computers that have more than 16 CPU cores due to a problem in the Apple H.264 compressor component.

 

Multi-Cam:
-If possible I do Multi-Cam on Avid with transcoded DNxHD36 media if more than 4 angles for the offline. For online relink to native media.
-Next best is Premiere with the same DNxHD36 media if more than 4 angles and native media with <4 angles.

Delivery:
You can edit without QuickTime but deliver in Quicktime. When a client asks for QuickTime & certain codec, I give them what they want. This means keeping a Mac around somehow for ProRes delivery.
Saving MXF clips to MXF container in Premiere
AVID – Fast Import Apple ProRes .mov to AVID Pro Res .mxf Workflow
Arri Alexa bypassing Quicktime using MXF instead

Interoperability with DAM systems:
Dalet supports import/ingest/export in MXF
-you have to test and update your workflow company wide to make use of this.
Delivery of MXF with Closed captioning

Uninstall QuickTime to avoid all kinds of conflicts with your NLE.
It forces your NLE to use their workarounds for play back Quicktime codecs.
Don’t worry, you can always put it back, the NLE usually has it’s own process to playback QuickTime sources. Premiere works fine on Windows without QuickTime installed.
It is a great trouble shooting step to try to improve the realtime capabilities of your system.

Uninstall QuickTime on Mac:
mac-qt-library

1. Internal HD>Library>Quicktime
2. Remove all or just the components you are trouble shooting, out of this folder.
3. restart apps using QuickTime.

Uninstall Quicktime on PC/Windows:
1. Control Panel>Add/Remove Software
2. Restart Computer.
3. click here to download latest version from Apple.

Resources:
Security Alert uninstall Quicktime
-Avid’s take on Quicktime
Do your own performance tests.
How do I speed up rendering, exporting, or encoding?
PREMIERE PRO AND MULTIPLE GPUS
-Adobe’s smart rendering workflow
Good article about how video codecs work
QuickTime is deprecated? What does that mean in practice?
What is Apple doing with QuickTime?
-Adobe says it buried on a support page: ProRes is not supported on Windows!
A Guide to Common Video Formats: Containers, Compression, and Codecs
Build a balanced system
GPU vs CPU how is the work shared?
Free Resolve-12: Great basic editor, fantastic color corrector > the standard for most TV & Film. Also has one feature no other NLE’S have: Export files from timeline as individual files!
Free Red CINE-X: Transcodes any Red file into an editable format: even Quicktime ProRes on Mac and PC!




Performance test: Codecs and QuickTime on PC.

I had an aha-moment recently after switching from a QuickTime based project on Premiere, to C300>native MXF. The system was all of a sudden snappy, no stalls, a lot more realtime performance and super fast renders and exports.
My system must have been struggling with something…
A codec that works with low CPU load on a Mac might have a high CPU load on PC.‬ On Mac QuickTime uses the 64-bit AV Foundations framework but on PC Premiere still uses the deprecated 32-bit QTkit framework. Adobe finally wrote an unofficial white paper March 2016 about performance. ProRes is not supported for export on Windows.
QuickTime on Mac works in 64-bit (use all your RAM), OpenCL GPU-accelerated (use your graphics card for playback, render and export), and support Grand Central Dispatch (use all of your CPUs and cores).
With QuickTime and most codecs on PC, I’m experiencing “mini crashes” > white screen lock ups, plus stuttery, slow performance and slow renders & exports. Why?
The QT-32 process on PC runs in a 4 GB memory space, even when lots of memory is installed. To keep up with playback, the system has to keep swapping banks of memory (pagefile.sys) to the hard-drive instead of using all installed memory. I think the white screen lock up is the pagefile.sys being written or read by Windows from the slow internal HD! (freezes about 30sec)
There is no 64-bit version of QuickTime, primarily because Apple has ceased development of QuickTime on Windows (except for security and minor updates). QuickTime has been 7.x for 10 years now, and 8.0 does not seem to be anywhere in the distance.
I urge you to spend one hour to do your own tests, on your system with your own media. Figure out what formats and codecs give you the best realtime performance. Each of the NLE’s has it’s own workarounds to bypass QuickTime depending on codec used. Save yourself and your clients thousands of hours not waiting anymore for renders, exports and restarts.
mi-smileMike

Resources
Security Alert uninstall QuickTime on PC.
-Avid’s advice about Quicktime
QuickTime is deprecated? What does that mean in practice?
What is Apple doing with QuickTime?
How do I speed up rendering, exporting, or encoding?
CPU vs GPU
PREMIERE PRO AND MULTIPLE GPUS
-Adobe’s smart rendering workflow
A Guide to Common Video Formats: Containers, Compression, and Codecs
-In Premiere> QuickTime for Windows cannot export H.264 on computers that have more than 16 CPU cores due to a problem in the Apple H.264 compressor component.

Delivery in MXF
You can edit without QuickTime but deliver in Quicktime. When a client asks for QuickTime & certain codec, I give them what they want. This means keeping a Mac around somehow for ProRes delivery.
Saving MXF clips to MXF container in Premiere
AVID – Fast Import Apple ProRes .mov to AVID Pro Res .mxf Workflow
Arri Alexa bypassing Quicktime using MXF instead
About smart rendering in Premiere

Here is what to do
1. Set yourself up for testing> press Ctrl+Alt+Delete and start Task manager> Resource Monitor >Overview tab.
Make visible
-CPU
-Disk
2. Observe CPU Usage & Disk I/O with different codecs.
With an optimized codec like AVID, Sony EX422 or Cineform your CPU load should be low 5-10% and your Disk I/O should match whatever the bit rate is of the codec. My conclusion is that the QT32 process doubles CPU load during playback, render and export. An Intra-frame, non-QuickTime codec plays at 5-10% CPU, as opposed to 50-60% CPU as QuickTime, while also adding to Disk I/O. It means that a timeline with filters will take twice as long to export and playback always stutters.
The most astonishing thing is that a Premiere timeline with an QuickTime codec, rendered green, will still stutter playing back on both Mac and PC! I think that is embarrassing and unacceptable!
The culprit is the  QT32 process multiplying itself for no reason, taking up more and more CPU cycles. As your project grows and your timeline becomes more complicated this keeps dragging your realtime capabilities down until it literally stops… Restart!

Below Playback the way it should be: Low CPU load and Disk I/O at the bit rate of the codec. Note the files on the timeline being read in the Disk I/O list.
1AVID_AVID-MXF_PLAY_NO_FILTER
Now look what happens when playing back a ProRes QuickTime in Premiere.
1. CPU cycles taken up by Premiere
2. CPU cycles taken up by the QT32 process
3-8. The files on disk being read by the QT32 process including:
9. pagefile.sys the Page File it has to keep writing and reading off the drive to compensate for the 4GB memory limit. >Causes “white screen lock up” (about 30 sec.)
PR_PRO_PLAY_NO_FILTER_pagefile




Free DCP

If you’re lucky enough to own Adobe Creative Cloud CC 2014/15 you can create a free 2K DCP.
For 4K, 3D, HFR or other frame rates than 24 or 25 you have to buy the pro version of Wraptor DCP
1. Create final file in the highest resolution possible with Audio as either:
A. Stereo
B. 3 channels (Left, Center, Right)
C. 5.1(L,R,C,LFE,Ls,Rs)
DCPs are True 24 fps or True 25 fps. Progressive!
For best results do your own conversion from 23.98 or 29.79 using Premiere or AE.
For interlaced footage 29.97, make a true 24 fps timeline >1998×1080. Set field options to >Always De-interlace.  Don’t use optical flow. Don’t stretch to 1998, just leave a little bit of black on both edges, scaling creates artifacts.
The best de-interlacing is done in AE or Red Giant’s Frames.
Speed up Audio by 0.1% (for 23.98) in Audition, then add to sequence.
DCP_WRAPTOR

2. Drag your final file into AMA and choose Wraptor DCP.
The only options are 24/25 frame and aspect ratio.
Choose “Flat” for 16×9 HD footage.
Bitrate is fixed to 250Mbps which is very good and creates files of 15GB per 60min of footage.
The encoder takes your file and creates a series of JPEG2000 files at true 24 fr/sec.
It takes 4x Realtime (1hour of film takes 4hr)
3. 2 tests before you send it out:
1. Test the Files by re-importing into Pr and re-combining the Video.mxf with the audio.mxf from the DCP folder.
2 Install DCP Player Free (WIN ONLY) 
4. Format USB stick as Linux ext2 or ext3 file system. Difference ext2, ext3, ext4
On MAC most people use the Paragon extension for osx.
or Ext2 File System Driver for WIN
NOTE: An Ext 2 formatted USB Drive will not be recognized by a WIN or MAC computer.
5. Copy your DCP folder to the EXT2 USB stick.
6. Ask your local theatre if you can stop by in in the morning for 15 mins and load on their D-Cinema server.
Transfer of 15GB/1hr takes about 5 min >USB3.
DCP-usb DCP-check

Most projectionists and festivals advice against using encryption.cru_dataport_6607_7100_0500_dx115_carrier_only_sata_1412203509000_1066163
Make sure you can deliver USB stick instead of old-fashioned CRU-115 Data-port carrier.

 

 

The Wraptor Lite DCP creator follows academy specs:
Framerate: 24fps (not 23.976)
Compression: JPEG2000
Color Space: XYZ
2K Format: Full Container 2048×1080, Flat (1.85:1) 1998×1080, or Scope (2.39:1) 2048×858
Audio Format: 48kHz uncompressed, 24 bit
Minimum of 3 Audio Channels: Left, Right, Center or 5.1
Encryption: Unencrypted material only
A DCP is nothing more than a set file structure in a folder that the D-cinema server can read.
Read more about DCP’s here or Filmmaker Magazine
How to prepare your project for a DCP Digital Cinema PackageDCP-files




Sample Level Audio Editing Mode in Premiere CC

By following these steps you create a “sample level edit mode” in Premiere CC.
You only want to be in this mode for precise audio edits. In this mode a 25th or 30th of a second is huge, it is like putting a microscope on the audio, the picture usually doesn’t even move!

1. Go into keyboard shortcuts >search for “time unit” and give all the “show audio time unit displays” the same shortcut (I did F10)

prem-aud-sam
2. Focus the timeline panel and click your new shortcut.
3. Right click the timecode display and make sure it’s on samples.prem-show samples
4. Now you can toggle with your new shortcut between sample & frame editing using all the normal controls.
in/out, zoom, rubber banding, keyframes etc. It works on both Source and Program side because you made all panels switch with your new shortcut.
Space will still playback, but that’s about the only navigation key still working normal.




Dealing with odd files like: .mpg, .mp4, Powerpoint, Animated GIF’s, Flash .swf .flv .fla

My secret weapon: The VLC player and screen recording software like Screenflow or Camtasia to make files into a Quicktime movie.
Flash
Read this to learn what an .swf file is.
When you receive .swf flies to use in an edit, ask the developer for the .fla files. Or better ask them to export a Quicktime in dimensions and frame rate of your edit to avoid alignment errors. Flash is vector based so in most situations it can be resized without getting pixelated. This is especially important when something was created for the web and now has to be sized up for broadcast.
The .fla file is the editable project file that was used to create the .swf file, it also holds the audio. When you re-import a .swf into Flash the audio does not get imported.
2 steps to pull audio and video from a .swf
1. VLC and screen recording software to record the audio.
2. Flash app (download free trial) to export the animation. Change the stage-color to black (or ignore stage and make Alpha) and use the Quicktime settings to match the dimensions and frame rate of your timeline.
.flv are flash video files that most software can import. MPEG Streamclip and VLC also open it.




Tips for getting out of password hell

The number one reason I get into password hell from time to time is because I’m juggling so many passwords and I mix them up. That said, I think it is sometimes hard to determine if my password hell is caused by human error or an error on the end of the entity I’m trying to log into. Company’s like Microsoft, Adobe & Apple juggle millions of passwords that they have to pass along between their websites, user accounts and applications. It doesn’t always work smooth, they are not honest about glitches in this system and there is not a lot of info on the web. Most web searches direct you to a page to reset your password, sometimes creating an infinite loop of resetting the password, trying to log in, resetting again etc.

Watch out, not every website lets you copy and paste a password or store passwords with browser or keychain.
Example: readkit & paypal don’t let you paste. Results will be wrong, that’s how their site is coded to prevent spammers.

Read this great times article about creating good passwords.

 

Apple
Apple is famous for it’s password loops, sometimes it happens on the iphone/ipad sometimes with appstore/itunes. They seem to have problems passing passwords along to the different parts of their servers. Biggest lesson I learned recently is: Reset your password on an Apple device! The browser iforgot solution is glitchy.

Problem: When changing your apple ID password in a browser you get message:
“your password contains unacceptable characters”
even though the green buttons are all lit and the password checker says “strong”
Solution : reset password on an Apple device instead.

Problem: Sometimes you unknowingly created multiple Apple ID’s because you thought you where giving an alternate email address.
Solution: pay attention to the email address Apple replies to, when you reset your password.

Problem: Even though you just successfully changed your Apple ID password, your iphone, ipad, mail application or itunes reject the password.
Solution: reset password on an Apple device.

iPhone email

Problem: Can’t send email using PoP/IMAP:A copy has been placed in your Outbox. The sender address “[email protected]” was rejected by the server.”
Solution: Fill username (your full email address) and password in on the smtp part of the email settings even though it says “optional”

 

 

 

 

 

 

WordPress (self hosted blog)

WordPress (wp-login.php) brute force attacks are common place lately.
Read this login trouble article
13 Vital Tips and Hacks to Protect Your WordPress Admin Area
In WP always try to change passwords from within the admin page.

If you’re caught in a loop, can’t get in or password new user doesn’t work then go in using phpMyAdmin. That should always work because it lets you edit the password database direct.
Make sure you edit the correct database! (I use unique email addresses for each user to make it easy to double check)

Adobe
Adobe creative cloud seems to have propagating issues similar to Apple’s.
Problem: You can log in to Application manager on one machine but not another.
Solution: Reset password on the computer that you want to work on. (and deal with the other machine later.)

Google
Google is great when it works and their servers are up, it sucks when they are down. They are not transparent about ongoing problems and because everything is free, there is nobody to yell at over the phone.

 

 




Shooting Video with the Nikon D600

When you buy a D600, you basically buy 2 camera’s in one box. You understand this when you notice that there are separate iso settings for video & stills.


pros
-great still camera
-audio with separate box to add  XLR, a little noisy, but ok for a DSLR.
-rolling shutter ok, comparable to 5Dm2
-haven’t noticed any overheating issues when shooting multiple 20 min takes.
-Nice dynamic range and low-light performance up to 1600 ASA. Grain, especially in the blacks looks bad at higher iso settings.


cons
-terrible moire & false colors etc. in almost every shot! I’m talking about 1080 24, 25 and 30 fps.

The 60fps 720p settings are even more useless. This is with sharpening turned down.
Sorry Nikon, you are still lightyears behind Canon & Sony.
The weird thing is that this full frame 24Megapixel sensor is made by Sony, the Nikon processing just screws it up. The internal H264 compression doesn’t help either and right now (Dec 2012)
you can’t use the HDMI out because it is not full screen. You can’t shoot any wide shots in focus because of the moire and watch out for anything shiny in a shot because it will be busy as hell.
-recording length limited to 20 mins.
-you can’t extend the dynamic range using a picture profile like the Technicolor profile




iTunes Match explicit/clean problem fixed.

For Mac OS X 10.6 or later!

1. Download SUBLER
2. In iTunes select the songs that are recognized as clean but you want the explicit version of.
3. Right-click and “Show in Finder”
4. Select all the songs again in the finder, or work in batches per album.
5. Drag the songs onto the Subler app to open.
6. In Subler set each song to explicit and close to save.

7. In iTunes, reset the cache.

8. Click the songs and see if they are now showing as explicit.
You might have to turn iTunes Match off/on to get it to recognize the new tags.

 




Shooting with the Canon C300

2.5 hr shot @ 50Mbits 1080p 29.97
Copy 60GB using USB-2 for the reader and FW-800 to the drive > 30min > 1/5 realtime

FCP-7 (with Canon XF plugin installed)
WARNING DO NOT UPGRADE TO OSX 10.8 Mountain Lion! 

Log & Transfer to Prores 422 using 2009 Core Duo macbookpro > Realtime, half realtime with i7 Quad-core
FCP-X can edit immediately (but secretly transcodes to Prores 422 when the machine is idle)

AVID (with Canon XF AMA plugin)
Can edit immediately with AMA
Transcode to DNxHD-45 > (using i7 2.7GHz) > half realtime

So:
-Transcoding in FCP or AVID seems to be half realtime with an i7 Quad-core CPU.
-Avid & FCP-X have the advantage of being able to read and edit the card immediately.
-I still have to test editing with DNxHD-45 and then relinking to the AMA files or the other way around. Don’t know what the limit is on how many hours you can link with AMA