Last year I volunteered to record an entire weekend’s worth of vintage computer talks.  Unsurprisingly, this also involved editing an entire weekend’s worth of vintage computer talks.  All of the footage was 1080p60, and was shot or delivered compressed (a mixture of AVCHD and H.264 MP4).  While this saves space, it is not always fluid to edit with, as compressed codecs arrange frames into groups that are highly dependent on each other.  The end result is that seeking around and cutting highly-compressed footage can feel sluggish even on extremely powerful systems.  My system was beefy for 2012, and can still surprisingly hold its own in 2017:  A 6-core/12-thread Core i7-980X with 24G of RAM, and 4TB of spinning-disk storage made up of 3x2TB disks in a RAID-5 array (capable of delivering up to 700MB/s sustained read speeds at the outer edges of the platter) with a 4th hot spare.  Despite the high specs, editing 1080p60 H.264 footage last year was sluggish to the point where I considered using an intermediate codec.

A quick primer on intermediate codecs

Intermediate codecs are used when you want to transcode your footage to something that is much easier for the computer to decompress, making it much faster to edit with.  There are two basic types, lossless, and lossy:

Lossless codecs exactly represent each original pixel in the source, and can be used interchangeably with the original footage through any number of processing or compression passes.  (Think of them as running the footage through PKZIP or 7-ZIP, but using more lightweight algorithms more suitable for decompression speed.)

Lossy codecs throw more information away, but do so in a manner that looks the same to human eyes.  They don’t match up with the original source pixel for pixel, but most people wouldn’t be able to tell the difference.  The resulting file sizes are much smaller than lossless codecs, but because some information is thrown away during the compression phase, you have to be careful running such footage through multiple processing or compression generations as the image could degrade unacceptably.  (For the old-timers: This is like what happens when you make a copy of a copy of a copy of a videotape: The end result is noisy, washed out, and barely watchable.)

Compressing into an intermediate codec takes time, so it is usually done during the ingestion process (such as with DaVinci Resolve or Adobe Prelude), or in a batch process overnight, or both.

Our test case

I decided at the time to pick a 10-minute piece of 1080p60 footage and transcode it to every intermediate codec that met the following criteria:

1. Worked in Windows without Quicktime (Quicktime for Windows was discontinued in 2016 due to security flaws).  This eliminated ProRes and Avid’s DNxHD.
2. Over 5 years old, compatible with Windows 7 or above, and relatively battle-tested
3. Evidence of recommendation online by video content producers (ie. something more professional than, say, people doing anime music videos)
4. Worked in Premiere Pro without crashing or odd behavior
5. Didn’t demand more than 200MB/s out of the I/O subsystem to maintain 60fps playback (this was more to keep filesizes down than anything else)

Additionally, since my source footage was not high enough quality to require HDR-level processing, I was realistic and considered codecs that had a YUV 4:2:2 configuration (and eliminated codecs that could only do 4:2:0).  This led to the following codec list:

• UTCodec
• Cineform
• Newtek SpeedHQ
• Blackmagic Design’s MJPEG codec (their other codecs are uncompressed)
• Lagarith
• Grass Valley HQX (both lossless and lossy variants)

To provide some interesting comparisons, I also included the following:

• YUV 4:2:2 8-bit uncompressed, to see what was possible given a fast I/O subsystem
• Cinepak (1991), possibly the fastest useful single-core decompression codec ever made
• HuffYUV (2000), the first popular free lossless codec for Windows

After my 10-minute 1080p60 sample was compressed into these codecs, I then defragmented my RAID array using mydefrag with a script that put all of the codec comparison files at the fastest area of the array (the beginning).  I then measured decompression speed using VirtualDub’s File->Run Video Analysis Pass feature, then measured I/O and CPU usage using Windows 7’s Resource Monitor.

Before reading on, I want to stress that the above criteria were the only factors considered.  There are many other things to consider if you have different production targets or workflow needs, such as 10-bit or 12-bit color depth, or iterative recompression stability.  These vary from codec to codec and are not discussed here.

The results

Here are the raw results (you might need a non-mobile browser to see this table correctly):

From this data, I eliminated codecs that were not capable of playing back 1080p60 footage at the bare minimum requirement of 60 frames per second.  I also eliminated Cinepak (which was only included for teh lolz) and Grass Valley HQX Offline, since the visual quality of these were unacceptable and I did not want to edit with proxies (I was not in the field on a laptop, but was on my actual editing desktop system).  I also eliminated YUV 4:2:2 uncompressed, because the storage requirements of transcoding everything to an uncompressed format was not practical.

From the remaining data, I concluded the following:

Playback performance (best to worst)

1. UTCodec 4:2:2 8-bit
2. Grass Valley HQX Online Standard
3. Grass Valley HQX Online Fine
4. Cineform (Medium HD)
5. Cineform (High HD)

File size (best to worst)

1. Grass Valley HQX Online Standard
2. Grass Valley HQX Online Fine
3. Cineform (Medium HD)
4. Cineform (High HD)
5. UTCodec 4:2:2 8-bit

Conclusion

For over a decade, I assumed that Cineform was the gold standard of intermediates, and was indeed the codec I used whenever I needed an intermediate workflow.  Recently in an online discussion, a colleague who is a professional cameraman and editor extolled the virtues of Grass Valley HQX.  HQX was not in my original comparison, so I added it and re-ran all of these tests in the same hardware configuration.  I was surprised to see that Grass Valley HQX was a hair better than Cineform in all areas.  I will definitely give it a spin for my next project.

To be absolutely fair, Cineform is (still) no slouch, and has served me very well over the years.  You could do much worse than either Cineform or Grass Valley HQX; either of them should suit your needs for an intermediate codec on Windows.

If you have a multi-core processor and lots of free disk storage, UT Codec remains the fastest lossless codec available on modern hardware.  However, evaluate your project needs before committing to it, as not every project needs 100% lossless compression.

Finally, if you only work with 24p/30p material, just about any of these codecs will serve you well.

Update: I’ve recently found that Cineform performs much faster if you can handle the quality the “Medium HD” preset provides, which can exceed the playback rate of GV HQX.  But they’re both good to use.

Update 2:  There is a wonderful, filterable list of intermediate codecs made available by David Kong.

Update 3: Cineform was open-sourced (woohoo!) and Virtualdub-filtermod already supports creating it using the official SDK.