The VFX Color Pipeline
Color is, well, hard. Even when you know everything about color, you know nothing about color. Aside from a few masters of the subject, even those with years of VFX experience can find the color world and all it’s intricacies a bit daunting.
The only way I learned any real concrete understanding of it was by coding a VFX flip book review application that included a color pipeline. I know most people don’t have time for that so I thought I would boil down what knowledge I have for those in a hurry as well as, for interested parties, a deeper dive into the wild world of digital color.
The Basics
Obviously, we can’t really start without the most general question…
What is a Colorspace?
(Warning, fancy description inbound)
A color space is a specific organization of colors. In combination with physical device profiling, it allows for reproducible representations of color, in both analog and digital representations. — Wikipedia
My definition: “Based on the way that image data is stored, the colorspace tells applications how to interpret the information and display/manipulate it properly.”
We’ll get into why they exist and we don’t just use the same colorspace all the time soon. For now, it’s safe to assume most image formats have a “default” colorspace. A proper jpeg, for example, will almost always be stored in the colorspace sRGB. OpenEXRs, on the other hand, lean towards always being in Linear.
tl;dr
For those who are hear to learn just a foundation of the color pipeline rather than the intricate math and definitions, let’s go through the absolute basics and an example “Utopian” color workflow.
At it’s most brief this common workflow does:
- Convert input image(s) to a common colorspace, typically Linear.
- Do our modifications on these images in the same colorspace. This makes it easy to do image manipulation.
- Then, optionally, apply a Look, or series thereof. A Look typically takes the form of a LUT. IMPORTANT: A Look does not change the colorspace!
- Finally, at the last second, we convert from that common colorspace where we did our manipulation to account for the way your display shows color. Most displays use the colorspace sRGB or something close to it. When writing images to disk, we may bake this information in (a-la jpeg) or leave it in the linearized space (a-la OpenEXR)
That’s it(ish)! If you can grasp that, you now have an above average understanding of color when it comes to the VFX world. There is no “one color workflow to rule them all” but rather dozens of ways to communicate through colorspaces. All of these ways, however, follow some form of this paradigm. Manipulation on all imagery is done in a single colorspace and converted back and forth where required.
Before You Go
Before you venture off with your new found color mastery, I recommend getting into the next couple sections to harden your understanding of color theory and the VFX color world. The next section get’s into some of the finer details without too much math and history.
The Not So Basics
For those who have decided to stick with me on this color trip, I salute you. Let’s start with some questions that you might ask “Why didn’t we cover this in the basics section?”
What is Linear Color?
Linear color is crucial when talking VFX. Thankfully it’s also the most straight forward. This is the “I am right!” colorspace. This simply means that a color value of 0 is black, 1 is white, and 0.5 is 50% grey. That’s vital because we do math “linearly” on a daily basis (e.g. 0.25 * 2 = 0.5). It’s the same math you’ve been taught since those fateful days in elementary school.
Images are saved in any number of colorspaces that a “Linear Color Application”, a term I’ve made up, can interpret and convert from said colorspaces to linear. Nuke, for example, is an LCA.
What is sRGB?
This is where the history begins!
Colorspace and much of color theory as we know it today find it’s roots in TV when it was first coming into the world. At that point we only had so much information that could be shipped from the tubes and sensors. We couldn’t waste any bandwidth.
Humans, the clever devils that they are, developed a standard to go from light entering the camera, which is linear, to image information that could be displayed by your average television with a half decent range of values.
Fast forward to slightly more modern times. Circa 1996. Mariah Carey was killing the billboard charts, grunge was still in for no good reason, and it was then that HP and Microsoft sat down and developed sRGB to standardize the way our average monitors displayed color.
While this colorspace is extremely outdated, it’s still widely used and will most likely stick around for a long time. In the sRGB colorspace 0 is still black, and 1 is still white, but to get to %50 grey, you have to use a value of ~0.73536 and 0.5 is 18% grey. Which, I know, sounds strange, but let’s look at a little math with pictures to describe what happens.
Note: There are dozens of other colorspaces that you’ll see out there but the concept is the same as sRGB.
What Does it Mean?!
At this point, we can put a some elements together to see why linearization is vital.
Remember that linear equation above?
color_a = 0.25 # linear value
color_b = 0.25 # linear value
color_a + color_b = 0.5 # linear value
Simple! Now, let’s say we did the same math in different colorspaces. := is what well use for assigning sRGB color.
color_a = 0.25 # linear value
color_b := 0.25 # sRGB value
color_a + color_b = 0.3008760881715568 # linear valueHuh?? How TF can 0.25 only equal 0.05?! Well, color_b has to be transformed into a linear value in order to add them together.
To convert to linear, we do the approximately the following (Note: you don’t have to know this. That’s what the LCAs are for):
convert_srgb_to_linear(srgb):
if srgb <= 0.0404482362771082: # ?
return srgb / 12.92 # ?!
else:
return pow(((srgb + 0.055) / 1.055), 2.4) # ?!?!That’s crazy right? That’s the “linearization” process for just sRGB. Each colorspace comes with it’s own equation to linearize and then transform it back to the original. This process is vital because otherwise, you’re doing the math on values that are not equal!
You Probably Skipped That Part
It’s complicated and boring. I get it. But, understanding that, even at a basic level, will unlock a whole new way to think about not only color but how we can manipulate it to do our bidding.
Once We’re Linear.
It’s VFX time!
Now that we’ve converted values to linear via their input colorspace, we can go crazy with any operations required. Comp our scene together, fire up the retime, get that building on fire, blow up that bridge! All of this within a linear color world.
That’s the best part — the math, while complicated, is rational and we get expected results because we’re not using values scrunched in some limiting colorspace.
There are colorspaces out there that account for the whole color spectrum (Check out ACES) but they are relatively uncommon as of this writing.
Showing The User
Skipping the Look section, let’s get right to the monitor. For more information on the Look, check out this article.
When we want to present the image on our device (e.g. monitor, projector, print media, etc.), we have to make sure that the colorspace that display uses is what our image data takes on, otherwise we’ll have widely inaccurate color. That’s where the Display part of our info graphic comes in. Just like the linearization step, we now just convert linear to the display colorspace.
We Made It!
That, with about a thousand caveats, is the color pipeline for VFX. If you’ve ready everything here, I recommend reading it again in a few days to let it sink in a bit more. When first learning this, it’s often something that clicks on the third or forth time going through it.
