So Many Colors

(A Belorussian translation of this post is available, courtesy of Designcontest)

This summer, I’ve been working on getting Mozilla’s color management backend ready for the prime time. We’re finally turning it on in tonight’s nightly builds, so I thought I’d give a bit of background on the history of color management in Mozilla and on color management in general. You should all go download the latest nightly when it comes out and give it a try.

In a nutshell, color management involves using information about a computer’s display, and in some cases information about certain images, to render pages more accurately. Contrary to popular belief, an RGB value (such as #ffffff) alone does not represent a color. Rather, the values for the red, green, and blue channels can be thought of as coordinates in a color space that contains the actual colors. The color that an RGB tuple represents thus depends on how the color space is defined (see Wikipedia for more information).

This type information is generally stored in a file called an ICC color profile. Computer displays often ship with ICC profiles that specify exactly how various RGB values will look on the screen. Additionally, JPG and PNG images are often tagged with ICC profiles to describe the color space of the pixels. In theory, every image should come with a color profile specifying its color space. However, many images do not contain embedded profiles, nor would it make sense for a CSS stylesheet to embed a color profile for CSS colors. As a result, the W3C determined that untagged web content should be assumed to be in a standard color space called sRGB. So color management in Mozilla is really a two-part process: we map incoming RGB values out of their respective color spaces, and into the color space of your display (which we obtain by querying the operating system).

Colors are more complicated than you might think.

About a year ago, Mike Beltzner unveiled Mozilla’s color management support in a blog post. Unfortunately, there were some major issues that kept the color management mode set squarely to ‘off’ for the release of Firefox 3.0.

First of all, there were quite a few rendering bugs filed for our color management implementation. I was able to confirm a good number of them, and discover a few more myself, and I’ve landed the necessary fixes. There were also quite a few issues with our Reftest rendering test suite that needed fixing so that the tests would pass with color management on. These issues have also been fixed. However, several people reported bizarre rendering behavior (whites showing up yellowish, blues showing up purple) that turned out to be the result of bogus display profiles on the reporter’s system (many thanks to Chris Murphy for lending his expertise on this issue). This is scary, because it raises the possibility that any number of people might see inexplicably weird colors in their browser once we turn on color management. As such, we’re hoping to get as many testers as possible in the nightlies and betas to see how widespread the problem is.

Another concern we had over color management had to do with plugins. Since plugins like Flash and Java do their own drawing, we don’t have any control over whether they use color management. While an extension to the plugin API to allow this is in the works, there’s currently no way for the browser and plugins to coordinate on color management. If the browser color corrects a color while the plugin doesn’t, they no longer look exactly the same side by side. At this point, it’s unclear exactly how much of a problem this is, but we’d appreciate feedback.

However, the biggest reason color management was disabled for Firefox 3.0 was that it regressed performance by about 20-30% (according to our Talos tests) when switched on. This was deemed unacceptable for a default setting. As such, I spent a great deal of time profiling code and then devising, implementing, and testing dozens of strategies to squeeze the performance overhead as close to zero as possible.

This turned out to be really hard. Color management is a per-pixel transform, meaning that we need to call a function for every pixel of an image (CSS colors are easier since repeated pixels are known to the rendering code). Such an embarrassingly parallel problem would be a perfect candidate for GPU acceleration, but we’re just beginning to draw up plans for OpenGL and Direct3D backends to Cairo, so it will be quite a while before we can just write a custom shader and make the entire problem disappear. As a result, I dug in my heels and worked on making the transformation of one pixel really, really fast. After some rearchitecting, various precaching strategies, and a whole lot of hand-tuned SSE2 assembly, the performance overhead of full-page color management is finally at about 4-5% on all platforms.

For the past few months, this seemed to be enough. Beltzner was receptive to the idea of turning it on, and nobody in either of the development meetings raised any gripes on the issue. Robert O’Callahan and David Baron had voiced some concerns about standards compliance and interoperability, but both were willing to turn it on for Firefox 3.1 Beta 1 in order to get feedback from the community. However, as Vlad and I were landing the final patch to turn color management on, we were beset by a flurry of naysayers who didn’t feel like the performance hit was worth it.

As a contingency, I had previously implemented a third mode for color management, one that color-corrects tagged images only, that has a performance hit of about 1%. Safari has been color-correcting tagged images for a while now, and this is certainly the dominant use case for color management. Given the pushback on full color management, it looks like we’ll have to settle for a default of “tagged only” for now. The capability for full correction is available in Firefox, and hopefully we’ll have an easy way to switch modes by the time 3.1 rolls out the door. For now, you can go to about:config and set gfx.color_management.mode to ‘0’ for no color management, ‘1’ for full color management, and ‘2’ (the default) for tagged-only correction (for technical reasons, you’ll need to restart your browser for this setting to take effect). Deb Richardson put together a great blog post where she highlighted some of the differences between color-managing and not color-managing tagged images. I’d encourage you to check it out.

Color Management Off, Color Management On (Courtesy of Deb)

Personal disappointment aside, I think it would be a good idea to open up discussion on the subject of full color management on the web. In a world where every browser supported color management and nobody had bogus display profiles (idealistic, I know), a web designer could pick a color scheme for his or her web page and rest assured that it would actually look the same for each viewer. Whites would really be white, and that soothing hue of green would look just as good on your mother’s laptop as it does on your desktop. I’d encourage people to try browsing the web with gfx.color_management.mode=1. Is the performance impact noticeable? Do you run across any web pages that don’t look right? How many color-matched plugins do you see on a daily basis? Do you care about color correction? As always, Firefox is your browser, and we want to know what you want. So tell us.