Yesterday morning, I landed a very major change to our Javascript, XPConnect, and DOM code. This change is commonly known as Compartment-Per-Global. Less formally, it’s CPG. Even less formally, it’s the ****-ing holy grail that blocks everything important. I wanted to give a few details about what’s going on here, and what to expect.

Before getting deep into history, I want to leave a quick up-front warning for frontend/chrome developers. Every scope, including JSMs, now has its own compartment. So objects that formerly touched each other directly now interact via wrappers. This should be pretty transparent, since chrome->chrome wrappers just forward everything along. Here’s the rub though: when chrome accesses content objects, it gets a special Xray (formerly known as XPCNativeWrapper) view on the underlying object, which bypasses content expandos and gives chrome a fresh slate. This wrapper also has a special expando holder to store expandos that chrome wants to put on the object. Xray wrappers are per-compartment. So previously there would only be one Xray wrapper (and thus one holder object) for a given content object, but now there can be several. This means that different chrome scopes no longer share expandos for content objects. You have been warned.

Back to the story.

Those who were around for the Firefox 4 release remember the hair-raising months when some of Mozilla’s most seasoned hackers struggled to land one of the release’s flagship features: Compartments. Compartments weren’t a hugely sexy feature from a marketing standpoint, but JaegerMonkey wouldn’t work without them, and shipping Firefox 4 without the new fast Javascript Engine would quite possibly have been the end of Mozilla. Remember Brain Transplants? That was the final piece of Compartments. Needless to say, it was one of the most stressful times to be involved in this area of the platform. I was quite thankful to be watching from the sidelines while finishing up my degree.

Compartments and Brain Transplants

mrbkap, circa October 2010

Andreas has a nice blog post explaining what compartments are and why they’re important. The details are complicated, but suffice it to say that all Javascript objects used to share a single heap, and Compartments gave us a separate heap for each origin. This was a win on a number of fronts, including decreased GC pauses and stronger security invariants.

Most importantly though, it made everything under the hood make much, much more sense. I use this phrase broadly to refer to both understandability (how easy is it for the programmer to grok what’s happening) and power (how well do the abstractions we use map to the things we want to do). The value of this can’t be overstated; It’s the difference between “we could fix that – I guess – It would probably take a month to implement and another month to track down all the regressions” and “sure, I can probably write and land that by the end of the day”. The more sense the code makes, the easier it is to diagnose and fix bugs, add new features, and make things fast. And as a rule, more compartments makes the code make more sense.

So having one compartment for every origin (whereas before, we effectively had one compartment for all of Firefox) was an amazing milestone. It helped us ship Firefox 4, paved the way for countless improvements and modernizations, and put us in a much better place with respect to things like security. But while having one compartment per origin was an amazing milestone, having one compartment per global object (think |window| in javascript) is a never-ending stream of goodness.

Rendition

Artist’s Rendition of Compartment-Per-Global
(by Demilion – http://fav.me/d3g2fue )

So why didn’t we do Compartment-Per-Global from the beginning? There are a number of reasons, but two stand out. First, compartments at that time imposed a much larger memory overhead, and so we didn’t want to be be creating them willy-nilly. More importantly though, ensuring that same-origin code shared compartments was a good way to avoid having too much stuff break. Objects in different compartments have a membrane between them that can only be crossed by Cross-Compartment-Wrappers. These wrappers are mostly transparent, but in practice there are a few ways in which they change behavior in subtle ways. Fortunately for us at the time, the security model of the web places pretty strong barriers between code running from different origins. So in general, code wasn’t expecting to do intimate and subtle things with objects from other origins. Therefore, by introducing compartments on a per-origin basis, we could avoid breaking too much existing code. Privileged browser code (aka chrome) was a particularly big beneficiary of this strategy. Chrome code all runs with the same principal (System Principal), so most of it saw no change.

So Firefox 4 landed with a splash, and everyone was happy. For a while. As time went on, it became apparent that more and more stuff that we wanted to do required a one-to-one mapping between compartments and global objects. A lot of these dependency chains are pretty long. For example, IonMonkey won’t work until we get rid of JSStackFrames, but we can’t get rid of JSStackFrames until we change our security architecture to pull principals directly off object and context compartments, but we can’t do that until the compartment principal exactly matches that of the scope (i.e. until we have CPG). And in general, there are lots of operations that can be made much faster with the help of CPG.

So about a year ago, Luke started looking into what it would take to make this a reality. A number of dependent bugs had to be fixed first, and other work got in the way. By early January, he had patches that would at least launch the browser. But there were lots of things that were broken. So I took the reins for a while, and pounded away at XPConnect, the JS Engine, the DOM, and the frontend to get things green. Some fixes were simple. Others were harder. Some had a very high comment-to-code ratio. But almost everything was landed as dependent fixes, meaning that the final patches that flipped the switch were quite manageable.

The final landing was small, but its effects rippled throughout the code. So every few days it was delayed, something new would be checked into the tree that broke it. This made the last stretch pretty strenuous. But thankfully, I wasn’t alone. Luke did a ton of work (and in fact holds authorship on eight of the nine patches that landed on Thursday). Mano, Mak, and zpao were hugely helpful in diagnosing browser-chrome issues. Blake was on the hook for reviewing all the really tricky stuff. Boris provided much-needed insight, and took some of my other work so that I could finish this stuff up. Johnny greased the wheels when they needed it. Kyle heckled. And Ed Morley helped out at the last minute to avert a backout. Go team.

Follow

Get every new post delivered to your Inbox.