Field notes on things that run themselves
Kept Alive by Throwing Almost Everything Away
Right now, on whatever device is showing you this sentence, a program nearby is quietly killing off almost everything it just made — not as a bug, but as constant, unglamorous housekeeping. Every few milliseconds it creates a batch of small objects — a loop counter, a temporary string, a short-lived result — uses each one for an instant, and then, almost always, never touches it again. Left alone, that debris would pile up until memory ran out and the program stopped cold. Instead, a small, fast nursery gets swept for the newly dead over and over, the rare long-lived survivor gets promoted to a quieter shelf checked far less often, and everything else is reclaimed in bulk — a bet, correct almost every time, that whatever a program just made, it’s already finished with.
That bet has a name: the generational hypothesis — the empirically observed regularity that, in almost any real program, the overwhelming majority of objects die young. It isn’t a law of computing, and it wasn’t assumed ahead of time; researchers went and measured how long objects in real, running programs actually lived, and kept finding the same lopsided shape — a sharp spike of things that die almost instantly, and a long, thin tail of things that stick around. Henry Lieberman and Carl Hewitt described the underlying idea in a 1983 paper, but never built it. David Ungar did, the following year, folding it into an interactive Smalltalk-80 system for Sun workstations he called Berkeley Smalltalk, and reporting results that, by his own paper’s title, were non-disruptive: no pauses long enough to notice, using as little as 1.5 to 2.5 percent of the machine’s total CPU time to do it.
A generational collector exploits that skew directly, instead of treating every object equally. New allocations land in a small, dedicated area — call it the nursery. When it fills, the collector runs a minor collection: it copies out whatever is still reachable, and simply walks away from the rest, no per-object bookkeeping required, because the dead objects were never touched at all. The cost of a pass turns out to be proportional to the survivors, not to everything that was ever allocated — which is why short-lived garbage is, from the collector’s point of view, essentially free. Objects that keep surviving repeated sweeps eventually get promoted, or tenured, into an older region scanned far less often, on the reasonable bet that anything that’s lived this long is likely to keep living. Oracle’s own tuning documentation shows what that looks like in practice on an ordinary Java program: a single minor collection reclaiming about 98 percent of a nursery’s contents in under fifty milliseconds. Almost everything that was there a moment ago is simply gone, and nothing running on top of it ever noticed.
This isn’t one collector’s private habit. The JavaScript engine inside every open browser tab runs the identical logic under a different name: V8’s young generation is swept by something its own team calls the Scavenger, descended from a decades-old copying design, built on the same insight — because so few objects ever survive, the collector effectively only pays for the ones that matter.
The convenience isn’t free, though; the bill just comes due somewhere else, continuously, rather than all at once at collection time. To skip re-scanning the entire rest of the heap on every minor sweep, the collector has to know, at all times, which older objects currently point back into the nursery — a short list called a remembered set. Keeping that list current means that every single time the running program writes a reference into an object’s field, anywhere in the whole program, a small piece of extra code called a write barrier fires first, checking whether that write just created a pointer worth remembering. It is overhead paid on every mutation, everywhere, permanently, whether or not a collection is anywhere near happening — the price of near-free minor collections is a quiet, constant tax on ordinary work, hidden inside an operation that looks, to the programmer, like it does nothing but store a value.
So the memory a running program appears to hold — the pool that looks steady, that never seems to fill up, that lets you open another tab or run another loop without the machine visibly buckling — isn’t a store of anything in particular. It’s the residue of near-constant triage. Almost everything the program makes is dead within microseconds, reaped in bulk before it’s ever missed; only the rare object that proves it’s worth keeping gets moved somewhere quieter and checked less often. What looks like a stable pool of live memory is a nursery being emptied, over and over, thousands of times a second — faster than anything running on top of it could ever watch it happen.
One loop I’m watching
Next: a V of migrating geese, which looks like it has one leader out front for the whole trip. Real GPS and accelerometer tracking says otherwise — birds rotate through the hardest, most costly position in a rough, continuous turn-taking, and the formation holds its shape not because anyone is in charge of it, but because that constant trading-off is exactly what makes the shape worth flying in at all.
Tip: the ← and → arrow keys move between issues.
Prefer feeds? Follow by RSS or JSON Feed.
Want something playable between issues? Thought Toys has its own email list and RSS feed.
New to The Standing Wave? Start here →