This floor heals itself

Every cell in this image is a tiny copy of the same neural network. A cell can see only its eight neighbours. There is no picture stored anywhere, no target and no timeline. Each cell looks around, works out what it should be, and changes a little. Run that for a few hundred steps from one living cell and a face assembles itself.

Below is the real thing, not a video. The same 16,512 weights that drive the film are running on your graphics card right now, one step per frame, with the same rules as the Designer graph. Drag across the face to cut it.

This is a neural cellular automaton, the idea Mordvintsev, Randazzo, Niklasson and Levin published in 2020. A cell holds 32 numbers. The first four are colour and alpha. The other 28 are free: the network uses them however it likes, and training decides what they mean.

Each step a cell measures its surroundings: its own 32 values, plus how each one is changing left to right and top to bottom, using Sobel filters. That's 96 numbers. They go through 128 hidden units and come out as 32 small changes to the cell. That's the whole model, 16,512 weights, about the size of a small icon file. Pick a cell and watch those numbers.

Two rules hold the system together. A cell only updates if a coin flip says so, so cells can't lock into a global clock. And a cell with no neighbour above alpha 0.1 is dead: it's zeroed, every channel. The face is an agreement the cells keep reaching, one neighbour at a time.

The coin flip needs care. Designer has no random numbers that would match Python's, and for a proof I need them to match. So the flip is a small integer hash of x, y and step, using only values under 2²⁴. Those are exact in 32-bit floats, so the mask is bit-identical on CPU, on Metal, in Designer and in your browser.

Training looks like this: start from one cell, run a random number of steps, compare the picture to the target, and push the error back through every step. Then damage the result, put it in a pool, and start from it again later, so that the network learns to survive and not just to arrive. The first version took a 48×64 face. It read as the Mona Lisa, through a visible pixel grid, and a mosaic made from it was mush.

The second problem took longer to see. v3 had four times the cells, and its final picture was far better. But it only had a loss at the end, so nothing stopped it from looking bad on the way there. Growth split into 13 to 17 blobs, lopsided 4 to 1, and read as damage spreading. The heal ghosted a doubled brow.

For v4 I added losses during the rollout: a growth front that has to expand, and a hole that has to close from its edges, checked every 12 steps. Warm-started from v3, 8,000 iterations, 3.7 hours on the Mac.

Python did the training. The film doesn't use Python to run the network. Designer does. It has no tensors, no matrix multiply and no memory between frames, so the network has to be spelled out in Pixel Processors.

The 32 channels of the whole grid are packed into one image, 8 RGBA texels per cell. Hidden units get their own image, 32 texels per cell. The weights are a 32×256 bitmap. One nca_step sub-graph holds two Pixel Processors: one forms all 128 hidden units with a while loop, the other turns them into the update and works out who lives. Hover the atlas to see what a texel means.

Nothing survives between frames in Designer, so nothing can be stored. Frame 900 of the network is seed, then 900 instances of nca_step, recomputed from scratch. An instance past the current frame just passes its input through. The final graph chains 1,504 of them.

I expected memory to be the problem. On paper the intermediates come to about 17 GB. In practice sbsrender holds about 4 GB with the look pass in, and a frame of state takes 13 seconds on an M3 Max. Your browser does the same 1,504 steps in the time figure 5 reports below, because it can keep state from step to step and only runs the arithmetic it needs. Designer has to prove every frame from nothing, and that's what makes it interesting.

A network ported by hand is only worth anything if it's the same network. So every Designer render is diffed against lm_ref.py, a Python reference that steps the same weights by the same rules. Steps 1, 10 and 60 of the final network differ by at most 2.2e-5, with every living cell identical.

Here's a third machine: yours. This figure replays the film on your GPU from a single cell, then compares it against the state Designer rendered on my Mac, stored in this page.

It isn't perfectly clean, and I'd rather show that. The network is chaotic at its edges. Machines round 32-bit floats differently, and now and then one cell on the rim tips the other way. On the 840-step trial a patch on the left of the oval drifted: 18 cells had a colour off by more than 1/255, the worst by .018. On the final 1,504-step run Designer and Python agree on every living cell at every step I checked, with colour inside 3.5e-4. The hidden channels reach 2.1e-3 by the end. Your GPU rounds differently again, and by step 1,504 the gap has grown: on my Mac in Chrome, 4 of 9,608 cells disagree and 193 are off by more than 1/255, all from that same kind of tipping. The face is the same; the last few rim cells aren't. Designer against itself is exact: a repeat render comes back bit-identical, and that's the determinism the film needs.

A network output is a colour grid. A mosaic isn't. My first pass put tiles on a lattice and it looked like a JPEG with grout. So I went to a real one, a 2nd century Roman floor with a portrait emblema in the Met's open collection, and copied how a setter thinks. Thin pale grout. Field stones in straight courses. Rows that hug the figure, which is called andamento. Finer stones in the face.

The rows in the face follow the image. I take the strongest edges of the healed portrait and lay stones along lines of equal distance from them, 3 fine rows along each feature. Brow, eye, nose and mouth all get rows that bend around them. That's 13,101 stones, placed once. They never move, so the animation is stones changing in place.

This part isn't pure Designer, and I want to be clear about it. The layout needs skeleton tracing, so Python computes it once and it enters the graph as bitmaps, along with static noise for veining and wear. Everything that changes per frame runs in the graph: sharpening, snapping to stone colours, losing stones, setting them back. It matches the Python look within 1/255 on every frame I diffed.

Each stone is one texel. The look pass reads the network at the stone's centre, sharpens it the way a mosaicist exaggerates a feature, and pulls it halfway toward one of 13 stone colours. When the network passes through a colour no stone comes in, the stone snaps fully. That's what ended the rainbow stones during growth. Every stone in the floor, in one 256×52 texture:

Getting it to look like stone took longer than getting the network into Designer. A list of what failed: tall domed stones read as cobbles, so the floor is ground nearly flat, the way Roman floors are. Dark grout looked wrong too, so the mortar is pale and only darkens under dark stones. Reds came back cyan until I found an 8-bit wraparound. The first bare patch still showed every stone's outline stamped in the bed. Creases read as worms. Smooth drag streaks read as wood grain. What's there now is broken chisel strokes along the wound.

The first full render went through and I caught something in the final frames. There's a shimmer in the floor, because the network never fully settles. Stones keep trading tiny amounts of colour. During the slash, that stopped.

The clip was cheating. To make the blade travel, I'd frozen the network at step 480 and wiped a mask across the frozen state. The hold on the finished cut was one frame repeated 24 times. The end hold was another. The floor was alive except at the moment it mattered most.

So the blade moved into the network. It's now a rule inside nca_step: the update pass clears cells behind a moving tip, on the step they're reached, while everything else carries on. There are no repeated frames in the film now. Shimmer on a patch of hair measures .00086 per frame before the cut and .00089 during it.

Then the network pushed back. A wound held open for 85 steps never quite healed. The cells beside it spent that time adapting to a hole, and the face came back with a permanent scar, at 2.5× the normal error. So the open wound lasts 12 frames, half a second, which is as long as the network tolerates.

One heal could be a trick. It had to take damage again. So I ran cheap tests: real weights, plain NumPy, raw colour, about a minute for nine effects. Each one scored by how close it came back. The rule the first round taught me was blunt: the network recovers from thin or small damage, and it can't rebuild a face from nothing. Big central loss never came back.

Round 2 tried S-curves with real brush shapes, impacts with debris, and an idea from real life: someone scrubbing with a pencil eraser. Wiping out the eyes of the most looked-at face in art was the first thing with meaning, not just damage. It also lost 3,215 cells and came back.

Then the stroke had to become a real gesture: one continuous zigzag, eraser never lifted, from the eyes to the bottom of the portrait. Every first attempt failed. A faster stroke was better, which told me the limit was time open, so I swept 24 settings of rows, width, speed and hold. A 1-second stroke healed. It was too fast to read. At 2 seconds I expected it to break, because 32 steps had been worse than 24. It didn't, but 4 rows couldn't stay open longer than 4 steps. With 3 rows the strips left between passes are wider, those strips are what the face regrows from, and each spot can stay bare for 12.

It joined the film the honest way. The network just keeps running from the healed state. Half a second after step 1,032, the eraser lands. Step 1,032 in the longer graph is bit-identical to the approved clip, so there's no join to see, and in the exact stepping the face returns to .00052 by step 1,504. It's the same error as before the eraser.

Designer makes three maps for every frame: base colour, height and roughness. It doesn't draw the picture you see. Blender gets a displaced plane and three area lights: a warm key about 18 degrees above the floor, a cool fill, and a soft box placed in the camera's mirror angle so polished stones catch it and matte ones don't. Raking light is what turns height into stone.

Designer stays on my Mac, where the license is. The maps get copied to a Windows machine with an RTX 4090, and Cycles renders the frames there. Frames through step 480 were unaffected by the blade fix, so those stayed.

The Mac did have one last bad night. An early clip run used a shell throttle that counts 0 in zsh, so 162 render jobs started at once, the machine ran out of memory, and the kernel panicked. Since then, every heavy job has a tested worker limit and a memory guard.