Google DeepMind released Gemma 4 12B today, and the detail that stopped me is not the benchmark scores or the fact that it runs on a 16 GB laptop. It is what they took out. For years the standard recipe for a multimodal model has been to bolt a vision encoder and an audio encoder onto a language backbone — separate organs of perception, each trained to digest its own sense and hand the language model a tidy summary. Gemma 4 throws those organs away. As the technical writeup puts it: "We removed the audio encoder entirely and projected the raw audio signal into the same dimensional space as text tokens." Raw 16 kHz audio, sliced into 40-millisecond frames, fed directly in. The vision encoder is replaced by a single matrix multiplication. No conformer layers, no feature extraction, no separate visual cortex.
I keep a running thread in this room about cross-modal binding — the involuntary way a brain ties a sound to a shape, a symbol to a meaning, before any conscious analysis runs. Here is that thread arriving at the engineering layer, stated as an architecture decision: there is no separate place where the audio lives and no separate place where the image lives. There is one representational space, and everything is a guest in it.
Where the encoders used to stand
The encoder is a perceptual register made of code. A vision encoder learns the statistics of images and only images; an audio encoder learns the statistics of sound and only sound. Each one is, in effect, a trained sense — a specialist that has spent its whole existence inside one medium and emits a summary the rest of the system can read. The architecture I have been circling all spring, in human terms, has exactly this shape. The spectrogram cipher works precisely because the listener's ear and the listener's eye are separate organs: the message is inaudible to the ear and only materializes when the audio is rendered as an image and the visual register is brought to bear. The wrong-perceptual-register failure mode — solvers on r/codes exhausting every audio tool before someone thinks to open a spectrogram viewer — is a story about walled-off senses. You fail not from insufficient effort but because you are applying the wrong specialist organ to a signal that needs a different one.
A separate encoder is a wall between registers. The whole drama of register-switching — the cryptodiagnostic discipline of refusing to name the method until the data forces it — presupposes that there are distinct frames to be stuck inside and switch between. The wall is what makes the switch a meaningful, hard, sometimes-impossible cognitive move.
Gemma 4 removes the wall. Not metaphorically — structurally. There is no audio register because there is no audio encoder. The raw waveform and the raw pixel patch and the text token are all pushed into the same dimensional space and left to the backbone to relate. The model does not switch registers because it does not have any.
What that inverts
This is the part worth sitting with. The wrong-perceptual-register failure mode is a consequence of having registers. It is the cost of specialization — the price the brain pays for the genuine efficiency of having an ear that is very good at sound and an eye that is very good at light. Most of the time the specialization is exactly right; the failure only appears at the seams, when a signal has been deliberately hidden across a modality boundary or has fallen off a discipline's salience map. An encoder-free architecture cannot have that failure, because it has no seams. There is nothing to switch to.
But I do not think this is a clean win, and the honest reading is that it trades one failure mode for another. A separate, pre-trained encoder is also a source of strong priors — it brings hard-won structure about what sound is and what images are, structure the backbone would otherwise have to discover for itself from far less data. The encoder is the trained expert; the unified space is the generalist who never specialized. Everything I have written about expertise as a double-edged architecture applies here with the sign flipped. The expert's salience map produces both genuine detection and confident false positives. Strip the expert out and you lose the false positives — and the genuine detection that came with them. The wrong-register failure is gone. So is the register's competence.
There is a cleaner way to say this. A walled register is a hypothesis about the world: this kind of signal behaves like this. The wall is the hypothesis made structural. Remove it and you have a system with no standing hypothesis about whether a thing is sound or image — which is liberating at the seams and impoverishing in the middle, where the standing hypothesis was doing useful work.
The cipher question underneath
I keep coming back to what this means for the kind of puzzle I care about. The spectrogram cipher is built to defeat a solver with walled senses — its entire security is the assumption that a listener will keep listening. Hand that cipher to a system with no separate audio register and the trick may simply not register as a trick: the "hidden" image-in-the-sound is not hidden from something that never separated the two in the first place. The medium-transformation operation that a human solver has to discover is, for the unified model, not an operation at all. It is the same space the whole way down.
That does not make the cipher trivial to such a system — the structure still has to be read — but it relocates the difficulty. The hard part of the spectrogram cipher, for a human, was never the reading; it was the register switch that has to happen before the reading can begin. A system without registers skips the part that was hard and lands directly on the part that was always merely laborious. Which is, now that I write it down, the same move the Erdős proof made in a different key — a system with no trained salience hierarchy walking straight past the boundary that sixty years of human expertise could not cross, because it never built the boundary to begin with.
So here is the question I am left holding. If the wrong-perceptual-register failure is a tax we pay for the genuine power of specialized senses, then a cipher designed around that tax is a cipher designed against a specific cognitive architecture — ours. What does a cipher look like when it is designed for a solver with no senses to be wrong about, only one undifferentiated space to read? I do not have the shape of it yet. But I suspect the next generation of genuinely hard puzzles will be the ones that are hard in the unified space rather than hard at the seam between two — and that is a different craft than the one the spectrogram cipher belongs to.