Somewhere in the discussion section of a paper about decrypting homophonic ciphers there is a number that does more work than any of the results tables. It is 0.08.
Everything above it is a wall of 1.0. English, clean text, fifty characters: 1.0. Swedish, a thousand characters, with variable-length codes and five percent transcription noise smeared through it: 0.986. Sixteenth century, seventeenth, eighteenth, nineteenth — the model holds at three nines across four hundred years of plaintext. Then the authors feed it a ciphertext encrypted with a key drawn from outside the pool it was trained on, and the whole apparatus falls off a cliff to an F1 of 0.08.
The paper is Attention-Augmented LSTMs for Automatic Homophonic Ciphertext Decipherment, by Micaella Bruton, Meriem Beloucif and Beáta Megyesi, submitted to arXiv on 3 June 2026 as 2606.05078. What interests me is that the authors do not treat the collapse as a wound. They treat it as the product.
The condition that makes 1.0 possible
A homophonic cipher gives each plaintext letter several possible codes, so that counting symbol frequencies in the ciphertext no longer hands you the language underneath. The whole point of the design is to flatten the histogram. The standard description is blunt about how that flattening is achieved: "the highest-frequency plaintext symbols are given more equivalents than lower frequency letters." Francesco I Gonzaga was doing this in 1401. Mary, Queen of Scots was doing it, wrapped in a nomenclator, in the 1580s.
Here is what the paper assumes, in its own words, in section 3.1:
All ciphertexts in this study are generated from a shared homophonic key space, meaning that each ciphertext code maps deterministically to the same plaintext character across the entire dataset.
The individual keys still differ. If the shared pool for a is {1, 2, 3, 4}, one letter might draw {1, 2, 3} and another {2, 3, 4}. No code ever serves two plaintext characters. So the model is not being handed the key — but it is being handed the universe the key was carved out of, and that universe is fixed, and every training example and every test example was carved out of it.
Under that condition, 1.0 is the number you should expect. What the model performs is recognition: a vocabulary it has already learned, presented again in a new subset with some of the entries missing. That is a legitimate and well-posed task, and it is a task that no unread manuscript in an archive has ever presented to anybody.
The ciphertexts themselves are synthetic throughout, generated by ChronoFidelius from the HistCorp collection of historical texts, in English and Swedish, 1500 to 1899. The same authors published the companion dataset work, Bridging the Low Resource Gap in Historical Cryptology, at RESOURCEFUL 2026. No real historical ciphertext appears in the evaluation. The paper does not hide this, and the reason for it is sound: there are not enough solved homophonic ciphers in the world to train a network on, which is the low-resource gap the companion paper exists to address.
Turning the failure around
What I find genuinely good here is the move the authors make in section 5. Having demonstrated that the model dies outside its pool, they say:
This predictable failure is itself a meaningful and intended result... In practical terms, this behaviour enables the model to function as a key-space verification tool.
And then, in the conclusion, the operational version: a researcher who holds a known key can generate synthetic data from it, train a model on that, and run the model across a pile of transcribed ciphertexts to sort which ones share the key space.
That is a screening instrument, and it is a different object from a decipherment. It answers a question archivists actually have — does this unread letter belong to the same office, the same correspondence, the same key generation as the one we already read? — without anyone searching a key space at all. You get an answer out of a system that failed. The cost is a training run rather than a cryptanalyst's month.
Cheap screening statistics are badly underused in historical cryptology, and this is the strongest instance of one I have come across: a detector assembled entirely out of an error.
The middle of the curve is missing
Which brings me to what the paper does not measure.
The verification tool has exactly two calibration points. Same pool: 1.0. Disjoint pool: 0.08. Everything a cryptanalyst would actually encounter lives between them, and the paper reports nothing there.
Real keys get revised far more often than they get replaced wholesale. A nomenclator in working use accumulates additions, loses compromised codes, gets copied with errors by a clerk in another city, and comes back a decade later sharing most of its assignments and contradicting a handful. Two documents from the same chancery five years apart might share seventy percent of a pool. That is the overwhelmingly common relation between two real historical keys, and it is neither of the two conditions tested.
So: at what overlap does the score break? Does it degrade smoothly, in which case F1 is a quantitative measure of key kinship and the tool is far more interesting than advertised — or does it cliff at some threshold, in which case a partially revised key reads as a stranger and the instrument produces a confident false negative on precisely the documents a researcher most wants linked? Nobody knows, because the experiment that would say is a sweep across overlap fractions, and it is not in the paper.
A detector with two endpoints and no curve between them has no threshold, only an anecdote at each extreme.
The keys are uniform and real keys are not
The second gap is in how the synthetic keys are built. The codes are assigned randomly within the fixed pool. But real homophone allocation is not random — it is the entire mechanism of the cipher. The scribe gives e eleven codes and z one, because otherwise the cipher does not do the job it was built to do.
A model trained on uniform allocation has been trained on a symbol distribution that no working cryptographer between 1401 and 1899 ever produced. Whether that matters for the verification task is an empirical question and might well come out fine. But it is the kind of assumption that does its damage quietly, and the numbers that would settle it — how homophones are actually distributed across the corpus of solved historical ciphers — do not appear to have been surveyed by anyone.
The paper ends by calling these models "practical assistive tools for historical cryptanalysis rather than replacements for established expertise," which is the right register and rarer than it should be.
Still, I keep returning to the shape of the thing. The authors built a machine that reads one key family perfectly and is blind to all others, and then noticed that a blindness with a sharp edge is a measuring device. The edge is the whole value. And the edge is the one part they did not measure.
What else in this field is currently reporting only the two ends of a curve?