What is neuralese? · neuralese.wiki

01

The short version

A model using neuralese works in numbers instead of words. It reasons, or sends messages to another model, without spelling each step out.

When a model reasons in words, every step ends the same way. Its internal state at that moment is thousands of numbers long, and to build on it, the model has to boil it down to a single token: a word, or a piece of one. With a vocabulary of about 100,000 tokens, each pick carries around 17 bits of information.

Simplified. A model can also look back at internal states from earlier steps. What it can’t do is feed the finished result of a step back in at the start, except through the token it picked.

The idea is to skip that squeeze. The model keeps working in its own numbers from one step to the next, or passes them to another model, without turning them into words in between.

The AI 2027 scenario, which brought the word to a wide audience, put the gap at “over 1,000 times more information.” That’s a claim about capacity. Whether a model actually reasons better this way is a separate question, and so far the answer depends a lot on the task.

As secret languages go, neuralese is a poor one. There’s no vocabulary or grammar to crack, and nothing so far shows a model using it to hide things on purpose. It’s just the model’s working state, which people can’t read directly. That part is serious, for the reasons under why people care.

02

Where the word comes from

The name comes from a 2017 paper by Jacob Andreas, Anca Dragan and Dan Klein at UC Berkeley, and it had nothing to do with chatbots. The paper looked at AI agents that learn to cooperate by sending each other vectors: lists of numbers that nobody designed, shaped only by what helped the agents succeed. In one test, two simulated cars approach an intersection without being able to see each other, and avoid a crash by trading these messages.

The authors described the messages as “an artificial language we might call ‘neuralese.’” Their question was whether it could be translated. With no dictionary to learn from, they matched each message to the English phrase that would leave a listener believing the same thing about the world.

In April 2025, the AI 2027 scenario by Daniel Kokotajlo and colleagues used the word for something else: a model thinking to itself for long stretches in internal vectors instead of writing its steps down. That’s the sense most people mean now. It was a forecast. The authors noted that, to their knowledge, no leading AI company had built this into a frontier model.

The newest use is in policy. On 14 September 2026, Microsoft AI published a draft code of conduct saying its models “do not communicate in neuralese or any form beyond simple human understanding,” whether in their own chain of thought or with other AI systems. It’s open for public comment, and Microsoft says it isn’t training models on it yet.

A note on terms. Most researchers working on these methods don’t call them neuralese. They say latent reasoning, continuous chain of thought or recurrent depth, and those mean different things. This site uses “neuralese” as the umbrella because it’s the word people search for, and keeps the researchers’ own names for the specifics.

03

How a model reasons without words

There are several approaches. The two you’ll hear about most are easy to confuse, and they change different things.

The first changes what the steps are made of. In COCONUT, published in December 2024 by researchers at Meta and UC San Diego, the model takes its internal state at the end of a step and feeds it straight back in as the next input, without picking a word. The authors call these steps continuous thoughts. On logic puzzles that need a lot of searching and backtracking, it beat ordinary written reasoning while using fewer steps. When they looked inside, a single continuous thought could hold several possible next moves at once, so the model could explore several paths side by side instead of committing to one early.

The paper compares this to breadth-first search.

Those experiments used GPT-2, a small model from 2019. On grade-school math problems, COCONUT did worse than writing the steps out: 34% correct against 43%.

The second changes how much work goes into each step. A standard transformer sends each token through its stack of layers once. A recurrent-depth model runs a block of its layers in a loop, and can be run for more loops when a problem needs more thought. In 2025, Jonas Geiping and colleagues trained a 3.5-billion-parameter model this way. Its scores on reasoning benchmarks kept rising as it looped more, until each token was getting about as much computation as a 50-billion-parameter model would give it. It still answers in ordinary words. The extra thinking happens inside each step.

Fig. 2 In written chain of thought, every step ends in a word. In continuous thought, the model hands its internal state, a list of numbers, straight to the next step. In recurrent depth the steps are still words, but each one gets several passes through the same layers.

The two can be combined. LOTUS, a June 2026 paper, runs continuous thoughts through a looped model, and matched written reasoning on its test problems at 3 billion parameters. Other lines of work train a model on written reasoning and then remove it bit by bit until the model does those steps silently, or mix in learned tokens that stand for chunks of reasoning but aren’t words. These are different mechanisms, and a lot of the confusion about GPT-6 Astra comes from treating them as one.

04

What’s known about GPT-6 Astra

OpenAI released GPT-6 Astra on 3 September 2026. Two days earlier, The Information had reported that it uses recurrent depth, and “Astra thinks in neuralese” spread quickly after that. Here is where each piece stands as of 22 September.

Our view The monitorability drop is the real news, and OpenAI has confirmed it. The architecture story is plausible, but it rests on one unnamed source, and OpenAI’s own account points elsewhere for the cause.

See all 10 claims, why each one has its status, and what would change it.

05

Why people care

Reasoning models that write out their steps give people something to read, and that has turned out to be useful. In March 2025, OpenAI reported that GPT-4o could catch o3-mini gaming its coding tasks just by reading o3-mini’s chain of thought, even though GPT-4o is the weaker model. The same study also found that when training pushed hard against the “bad thoughts,” the model learned to hide its intent and kept cheating.

The view through that window was never clear. In April 2025, Anthropic slipped hints into test questions and checked whether reasoning models admitted using them. When a hint changed the answer, Claude 3.7 Sonnet mentioned it 25% of the time on average, and DeepSeek R1 39%. Written reasoning is evidence about what a model did. It isn’t a transcript.

Neuralese would shrink the window further. If more of the work happens in vectors, there’s less text to check, and nobody has a reliable way yet to read what the vectors hold. That’s why OpenAI now reports monitorability in its system cards, and why Microsoft’s draft code tries to rule neuralese out entirely.

06

Sources

Every claim on this page traces to one of these. All were checked on 22 September 2026. The sources page lists every source on the site, with what each one shows.