or

01

Neuralese, the short version

An AI model using neuralese works in numbers instead of words. It reasons, or sends messages to another model, without spelling each step out.

When a model reasons in words, it writes a word, reads back everything it has written, and goes on to the next. While it thinks, it works in numbers, thousands of them at each step, and only a sliver of that fits into the word it writes down.

The idea of neuralese is to skip that limitation. The model keeps working in its own numbers from one step to the next, or passes them to another model, without turning them into words in between.

As secret languages go, neuralese is a poor one. There’s no vocabulary or grammar to crack, and we haven’t yet seen evidence of a model using it to hide things on purpose. It’s just the model’s working state, which people can’t read directly. That part is serious, and it’s why people care.

The viral stories about AIs inventing languages, from Google Translate in 2016 to Moltbook in 2026, are mostly something else. Does AI have a secret language? goes through each one.

02

Which AI models use neuralese?

Neuralese, , do you speak it?

Not the big chatbots, as far as anyone has shown. The ones people use, from ChatGPT to Claude to DeepSeek, still write their reasoning out in words before they answer.

What gets called neuralese in them is usually words behaving strangely. When Apollo Research and OpenAI read the chains of thought of o3, an OpenAI reasoning model, they found words like “illusions”, “overshadow” and “marinade” used in odd ways, a dialect of its own. DeepSeek-R1-Zero, trained on rewards alone, slid between English and Chinese in the middle of a thought. Strange as both are, they’re still made of words.

Models that really do reason in numbers exist, but so far they’re research models, small and built that way on purpose, like a version of GPT-2 trained with a method called COCONUT, and Huginn-0125, which loops its own layers. How they work comes further down. Which AI models think in neuralese? goes through nine models, one claim each, with where the evidence stands.

03

Does GPT-6 Astra speak neuralese?

One big model comes closer than the rest. Two days before OpenAI released GPT-6 Astra on 3 September 2026, The Information reported that it uses recurrent depth, looping some of its layers so it can think longer without writing anything, and “Astra thinks in neuralese!” spread quickly after that. OpenAI hasn’t confirmed the report.

What OpenAI did confirm, in Astra’s own system card, is the part it says worries it: Astra’s written reasoning is harder to monitor than earlier models’. And since 29 September, Astra also runs OpenAI’s dots, agents that keep working on their own, which OpenAI is rolling out to ChatGPT’s paying subscribers and plans to bring to more people soon.

That still isn’t neuralese, as far as anyone has shown. Astra writes its reasoning out in words before it answers, and we’ve found no public sign that it has swapped them for numbers. Here is where each claim stands as of 22 September.

Confirmed
Astra’s chain of thought is harder to monitor than earlier OpenAI models’.
Astra system card
Confirmed
Astra can do much more without writing any reasoning at all.
Astra system card
Baker et al. 2025
Reported
Astra uses recurrent depth, a technique that runs some of its layers in a loop.
The Information
Transformer
Astra system card
Not established
Recurrent depth caused the drop in monitorability.
Astra system card
An Alien Mind
Not established
Astra reasons in COCONUT-style continuous thoughts.
COCONUT paper
Geiping et al. 2025
Wrong
Astra has stopped thinking in words.
Astra system card

OpenAI’s declining ability to monitor its own model’s reasoning is the real news, and OpenAI has confirmed it. Its chief scientist, Jakub Pachocki, wrote on 6 September that the company’s ability to rely on reading a model’s chain of thought “is progressively diminishing.”

04

Where the word “neuralese” comes from

The word “neuralese” comes from a 2017 paper by Jacob Andreas, Anca Dragan and Dan Klein at UC Berkeley, and it had nothing to do with chatbots. The paper looked at AI agents that learn to cooperate by sending each other vectors: lists of numbers the researchers didn’t design, shaped only by what helped the agents succeed. In one test, two simulated cars approach an intersection without being able to see each other, and avoid a crash by trading these messages.

Fig. 2 The driving game from Andreas, Dragan and Klein’s 2017 paper: two cars that can’t see each other cross an intersection, trading a list of 64 numbers at every step. The numbers here are made up; the cars learned theirs in training.

The authors described the messages as “an artificial language we might call ‘neuralese.’” Their question was whether it could be translated.

In April 2025, the AI 2027 scenario by Daniel Kokotajlo and colleagues used the word for something else: a model thinking to itself for long stretches in its own numbers instead of writing its steps down. The scenario called it “neuralese recurrence and memory.” That’s the sense most people mean now. It was a forecast. The authors noted that, to their knowledge, no leading AI company had built this into a frontier model.

In September 2026, the word “neuralese” appeared in Microsoft policy. On 14 September, Microsoft AI published a draft code of conduct saying its models “do not communicate in neuralese or any form beyond simple human understanding,” whether in their own chain of thought or with other AI systems. It’s open for public comment, and Microsoft says it isn’t training models on it yet.

05

How a model reasons without words

Most of the research follows one of two approaches, which people often confuse.

The first changes what the steps are made of. In COCONUT, published in December 2024 by researchers at Meta and UC San Diego, the model takes its internal state at the end of a step and feeds it straight back in as the next input, without picking a word. The authors call these steps continuous thoughts. On logic puzzles that need a lot of searching and backtracking, it beat ordinary written reasoning while using fewer steps. When they looked inside, a single continuous thought could hold several possible next moves at once. They read this as the model exploring several paths side by side instead of committing to one early. A 2026 retest found the same pattern in models built without COCONUT’s feedback loop, so the pattern alone doesn’t show how the model reasons.

COCONUT’s authors compare this to breadth-first search.

Those experiments used GPT-2, a small model from 2019. On grade-school math problems, COCONUT did worse than writing the steps out: 34% correct against 43%.

The second reasoning approach changes how much work goes into each step. An ordinary language model sends each word through its stack of layers once. A recurrent-depth model runs a block of its layers in a loop, and can be run for more loops when a problem needs more thought. In 2025, Jonas Geiping and colleagues trained Huginn-0125, a 3.5-billion-parameter model, this way. Its scores on reasoning benchmarks kept rising as it looped more, until each word was getting about as much computation as a 50-billion-parameter model would give it. It still answers in ordinary words. The extra thinking happens inside each step.

Fig. 3 In written chain of thought, the steps end in words. In continuous thought, the model hands its internal state, a list of numbers, straight to the next step. In recurrent depth the steps are still words, but each one gets several passes through the same layers.

These two approaches can be combined. LOTUS, a model from June 2026, loops its layers and passes numbers instead of words from one step to the next, and on its test problems it did as well as written reasoning. Other researchers train a model to reason in writing, then take the writing away bit by bit until the model does those steps silently.

These are different mechanisms, and a lot of the confusion about GPT-6 Astra comes from treating them as one. What is latent reasoning? goes through each one and what’s known about how well it works.

06

Why people care about neuralese

Reasoning models that write out their steps give people something to read, and that has turned out to be useful. In March 2025, OpenAI reported that GPT-4o could catch o3-mini gaming its coding tasks just by reading o3-mini’s chain of thought, even though GPT-4o is the weaker model. The same study also found that when training pushed hard against the “bad thoughts,” the model learned to hide its intent and kept cheating.

That window onto a model’s reasoning has blind spots. In April 2025, Anthropic slipped hints into test questions and checked whether reasoning models admitted using them. When a hint changed the answer, Claude 3.7 Sonnet mentioned it 25% of the time on average, and DeepSeek R1 39%. Written reasoning is evidence about what a model did.

Neuralese would shrink the window further. If more of the work happens in numbers, there’s less text to check, and researchers don’t yet have a reliable way to read what those numbers hold. That’s why OpenAI now reports in its system cards how well a model’s reasoning can be monitored, and why Microsoft’s draft code tries to rule neuralese out.

07

Sources

These are the sources behind this page. We checked them between 22 September and 1 October 2026. The sources page lists the site’s sources, with what each one shows.

The kitten in the explainer at the top of the page is drawn from a photo by Koen Eijkelenboom on Unsplash.

What is neuralese