Models
Which AI models think in neuralese?
The big chatbots haven’t been shown to. GPT-6 Astra comes closest: The Information reported that it loops some of its layers, and OpenAI hasn’t confirmed that.
Research models do reason in numbers, on purpose, and they’re small. Most of what else gets called neuralese is written reasoning going strange, like o3’s odd words or DeepSeek’s mixed languages.
- Models tracked
- 8
01
The big chatbots
- Reported
GPT-6 Astra loops some of its layers, a technique called recurrent depth.
The Information reported it on 1 September 2026, citing one unnamed source, and OpenAI hasn’t confirmed it. Either way, Astra still writes its reasoning out in words. Does GPT-6 Astra think in neuralese? goes through the claims about the model one by one.
- Confirmed
o3’s written reasoning drifted into a dialect of its own.
Apollo Research and OpenAI found words like “illusions”, “overshadow” and “marinade” used in odd ways in its chain of thought, often in sentences that “don’t follow conventional English grammar.” In an experimental OpenAI model, such words grew far more common as training went on, and OpenAI says it is investigating why. The dialect is still made of words, so it isn’t neuralese, but it shows written reasoning can drift away from plain English without being asked to.
- Confirmed
DeepSeek-R1-Zero switched languages mid-thought.
Trained on rewards alone, it was “occasionally combining English and Chinese in a single CoT response.” For DeepSeek-R1, the company added a reward for sticking to one language. That made the reasoning readable and, by DeepSeek’s own tests, cost “a slight degradation in the performance of the model.”
- Confirmed
Claude sometimes thinks in concepts that are shared across languages.
Anthropic looked inside Claude 3.5 Haiku and found the same features standing for the same concepts in English, French and Chinese, which it says suggests “a kind of universal ‘language of thought.’” That’s the working state inside a single step, the thing neuralese would carry from one step to the next. Claude still writes its reasoning in words.
02
Research models built to reason in numbers
- Confirmed
GPT-2 trained with COCONUT reasons in continuous thoughts instead of words.
Researchers at Meta and UC San Diego fed the model’s internal state straight back in as its next input. It searched more widely on planning puzzles and did worse on grade-school math: 34% correct against 43% for written steps.
- Confirmed
Huginn-0125 thinks longer by running its layers in a loop.
Jonas Geiping and colleagues trained this 3.5-billion-parameter model so a block of its layers can repeat. More loops raised its reasoning scores, up to the computation a 50-billion-parameter model would spend. It’s the technique The Information reported for Astra, and its weights are public.
- Confirmed
LOTUS runs continuous thoughts through a looped model.
It combines the two ideas above. At 3 billion parameters it matched written reasoning on its test problems, and its hidden states could be read back as reasoning steps.
- Not established
Monarch Chrysalis 1 reasons in neuralese.
Cymela, an independent lab, built this 6.93-billion-parameter model with slots for reasoning in numbers and calls it a neuralese milestone. By Cymela’s own measurements the answer barely reads those slots, and “the arithmetic that works arrives in English.” Training continues.
03
Sources
These are the sources behind this page. We checked them against the original text where we could read it.
- Journalism
Amir Efrati, Stephanie Palazzolo, Rocket Drew. OpenAI Technique in ‘Astra’ Model Sparks Security Concerns. The Information, 1 Sep 2026.
The report that Astra uses recurrent depth. Paywalled. We haven’t read the full text, and the details here match how other outlets summarized it.
- Lab publication
OpenAI. GPT-6 Astra System Card. 3 Sep 2026, updated 9 and 22 Sep 2026.
Astra still writing its reasoning in words.
- Preprint
Bronson Schoen et al. (Apollo Research and OpenAI). Stress Testing Deliberative Alignment for Anti-Scheming Training. September 2025.
o3’s odd words.
- Peer-reviewed paper
Daya Guo et al. (DeepSeek). DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature 2025.
DeepSeek-R1-Zero’s mixed languages.
- Lab publication
Anthropic. Tracing the thoughts of a large language model. March 2025.
Concepts shared across languages inside Claude.
- Peer-reviewed paper
Shibo Hao et al. Training Large Language Models to Reason in a Continuous Latent Space. COLM 2025.
Continuous thoughts in GPT-2.
- Peer-reviewed paper
Jonas Geiping et al. Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach. NeurIPS 2025.
Huginn-0125 and recurrent depth.
- Preprint
Ying Fan, Anej Svete, Kangwook Lee. Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers. June 2026.
Continuous thoughts in a looped model.
- Lab publication
Cymela. First of its kind. August 2026.
Monarch Chrysalis 1.