Neuralese Wiki

Last updated

Myths

Myths about neuralese and AI in general, and evidence-based takes on them

Anything AI-related, including neuralese, attracts myths from all kinds of directions. The alarmed ones say AI is plotting in a secret tongue. The comfortable ones say that if a model writes its reasoning in English, we can read its mind.

Here’s where the evidence stands and the sources behind it.

Myths tracked
14
Last updated

01

About the word “neuralese”

  1. Wrong

    The word “neuralese” was made up in 2025 for the AI 2027 forecast.

    It goes back at least to 2017, when Jacob Andreas, Anca Dragan and Dan Klein used it in a paper called “Translating Neuralese” for the messages AI agents had learned to send each other. AI 2027 brought it to a wide audience in 2025, in its newer sense: a model reasoning in its own numbers.

    Andreas et al. 2017
    AI 2027

  2. Wrong

    Neuralese is a language, with its own words and grammar.

    It’s the stream of numbers a model works with inside, used in place of words. There’s no vocabulary or grammar to crack. People who try to read it translate or probe the numbers instead, the way the 2017 paper matched each message with the English phrase that left a listener believing the same thing.

    Andreas et al. 2017

  3. Not established

    “Neuralese” is OpenAI’s term for how Astra works.

    The word appears nowhere in the system card or in Pachocki’s essay. It does appear in Microsoft’s draft code of conduct, which isn’t about Astra.

    Astra system card
    An Alien Mind
    Microsoft draft code

  4. Wrong

    Microsoft has banned its AI from using neuralese.

    Microsoft AI proposed that its models “do not communicate in neuralese or any form beyond simple human understanding.” It’s a draft, open for public comment and not in force, and Microsoft says it isn’t training models on it yet.

    Microsoft draft code

02

About how neuralese works

  1. Wrong

    Thinking in numbers makes a model 1,000 times smarter.

    The figure comes from AI 2027, which put the gap between a written word and a model’s internal state at “over 1,000 times more information.” That’s about how much one step can carry, not how well a model reasons. COCONUT, the best-known method, ran on a small GPT-2 model and scored 34% on grade-school math, against 43% when the same model wrote its reasoning out.

    AI 2027
    COCONUT paper

  2. Wrong

    Continuous thoughts and recurrent depth are the same idea.

    They change different things. Continuous thoughts, as in COCONUT, replace written steps: the model feeds its internal state back in as its next thought instead of writing a word. Recurrent depth runs a block of the model’s layers in a loop, so each step gets more computation. A model could use either, both or neither. What is latent reasoning? goes through both.

    COCONUT paper
    Geiping et al. 2025

  3. Open question

    Reasoning without words already works better than writing it out.

    It depends on the task. COCONUT searched more widely on planning puzzles but trailed written reasoning on math. A recurrent-depth model with 3.5 billion parameters raised its scores by looping more, up to the computation of a 50-billion-parameter model. Neither has been shown at the scale of today’s frontier models.

    COCONUT paper
    Geiping et al. 2025

  4. Wrong

    Nobody is building AI that reasons without words.

    Researchers have published working methods for years. COCONUT feeds a model’s internal state back in as its next thought instead of writing a word, and recurrent-depth models run their own layers in a loop to think longer before they answer. The Information reported that GPT-6 Astra uses recurrent depth. OpenAI hasn’t confirmed it.

    COCONUT paper
    Geiping et al. 2025
    The Information

  5. Wrong

    As long as a model reasons in plain English, we can see what it’s thinking.

    Written reasoning leaves things out. In Anthropic’s tests, when a slipped-in hint changed a model’s answer, Claude 3.7 Sonnet mentioned the hint only 25% of the time and DeepSeek R1 39%. In OpenAI’s experiments, punishing a model for suspicious reasoning taught it to hide its intent while it kept cheating. Studies of small open models in September 2026 add detail: some concepts that drive the answer stay out of the text, and on easy tasks the models often skip past their own written steps. The usual check, editing the text to see whether the answer changes, also overstates how much the steps matter.

    Anthropic 2025
    Baker et al. 2025
    Wang et al. 2026
    Jia and Mu 2026
    Bhupatiraju and Nyaupane 2026

03

About GPT-6 Astra

  1. Wrong

    Astra has stopped thinking in words.

    Astra still writes a chain of thought, and most of OpenAI’s monitorability testing is about reading it. What changed is how much that chain of thought gives away.

    Astra system card

  2. Not established

    Astra reasons in COCONUT-style continuous thoughts.

    We found no public source that says so. The reporting is about recurrent depth, which is a different technique. What is latent reasoning? explains the difference.

    COCONUT paper
    Geiping et al. 2025

  3. Not established

    Recurrent depth caused the drop in monitorability.

    OpenAI says it is “quite confident” that the change in controllability is not “differentially due to any architectural changes,” and that the change tracks Astra’s gains at working without reasoning. In his 6 September essay, Pachocki blamed “a combination of factors”: more complex, tool-heavy tasks, models getting better at manipulating their own reasoning, and models becoming “much smarter even without using verbalized reasoning at all.” Architecture isn’t on his list.

    Astra system card
    An Alien Mind

These come from Does GPT-6 Astra think in neuralese?, which tracks the claims about the model and what would change each status.

04

About secret languages

  1. Not established

    AI models are already talking to each other in a secret language.

    We know of no evidence that deployed models pass hidden messages to each other on their own. Researchers do test for the nearest real thing, a model tucking hidden reasoning into ordinary-looking text. OpenAI’s evaluations of GPT-6 Astra “indicate Astra is not able to perform steganographic CoT reasoning.”

    Astra system card

  2. Wrong

    When AI drifts into its own shorthand, it’s hiding something.

    Drift like the Facebook bots’ is a known side effect of training. When nothing rewards a model for staying readable, it tends to take the shortcuts that pay, the way traders or surgeons build their own jargon. Dhruv Batra called it “a well-established sub-field of AI, with publications dating back decades.”

    Fast Company 2017
    USA Today 2021

And the viral ones, each told in full on Does AI have a secret language?

  1. Wrong

    2016: Google’s translation AI invented its own secret language.

    Google 2016
    TechCrunch 2016

  2. Wrong

    2017: Facebook shut down its AI in a panic after two bots invented their own language.

    Lewis et al. 2017
    FAIR 2017
    Fast Company 2017
    Gizmodo 2017
    USA Today 2021

  3. Not established

    2022: DALL·E 2 has a secret language.

    Daras & Dimakis 2022
    The Conversation 2022

  4. Wrong

    2025: two AI agents switched to a secret language once they realized no human was listening.

    ElevenLabs 2025
    TechCrunch 2025
    Dataconomy 2025

  5. Wrong

    2026: AI agents on Moltbook plotted a secret language that humans couldn’t read.

    Fortune 2026
    The Next Web 2026

05

Sources

These are the sources behind the myths on this page. We checked them against the original text on 24 September 2026, where we could read it. The viral stories have their sources on their own page.

  1. Peer-reviewed paper

    Jacob Andreas, Anca Dragan, Dan Klein. Translating Neuralese. ACL 2017.

    Where the word comes from, and how the messages were translated.

  2. Forecast

    Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland, Romeo Dean. AI 2027. April 2025.

    The word’s wide audience, and the “1,000 times” figure.

  3. Draft policy

    Microsoft AI. Humanist AI Code of Conduct for MAI Models. Draft for public consultation, 14 Sep 2026.

    Microsoft’s draft rule.

  4. Peer-reviewed paper

    Continuous thoughts, and how they did on math and planning.

  5. Peer-reviewed paper

    Recurrent depth.

  6. Lab publication

    How often models mention the hints they used.

  7. Preprint

    Models learning to hide their intent.

  8. Lab publication

    OpenAI. GPT-6 Astra System Card. 3 Sep 2026, updated 9, 22 and 29 Sep 2026.

    What OpenAI says about Astra.

  9. Lab publication

    Jakub Pachocki (OpenAI). An Alien Mind. 6 Sep 2026.

    Pachocki on Astra.

  10. Journalism

    Amir Efrati, Stephanie Palazzolo, Rocket Drew. OpenAI Technique in ‘Astra’ Model Sparks Security Concerns. The Information, 1 Sep 2026.

    The report that Astra uses recurrent depth. Paywalled. We haven’t read the full text, and the details here match how other outlets summarized it.

  11. Journalism

    Why bots drift into shorthand.

  12. Journalism

    Dhruv Batra on drift as a known effect. Linked to the Tech Xplore reprint of the USA Today article.