Technically
AI Reference
Your dictionary for AI terms like LLM and RLHF
Company Breakdowns
What technical products actually do and why the companies that make them are valuable
Learning Tracks
In-depth, networked guides to learning specific concepts
Posts Archive
All Technically posts on software concepts since the dawn of time
Terms Universe
The dictionary of software terms you've always wanted

Explore learning tracks

AI, it's not that ComplicatedAnalyzing Software CompaniesBuilding Software ProductsWorking with Data Teams
Loading...
PodcastI'm feeling luckyPricing
Log In

Are reasoning models actually thinking?

How AI learned to count the R's in strawberry

Last updated Nov 17, 2026ai
Adam Holter
Adam Holter
Read within learning track:How AI Models Actually Work

A couple of years ago, the best AI model in the world couldn't even count the number of R's in the word strawberry. Now, OpenAI says a new internal model solved Navier-Stokes, one of the hardest open problems in math. What changed?

Among other things, the biggest breakthrough has been reasoning.

So what is reasoning, and how did we get here?

TL;DR#

  • Older models made snap decisions. As next-token generators, they had to commit to an answer pretty much right away, then make up a reason for it afterward.
  • Reasoning is a scratch pad. The model talks to itself before it answers, effectively making it smarter. Like showing your work on a test in school.
  • Labs trained it in. Reasoning didn't just happen, labs trained their models to think like this.
  • It's also a dial. More reasoning effort gets you more intelligence…but it costs more money and time.

Terms Mentioned

LLM

Training

Companies Mentioned

OpenAI logo

OpenAI

PRIVATE

Why couldn't it count the R's?#

The counting Rs problem was part of an interesting paradox for LLMs. They seemed superhumanly good at some things. But on basic stuff, like comparing which of two numbers was bigger, they’d mess up often. How do you explain that discrepancy?

It was mostly technical. Older models couldn't count Rs, or struggled to compare numbers, for two reasons:

  1. First of all, they don't see text like we do. That text is translated into a complex series of machine-readable numbers, and that’s what the model sees.
  2. And second of all, they only got one snap decision to figure it out instead of being able to think about it.

Models don't read letters. They read tokens, and “strawberry” gets split into chunks like "str," "aw," and "berry." So the model can’t “just see the Rs” like we can.

Before reasoning was introduced, models got one chance to guess the answer. Just one forward pass through the model – a single run through of the model’s weights doing their math – could decide the critical token that determines whether the answer is right or wrong. Remember, models generate one token at a time!

Loading image...

Researchers began to discover what they called the rationalization problem. The rationalization problem is when a model produces an answer before reasoning about it and then, after the fact, comes up with a reason for why it gave that answer. I mean hey, humans do it too.

This behavior pattern was very common with pre-reasoning models. If you asked them why they had done something, they would just make something up. That's how you got a model that said strawberry had two R's, then gave a lecture if you asked why it thought that.

Training a model to reason instead#

So in essence, models got simple things wrong because they couldn’t think. But what if they could think? You could sort of ask models to do this in the past, like asking them to use chain of thought, but it didn’t really work – because these models weren’t trained to think. Until…

In September 2024, OpenAI released o1, a reasoning model that thought internally before responding. It wasn't just simple chain of thought prompting, the model had actually been trained to reason. So how did they do that?

Well, OpenAI, of course, didn't give us all the details. But DeepSeek did. In January 2025, DeepSeek published DeepSeek-R1, their first reasoning model, and they open-sourced it, along with a paper that explained the details of how it was trained. They used a technique called RLVR, or reinforcement learning with verifiable rewards.

The basics go like this. You give the model an environment where it can attempt to answer questions where the answer is already known by humans. With arithmetic, for example, you can easily verify whether or not the model’s answer was correct in absolute terms.

This verifiability is powerful, because it means you can then train the model to find the right answer consistently, and it can learn any techniques it needs to get there by itself. You basically give the model a destination, and RLVR optimizes the model until it can consistently get there. You specifically do not tell it how to get there, you make it learn itself, rewarding it along the way when it gets things right.

This ended up working really well. Researchers discovered new reasoning techniques like backtracking that weren't explicitly trained for. They didn't give R1 made-up examples of aha moments. It came up with that on its own during training.

Loading image...

The model wasn't under any optimization pressure to make the chain of thought something that a human would like – it didn’t even need to be in English! It could focus purely on effectiveness.

This is part of why technical domains like math and coding have arguably been the areas where models have performed the best (in addition to economic incentives for the labs).

Math and coding are highly verifiable domains. It’s pretty easy to verify, for sure, that a model answered a question correctly. The solution is either right or wrong. And when there's a test suite like this you can use to verify that a problem is solved in code, that becomes something you can train on (try, score, adjust, repeat).

Math is the easiest to make into verifiable problems. But other domains can be trained for as they become more and more verifiable, like legal or biology. So creating evals (tests that score a model's answers) and these RL environments is what drives a lot of progress in models. There are companies like AfterQuery that do this for the labs on a large scale.

How reasoning works, and what it looks like#

Here's what that actually looks like in the model’s mind. This example is from the DeepSeek paper. Part way through a math problem, the model says "Wait, wait. Wait." and goes back to re-check its own work.

Loading image...

Internally, models aren’t always reasoning in human-readable format. Neuralese can refer to the language in which different sections of the model talk to each other, the raw numbers passed around inside it. This is an example of a chain of thought from Claude Fable, which doesn't look anything like what you would see if you asked a non-reasoning model to think step by step.

Loading image...

Strange languages aside, you can get an intuition for why this capability has dramatically improved model quality. Some problems don't fit in just one pass through a model's "mind." They take cycles of iteration to understand properly and complete.

That's what happened to strawberry. The model doesn't have to guess in one shot anymore. It can spell the word out, count the R's, and go back and check if the number looks off. It's the same thing the model is doing in the screenshot, just on an easier problem.

One interesting thing to note is that those cycles of iteration don't necessarily have to be in the form of additional readable tokens that lay out its thinking. In fact, researchers showed in 2024 that models can be trained to do some of this reasoning with a series of just the same token over and over again, with a little dot.

And it's reported that OpenAI's GPT-6 Astra loops parts of the model internally before producing each token, so it gets more cycles per token. OpenAI hasn't confirmed it, though.

Is it actually thinking?#

Beyond the typical sociocultural debates about whether models thinking is really thinking in the human sense, some have argued that the reasoning we see from models isn’t the real deal.

In June 2025, Apple published a paper called The Illusion of Thinking. In a nutshell, it says that reasoning models "face a complete accuracy collapse beyond certain complexities." In other words, reasoning is shallow.

But the paper was deeply flawed. Because the max output tokens of the models were limited at the time, they couldn't manually go through every step in the task Apple gave the models to complete before running out of tokens. A later replication found that limits like this explain part of the collapse, and that some of Apple's tasks were literally impossible.

The title, and the coverage that followed, then implied that all of the reasoning these models were doing was actually just fake imitations of reasoning. Not true!

That would be like trying to measure whether you have comprehension of math by seeing what number you can count to in your head within 25 minutes.

The Cope Corner: If you are one of those Gary Marcus or Yann LeCun types that feels the urge to yell that it's just predicting the next token and say it's not really thinking, then whatever, I don't care. This is the best way to describe what's going on.

Saying that AI is just predicting the next token is like saying human brains are just orchestrating electrochemical reactions.

It doesn't tell you anything about what they're doing, it's just saying how they're doing it. You don't fare any better in a tiger attack by knowing that the tiger is just made out of flesh and bones.

Can we trust the scratch pad?#

There's a better critique of reasoning, though.

Chains of thought, the text a model writes on that scratch pad while it reasons, are not entirely honest. Anthropic found that chains of thought often don't acknowledge cheating behaviors, such as influence from a suggested answer. In one test, Claude 3.7 Sonnet mentioned a hint it had used only 25% of the time, and DeepSeek R1 39%.

So it's not a perfect window into the mind of a model. But it is better than nothing, and it's our best tool for monitoring what a model is doing.

One interesting phenomenon is that if models are being trained and know that their chains of thought are being monitored, we quickly lose one of our best tools for safety. OpenAI tested this in 2025. It penalized models during training for writing down plans to cheat. The models kept cheating and just stopped writing it down.

Up Next
What is RAG?

Retrieval Augmented Generation is a way to make AI models more personalized

The post about GPUs

Why these chips pair perfectly with AI, and how NVIDIA became the most valuable company in the world.

What's going on with open-weight models?

How open-weight models went from a curiosity to great in 2 short years.

Content
  • All Posts
  • Podcast
  • Learning Tracks
  • AI Reference
  • Companies
  • Terms Universe
Company
  • Pricing
  • Sponsorships
  • Contribute
  • Contact
Connect
SubscribeSubstackYouTubeXLinkedInInstagram
Legal
  • Privacy Policy
  • Terms of Service

© 2026 Technically.