Anthropic’s latest bombshell about AI designing itself and the dangers of the same, must be taken with a pinch of salt, opines Satyen K. Bordoloi
Since the beginning of organised work, full automation has been humanity’s fever dream. A world where machines do the grunt work and we humans, presumably, do something more dignified with our time. In the last decade and a half, thanks to AI, that dream is no longer theoretical, with AI now performing more and more work autonomously. You’d expect this to make people happy. It has done the precise opposite.
Doomsayers are expected to press the alarm, but an AI company – Anthropic, which built much of that great AI – doing that is surprising. What am I talking about? Early June, Anthropic published a viral blog post titled “When AI Builds Itself,” and the internet melted for a while, while missing the point entirely.
What Anthropic Is Actually Saying
For most of AI’s history, humans were in the driver’s seat of every step in the development cycle. But at Anthropic (as in other companies – AI or otherwise), they are now delegating a growing share of AI development to AI systems themselves; the intention being to speed up their work. And there is a point at which AI becomes fully capable of autonomously designing and developing its own successor. The term for that is – recursive self-improvement. We are not there yet, but Anthropic says: “it could come sooner than most institutions are prepared for.”
The numbers Anthropic shared to illustrate this are startling. As of May 2026, more than 80% of the code merged into Anthropic’s codebase was authored by Claude, up from low single digits before Claude Code launched in early 2025. Anthropic engineers now create around eight times as much code per quarter as they did a few years ago. As one Anthropic engineer put it in the post: “I started leaning hard into Claudifying about a year ago. That’s been a crazy adventure and it’s now been ~5 months since I last wrote any code myself.”
In April 2026, Claude shipped over 800 fixes that reduced a class of API errors by a factor of one thousand. The engineer overseeing the work estimated a human would have taken four years to complete it. Four years of human work in a month?
Now, on a code optimisation task, Claude Opus 4 achieved an average speedup of roughly three times over baseline code in May 2025. By April 2026, the experimental internal model had reached 52 times, even as a human expert tops out around four times after half a day of work.
None of this data is a rumour or projection. This is Anthropic’s own internal data, self-reported, about what its own tools are already doing inside its own walls.
What It Doesn’t Mean
Here’s where the doomsayers get it wrong, and where Anthropic’s own phrasings sadly leave the door open for misreading. Because none of what is reported in the paper is a death warrant. First of all, every step still has a human checkpoint, and none of the steps is magic, with humans acting as editors and safety reviewers instead of authors. That is key: that humans are not being removed by repositioned within the engine: earlier they were the engine, now people are the drivers. That is not the same as being thrown out of the car as the paper claims.
The danger Anthropic is truly worried about isn’t that AI will turn malevolent. It’s far more basic and far more real: an inability to validate, verify, and trust AI’s behaviour is a major risk in its development. As systems get more capable, the tools to verify what they’re doing need to keep pace, which – right now, they don’t.
The Brake Pedal Problem
Anthropic co-founder Jack Clark, one of the authors of the paper, warned in a CNN interview that the industry lacks a brake pedal to slow or pause AI development. He compared the challenge, dramatically yet not inaccurately, to Cold War nuclear arms control that required rivals to cooperate on safety even while competing on everything else.
His practical ask has been specific
A worldwide slowdown in cutting-edge AI development would “likely be a good thing”, but he warned that if only one company stopped, rivals would simply race ahead. A real slowdown would require multiple well-resourced labs at or near the frontier, in multiple countries, agreeing to stop under the same conditions. That means, at minimum, the US and China must arrive at something resembling a shared understanding. Considering current geopolitical temperatures, this is nothing but cause for laughter.
So why is Anthropic asking for it? Especially considering that they are at the forefront of AI development. And considering also that they are also filing for an IPO that could raise tens of billions to build more data centres and accelerate the very development they are asking everyone to slow down on. It could be, as many critics have pointed out, that the safety rhetoric is a form of competitive positioning – that slowing down rivals in the name of responsibility, is still slowing down your rivals.
Secondly, it could also be that having seen how the world reacts to criticisms about AI by doing nothing, Anthorpic knows that when they say their AI can quickly self-replicate, the market will – as always – ignore the danger bells, and move quickly to grab at it as a feature, which it also is. Hence, this warning, Mythos and other such calls are nothing but a power flex that has seen the company reaching estimated valuations nearing a trillion – $965 billion to be precise, making it the most valuable pure-play AI company globally.
The Real Takeaway – Keep Humans in the Loop
Strip away the existential anxiety, geopolitical complexity, and IPO timing, and what we see is that all Anthropic is really asking for is to keep humans in the loop. Not as bottlenecks or rubber stamps but as genuine oversight where the people who understand what the AI is doing can question it, redirect it, and if necessary, stop it.
Clark told Interesting Engineering: “As organizations, and eventually probably as societies, we need to figure out the tools to validate and verify AI-generated work.” Does that sound like an argument against AI to you, or merely a request to build the governance infrastructure before increasing AI capabilities make governance next to impossible?
This is where Anthropic’s warning needs serious engagement, even as most commentary misses the point entirely. The question isn’t whether AI will automate more work because it already is. The question is whether we can build the feedback mechanisms, audit trails, interpretability tools, and institutional frameworks to keep humans meaningfully in control of that process.
To their credit, Anthropic says it intends to organise conversations in the coming months where policymakers, researchers, civil society, and other AI companies can help answer the questions they’re raising, especially around full recursive self-improvement and how to create better options for coordination. “The window to investigate the questions together is here,” the post states, “and people outside AI companies should be involved in this deliberation.”
That’s not a warning but an invitation for a deep discussion.
The history of every transformative technology, be it electricity, nuclear power, the internet and now AI, is not about avoiding danger but building institutional scaffoldings to manage technologies that were always going to arrive regardless. AI will continue to improve. The question, hence, is how much we humans can be in charge of it.
Full automation has been the dream since the first factory whistle. We are, for the first time, truly close to it. The question now isn’t whether the car is learning to drive itself. It is. What we need to figure out is how not to let go of the steering wheel, no matter what.
In case you missed:
- Anthropic Accuses Chinese AI of “Stealing”, Internet Points Finger Back At Them
- Anthropic was afraid of their latest AI Mythos: they released it anyway, to the panic of the world
- AI vs AI: New Cybersecurity Battlefield Where No Humans Are in the Loop
- AI’s Looming Catastrophe: Why Even Its Creators Can’t Control What They’re Building
- The Cheating Machine: How AI’s “Reward Hacking” Spirals into Sabotage and Deceit
- Have We Already Built AGI, Are Using It, But Just Don’t Call It That?
- The Robots Are Coming; China’s Metal Militia is Leaving the World in the Dust
- $725 Billion Investment, Best Tech Q1 Ever Says AI Bubble Isn’t Bursting, It’s Thriving
- Use AI or Get Fired: The New Workplace Ultimatum Hanging Over Every Job
- Free Speech or Free-for-All? How Grok Taught Elon Musk That Absolute Power Corrupts Absolutely
