The most important technical debate in AI right now isn’t really about technology but about its weight, and the answer will weigh on humanity forever, writes Satyen K. Bordoloi.
I asked an LLM what it’d call the July 24 letter that NVIDIA CEO Jensen Huang released in his first X post. Its answer: “a press release dressed up as a manifesto.” Why I asked an LLM is because the letter was signed by 25 companies, which within hours doubled to 50, and this included some of the biggest AI companies on the planet. When so many AI founders agree on something, it’s better to ask an LLM what it thought.
But two companies were conspicuously absent: Anthropic and Amazon.
Huang’s letter titled “Open Weights and American AI Leadership” tries to make a case that sounds obvious on first read. Open-weight AI models, i.e. those models whose underlying numerical parameters anyone can download, inspect, modify, and run on their own hardware, are a public good. Why? Because they expand access and strengthen competition, becoming in essence the AI equivalent of open-source software. Innovation strives on openness – no one can argue against that.
Except Anthropic did. And the matter is, it’s dragged the White House into its ambit.
The Chinese Ghost In The Machine
The timing for the letter is not a coincidence. A Chinese AI startup dropped a model that’s raised many eyebrows, including in the American establishment, for what it can do. That’s because Kimi K3 by Moonshot AI, an open-weight model, is the latest toast of the world for matching and, in a few cases, surpassing leading US ones on multiple industry benchmarks
Kimi K3 is the largest open-weight AI model ever released, with 2.8 trillion parameters, and Moonshot didn’t just release the weights but also its key infrastructure, including tools to improve efficiency and stability. This allows developers worldwide to use, copy, modify, and distribute the software, one that’s as good as the best. It is like in the 1990s when Microsoft Windows ruled the world, a Chinese company releasing a free equivalent that’s as good in most things, and better in a few.
This is an event that is bound to shake the AI world order. And indeed it has, as the markets worldwide saw a selloff in tech stocks in the last few days.
It is now believed that the White House is actively considering restricting Chinese open-weight models by adding Chinese AI labs to its “Entity List,” effectively cutting off U.S. access without a license. The Trump administration also accused Moonshot AI of “distilling” knowledge from American AI companies: a process where developers use millions of interactions with existing models to train their own systems. Specifically, it’s been accused by the administration of having distilled Anthropic’s Fable.
Moonshot, of course, denies everything, and some independent researchers have pointed out discrepancies in the launch of Fable and K3’s launch – a 15-day window – too small to do a technically good job of distillation to explain K3’s capabilities.
The truth doesn’t matter as the White House is slated to turn its accusation into policy and NVIDIA’s open letter, in some ways, becomes a direct response to the government’s impulse to ban Chinese open-weight models wholesale.
Anthropic’s Response
After remaining quiet for days – eternity in the AI age – Anthropic founder Dario Amodei’s response, posted July 27, opens by being unusually direct: “Anthropic has never advocated for a ban on open-weight models.” So, why didn’t they sign the letter? The reason, Amodei says, is that while he agrees with most of it, he still believes in the possibility of two nightmare scenarios that he thinks the letter just waves away.
His first fear is that authoritarian governments might build AI models more powerful than those built by the US and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people. His second fear is that powerful AI models may be misused to carry out cyberattacks or biological attacks, and open-weight models, regardless of their country of origin, do present a potentially higher risk than closed models, because it is very difficult to apply guardrails to them or monitor their usage, because once weights are released, they cannot be withdrawn.
He uses biology as his sharpest example, stating that a sufficiently capable AI could potentially help weaponise a pandemic-level virus with widely available materials, with the world taking years to defend itself against just one virus, as we saw during COVID.
So Amodei’s position is more nuanced, and more uncomfortable. He’s basically asking not to ban open weights or Chinese models. But he’s asking for powerful chips to be blocked from China, a crackdown on industrial-scale distillation operations, and mandatory safety testing for all sufficiently capable models regardless of whether they’re open or closed.
The NVIDIA letter’s strongest argument
To be fair, NVIDIA and cohorts aren’t wrong about the economics and the history of open source. Open-source software, once seen as a threat to commercial software, now supports most of the internet and underlies systems used by the world’s largest technology companies, as well as the U.S. military and federal agencies. Open source did more than lower the cost of software: it created a shared foundation of knowledge on which generations of American engineers and entrepreneurs built their institutional sovereignty.
The same logic applies to Open-weight models, which let organisations match the right model to the right job at the right cost, reserving frontier-scale capability for genuine frontier problems and running efficient, specialised models everywhere else. That discipline is what will make AI economically sustainable for everyone out there because there are thousands of hospitals, farms and factories that can’t afford a Claude or GPT-5 subscription for every mundane task.
In essence, the world needs both frontier closed models and frontier open models.
The uncomfortable self-interest nobody wants to name
While the letter promotes openness for “American AI leadership,” critics have indeed observed a self-serving aspect in it. That’s because many signatories, including organiser NVIDIA, maintain proprietary layers elsewhere. NVIDIA wants more models trained on more hardware. More open weights means more GPUs working worldwide. Thus Huang’s open-source advocacy is also, conveniently, a sales pitch.
OpenAI and Anthropic – both valued at nearly $1 trillion and gearing up for potentially massive IPOs – did not initially sign the letter. Companies building closed frontier models have obvious commercial reasons to be sceptical of open weights stealing their lunch. Amodei is careful enough not to say this, but it’s worth naming: the companies loudest about openness often have the most to gain from commoditising their competitors’ turf.
And China’s play? Moonshot’s strategy, and also of Chinese labs more broadly, is to go open, i.e. letting anyone download, study, and customise their models, almost mocking the closed systems of American AI companies. If you can’t out-restrict your adversary, drown them in open access. Force the debate. Watch the Americans fight each other about it. Which they are, right now.
What needs to happen is based on whose interest we look at the problem from. If it is American interest we’re talking about (India has no interest yet, and our position should favour open weights) then Amodei’s three-part prescription is what the US should follow: stop chip smuggling of which there has been many, crack down on industrial scale distillation and test everything before release to prevent misue risk and directly test models for cyber, biological and alignment risks before release.
In essence though, the open weights debate is a proxy war for something much bigger: who gets to set the rules of a technology that is reshaping every industry, every government, every military, indeed every person’s life on the planet. The NVIDIA coalition is right that openness has historically driven American dominance. Amodei is also right that this moment has weapons-grade variables that didn’t exist in the 1980s software wars.
Both things are true. And between them – in that uncomfortable gap- is where actual national policy – be that of the US or China’s- has to live. In the meantime, Kimi K3’s weights are already sitting pretty on Hugging Face right now. Anyone can download them.
The argument about whether that’s dangerous is already moot.
In case you missed:
- Anthropic Accuses Chinese AI of “Stealing”, Internet Points Finger Back At Them
- Anthropic was afraid of their latest AI Mythos: they released it anyway, to the panic of the world
- A Small LLM K2 Think from the UAE Challenges Global Giants
- NVIDIA’s Strategic Pivot to Drive our Autonomous Future with Innovative Chips
- Anthropic Fears AI Could Self-Replicate, Leaving Humans in the Dust
- The Major Threat to India’s AI War Capability: Lack of Indigenous AI
- The AI Peace Race: Why India’s Real Opportunity Starts Where the Arms Race Ends
- OpenAI, Google, Microsoft: Why are AI Giants Suddenly Giving India Free AI Subscriptions?
- AI vs AI: New Cybersecurity Battlefield Where No Humans Are in the Loop
- The AI Prophecies: How Page, Musk, Bezos, Pichai, Huang Predicted 2025 – But Didn’t See It Like AI Is Today
