The AI rivalry that safety cannot tame

Published on: Sep 14, 2026
Author: Nigel Trimmer

What if the most dangerous thing about artificial intelligence is not its speed, but the belief that its builders can slow it down on command? That is the paradox now sitting at the center of the OpenAI-Anthropic rivalry. These firms were born from a warning and have become a contest. One side says restraint is necessary. The other side says restraint is no longer credible. In markets, in engineering, and in politics, the gap between principle and incentive is where brittle systems break.

The latest dispute began with a claimed AI-assisted answer to the 200-year-old Navier-Stokes equations problem, a prize far too symbolic to remain merely academic. On one side stood a New York mathematician working with Anthropic. On the other was OpenAI, which reportedly spent millions and used nearly 10,000 AI agents. Even before any technical truth is settled, the episode shows the deeper problem: once status, credit, and strategic advantage are on the table, every breakthrough becomes a chess piece. The question is no longer whether the model can solve the problem. It is whether rivalry turns every result into an escalation.

The logic is familiar. In nature, a predator and prey can co-evolve into ever more extreme forms. In game theory, a repeated contest can produce mutual escalation even when both players know the end state is worse. Markets do this too. A bank may know leverage is dangerous, yet borrow more because the rival across the street is borrowing more. The AI race has the same shape. The language of safety remains, but the incentives belong to competition. And when competition becomes the organizing principle, caution begins to look like surrender.

The strange part is that both firms grew out of distrust. Anthropic was founded in 2021 by former OpenAI employees who believed AI needed more responsible stewardship than OpenAI provided. That origin story sounds like a moral split, but history suggests such splits rarely stay pure. Dissidents often become incumbents with better branding. The institution formed to resist one risk soon inherits its own. The deeper a company climbs into the frontier, the more it must prove itself in the same arena it once criticized. Idealism, in other words, is often a bridge to the same old machinery.

That is why the recent calls for slowing AI sound less like a coordinated safety effort than a recognition of mutual vulnerability. Anthropic co-founder Dario Amodei recently called for slowing AI’s pace after a series of safety warnings from current and former employees. Sam Altman and Elon Musk backed the call in a rare show of unity. But unity in public does not erase rivalry in practice. The market does not reward hesitation for long. If one firm pauses while the other advances, the pause starts to resemble self-harm. That is the central fragility: safety is a collective good, while advantage is private and immediate.

Safety as a Public Good

Anthropic’s founder has not argued for stillness. Amodei’s three-step plan, as described by The Paper, is a blueprint for managed risk rather than moral purity. It begins with independent external evaluators embedded long-term in companies. Then it calls for coordinated common safety standards among democratic nations’ AI firms. Finally, it reaches for global coordination. Each step is rational. Each step is also hard. The first depends on access. The second depends on trust among competitors. The third depends on states that often cannot coordinate even when the danger is visible. This is the old tragedy of the commons in a new uniform.

Anthropic has already shown how hard unilateral restraint is to maintain. It publicly walked back its responsible scaling policy commitments this year, citing competitive pressure and lack of faith in unilateral commitments. That retreat matters more than any polished safety speech. It is the engineering lesson hidden inside the ethics debate: a system designed with noble assumptions can fail when one input changes. In this case, the input is competitive pressure. Once that arrives, the moral architecture begins to deform. Companies do not usually announce the collapse of their principles. They reframe them as practicality.

There is also the human factor, which is always less heroic than the press release. Anthropic researcher Jacob Coxon resigned, warning that “the people building AI earnestly believe that it could kill us all by the end of the decade.” Stuart Russell, the UC Berkeley AI professor, put the competitive problem even more directly: “There’s no question that competition between companies causes them to take shortcuts on safety.” Those are not the words of traders or marketers. They are the language of insiders describing a pressure system. When people closest to the machine speak this way, the argument is no longer abstract. It becomes a question of whether the machine can be governed by those racing to build it.

The market logic is equally unforgiving. Both companies filed IPO paperwork with the SEC about one week apart, each seeking valuations above $1 trillion. That detail matters because public markets do not merely finance behavior; they fossilize it. Once a company is priced for perfection, the distance between “responsible” and “competitive” narrows. Every delay has a cost. Every safety measure has to justify itself against growth. In such a setup, the safest speech is often the least decisive. Investors do not buy caution when they think the frontier is still open.

The Credibility Problem

There is a reason ancient thinkers distrusted power that claimed to be benevolent. Plato understood that guardians can become rulers. Roman history shows how republics can slide into empire while still talking about duty. The AI sector now displays a modern version of the same problem. It wants to govern itself because it fears government. Yet self-governance works only when the participants accept limits that hurt them. The moment those limits threaten relative position, the arrangement becomes fragile. A vow that depends on everybody keeping it is only as strong as the least patient competitor.

That fragility is visible in the performative symbolism of the rivalry. In February, Altman and Amodei refused to shake hands at a photo session with Indian Prime Minister Narendra Modi, raising fists instead. It is a small scene, but small scenes often reveal large truths. Publicly, the industry can talk about civilization-scale risk. Privately, it still behaves like a factional contest. The gesture matters because the audience is global. Leaders understand that the image of unity can be useful even when real trust is absent. But a raised fist is also an admission: the contest is not over, and neither side expects détente.

The industry’s own language keeps circling back to the same truth. The story is not really about one disputed scientific result, one resignation, or one photo session. It is about the structure of incentives in a field that is racing toward systems no one fully understands. In such systems, the first danger is not catastrophe. It is normalization. Once ever larger models, ever larger spending, and ever larger claims become routine, the extraordinary starts to look ordinary. That is when the margin of safety shrinks without anyone announcing the shrinkage.

What happens next will not be decided by a single summit or a single founder’s warning. Sen. Bernie Sanders is set to convene a closed-door briefing of lawmakers and top AI safety experts on Sept 16. On Sept 17, King Charles III is due to meet AI leaders, including OpenAI and Anthropic executives, Nvidia’s Jensen Huang and Demis Hassabis, for a frontier prudence session on red lines for bioweapons and cyberattacks on critical systems. These gatherings matter, but not because they can eliminate rivalry. They matter because they may reveal whether society can build guardrails around actors who remain locked in mutual suspicion.

The sober conclusion is uncomfortable. OpenAI and Anthropic may both sincerely believe they are acting for the common good, yet sincerity is not the same as stability. The market rewards speed, the audience rewards confidence, and the system rewards whichever side appears least constrained. That is why the AI race feels less like a policy debate than a test of whether civilization can impose limits on a competition that keeps finding reasons to continue. In the old world, empires overreached because they thought they were exceptional. In the new one, the danger may be more banal: each side thinks it can afford one more shortcut.

AI