When the Builders Sound the Alarm: What Anthropic's Insider Warnings Mean for Every Board

On the evening of September 8, a 27-year-old researcher named Jacob Coxon posted his resignation from Anthropic — one of the three companies at the frontier of artificial intelligence — and told the world that the people building these systems "earnestly believe" AI could kill everyone by the end of the decade. His post drew 76 million views overnight. Within hours, Anthropic's own alignment science lead, Evan Hubinger, publicly agreed and put his personal estimate of AI-caused human extinction above 10 percent within ten years.

That would be alarming enough on its own. But it landed in a week when the world's top financial regulator had just named frontier AI cyber risk the most immediate threat to global financial stability, and when a bipartisan group of U.S. senators began advancing legislation to turn the AI industry's voluntary safety pledges into enforceable legal duties. Three signals, three sources, one message: the gap between what insiders know and what boards are doing about it has never been wider.

The Insider Warning

Coxon spent three years on pretraining research at OpenAI and Anthropic — the two companies closest to the frontier. His resignation was not a complaint about working conditions or a grudge. It was a substantive claim: both companies are racing toward self-improving superintelligence without adequate safeguards, and the competitive pressure between them makes voluntary caution insufficient.

In a Slack message to colleagues, Coxon warned that without more caution and cooperation, superintelligent AI posed "a risk of causing human extinction." Hubinger's public confirmation was even more pointed: Anthropic still has no plan for how to keep superintelligence under human control.

This is not science fiction. This is the assessment of people with direct access to the models, the training runs, and the capability curves. For boards still treating AI as a productivity story with a compliance footnote, that should change the conversation.

The Regulator Warning

Two weeks before Coxon's resignation, Financial Stability Board Chair Andrew Bailey — who also serves as Governor of the Bank of England — sent a letter to G20 finance ministers and central bank governors that should be required reading for every audit committee in America.

Bailey's core warning: "Frontier AI may have the ability materially to alter the speed, scale and economics of cyber risk, which could undermine market confidence system-wide." He singled out concentration risk — the financial sector's dependence on a small number of technology providers — as a force multiplier. A major disruption at one provider could affect multiple financial institutions simultaneously.

Bailey called on authorities to support safe model release and for financial institutions to ensure "strong response and recovery capabilities." Read between the lines: the global body responsible for financial stability believes frontier AI has already changed the threat model, and institutions that haven't adapted are exposed.

The Legislative Pivot

Congress has spent years holding hearings and writing pointed letters about AI. This month, something changed.

Senators John Thune, Ted Cruz, and Amy Klobuchar — a rare bipartisan coalition spanning the Senate Commerce Committee's leadership — are negotiating legislation that would impose a legal duty of care on frontier AI developers. The emerging bill would require mandatory pre-release safety testing, independent third-party audits, and standardized reporting. Most significantly, it would give the government power to block the release of models deemed hazardous. Voluntary pledges become enforceable law.

The Coxon-Hubinger disclosures landed at exactly the moment this legislation was taking shape, making it politically difficult to argue that industry self-governance is sufficient.

Meanwhile, former President Obama privately urged House Minority Leader Hakeem Jeffries to make AI oversight the top Democratic priority for the midterm elections, telling him to organize a "prominent public conversation" on AI policy. The AI Kill Switch Act, the FRONTIER Act, and the Thune-Cruz-Klobuchar bill are all moving simultaneously — an unprecedented concentration of legislative activity around a single technology risk.

The Defensive Side Keeps Expanding

Amid the alarm, the defensive infrastructure is also growing. On September 3, application security firm Checkmarx announced it had joined Anthropic's Project Glasswing, gaining access to Claude Mythos 5 for defensive cybersecurity work. Checkmarx will use the model to strengthen vulnerability detection and share findings with the broader security community.

This matters because Glasswing's value proposition rests on a paradox boards need to understand: the same capabilities that make Mythos dangerous make it indispensable for defense. Checkmarx noted that roughly 80 percent of exploitations now occur on or before the day a vulnerability becomes public. Without AI-powered detection operating at machine speed, defenders are structurally behind.

The question is no longer whether to use frontier AI for defense. It's whether your organization has access — and whether access alone is enough when the people who built the model say they can't guarantee control over what comes next.

What to Ask Your CISO This Week

The convergence of insider warnings, regulatory alerts, and legislative action creates a board-level obligation that didn't exist six months ago. Here are four questions for your next committee meeting:

1. Have we assessed our exposure to frontier AI-powered threats? The FSB's warning about concentrated third-party risk applies to every company with cloud dependencies, well beyond financial institutions. Do you know which providers your critical systems depend on, and what happens if one of them is compromised at machine speed?
2. Are we participating in — or at least monitoring — Glasswing-class defensive programs? If your organization isn't using frontier AI for defense, your adversaries may already be using it for offense. The asymmetry is the risk.
3. What's our position on the incoming regulatory requirements? The Thune-Cruz-Klobuchar bill, the Kill Switch Act, and the FRONTIER Act could all move this fall. If your company develops, deploys, or depends on frontier AI models, you need a regulatory-readiness posture now, not after the vote.
4. Has the board discussed the insider warnings? When researchers at the leading AI labs publicly state they believe their own products could cause human extinction, that's material risk information. Your risk committee should have a documented position on how the organization interprets and responds to these disclosures.

For the full framework on structuring these board conversations — including the Three Questions every director should ask — see Chapter 8 of Cyber Risk Is Business Risk.

The Bottom Line

There's a pattern in technology risk: insiders warn, regulators notice, legislators act, and boards adapt — usually in that order, and usually too slowly. This month, all four are happening at once. The people building the most powerful AI systems are publicly saying they don't know how to keep them under control. The world's top financial stability body is calling frontier AI cyber risk the most immediate systemic threat. And Congress is drafting laws that will turn voluntary commitments into legal obligations with real penalties.

The question for every board in America is no longer "should we be paying attention to AI risk?" It's "can we demonstrate that we are?"