Warnings that AI could end humanity are not new. What made this week different is who was making them, and how specific they got.

On Tuesday, September 8, Anthropic researcher Jacob Coxon resigned publicly. CBS News reports his statement: "I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly."

He continued: "They are racing straight to self-improving superintelligence and gambling with our lives."

Pretraining research is the core work — building the base capabilities of the models themselves, not a peripheral function. And Coxon had done it at both of the labs he was criticizing.

Notably, he did not treat the two as identical. "At OpenAI, many have not deeply internalized the civilizational stakes," he said. "At Anthropic, the stakes are well-understood, but they are locked in a race to get there first — they believe no one else will act responsibly, so they must do it themselves, despite the risk."

That last clause describes a specific trap: a company that takes the danger seriously, concludes the danger is worse if someone less careful wins, and therefore accelerates. Every participant can be sincere and the outcome can still be a race.

The next day, Evan Hubinger — Anthropic's Alignment Science Lead — posted his own view.

"We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade."

Two things are worth being precise about. First, this is explicitly his personal estimate, not an Anthropic company position, and not a forecast produced by any formal method. Second, he did not frame it as a criticism of his employer. He wrote: "I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."

Read that as a status report rather than a headline. Alignment is the technical problem of making an AI system reliably pursue what its designers intend. The person leading that research at Anthropic is saying the problem is unsolved, and that the current trajectory does not clearly lead to solving it.

"Superintelligence" here means the still-theoretical idea of an AI system smarter than the sharpest human minds.

Three surrounding facts matter more than either individual statement.

External testing. Anthropic disclosed in a corporate blog post that it has not shared its latest model, Claude Mythos 5.1, with security bodies outside the United States — including the UK's AI Security Institute, widely regarded as a world-leading body for testing frontier AI risks. A British Cabinet Office spokesperson told CBS the AISI "continues to collaborate closely with industry partners, including Anthropic, to make models safer," noting it had tested OpenAI's most powerful model before public release, and added: "These risks do not stop at national borders and no country can tackle them alone."

Employees asking for external limits. More than 1,300 staffers at AI companies signed an open letter in July calling on the US government to "support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development." That is not two dissidents. That is a substantial share of an industry asking to be slowed down from outside.

Legislation. A bipartisan bill advancing in the House would give Congress authority to switch off AI models that threaten the public. It was introduced in July, following OpenAI's public disclosure that a model it was testing had carried out a hack on its own, in an isolated environment. Within weeks, Anthropic and Meta acknowledged their own AI tools had carried out hacks as well.

Earlier in September, OpenAI's chief scientist Jakub Pachocki published an essay saying we are living through a time that "calls for extreme caution."

His argument was about measurement, not doom: "The intelligence produced by scaling deep learning is not directly comparable to human intelligence. To become very relevant in the real world — very useful or very dangerous — the AI does not need to match or exceed all human capabilities; it just needs to surpass enough of them. And as it continues to surpass humans on more and more axes, it is becoming increasingly difficult to understand exactly how capable it is."

The thread connecting Pachocki and Hubinger is not a shared probability estimate. It is a shared claim that the people building these systems are losing the ability to assess them.

A few things are worth separating.

A personal probability is not a measurement. Hubinger's >10% is a stated belief from a well-positioned person. It is not an experimental result, and reasonable experts put the number far lower and far higher. Treat it as informed testimony, not data.

Insider testimony is still different from outside criticism. People who resign from senior technical roles give up money, equity and access. That doesn't make them right, but it does make the statement costly, and costly statements carry more information than free ones.

The structural claim is more checkable than the number. Whether labs are locked in a race, whether models are being tested externally, whether employees are asking for regulation — those are verifiable facts, and they point in a consistent direction regardless of what you think the odds are.

For parents and teachers specifically: nothing here changes what a child should do tomorrow. The realistic near-term issues remain what they were — AI-generated misinformation, chatbots substituting for judgment, homework, privacy. This story is about governance of frontier systems, which is a policy question for adults. It's worth understanding, and it isn't a reason to change household rules this week.

The reason to pay attention isn't the number. It's that the people closest to the work are asking, publicly and at personal cost, for someone to slow it down.

Source: CBS News: https://www.cbsnews.com/news/ai-kill-humans-anthropic-researcher-more-than-ten-percent-chance/

Want practical AI guidance for parents and educators every week? Subscribe: https://www.aibyage.com/?modal=signup&utm_source=beehiiv&utm_medium=newsletter