The hardest question to answer, according to Coxon, was what an AI apocalypse would look like.
One of the risks outlined in Amodei’s comments was of a swarm of bots acting like a supercomputer that could take over the internet.
Coxon said this scenario could be realistic in six months to a year.
In response to Coxon’s departure, an Anthropic spokesperson told BBC News: “We have always been transparent that AI will bring both enormous benefits and unprecedented risks.
“To address these risks, we continue to build models with some of the strongest safeguards in the industry.”
The firm has been a pioneer in studying how AI models work, the spokesperson added. It was the first to publish a framework for mitigating the risks posed by their development – and it also “aggressively” tests its models and publishes the findings to aid scrutiny and prevent incidents of “AI misalignment”.
“This work is also why we believe the world would benefit from the industry adopting a lawful, verifiable way to work together to pace how we release powerful models,” they said.
Coxon said his peers feared the danger could even arrive in the next two years.
He said staff at AI firms were “planning what to do with their lives and thinking about the impacts of their work”, while some were “considering buying land somewhere because they’re so scared of the instability as a result of rapid AI progress”.
“They all keep this in their head on a daily basis while working on the technology.”
But Coxon did have some optimism about the future of AI, telling the BBC that the people researching the technology “genuinely want to see the upside” of things like “solving diseases and improving everyone’s lives”.
