Former Anthropic engineer raises the alarm about the fear AI hides
Former AI engineers privately admit they fear an existential risk — yet billions of dollars and US economic growth depend on roughly ten companies racing to reach AGI first.
Today's piece is about a rather uplifting story involving Jacob Coxon, a former AI engineer at Anthropic.
From what he wrote on X, I want to highlight:
> "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately.
> No other human activity poses this level of danger"
Right now there are many billions of dollars invested in AI, and the growth of the American economy depends on this investment, which is essentially in the hands of about 10 companies.
There's a race between them, and another with the Chinese AI models. That race is to reach what they call AGI — a superior artificial intelligence, one that would actually have the capacity to reason, to learn, and therefore to discover new things they believe would be revolutionary.
What they also believe is that whichever company gets there first will have such a huge advantage over the others that it will be the only one left standing. Some sources even say the CEO of Anthropic has repeatedly said in private that Anthropic will one day be the only private company left in the world.
My opinion on this is that, just like with nuclear weapons, there should be an international agreement on AI development. With today's world leaders? Yes, I think that's impossible.
But on the other hand, I also think all these Big Tech leaders have a very "particular" temperament — they all watched a lot of science fiction movies. Some of them, given the environment they grew up in when they watched those films, ended up siding with the "dark side of the force."
And basically, these people are out of their minds and need to be controlled. And they aren't. And they hold billions of dollars in their hands. And even more billions depend on the decisions they make.
As a Portuguese TV ad from the 90s used to say, "they think they're the best drivers in the world, and yet..."
*Note: for anyone who never saw the ad, it ends with the pencil (playing the role of the car) skidding off and snapping in two.*
Sources:
- X post quoted above
- WSJ — Anthropic Researcher Quits Over 'Out of Control' AI Fears
- Yahoo Finance — Anthropic left the only private company