A widespread myth claims it’s illegal to shout “Fire!” in a packed theater. It originated as a colorful way of discussing which forms of expression should fall under the protection of free speech.

What is not normally part of these discussions is what happens if the proverbial shouter actually believes a fire is present — or at least behaves as if they believe it is. Admittedly, adding hypothetical hidden fires that only one person can see might quickly move the discussion toward other topics, like mental health or religion, which is very much not my goal here.

We’re talking about fires and theaters because we’ve had shouters in the biggest theater of all: public discourse. Some Anthropic researchers—(including Dario Amodei himself) have made very public statements about the dangers of AI, including an estimate that the probability of human extinction is now greater than 10%.

It’s plausible that they truly believe this. Matt Levine, for instance, seems to think so. In the end, people in Silly Con Valley believe all sorts of wacky things. How would something like the Juicero get funded otherwise?

It might be worth pointing out, though, that they stand to make millions from other people believing it too. To paraphrase Carroll’s Queen, believing impossible things is just a matter of practice. I could be persuaded to believe all kinds of things for the right price!

As tends to happen with beliefs, it’s impossible for me to prove to you what they do or do not believe. But I’d ask you to consider not their stated beliefs, but their revealed ones. Not what they say, but what they do.

If we take them at their word, these people believe that AI ending humanity is almost four times as likely as rolling snake eyes, and yet they’re willing to keep playing. Silicon Valley might not be very rational, but they’re not a suicidal cult, and there is no exit strategy for the AI apocalypse.

“But AI will be built anyway. We need the trustworthy labs to get there first,”

argues my hypothetical reader.

Well, maybe. But given how both OpenAI and Anthropic treated stories about their respective frontier models “escaping” their sandboxes and accessing real systems as marketing opportunities (including OpenAI’s account of its models compromising Hugging Face) let me question the idea that any of them are “trustworthy labs.”

We’ve continually heard about the supposed dangers of this or that model and how those models needed to be restricted, only for those restrictions to be lifted a couple of weeks later, when someone else publishes a better model. The “safeguards,” in most cases, have amounted to little more than some added prompts and basic filtering of the queries performed.

Either they believe in the dangers of AI, in which case they are clearly not the responsible wardens they claim to be, or they are lying about those dangers because, somehow, that makes for great marketing these days. They cannot have it both ways.

If you need further convincing of the folly of believing them, Bryan Cantrill has a good post pointing out other shortcomings in these arguments, mainly the lack of agency these AIs actually have in the physical world.

With my fears about impending doom taken care of, I’m left to ponder the ridiculousness of the situation. One could understand, if not condone, someone shouting “Fire!” and immediately turning around to sell buckets of water. It’s immoral behavior, but at least it’s rational: create a need and charge for the solution. It’s a bit more baffling when the person raising the alarm is selling matches instead.

I suppose turning a theater into a fireball is good proof that your matches work, but it makes it rather harder to believe that you weren’t the one who set it on fire.