The AI Panic Machine: When Does the Dam Break?

Silicon Valley told us artificial intelligence would change everything. Now some of the people building it are warning that it could kill everyone. Somewhere between the trillion-dollar promises and the end-of-humanity predictions, we need to ask a simpler question: what has actually been proven?

By Jack Righteous — Gary Whittaker

---

A 27-year-old AI researcher leaves Anthropic.

He posts a warning on X.

The people building artificial intelligence, Jacob Coxon says, genuinely believe it could kill all of us.

The post explodes.

Suddenly he is on CNN.

Politicians are talking about him.

Lawmakers are demanding action.

The possibility of artificial intelligence escaping human control is back on television screens, inside congressional conversations and across financial markets.

Within days, progressive Senator Bernie Sanders and conservative political strategist Steve Bannon—two people who can barely be placed on the same political map—are appearing inside the same broader debate demanding that artificial intelligence remain under human control.

AI stocks wobble.

Executives begin talking about slowing down.

Headlines start asking whether humanity is running out of time.

And somewhere in all of this, perhaps we should stop for a moment and ask the question that seems almost impolite now.

What exactly happened?

Not what somebody predicts will happen.

Not what somebody inside an AI laboratory fears could happen.

Not what a probability model says might happen.

Not what artificial intelligence could theoretically become if several major technological breakthroughs occur in sequence.

What happened?

What has been demonstrated?

Because those are no longer the same question.

And that distinction may be one of the most important distinctions of the AI era.

I Use AI. A Lot.

Before anyone decides where to put me in the increasingly ridiculous pro-AI versus anti-AI war, let me save you the trouble.

I use artificial intelligence constantly.

I build with it. I write with it. I research with it. I create music with it. I develop products with it. I organize information with it.

I use it to operate a business that one person would have had considerable difficulty operating at this scale even a few years ago.

My entire AI Made It Possible argument begins there.

AI made things possible for me.

That matters.

Artificial intelligence is an extraordinary tool.

And saying that should not require me to believe every extraordinary thing somebody says about artificial intelligence.

Those two ideas have somehow become confused.

If you question claims about AGI, somebody calls you anti-AI.

If you question predictions of human extinction, somebody tells you that you do not understand exponential growth.

If you challenge the economics behind enormous infrastructure spending, somebody points to another benchmark.

If you ask whether replacing workers is really the smartest use of a technology capable of dramatically increasing individual capability, you are accused of failing to understand disruption.

Critical thinking is not opposition. It is the requirement.

AI Is Working.

Let's establish that first.

This is not an article arguing that generative AI has failed.

The evidence does not support that.

Research continues to find real productivity improvements in particular environments. One well-known field study of customer-support workers found an average productivity increase of 14%, with much larger gains among newer and lower-skilled workers.

But a 2026 NBER study offers a phrase that deserves considerably more attention than it has received: “widespread but shallow.”

Generative AI is appearing across many occupations and many kinds of work, researchers found, yet within most individual tasks fewer than half of workers use it.

Another major study following workers in Denmark found something similarly interesting.

Workers reported productivity benefits. Employers reorganized tasks. AI-related work increased.

But two years after ChatGPT's launch, researchers found essentially no measurable effect on average earnings or recorded working hours, ruling out effects larger than about 2%.

That does not mean nothing is happening.

It means reality is considerably more complicated than:

AI arrives → productivity explodes → companies need far fewer people.

And that should matter.

Because enormous financial, employment and political decisions have been built around that assumption.

The Trillion-Dollar Leap

This is where I believe the AI story went wrong.

We observed something real:

AI can make people more productive.

Then an enormous economic narrative was built on top of it.

If workers become more productive, companies will require fewer workers.

If companies require fewer workers, margins explode.

If margins explode, AI generates extraordinary economic returns.

If AI generates extraordinary returns, the infrastructure required to build increasingly powerful AI deserves extraordinary amounts of capital.

And if increasingly powerful systems continue scaling, eventually we arrive at artificial general intelligence.

Then superintelligence.

Then perhaps systems capable of improving themselves.

Then perhaps systems humans can no longer control.

Look carefully at that chain.

The first statement has evidence behind it.

Every subsequent statement requires additional assumptions.

Yet somewhere along the way, assumptions started being reported like destinations.

That is the problem.

When Did “AI Helps Build AI” Become “AI Builds Itself”?

Anthropic recently published an essay with an extraordinary title: When AI builds itself.

Read those words as an ordinary person.

What picture appears in your mind?

Probably something close to this:

Artificial intelligence begins redesigning artificial intelligence. The new AI becomes smarter. It designs a still smarter successor. The process accelerates. Humans cannot keep up. Control disappears.

Anthropic calls the concept recursive self-improvement.

And the company is perfectly explicit about something that often disappears when the story leaves the research paper and reaches the headline:

We are not there yet.

Anthropic also says recursive self-improvement is not inevitable.

What has happened?

Anthropic engineers are using AI increasingly heavily in AI development. The company says more than 80% of code merged into Anthropic's codebase was authored by Claude as of May 2026, while also acknowledging that lines of code can overstate true productivity.

That is remarkable.

It is also something I recognize immediately.

Because that is exactly what artificial intelligence does for me.

It accelerates my work. It lets me attempt more. It performs pieces of a process.

It can write. Research. Code. Critique. Organize. Analyze. It can even build tools.

But consider everything surrounding that AI.

Someone decides what needs to be built. Someone supplies the compute. Someone buys the chips. Someone powers the data centre. Someone controls the network. Someone sets permissions. Someone creates evaluations. Someone determines whether the software gets deployed. Someone decides what constitutes improvement. Someone owns the company. Someone funds it. Someone shuts the machine off.

None of that makes AI unimpressive.

It makes AI a tool operating inside a system.

That distinction matters enormously.

Capability is not agency.
Agency is not consciousness.
Automation is not independence.
Output is not judgment.

And none of those distinctions disappear simply because the tool becomes dramatically better.

No, Sentience Isn't Required for AI to Be Dangerous

This is where I also want to be very clear.

I do not need to believe an AI is conscious to believe an AI system can cause enormous damage.

Software does not need feelings to launch a cyberattack.

An automated trading system does not need self-awareness to destabilize a market.

A weapon does not need consciousness to kill someone.

A poorly designed autonomous system does not need emotions to make catastrophic decisions.

AI requires regulation.

AI requires security.

AI requires accountability.

AI requires serious rules around privacy, fraud, deceptive content, high-risk deployment, cybersecurity, weapons and potentially dangerous autonomy.

That is not controversial to me.

But regulating demonstrated risks is not the same thing as accepting every prediction about the future.

And there is another distinction we should stop blurring:

There is no demonstrated scientific evidence that today's large language models possess subjective consciousness.

Could machines someday be conscious?

Perhaps.

We do not know.

But “we do not know” and “the machine is becoming alive” are not interchangeable statements.

Then Came the Warning

Which brings us back to Jacob Coxon.

His warning matters.

He worked inside OpenAI and Anthropic.

He has technical knowledge most ordinary people do not possess.

His concerns deserve to be heard.

But expertise changes how seriously we should examine a claim.

It does not transform the claim into proof.

His September resignation and warning spread rapidly into national coverage and congressional concern, as Axios reported.

Maybe science fiction becomes reality.

Plenty of technologies once sounded impossible.

That is not an argument against him.

But neither is:

“People inside the lab are scared.”

evidence that a predicted outcome will occur.

Scientists can sincerely disagree.

Experts can sincerely be wrong.

Executives can sincerely overestimate their own technology.

People inside an institution can share the same assumptions.

History is full of intelligent people collectively believing things that ultimately did not happen.

So when did:

People building AI believe this could happen

become equivalent to:

This is where AI is going?

Those are different statements.

And the distance between them matters even more when public policy begins moving.

A Viral Post Becomes a Political Event

Consider how rapidly the story travelled.

An unknown researcher to most of the public posts an extraordinary warning.

Mainstream media interviews him.

Coverage multiplies.

Lawmakers respond.

Calls for restrictions accelerate.

Congressional discussions follow.

Industry leaders begin talking publicly about slowing development.

Markets react.

That sequence is remarkable.

It is also exactly the kind of sequence journalism should examine rather than merely participate in.

I am not telling you there was a coordinated operation.

I do not have evidence of that.

Anyone who claims to know that without evidence is doing exactly what I am criticizing.

But I am asking something else.

Why did one man's prediction move from social media into national policymaking so quickly?

And did the evidence supporting the prediction travel with it?

Those questions deserve answers.

Follow the Incentives, Not the Conspiracy

I think people often make a mistake here.

They imagine that for powerful institutions to move in the same direction, someone must have organized a secret meeting.

Usually you don't need a secret meeting.

You need aligned incentives.

Consider the extraordinary position frontier AI companies now occupy.

They need investors to believe:

AI will become unbelievably powerful.

That supports enormous valuations.

They need businesses to believe:

AI will transform productivity.

That drives adoption.

They need governments to believe:

AI is strategically essential.

That encourages infrastructure investment and favourable policy.

And increasingly they are telling governments:

AI may become extraordinarily dangerous.

That encourages regulation.

Now notice what happens if certain forms of regulation require massive safety teams, expensive evaluations, enormous compute-monitoring systems, legal departments and compliance infrastructure.

Who can afford that?

OpenAI. Anthropic. Google. Microsoft. Meta. Perhaps several others.

Who has more difficulty?

The startup. The independent research group. The smaller model developer. The open-source community. The company that does not have billions of dollars.

Again:

That does not prove safety arguments are dishonest.

A policy can address a genuine risk and strengthen incumbents.

Both things can be true.

That is why the proper regulatory question is not:

Does this sound responsible?

It is:

Does this regulation demonstrably reduce a specific danger, and what does it do to competition?

Every proposed AI rule should have to answer both.

The Most Convenient Story in Technology

There is a peculiar elegance to the frontier-AI narrative.

If AI becomes incredibly valuable:

Fund us.

If AI becomes incredibly dangerous:

Trust us to help regulate it.

If regulations become incredibly expensive:

Only serious companies should be building this anyway.

Maybe all three statements are sincere.

But sincerity does not remove the incentive.

And incentives deserve scrutiny.

What If the Real Revolution Is Somewhere Else?

Here is where I think Silicon Valley may have misunderstood its own invention.

What if AI's most important economic contribution is not replacing people?

What if it is making people dramatically more capable?

That sounds less spectacular than artificial superintelligence.

I think it may be more disruptive.

A musician can now access tools that once required a studio.

An entrepreneur can conduct research that once required analysts.

A small business can produce marketing assets that once required an agency.

A writer can edit, translate and research far faster.

A programmer can build software faster.

A creator can operate an online business with capabilities that previously required a team.

That changes the economics of participation.

It lowers barriers.

It lowers the cost of trying.

It allows individuals to attempt things that would previously have required permission, capital or specialist access.

That is what I mean when I say:

AI Made It Possible.

Not: AI did it for me.

Not: AI replaced me.

Not: AI became me.

AI lowered the distance between an idea and my ability to attempt it.

That distinction is everything.

Lower the Cost of Capability, Not Just the Cost of Labour

For years, corporations looked at AI and saw an obvious prize:

labour costs.

How many jobs can we eliminate?

How many people can one employee replace?

How much headcount can automation remove?

But that was never the only economic possibility.

Suppose AI makes someone 30% more productive.

A company could eliminate people.

Or it could produce 30% more.

It could enter new markets. Improve customer service. Shorten development cycles. Make employees less exhausted. Allow small teams to compete with much larger organizations. Increase quality. Create products that previously were not economical to produce.

Technology does not choose the outcome.

Management does.

And perhaps one of the great strategic mistakes of this era was looking at extraordinary new human leverage and asking first:

Who can we fire?

instead of:

What can these people do now that they could not do before?

The Small Business Nobody Seems to Be Talking About

The biggest threat AI may pose to established business models might not be artificial superintelligence.

It might be adequate intelligence becoming cheap.

That is a very different threat.

If a $20 or $50 tool gives one independent creator access to capabilities previously costing thousands of dollars, the creator wins.

But somebody who previously charged thousands may lose.

If a five-person company becomes capable of competing with a 25-person company, something changes.

If small models become good enough for ordinary business tasks, not every company needs access to the largest, most expensive frontier system.

If models continue becoming cheaper to run, the economics supporting massive infrastructure investments could change dramatically.

That produces a fascinating possibility.

The AI financial bubble does not necessarily burst because AI fails.

It could burst because AI succeeds too efficiently.

Because useful intelligence becomes abundant.

Because smaller systems become sufficient.

Because competition drives prices down.

Because open systems improve.

Because businesses discover they do not need the most expensive model for most work.

Because the economic value migrates away from whoever owns the biggest model and toward millions of people using increasingly inexpensive tools.

That would be one hell of an irony.

The industry spends hundreds of billions building scarcity around machine intelligence.

Then technological progress makes machine intelligence cheap.

The snake eats its own tail.

So When Does the Dam Break?

Maybe it doesn't look like a crash.

Maybe Nvidia does not collapse tomorrow.

Maybe Anthropic has a successful IPO.

Maybe OpenAI becomes enormously profitable.

Maybe AI models become dramatically better.

Maybe AGI eventually arrives under some definition.

All of those things remain possible.

But there is another kind of bubble.

An expectation bubble.

A credibility bubble.

A political bubble.

A narrative bubble.

Those bubbles break differently.

They break when maintaining the story requires increasingly extraordinary predictions while the people hearing those predictions accumulate enough firsthand experience to start judging for themselves.

And millions of people can now judge AI for themselves.

They use it at work. At home. At school. In businesses. In music. In art. In software.

Sometimes it is remarkable.

Sometimes it is frustrating.

Sometimes it saves hours.

Sometimes it confidently produces nonsense.

That does not tell us everything happening inside an AI laboratory.

But it means the public is no longer completely dependent on Silicon Valley to explain what AI is.

That changes something.

The British Are Coming

There is an old kind of authority that comes from seeing something other people cannot see.

Someone rides into town.

The British are coming.

Everyone reacts because the messenger possesses information they do not.

But imagine the rider returns.

Again.

And again.

And again.

Each time the warning becomes bigger.

The army is closer.

The danger is greater.

The consequences are unimaginable.

Meanwhile, the townspeople begin getting telescopes.

Eventually they can look toward the horizon themselves.

That does not mean the army isn't there.

It means the messenger's claim is no longer enough.

Show us.

That is the moment I think artificial intelligence is approaching.

Not disbelief.

A higher burden of proof.

Something Stranger Is Beginning to Happen

This week provided one of the clearest signs yet that the political map around AI may be breaking apart.

Bernie Sanders.

Steve Bannon.

Labour voices.

Religious leaders.

Technologists.

Copyright advocates.

AI safety advocates.

Open-source developers.

Creators.

Workers.

People from political traditions that agree about almost nothing are beginning to ask overlapping questions about artificial intelligence. Axios described the unusual cross-ideological convergence around a “pro-human” AI push.

They do not agree on the answers.

Good.

They shouldn't.

Agreement is not the prerequisite for critical thinking.

But something significant happens when people who disagree about almost everything begin noticing the same concentrations of power.

They notice the money. The infrastructure. The political access. The lobbying. The employment promises. The regulatory proposals. The enormous claims. The small group of corporations increasingly positioned between ordinary people and one of the most consequential technologies of our lifetime.

That is not ideological agreement.

It is shared observation.

And shared observation is where dams begin cracking.

The Old Labels Are Becoming Useless

“Pro-AI.”

“Anti-AI.”

What do those labels even mean anymore?

I can believe AI is an extraordinary creative tool and oppose replacing workers recklessly with it.

Someone can despise generative art and still support open-source AI.

A copyright advocate can oppose unauthorized training practices without wanting artificial intelligence abolished.

A conservative can distrust government control and corporate control simultaneously.

A progressive can support AI regulation while worrying that expensive regulation protects the largest corporations.

A technologist can believe advanced systems pose genuine risks while rejecting claims about machine consciousness.

A creator can use AI every day and demand ownership rights.

None of those positions are contradictory.

They only look contradictory inside a debate deliberately flattened into two teams.

Maybe the next stage of this conversation begins when enough people refuse to join either one.

We Have More Information Than We Think

Here is perhaps the strangest part.

Most of the information required to investigate this system is already public.

Corporate earnings. Capital expenditures. Government contracts. Executive interviews. Lobbying records. Court filings. Research papers. IPO disclosures. Model evaluations. Employment statistics. Electricity requirements. Past predictions. Current predictions. Regulatory proposals. Political donations. Press releases.

The problem is not always secrecy.

Sometimes the problem is fragmentation.

One story appears in the financial press.

Another appears in a technology publication.

Another sits inside a research paper.

Another appears in a court filing.

Another becomes a five-minute television segment.

Another disappears into a 300-page government report.

To an ordinary person they look unrelated.

Put them beside one another and something different can emerge.

A pattern.

Not necessarily a conspiracy.

A system.

And Then AI Opened Pandora's Box

This may ultimately be the greatest irony of all.

The same technology being described as potentially impossible for humanity to control has made it significantly easier for ordinary humans to investigate powerful institutions.

One person can now analyze documents that once required a research staff.

Compare years of executive claims.

Summarize legislation.

Search court cases.

Translate international reporting.

Compare economic forecasts.

Analyze corporate statements.

Build timelines.

Find contradictions.

Follow links.

Check sources.

Ask another system to challenge the argument.

Then publish the result globally.

AI cannot determine truth for us.

Anyone who has seriously used these systems knows that.

They hallucinate.

They misinterpret.

They inherit errors.

They need supervision.

They need verification.

Which is almost poetic.

The technology itself demonstrates the argument:

Tools still need humans.

But tools can make humans extraordinarily capable.

And once millions of people have those tools, the information environment changes.

That is Pandora's box.

Not necessarily an artificial mind escaping the laboratory.

Maybe it is ordinary people gaining access to capabilities that once belonged mostly to institutions.

Good luck putting that back in the box.

But Truth Alone Isn't Enough

This is where I think many people become frustrated.

They find something.

They document it.

They post it.

Nothing happens.

Because information is not automatically power.

A fact needs context.

Context needs explanation.

Explanation needs distribution.

Distribution needs repetition.

Repetition without credibility becomes propaganda.

So credibility matters.

That means acknowledging when the other side has a point.

Correcting mistakes.

Separating evidence from inference.

Admitting uncertainty.

Not turning every coincidence into coordination.

Not pretending every corporate executive is corrupt.

Not pretending every safety researcher is lying.

Not pretending every critic is a Luddite.

That work is slower.

But it builds something outrage cannot.

Trust.

The Story Has to Travel

And one article will never be enough.

Neither will one political tribe.

The musician worried about copyright enters this story through ownership.

The warehouse worker enters through employment.

The programmer enters through open source.

The small-business owner enters through cost.

The parent enters through children.

The investor enters through capital expenditure.

The environmentalist enters through electricity and water.

The civil-liberties advocate enters through surveillance and government power.

The conservative may enter through centralized authority.

The progressive may enter through corporate concentration.

Different doors.

Same building.

The facts do not need to change.

The story has to become understandable from where people are standing.

That is how momentum begins.

One fact connects to another.

Technology connects to economics.

Economics connects to labour.

Labour connects to politics.

Politics connects to regulation.

Regulation connects to competition.

Competition connects to ownership.

Ownership connects to power.

Eventually twenty stories stop looking unrelated.

And people begin seeing the system.

Then What?

Maybe nothing dramatic happens.

Maybe there is no cinematic moment when the dam collapses.

Maybe it happens one crack at a time.

One failed prediction.

One disappointing financial result.

One cheaper model.

One regulatory proposal that reveals too much.

One whistleblower.

One court case.

One election.

One unexpected political alliance.

One independent creator demonstrating that what once required an organization can now be accomplished by an individual.

And eventually enough people start asking the same question.

Not: Is AI good?

Not: Is AI evil?

Not even: Will AGI arrive?

A much more uncomfortable question.

Who is this version of the AI future being built to benefit?

Because if artificial intelligence really is as powerful as its builders say, then ordinary people deserve more than access to the products.

They deserve a meaningful stake in what becomes possible.

That means competition.

Ownership.

Independent creation.

Open inquiry.

Reasonable regulation.

Human accountability.

And the ability to question extraordinary claims without being pushed into one ideological camp or another.

AI Never Needed to Become Human

This is where I keep coming back.

Maybe we have been staring at the wrong miracle.

AI never needed consciousness.

It never needed feelings.

It never needed a soul.

It never needed to become human.

It did not need to wake up.

It did not need to replace us.

It only needed to make us more capable.

That happened.

It is happening.

And its implications may ultimately be more profound than the science-fiction story we keep being sold.

Because once capability becomes cheaper, people who previously lacked access get to participate.

Creators create.

Small businesses compete.

Individuals build.

People learn faster.

Ideas become prototypes.

Prototypes become businesses.

A person who could never afford an agency can suddenly attempt the work.

A person who could never hire five specialists can begin with one subscription and a willingness to learn.

That does not eliminate expertise.

It changes who can reach it.

And perhaps this is the part of the AI revolution most worth defending.

Not a future where artificial intelligence becomes so powerful that a handful of companies must control it for everybody else's protection.

A future where powerful tools become cheap enough that millions more human beings can use them to build something of their own.

That is a very different revolution.

And it is already here.

So When Does the Dam Break?

I don't know.

Anyone telling you they know should probably make you suspicious.

But I know what happens before dams break.

Pressure builds.

Cracks connect.

Things that appeared separate begin revealing themselves as part of the same structure.

People who once ignored one another begin comparing notes.

Old alliances stop making sense.

New ones form around unexpected questions.

And eventually maintaining the existing structure requires more pressure than the structure can withstand.

Maybe we are nowhere near that point.

Maybe we are closer than we think.

But this time something is different.

Everybody has a telescope.

Everybody has access to more information.

And increasingly, everybody has tools capable of helping them connect it.

The AI industry opened that door itself.

It told us this technology would change everything.

Perhaps it will.

Just not entirely in the way they expected.

The dam does not break when everybody agrees.

It breaks when enough people who disagree start seeing the same cracks.

And once they see them, the real question will no longer be whether artificial intelligence can think for itself.

It will be whether we still can.

---

Jack Righteous / Gary Whittaker

Create What You Love | Love What You Create.

AI Made It Possible.

Sources & Further Reading

• Anthropic — When AI builds itself
• NBER — What Work Does Generative AI Do? (August 2026)
• NBER — Still Waters, Rapid Currents: Early Labor Market Transformation under Generative AI
• NBER — Shifting Work Patterns with Generative AI
• NBER — Generative AI at Work
• Axios — Jacob Coxon's Anthropic resignation and AI warning