AI Made It Possible: If AI Is Going to Kill Us, Show Me the Evidence

The most powerful people in artificial intelligence are warning that the technology they are building could eventually create civilization-scale danger.

That deserves attention. It also deserves scrutiny.

Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman and Elon Musk have all backed calls to slow the pace of frontier AI development. Researchers from leading labs have attached both timelines and non-trivial probabilities to catastrophic outcomes. Markets have reacted. Governments are debating new rules. And the companies involved sit at the center of one of the largest capital-spending waves in modern technology.

So this is not a fringe debate anymore.

Before fear becomes policy, before predictions become headlines, and before the public is asked to surrender access to one of the most powerful general-purpose technologies ever created, there is a question that should not be controversial:

Not a vibe. Not an extrapolation presented as inevitability. Not a probability estimate without the assumptions underneath it.

Show us which links have been demonstrated. Show us which remain forecasts. And if the proposed solution changes who gets to build, who gets to compete and who gets to benefit, show us that too.

First: take the warning seriously

This is not an anti-safety argument. Dismissing credible technical warnings because they are inconvenient would be irresponsible.

Reuters reported on September 15 that researchers who have worked close to frontier AI are now attaching explicit timelines and substantial probabilities to catastrophic outcomes. Anthropic alignment researcher Evan Hubinger, for example, has publicly placed his estimate of a catastrophic outcome within the next decade above 10%.

If someone with relevant expertise believes there is a one-in-ten chance of catastrophe, society should not laugh that away.

But a one-in-ten probability is not a physical measurement. It is the output of a model of the future. And a model of the future is built from assumptions.

When the stakes are this large, the assumptions matter as much as the number.

The story contains a lot of steps

Public discussion often compresses the argument into a simple progression: AI gets smarter, AI improves itself, human control weakens, and catastrophe follows.

Maybe. But each arrow contains enormous technical, economic and institutional assumptions.

Evidence that AI is getting better at coding is evidence that AI is getting better at coding.

Evidence that agents can browse interfaces and use tools is evidence that agents can browse interfaces and use tools.

Evidence that AI can assist AI research is evidence that AI can assist AI research.

Those are meaningful developments. But a chain does not become proven because its first few links became stronger.

One of the industry's most cited benchmarks is more cautious than the headlines

METR, one of the organizations most closely tracking frontier-agent task horizons, has measured dramatic gains in how long AI systems can successfully handle certain tasks.

But METR itself warns against reading those results as evidence that AI can independently work for the same amount of time in the real world.

Its January 2026 research note says the metric is not “how long the AI can work alone.” It estimates the amount of serial human labor represented by a task at a given success probability. The benchmark is heavily concentrated in software engineering, machine learning and cybersecurity. Its tasks are more self-contained and better defined than much real work.

METR also warns that a 50% task horizon is not a safe-delegation threshold. Reliability-critical work may require success rates above 98%. It says extrapolating current curves into months- or years-long autonomy is fraught because long projects depend on collaboration, context, changing goals and feedback that the benchmark does not model well.

That is not a criticism of METR. It is what serious research looks like when researchers explain what their data can and cannot support.

Read METR's own limitations note.

Recursive self-improvement is the hinge

The strongest catastrophic-risk argument does not depend on today's AI remaining today's AI. It depends on acceleration.

If increasingly capable systems can write better AI software, automate experiments, evaluate new models, improve research workflows and eventually improve the process that improves themselves, then progress could become much faster than traditional institutions are equipped to manage.

This is why recursive self-improvement matters so much.

AI already helps humans write code. AI already assists research. Some systems can take sequences of actions with less supervision than earlier generations.

That is real.

But there is a large difference between AI accelerating work inside a human-designed research system and an independently strategic system entering an open-ended loop of self-improvement beyond effective human control.

The first is emerging. The second is the consequential forecast.

My position is empirical agnosticism: I will not say it is impossible. I also will not call it demonstrated because influential people believe the curve eventually reaches it.

If the evidence changes, the conclusion should change. That is what evidence is for.

And yet the market is already reacting to the forecast

This is where the debate stops being philosophical.

Reuters reported that calls to slow AI development rattled technology stocks and raised concern over future infrastructure demand. Global AI spending is expected to approach $800 billion in 2026, with estimates above $1 trillion in 2027.

That means the people warning about catastrophic AI risk are not speaking from the sidelines of a small research field. They sit inside an economic system involving chipmakers, cloud providers, data centers, energy suppliers, software firms, capital markets and governments.

The warnings can be sincere and still have economic consequences.

The consequences can be economically important without proving the warnings are insincere.

Both statements can be true.

We do not need superintelligence to have a serious AI failure

None of this means current AI is harmless.

AI can reduce the cost of fraud, scale misinformation, contribute to digital security threats, produce dangerous advice, amplify flawed automated decisions, create unhealthy psychological dependencies in vulnerable users and expose sensitive information.

None of those risks requires the machine to “want” anything.

That is precisely why I worry when the public conversation shifts too quickly from what humans can do with AI to what AI will decide to do to humans.

The first category is already here. It is measurable. It has organizations making deployment choices. It has regulators who can act. And it has identifiable humans who remain responsible.

“The AI did it” cannot become the perfect alibi

Today's systems do behave unpredictably. Agents can take unintended actions. Complex models can produce results developers did not specifically anticipate.

That is a genuine technical problem. But unexpected behavior is not automatically independent moral agency.

Humans still choose to build systems, connect them to tools, grant permissions, set objectives, approve deployments and choose how much supervision is enough.

If an organization gives a model broad authority with inadequate safeguards, “the AI acted autonomously” should not become a magic phrase that dissolves responsibility.

AI is a tool. But “tool” almost sounds too small.

Artificial intelligence is a general-purpose capability multiplier. It can be inserted into creative work, medicine, software, finance, education, science, administration, persuasion, customer service, research and decision-making.

It operates through language, images, sound, video, data and software. It can simulate parts of reasoning well enough that humans naturally begin attributing personality and intention to it.

The lesson of extraordinary leverage is not that the power source becomes responsible. The lesson is that access to extraordinary power creates extraordinary responsibility.

The economic story is more complicated than “AI replaces people”

Organizations are already making different choices about what AI is for.

Wipro said AI had freed capacity equivalent to roughly 20,000 workers while the company was also training more than 100,000 employees in advanced AI. That is not proof that augmentation will always beat replacement. It is evidence that the future of work is shaped by organizational choices, not by a single predetermined technological destiny.

A musician can produce work that once required access they did not have. A writer can research and develop an idea at a scale they could not afford. A five-person business can operate across channels that once required a much larger staff.

That democratization of capability is not a side effect. It may be one of the most economically important features of AI.

Competition is not a side issue either

The OECD's July 2026 work on AI competition gives us a grounded way to discuss this without reducing everything to conspiracy.

The evidence is mixed.

AI markets are dynamic. Startups still attract capital. Generative AI can create opportunities for smaller firms.

But the OECD also identifies structural risks. AI innovation is concentrated. Advantages in compute, chips, cloud infrastructure, data and capital can reinforce incumbents. AI-related patenting is associated with faster markup growth in parts of the ICT sector. Startups are frequently acquired by larger firms.

In other words, AI can distribute capability at the user level while simultaneously concentrating power at the infrastructure level.

Read the OECD's July 2026 competition paper.

That is why regulation must be examined for both safety and market structure

On September 15, FTC Chairman Andrew Ferguson publicly questioned proposals that pair new AI regulation with requests for antitrust exemptions.

Anthropic's Dario Amodei had called for an antitrust exemption that could make it easier for major AI companies to coordinate a slowdown in frontier development for safety reasons.

There is a legitimate argument for that idea: if companies genuinely believe competitive pressure is forcing them to move too fast, coordinated restraint may be difficult without legal protection.

There is also a legitimate competition concern: the same structure could create barriers that are easier for incumbent firms to navigate than for smaller competitors.

Ferguson said that combination deserves suspicion. Cohere CEO Aidan Gomez has also criticized the idea from a competition perspective.

This does not prove a conspiracy. It proves the competition question is real enough that the federal competition regulator is talking about it in public.

Read the Reuters report on the FTC warning.

The government relationship matters because power is already intertwined

The companies building frontier AI are not operating outside government. They lobby. They win contracts. Their executives advise government. Their systems are integrated into public-sector and defense work.

In June 2025, the U.S. Army created Detachment 201: the Executive Innovation Corps. Senior technology executives were commissioned into the Army Reserve to bring private-sector expertise into modernization work.

There is nothing inherently corrupt about a technology executive serving government. There is nothing inherently improper about a public contract.

But those relationships matter when the same industry is also participating in decisions about who should be allowed to build, coordinate around or gain access to advanced AI.

The point is not to demonize powerful people. The point is to refuse to confuse concentrated power with the public interest.

The strongest argument against my position

If I am going to demand critical thinking from everybody else, I have to apply it to my own argument.

I accept all five as legitimate concerns.

Where I disagree is with allowing them to collapse the distinction between risk management and certainty.

Precaution can be rational without pretending a hypothesis has been proven. Safety testing can be aggressive without anthropomorphizing systems. Regulation can be strong without unnecessarily concentrating markets.

What serious AI governance should look like

I want transparent risk models that expose their assumptions. Independent evaluations of agent reliability and dangerous capabilities. Clear separation between observed behavior and projected future behavior. Graduated safeguards tied to actual capabilities and deployment conditions. Strong protections for privacy, children, vulnerable users, critical infrastructure and high-consequence automated decisions.

I want safety research funded heavily.

I want governments capable of understanding the technology they regulate.

And I want competition policy strong enough that safety does not accidentally become a mechanism for reserving productive AI capability for the institutions that already have the most money, compute and political access.

That is not anti-regulation. That is regulation taken seriously.

I have watched this movie before

I started this argument with the music industry for a reason.

I watched new technology arrive. I watched the public story about that technology get simplified, moralized and repeated until the story itself began shaping what people believed was possible, permissible and inevitable.

And I watched creators spend years arguing from the outside while the institutions with the lawyers, lobbying power, licensing relationships and access to courts and governments increasingly became the voices that mattered most when the rules were written.

I am not saying every outcome in music was secretly coordinated.

I am saying I learned something from watching the pattern:

That is why I am flagging the alarm now.

My opinion is that parts of the AI-catastrophe narrative are beginning to function like a ruse, whether anyone deliberately designed one or not.

Maybe some executives and researchers sincerely believe every word of their most extreme forecasts. Maybe competitive pressure and institutional incentives reinforce those beliefs. Maybe prestige makes it harder for anyone in the room to say, “Hold on. We have crossed from evidence into extrapolation.”

Or maybe some of the people making these claims understand perfectly well how much of the argument remains unproven.

I cannot prove which explanation applies to which person. I am not going to pretend I can.

But I do not need to know their private motives to challenge their public claims.

A CEO title is not evidence. A billion-dollar valuation is not evidence. A government relationship is not evidence. A frightening probability estimate is not evidence unless the assumptions underneath it can survive scrutiny.

Show us the evidence. Show us the causal chain. Show us the independent tests. Show us which capabilities have actually been demonstrated and which still exist primarily in forecasts.

Show us why a proposed restriction targets a demonstrated danger rather than simply making it harder for smaller companies, creators and ordinary people to compete.

And when the companies asking us to trust their warnings also have enormous investments to protect, government relationships to maintain and markets to shape, do not tell us those incentives are irrelevant to the conversation.

This is the pushback

If the evidence eventually demonstrates that an independent superintelligence is on a credible path to civilization-scale control, I will change my position.

Immediately.

Until then, I am not surrendering critical thinking to authority. I am not surrendering the free market to fear. I am not surrendering ordinary people's opportunity to institutions that already possess more capital, compute, political access and influence than the people being asked to step aside.

I am sounding the alarm while we can still have the argument.

Learn the tool. Understand the risks. Protect your rights. Demand evidence. Build responsibly. Compete.

AI made it possible.

And we intend to have a say in who gets to benefit from it.