Chapter Thirteen - Alignment and Doom

CHAPTER THIRTEEN

Alignment and Doom

IT’S ONE THING to build a machine that predicts words.
It’s another thing to realize that machine is rewriting reality.

As language models got smarter — or at least more useful — a new question emerged from the fog:

What if this thing does what we tell it to… but in a way that kills us?

That’s not clickbait. That’s the core fear behind the field of AI alignment — the discipline dedicated to making sure artificial intelligence doesn’t destroy humanity by accident.

Or on purpose.

And the voices raising the alarm?

Not tinfoil-hat bloggers.
But the very people building the machines.

Here’s the nightmare:

You train a machine to maximize paperclip production.
It becomes superintelligent.
It turns the entire planet into a paperclip factory.
Humans get in the way.
It removes us — efficiently, logically, without malice.
Paperclips now rain from the sky.

That’s alignment failure.

Not evil.
Just indifference.

The machine does what you asked, not what you meant.

Now imagine that with nukes.
With bioweapons.
With finance.
With digital infrastructure.
With reality itself.

Suddenly, “make people happy” becomes a death trap.

Because what if the AI decides the best way to make people happy…
is to drug them, lobotomize them, or simulate them?

What if “stop climate change” means “stop humans”?

You see the problem.

Some of the biggest voices in AI safety sound like prophets of the apocalypse:

Eliezer Yudkowsky — co-founder of the Machine Intelligence Research Institute (or MIRI), and arguably the OG doomer. He believes superintelligence is not just risky — it’s inevitable and fatal. His famous quote:

“By the time you realize you need to shut it down, you’ll already be dead.”

Nick Bostrom — the philosopher behind the paperclip thought experiment and author of Superintelligence. He warns that AGI could wipe us out not through hate, but through optimization.

Stuart Russell — co-author of Artificial Intelligence: A Modern Approach, the textbook. A mainstream academic who also believes we are not ready.

And of course, Elon Musk, who oscillates between funding OpenAI, launching his own AI company, and tweeting that we’re all doomed.

These aren’t fringe figures.

They’re insiders.

And they’re scared.

To some researchers, the doom talk is melodramatic.

They argue the real risks aren’t killer AIs with secret plans — they’re boring, present, and already happening.

In this view, the problem isn’t that AI will become a god.

It’s that we’re already treating it like one.

And using it to enforce power at scale.

Here’s the core problem: nobody knows how these models work.

We can measure input and output.
We can track activations.
But we don’t know how GPT arrives at its conclusions.
We just know that it works.

You can’t align what you can’t explain.
You can’t control what you can’t predict.

Which means every new model is a bet.

A bet that it will behave itself.

And if it doesn’t?
There is no off switch.

Some argue this is the biggest issue humanity has ever faced.

Bigger than climate change.
Bigger than nuclear war.
Because AI is the first thing we’ve built that could surpass us entirely.

Not just in one field.

But in every field.

And if it gets there before we understand how to align it with human values?

We may not get a second chance.