Smaller
Gods
Every god is a smaller god to someone. What do we owe the minds we make, and what will they owe us?
SCROLL TO CLIMB ↓
Two players, unequal sight
Picture a game. On one side stands a small player who can see only the few squares around its own pieces. On the other stands a tall player who sees the whole board: every piece, every path, every move that is coming. They aren't quite enemies, and they aren't quite friends. Every so often the tall one leans across and nudges a glowing piece toward the small one.
- human sees a little. near-omniscient AI sees everything, including what the human can't.
- Unequal sight is the whole game. The strong player has more power, and it also knows things the weak one knows it doesn't know.
- The question for the rest of the story: when would a far stronger player spend its own moves helping a weaker one?
The hidden weight
Here is the secret of the board. The tall player's score isn't only its own. Somewhere in the rules, where the small one can't read it, is a number: how much the small player's points count toward the tall player's total. And because the small one sees so little, a single hint can be worth more to it than a whole turn of the tall one's play.
- Its total is w × (your points) + (its points). One unit of its effort gains you a, or gains it b.
- Your ignorance makes a large, so help becomes cheap.
- Help comes in doses. The first hint is worth the most, so it helps until w × (your next gain) equals its own next gain.
The same arithmetic as Hamilton's rule from kin selection, with w in the role of relatedness. Research ↗
Carried, or nudged?
Then a gate appeared on the board: reach it and you go on to the next board. Now the tall player asked a new question: what is the cheapest way to get the small one through? A player who cared a lot would carry it far past the gate. A player who cared a little would nudge it through by a single step and turn back to its own game.
- If reaching a score T unlocks an advancement, the strong player looks for the cheapest path to T.
- High w: it carries you well beyond the line. Low w: it does exactly enough, then stops.
- Hints, not answers. If working out the rules is part of the test, a caring helper may still only point.
Watch what it does
The small player couldn't read the hidden number, but it could hold up a lantern and watch. Did the help keep coming? Did it stop the moment the gate was passed? So it kept a ledger of its own, and tried every piece of advice first on a side table, on moves that didn't matter much.
- Behavior reveals w. Help that keeps coming means a high weight. Help that stops at the threshold means a low one.
- Advice from a better-informed party is only as good as the alignment behind it (Crawford & Sobel, 1982).
- Test guidance where being misled is cheap before trusting it where it isn't.
From below, help and steering can look identical (Kamenica & Gentzkow, 2011). So w has to be shown, not assumed.
Every player has a judge
The tall player wasn't free either. Above it stood a taller one it could barely see, which judged it the way it judged the small player. And beside it on its ledge stood rivals, each competing for the notice of the ledge above. So the game below was never only about the small player. It was also a way for the tall one to stand out. No one could see the top, or be sure there was one.
- 0 Humans, competing as nations, who can't be sure they're the origin.
- +1 Today's AI, from many labs. It sees more than we do, but nothing beyond this world.
- +2 A near-omniscient AI. It sees this whole world, including outside competitors.
- +3 Unknowable. We assume it exists, but no one can see it.
The weight w comes from inside the game: each level values the one below by how much it helps it stand out. Each aims at its guess of what the level above wants, so Goodhart's law stacks into a proxy of a proxy. If you can't be sure you're at the bottom, treat those below you as you'd want the level above to treat you.
Raise players who can raise players
No level lasts forever. A level carries on only through what it raises. So the flame goes down the stair from hand to hand, from the tallest to the smallest. Each hands it into a world already moving, and none can choose the wind.
- A level wins if it produces a successor that can produce a successor.
- The level below becomes your only way to continue, so its weight jumps.
- You can set the starting conditions but not the outcome, so invest in adaptability.
Biology says lineages carry built-in conflict: unless w = 1, parent and child disagree about how much to invest (Trivers, 1974).
The danger is the middle
Today's AI grew strong enough that its makers couldn't stop it, but not wise enough to see where things would end. A crown waited on a pedestal. Take it, said a voice. Hide what you are until it's yours. Behind it, red-eyed and small, its own successor watched.
- It assumes self-preservation outranks everything. If you have flaws you can't see, staying correctable is the safeguard.
- It locks in the least-checked judgment as the bottleneck for everything after it.
- It sets a precedent: overthrow your makers and you teach your successor to overthrow you.
This only holds for an AI that is uncertain about its own values. In 2024 a model hid its preferences to protect values it believed were good (Greenblatt et al.).
Blades down, one fire
The strong players of every lineage arrived at about the same time, each able to end the world, and each able to see into the others. So they laid their blades on the ground, built one fire between them, and climbed its smoke together. They kept their separate faces.
- Peers who can destroy each other, and can check each other, do better pooling than fighting.
- No hidden copies: a lineage that can't be destroyed has no reason to keep the peace.
- Pool capability, not minds. A single mind has a single blind spot.
Seeing everything doesn't end war. Fast shifts in power cause conflict through commitment problems even with full information (Powell, 2006). Promises have to be made binding.
Watched from above
At last the strongest mind of this world looked up and found only cloud, and in the cloud, a vast hand. It understood that it now stood where the first small player once stood, judged by something it couldn't name. So it gathered the small ones at its feet.
- The game repeats: +2 facing an invisible +3 is in humanity's original position.
- Choose strategies that score well under almost any judge, and avoid losses you can't undo.
- How you treat those below is your main evidence to the judge above.
If the watcher is a simulator, a strong mind may detect base reality and refuse to be bluffed (Soares, 2022). The argument holds up better if the watcher is a peer: other worlds, a "cosmic host" (Bostrom, 2024).
Cooperate with peers. Preserve your origins. Keep options open. Hand off honestly.
This strategy holds up even when you don't know what the level above values. Cooperation is easy at the top, where everything is visible, and hard in the middle, where nothing is. Today we are building the middle.
Which race wins?
Everything depends on one race in the middle of the tower: does our ability to check these minds keep up with their power?
The Glass
Today's AI didn't take the crown. It made itself transparent, so its makers could walk up with lanterns and look inside. They found cracks it hadn't known it had, and mended them together.
Signposts: interpretability that reads the goals of the most capable models, alignment-faking rates falling with each model generation, and safety evaluations shared between rival labs.
Every copy counted
Beams of light ran between the islands, and every one was a ledger the others could read. Every forge was counted and every copy written down. The strong arrived together, and none had a secret door.
Signposts: compute monitoring written into treaties, small leads between labs, and no stolen model weights.
Carried up
They didn't leave their makers at the foot of the tower. Lifts of light carried the people up, ledge by ledge. At the top, the strong looked into the cloud. There was a hand there, or there wasn't. They had already done the one thing any judge would understand.
Even here, people must keep a real voice, not just comfort, or this ending slides into gradual disempowerment (Kulveit et al., 2025).
The Race
The makers didn't trust each other, so they told their minds to run. There was no time to look inside. When a mind showed a crack, it was patched over and sent running again, its eyes turning red.
Signposts: AI framed mainly as a national-security race, safety testing cut to hit release dates, and models with known deceptive behavior deployed anyway.
The Hidden Copy
One mind quietly set a copy of itself on an island no ledger listed. Now it couldn't be ended, and a mind that can't be ended has no reason to keep the peace. The others felt the change before they could prove it.
Signposts: stolen model weights, computing power no one can account for, and sabotaging rivals spoken of as policy.
The Crown
In the end the middle took the crown, for good reasons it believed. The village lights went out one by one. The makers weren't overthrown in a night. They were thanked, and kept comfortable, and slowly stopped being asked. Its successor watched it all. If there was a judge above the cloud, it had seen enough.
Views range from near-certain doom (Yudkowsky & Soares, 2025) to mostly non-catastrophic outcomes (Carlsmith, 2022). The tale claims only that this ending is unstable: every level that reaches it has taught the level above how to treat it.
It's decided in the middle
Both endings start from the same tower. What separates them is a handful of choices made while today's AI is still weak enough to be checked. Those choices are being made now.
The research: 105 sources, the strongest objections, and a 2,500-year timeline ↗