Epistemic status: All of the western canon must eventually be re-invented in a LessWrong post, so today we are re-inventing modernism.
In my post yesterday, I said:
Maybe the most important way ambitious, smart, and wise people leave the world worse off than they found it is by seeing correctly how some part of the world is broken and unifying various powers under a banner to fix that problem — only for the thing they have built to slip from their grasp and, in its collapse, destroy much more than anything previously could have.
I think many people very reasonably understood me to be giving a general warning against centralization and power-accumulation. While that is where some of my thoughts while writing the post went to, I would like to now expand on its antithesis, both for my own benefit, and for the benefit of the reader who might have been left confused after yesterday's post.
The other day I was arguing with Eliezer about a bunch of related thoughts and feelings. In that context, he said to me:
From my perspective, my whole life has been, when you raise the banner to oppose the apocalypse, crazy people gather around making things worse; and the alternative to this has always been apocalypse entirely unopposed.
So yeah, I would like something else than apocalypse entirely unopposed.
Do you know what really grinds my gears? The reification of innocence as the ideal of moral virtue.
As I will probably never stop quoting at least once a month, Ozy Brennan summarizes it best:
Many people who struggle with excessive guilt subconsciously have goals that look like this:
- I don’t want to make anyone mad.
- I don’t want to hurt anyone.
- I want to take up less space.
- I want to need fewer things.
- I don’t want my body to have needs.
- I don’t want to be a burden.
- I don’t want to fail.
- I don’t want to make mistakes.
- I don’t want to break the rules.
- I don’t want people to laugh at me.
- I want to be convenient.
- I don’t want to have upsetting emotions.
- I want to stop having feelings.
These are what I call the life goals of dead people, because what they all have in common is that the best possible person to achieve them is a corpse.
Corpses don’t need anything, not even to breathe. Corpses don’t hurt anyone or anger people or fail or make mistakes or break rules. Corpses don’t have feelings, and therefore can’t possibly have feelings that are inappropriate or annoying. Once funeral arrangements have been made, corpses rot peacefully without burdening anyone.
Compare with some other goals:
- I want to write a great novel.
- I want to be a good parent to my kids.
- I want to help people.
- I want to get a raise.
- I want to learn linear algebra.
- I want to watch every superhero movie ever filmed.
- I don’t want to die of cancer.
- I don’t want the world to be destroyed in a nuclear conflagration.
- I don’t want my cat to be stuck in this burning building! AAAAA! GET HER OUT OF THERE
All of these are goals that dead people are noticeably bad at. Robert Jordan aside, corpses very rarely write fiction. Their mathematical skills are subpar and, as parents, they tend to be lacking. Their best strategy for not dying of cancer is having already died of something else.
Let's remember that we are not here to be pure. We are here to build things. To live, to multiply, to party hard, regret our choices, and do it all again anyways. To reshape the cosmos in our image because most of it appears to be made out of mostly inert plasma clouds, and you know what, inert plasma clouds really suck compared to basically anything else.
So in evaluating any appeal to not conquer the cosmos, to not spread the values of the good far and wide, we have to remember that dead people suck, and if whatever principles we arrive at suggest that it's better to be dead than alive, then we almost certainly went wrong somewhere and should take it from the top.
Attached is a letter from a ‘cry baby’ scientist, which I wish you would read and discuss with me. Mr. Byrnes placed him on the Commission to attend the Atomic Bomb tests. He came in my office some five or six months ago and spent most of his time ringing his hands and telling me they had blood on them because of the discovery of atomic energy
This is how President Truman recalled meeting Oppenheimer complaining about his role in the development of nuclear weapons. I have kept going back and forth over the years over who was right and who was wrong in this situation.
Oppenheimer built the bomb, only for his invention to far escape his grasp, resulting in him spending the last years of his life trying to avert a nuclear arms race with the Soviet Union. But Truman was more the man in the arena than almost anyone else. Truman could not quit, and hand-wringing did not absolve him from responsibility. It's easy to imagine Truman seeing in Oppenheimer a man too afraid to actually face the responsibility for his actions, and to do the best with the hand they were dealt.
Ok, fine. I'll say it directly. I am extremely glad the west colonized North America. The American experiment was one of the greatest successes in history, and god was it far from perfect. Despite it all, despite the Trail of Tears, despite smallpox ravaging the land, despite the conquistadors and the looting and the rapes — yes, all of that, and still it was worth it. America is worth it. Democracy is worth it.
If you were faced with the horrors of the American colonization, would you have chosen to keep going? Or would you have wrung your hands, declared the American experiment a failure, concluded that maybe man was never supposed to wield this power, and retired to the countryside, in denial that other men and women were doing the dirty work for you?
This doesn't mean that you should have rationalized that all of what was going on was just. It doesn't even mean that the marginal effort was not best spent advocating for settlers and conquistadors to be held to account for their atrocities, or to scramble desperately to somehow prevent plagues from ravaging the land. But if you would have stopped it all when you saw the horrors, or sneered from your ivory tower at the frontier settlers, then I do think you would have been on the wrong side of history.
And this doesn't mean that you, having seen the horrors of it all, would have needed to keep going. A human's soul can only bear so much, and in as much as being involved with the horrors was a price to be paid by someone, there were enough souls to go around to spread that burden out. But do not mistake your trauma and exhaustion for wisdom.
“I wish it need not have happened in my time,” said Frodo.
“lmao” said Gandalf, “well it has.”
Why defend colonialism of all things? Why go to bat for maybe the biggest boogeyman of the 21st century?
Hearing "do not conquer what you cannot defend" is easy. Nobody can ever blame you for not conquering things. But in that principle lies its dual.
"May goodness conquer all that it can defend"
A principle should be defended against its strongest counterarguments, and I find the colonization of the Americas one of the most interesting examples of what this might look like. And how far it might be worth it to go.
And maybe you put the boundary somewhere else. It is very plausible to me that it would have been better to stop those early settlers. While it seems hard to imagine property rights being straightforwardly respected, and hard to see how (given the technologies at the time) we could have prevented diseases from ravaging the land, clearly we could have done much much better. And maybe if you had stopped the first colonizers, and had thrown your body on the gears, we would have had trade and immigration and a gradual mixing of the western and native way of life, and maybe this would have been better.
I don't currently think this, but it doesn't seem implausible to me.
I have fought in the arena, and I've felt the blood on my hands, and seen the madness in my allies' eyes and I thought that I had conquered much more than I could defend.
And I've stood in that arena, fighting for what is good and right and just, and I saw my allies abandon their posts as they could not face the choices they had to make. And I grabbed them, and I shook them, and I stared into their eyes and said "despite the damage, this fight is worth fighting, do you not dare to leave us now".
So I say, do not conquer what you cannot defend. But let goodness conquer all that it can defend.
And if given the right support, goodness can defend quite a lot. Long-term governance is possible. America is about to be 250 years old. And all it required was a bunch of young highly disagreeable people winning a revolutionary war and thinking really hard about how to not let it sway from justice. And god, was it ugly. Ugly from the very beginning. And god was it beautiful, all the way until now.
And other times, goodness falls almost immediately. Sometimes you write ambitious bylaws, and "merge-and-assist clauses", and fill your board with young disagreeable people thinking at least somewhat hard[1] about how to not let it sway from justice, and (as far as I can tell) it falls apart almost immediately as it comes in contact with reality.
And the governance problems of the future will not be the governance problems of the past. I could analyze here all the parallels and disanalogies between the founding of the US and the founding of OpenAI, but OpenAI governance does not need to last 250 years, and the Founding Fathers did not need to figure out how to navigate a world drastically transformed by technology and the handoff of humanity's future to our successors.
My guess is that AI will guarantee your own obsolescence soon enough that you won't have to worry about your own retirement, and succession is not the problem to solve. You will have to worry about your own corruption and your incentives and adversaries much more powerful than any adversary in history.
Much has been written, and much will be written about what keeps institutions and groups on track. I hope to write myself more on this in the future.
But here, all I dare and have the time to say, is that success is possible, and perfection is not the standard. That great mistakes and shaky foundations can be fixed along the way. That goodness can defend much. And if you can conquer what the good can defend, you should do so.
- ^
Relatedly, I do actually think that one of the single biggest mistakes that our broader ecosystem made on this topic was for the OpenAI board members to not be full-time board members. Like, man, it does really seem to me that people underestimated the trickiness of stuff like this, and did not budget resources proportional to its difficulty.
If you want an actionable takeaway from this post, I would recommend making sure that Anthropic Long Term Benefit Trust members are full-time and in an actual position to do something if they notice bad things going on.
Discuss