The government, the economists, your colleagues, and your neighbors are not giving mass automation and its potential implications their attention. The world's most well-resourced companies are burning hundreds of billions of dollars trying to build AGI. If successful, they might unlock unprecedented abundance—a good thing. However, the current plan for AGI also takes away everyone's main lever of power: their ability to create value in the world.
How would people fare in such a world? You could trust the politicians and CEOs—and AIs—that end up in power. Maybe they'll be benevolent. But neither history nor incentives look kindly on the stability of this arrangement. The best world is one where people are not subjects, but players.
Wishful thinking dominates, motivated by a mix of denial and self-importance. Many of those who see this coming expect that they personally will be fine. Perhaps they have high-paying jobs at AI labs, or think that they have the uniquely human job that AIs can't take.
Others are ignoring the AI capabilities in front of them, writing manifestos about what AIs cannot do followed shortly thereafter by proof that they can. Rather than changing their beliefs, they ignore the evidence.
But among a third group—those who can see the wave and know it will subsume them—unbounded optimism has replaced critical thinking.
In a recent conversation, we raised the concept of the intelligence curse. We hadn’t fleshed it all out yet, but their response convinced us that we needed to. This person, a well-connected person in the AI space, agreed technological displacement was the most likely outcome of AGI, but believed that it would default to utopia.
“We won’t need jobs – we’ll be free to self-actualize. We’ll pursue meaningful goals and write poetry.”
You do not get to utopian poetry writing by having faith that someone else will figure it out. You are not praying to God, you are praying to men more ignorant than you.
Instead of turning to others for answers, take action yourself:
- If you are in governments, you should be forecasting AI capabilities and thinking through solutions to the intelligence curse.
- If you’re at a think tank, start turning out policies designed to get us ready for a post-AGI world.
- If you’re at an AI lab, critically examine your organizations’ incentives and help build better internal governance structures to overcome them.
- If you are young, get ambitious. The traditional prestige paths are closing anyways. Start companies trying to design tech that will keep humans economically relevant and spread abundance.
- If you’re a VC, fund projects and products that will keep humans in charge.
The direction of civilization is not fixed. History is yours to write. Get to work.