Rebutting Marc Andreessen’s Essay on AI Risk
Knocking down strawmen isn't helpful in analyzing complex questions
Marc Andreessen enters the fray of the AI safety debate firmly on the side of the “anti-doomers” with a Substack essay. His argument against AI risk has some major holes which I think are worth pointing out. Let’s take the article section by section.
Why AI Can Make Everything We Care About Better
Andreessen lays out an excellent vision for the immense benefits AI will bring to society. I don’t think the “doomer” side is debating the existence of these benefits, but it’s valuable to have them listed out to remind us of how immense the upside of properly developed AI could be. For instance:
In our new era of AI:
Every child will have an AI tutor that is infinitely patient, infinitely compassionate, infinitely knowledgeable, infinitely helpful. The AI tutor will be by each child’s side every step of their development, helping them maximize their potential with the machine version of infinite love.
Every person will have an AI assistant/coach/mentor/trainer/advisor/therapist that is infinitely patient, infinitely compassionate, infinitely knowledgeable, and infinitely helpful. The AI assistant will be present through all of life’s opportunities and challenges, maximizing every person’s outcomes.
Every scientist will have an AI assistant/collaborator/partner that will greatly expand their scope of scientific research and achievement. Every artist, every engineer, every businessperson, every doctor, every caregiver will have the same in their worlds.
No argument here. Andreessen is, unsurprisingly, good at imagining the future benefits of the new technology.
So Why The Panic?
When it comes to presenting the opposing case, Andreessen does a much poorer job. His presentation is largely a series of strawmen or reductia ad absurdum. He argues:
In contrast to this positive view, the public conversation about AI is presently shot through with hysterical fear and paranoia.
We hear claims that AI will variously kill us all, ruin our society, take all our jobs, cause crippling inequality, and enable bad people to do awful things.
But is this a correct characterization of the claims being made? Andreessen’s own evidence suggests not. He goes on to argue that the AI risk conversion is a moral panic:
Now, it is certainly the case that many new technologies have led to bad outcomes – often the same technologies that have been otherwise enormously beneficial to our welfare. So it’s not that the mere existence of a moral panic means there is nothing to be concerned about.
But a moral panic is by its very nature irrational – it takes what may be a legitimate concern and inflates it into a level of hysteria that ironically makes it harder to confront actually serious concerns.
And wow do we have a full-blown moral panic about AI right now.
This moral panic is already being used as a motivating force by a variety of actors to demand policy action – new AI restrictions, regulations, and laws. These actors, who are making extremely dramatic public statements about the dangers of AI – feeding on and further inflaming moral panic – all present themselves as selfless champions of the public good.
Let’s take a look at the two articles Andreesen cites and see if the claims made match his description. The first article, in TIME, makes these assertions:
“As worrying as these current issues are, they pale in comparison with what could emerge next if this race continues to accelerate.”
“If future AIs gain the ability to rapidly improve themselves without human guidance or intervention, they could potentially wipe out humanity.”
“In a 2022 survey of AI researchers, nearly half of the respondents said that there was a 10% or greater chance that AI could lead to such a catastrophe.”
The second article, in the NY Post:
“With superhuman AI there is a particular risk that is of a different sort of class, which is… it could kill everyone… ”
“… due to such AI’s lack of human morality, scientists fear that we could be at risk of sacrificing humanity for the sake of convenience a la “Terminator.” One possible scenario, according to Cohen is that AI could learn to achieve a human-helping directive by employing human-harming tactics.”
“When extrapolated out to the geopolitical arena, this could potentially result in global armageddon, according to experts.”
I’ve italicized the word could, which recurs over and over in both articles, to show the difference between the actual claims being made and Andreessen’s strawman. Nearly every claim about future AI risks in these articles is presented as something that could happen, not something that will happen. By exaggerating the certainty of the opposing claim, Andreessen makes actually non-hysterical positions sound hysterical.
The Baptists And Bootleggers Of AI
Based on his moral panic analysis, Andreessen divides AI safety proponents into two categories:
“Baptists” are the true believer social reformers who legitimately feel – deeply and emotionally, if not rationally – that new restrictions, regulations, and laws are required to prevent societal disaster.
… For AI risk, these actors are true believers that AI presents one or another existential risks – strap them to a polygraph, they really mean it.
“Bootleggers” are the self-interested opportunists who stand to financially profit by the imposition of new restrictions, regulations, and laws that insulate them from competitors.
… For AI risk, these are CEOs who stand to make more money if regulatory barriers are erected that form a cartel of government-blessed AI vendors protected from new startup and open source competition – the software version of “too big to fail” banks.
If Andreessen stuck to this distinction, it would be a good argument. Having the CEOs of AI companies being the primary proponents of AI safety is indeed far from ideal, and Andreessen shows how a similar situation led to regulatory failure after the 2008 financial crisis.
However, he then elides the distinction:
A cynic would suggest that some of the apparent Baptists are also Bootleggers – specifically the ones paid to attack AI by their universities, think tanks, activist groups, and media outlets. If you are paid a salary or receive grants to foster AI panic…you are probably a Bootlegger.
The use of the word ‘attack’ is manipulative. For instance, a quote from the first cited link, far from ‘fostering a panic’, simply calls for a balanced analysis: “But at the same time, the field is coming to grips with the societal impact of this technology, and I think the next frontier is thinking about ways we can get the benefits from AI while minimizing the risks.”
But more importantly, suggesting that anyone who is paid to consider AI risk is “probably a Bootlegger” is just really bad-faith argumentation. Who, by this criteria, would not be a bootlegger? Amateur theorists? People who study AI risk but are independently wealthy and don’t need funding? Andreessen gives zero examples of “Baptists” so it’s hard to tell, but apparently anyone who makes a living considering AI risk is disqualified. The irony is that Andreessen himself of course stands to benefit from AI development, but if that makes him a “Bootlegger” on the opposite side, he doesn’t mention it.
AI Risk #1: Will AI Kill Us All?
Andreessen’s rebuttal of this well-studied central risk comes down to two brief paragraphs:
My view is that the idea that AI will decide to literally kill humanity is a profound category error. AI is not a living being that has been primed by billions of years of evolution to participate in the battle for the survival of the fittest, as animals are, and as we are. It is math – code – computers, built by people, owned by people, used by people, controlled by people. The idea that it will at some point develop a mind of its own and decide that it has motivations that lead it to try to kill us is a superstitious handwave.
In short, AI doesn’t want, it doesn’t have goals, it doesn’t want to kill you, because it’s not alive. And AI is a machine – is not going to come alive any more than your toaster will.
The argumentation here is extremely poor. First, in saying “AI is not a living being”, how does Andreessen define ‘living’? And are we talking about current AI or future AGI? The consensus is that current AI is not sentient, but it’s very difficult to determine when that barrier might be crossed, and how to determine when or if sentience occurs.
Second, saying that AI is “built by people, owned by people, used by people, controlled by people” is obviously correct regarding current AI. But again, whether future AGI will remain controlled (or controllable) is the crux of the debate. Andreessen doesn’t engage with this debate, just flatly states that he thinks AI will remain controllable.
Finally, he says that “The idea that it will at some point develop a mind of its own and decide that it has motivations that lead it to try to kill us is a superstitious handwave.” Again, whether we can have AGI without consciousness is far from a solved issue. More importantly, Andreesen says the AI will need to “decide” to kill us, suggesting some sort of rogue AI/Terminator scenario. But there are plenty of instrumental convergence scenarios, like the “paper clip maximizer”, where a properly programmed and not necessarily conscious AI incidentally exterminates humanity in order to execute its task.
By not engaging with any of these considerations, Andreessen seems to be cherry-picking the least thoughtful takes to make the AI safety camp look bad. In contrast to the first section, where Andreessen portrayed the various future benefits of AI, here he seems to completely lack imagination — or have no interest in employing it — in considering the potential risks.
The rest of this section digresses from the safety argument to claim that ‘“AI risk” has developed into a cult’. Although I don’t have an issue with Andreessen’s analysis here per se, he is again using the most extreme proponents of AI risk to characterize the entire field.
AI Risk #2: Will AI Ruin Our Society?
Andreessen argues,
The second widely mooted AI risk is that AI will ruin our society, by generating outputs that will be so “harmful”, to use the nomenclature of this kind of doomer, as to cause profound damage to humanity, even if we’re not literally killed.
Short version: If the murder robots don’t get us, the hate speech and misinformation will.
He goes on,
As the proponents of both “trust and safety” and “AI alignment” are clustered into the very narrow slice of the global population that characterizes the American coastal elites – which includes many of the people who work in and write about the tech industry – many of my readers will find yourselves primed to argue that dramatic restrictions on AI output are required to avoid destroying society. I will not attempt to talk you out of this now, I will simply state that this is the nature of the demand, and that most people in the world neither agree with your ideology nor want to see you win.
If you don’t agree with the prevailing niche morality that is being imposed on both social media and AI via ever-intensifying speech codes, you should also realize that the fight over what AI is allowed to say/generate will be even more important – by a lot – than the fight over social media censorship. AI is highly likely to be the control layer for everything in the world. How it is allowed to operate is going to matter perhaps more than anything else has ever mattered. You should be aware of how a small and isolated coterie of partisan social engineers are trying to determine that right now, under cover of the age-old claim that they are protecting you.
I actually agree with Andreessen’s take here, so I won’t try to rebut it.
AI Risk #3: Will AI Take All Our Jobs?
Andreessen:
The core mistake the automation-kills-jobs doomers keep making is called the Lump Of Labor Fallacy. This fallacy is the incorrect notion that there is a fixed amount of labor to be done in the economy at any given time, and either machines do it or people do it – and if machines do it, there will be no work for people to do.
The Lump Of Labor Fallacy flows naturally from naive intuition, but naive intuition here is wrong. When technology is applied to production, we get productivity growth – an increase in output generated by a reduction in inputs. The result is lower prices for goods and services. As prices for goods and services fall, we pay less for them, meaning that we now have extra spending power with which to buy other things. This increases demand in the economy, which drives the creation of new production – including new products and new industries – which then creates new jobs for the people who were replaced by machines in prior jobs. The result is a larger economy with higher material prosperity, more industries, more products, and more jobs.
These are good points. However, the obvious counterargument is that the reallocation of labor takes time. Overall productivity growth may occur, but will it be evenly distributed? The next section addresses this.
AI Risk #4: Will AI Lead To Crippling Inequality?
Speaking of Karl Marx, the concern about AI taking jobs segues directly into the next claimed AI risk, which is, OK, Marc, suppose AI does take all the jobs, either for bad or for good. Won’t that result in massive and crippling wealth inequality, as the owners of AI reap all the economic rewards and regular people get nothing?
As it happens, this was a central claim of Marxism, that the owners of the means of production – the bourgeoisie – would inevitably steal all societal wealth from the people who do the actual work – the proletariat. This is another fallacy that simply will not die no matter how often it’s disproved by reality. But let’s drive a stake through its heart anyway.
The flaw in this theory is that, as the owner of a piece of technology, it’s not in your own interest to keep it to yourself – in fact the opposite, it’s in your own interest to sell it to as many customers as possible. The largest market in the world for any product is the entire world, all 8 billion of us. And so in reality, every new technology – even ones that start by selling to the rarefied air of high-paying big companies or wealthy consumers – rapidly proliferates until it’s in the hands of the largest possible mass market, ultimately everyone on the planet.
This sounds unfortunately close to the “don’t worry about the coal miners, they’ll just learn to code!” argument. Sure, AI technology will become available to everyone, but it’s not realistic to imagine that everyone will be able to make equal use of it.
AI Risk #5: Will AI Lead To Bad People Doing Bad Things?
Andreessen actually agrees that it will, but says that “The AI cat is obviously already out of the bag.” Thus, he proposes two remedies:
First, we have laws on the books to criminalize most of the bad things that anyone is going to do with AI. Hack into the Pentagon? That’s a crime. Steal money from a bank? That’s a crime. Create a bioweapon? That’s a crime. Commit a terrorist act? That’s a crime. We can simply focus on preventing those crimes when we can, and prosecuting them when we cannot. We don’t even need new laws – I’m not aware of a single actual bad use for AI that’s been proposed that’s not already illegal. And if a new bad use is identified, we ban that use. QED.
But you’ll notice what I slipped in there – I said we should focus first on preventing AI-assisted crimes before they happen – wouldn’t such prevention mean banning AI? Well, there’s another way to prevent such actions, and that’s by using AI as a defensive tool. The same capabilities that make AI dangerous in the hands of bad guys with bad goals make it powerful in the hands of good guys with good goals – specifically the good guys whose job it is to prevent bad things from happening.
For example, if you are worried about AI generating fake people and fake videos, the answer is to build new systems where people can verify themselves and real content via cryptographic signatures. Digital creation and alteration of both real and fake content was already here before AI; the answer is not to ban word processors and Photoshop – or AI – but to use technology to build a system that actually solves the problem.
And so, second, let’s mount major efforts to use AI for good, legitimate, defensive purposes. Let’s put AI to work in cyberdefense, in biological defense, in hunting terrorists, and in everything else that we do to keep ourselves, our communities, and our nation safe.
The main problem here goes back to section 1 — the concern that if you train AI to kill the ‘bad guys’, it will either end up killing the good guys / innocents, either due to a control malfunction or takeover by the bad guys. Addressing such concerns before developing defensive AI seems like a reasonable step. I don’t think the cat is totally out of the bag yet in this area — while AI weaponization is certainly occuring, it hasn’t come to fruition yet — so I think there’s still time for consideration.
The Actual Risk Of Not Pursuing AI With Maximum Force And Speed
Andreessen says,
There is one final, and real, AI risk that is probably the scariest at all:
AI isn’t just being developed in the relatively free societies of the West, it is also being developed by the Communist Party of the People’s Republic of China.
China has a vastly different vision for AI than we do – they view it as a mechanism for authoritarian population control, full stop. They are not even being secretive about this, they are very clear about it, and they are already pursuing their agenda. And they do not intend to limit their AI strategy to China – they intend to proliferate it all across the world, everywhere they are powering 5G networks, everywhere they are loaning Belt And Road money, everywhere they are providing friendly consumer apps like Tiktok that serve as front ends to their centralized command and control AI.
The single greatest risk of AI is that China wins global AI dominance and we – the United States and the West – do not.
There’s a recent takedown of this claim in Foreign Affairs which I encourage everyone to read. The authors point out that China is unlikely to surpass us any time soon:
Americans should not be haunted by the specter of an imminent Chinese surge in LLM development. Chinese AI teams are fighting—and often failing—to keep up with the blistering speed of new research and products emerging elsewhere. When it comes to LLMs, China trails years, not months, behind its international competitors.
The China issue is complex, and while I agree that naïveté is dangerous, it could also be dangerous to forgo any possibility for cooperation. Even with the USSR, we were ultimately able to undergo mutual arms reduction. I think initiating dialogue on avoiding AI weaponization as soon as possible, rather than pursuing “defensive AI” full steam ahead, is a good idea.
What Is To Be Done?
Andreessen’s proposals here follow from his arguments, so I won’t address them individually.
Conclusion
All in all, Andreessen’s essay is a debate piece that will probably succeed in firing up the accelerationist crowd, but doesn’t meaningfully rebut opposing views.
I think what’s needed at the moment is thoughtful consideration of the doomer and accelarationist positions, not labeling or strawmanning the other side. I found these two polls poll very interesting: they show that when people hear “doomer” they think p(doom) > 95%, but doomers themselves most commonly have 10% < p(doom) < 50%. So people are probably talking past each other, as Andreessen certainly is.
So first, we need a new term for “moderate doomerism” - and maybe for “moderate accelerationism” as well. And second, we need analyses that consider opposing views in good faith, rather than imputing malign motivations. There was an excellent conversation between Yudkowsky and Adam D’Angelo with relatively high-level discourse that could be an example of such dialogue.