‍This post was cross-published on the author’s Substack, Rising Tide.

____

‍

Historically, the way we’ve dealt well with rapidly evolving uncertain processes is classical liberalism.
-Dwarkesh Patel, X

I wasn’t expecting a book from 1998 to explain the 2023-2024 AI safety wars, but Virginia Postrel’s The Future and Its Enemies — which I picked up at the recommendation of libertarian AI policy wonk Adam Thierer — does a surprisingly good job.

Postrel’s book helped reframe something that had been bothering me. I think there’s plenty to critique about AI safety ideas and the AI safety community, but when they come under fire — as has happened a lot over the past couple of years — the critiques often miss the mark. One common theme is portraying AI safety advocates as anti-technology, which is totally out of step with the nerdy, early-adopter, industrial-revolution-enthusiast crowd I know. Another recurring criticism is that AI safety fits in with a broader tradition of “safetyists,” who want to eradicate any source of risk-taking in the world, cost-benefit be damned.¹ But this doesn’t fit either — the AI safety community, perhaps due to its close connections with the world of effective altruism, is extremely into thinking in terms of expected value, VC-style bets with a small chance of a huge payoff, and the like. 

These dichotomies — pro-tech vs. anti-tech, risk-embracing vs. risk-averse — are awkward fits for the arguments people want to have about AI safety ideas. In her book, Postrel sets out to describe a different dichotomy, one that in my view much better captures what it is about AI safety ideas that gets critics’ hackles up.

The Future and Its Enemies is a manifesto for what Postrel calls “dynamism — a world of constant creation, discovery, and competition.” Its antithesis is “stasis — a regulated, engineered world... [that values] stability and control.” 

Postrel describes how dynamism vs. stasis is a clearer lens for some of the political debates of the late 1990s than the traditional left-right political spectrum. On trade, dynamist free-traders clashed with a stasist coalition worried about trade’s implications for sovereignty (on the right) and social justice (on the left). On immigration, dynamists favoring higher quotas come up against stasists motivated in some cases by fears of population growth, in others by xenophobia. 

She couldn’t have known it at the time, but dynamism vs. stasism is also a useful lens for disagreements about how to handle the challenges we face on the road to superhuman AI.

Stasism: Finding the “One Best Way”

Lest I be misunderstood, I’ll be clear: I don’t think AI safety is inherently stasist. But I do think that a subset of the policies championed by the AI safety community are quite stasist, and that the community overall can have a stasist vibe.

What do I mean by this? Let’s hear some more from Postrel about stasism:

Stasists seek specifics to govern each new situation and keep things under control [...] [they] want their detailed rules to apply to everyone.

"[Stasist William Greider] frets that someone should be in charge; a juggernaut of global capitalism is loose and ‘no one is at the wheel.’ Yet this begs the question. Whom do we really want at the wheel of the global economy?"²

[Stasism] does not allow such turbulence. It is centralized and inflexible. It asks people with new ideas to justify them to boards and commissions. It establishes rules, from broadcasting regulations to laws against working at home, that assume that neither technologies nor tastes will change.

To me, these descriptions call to mind a number of things associated with AI safety. 

First among them is probably my least favorite existential risk paper of all time, Nick Bostrom’s “Vulnerable World Hypothesis.” This is a philosophy paper built out of abstractions and thought experiments, centered around the idea that the urn of possible technological inventions could contain a “black ball,” which would destroy civilization. The policy implication:

In order for civilization to have a general capacity to deal with "black ball" inventions of this type, it would need a system of ubiquitous real-time worldwide surveillance. In some scenarios, such a system would need to be in place before the technology is invented.

It’s hard to get more stasist — focusing on stability and control — than ubiquitous real-time worldwide surveillance.

Because it’s a philosophy paper written by a philosopher, it is not clear how much the Vulnerable World Hypothesis is intended as a neutral exploration of possibility space vs. a sincere set of prescriptions. But regardless of how it was intended, it’s an influential text in the existential risk community, which is closely connected to the AI safety community. In my experience, the underlying tone of “the only way to deal with sufficiently dangerous technology is top-down control” suffuses many (though certainly not all) parts of the AI safety community.³

Other manifestations of stasist tendencies in the AI safety world include:

• The widespread (and long-unquestioned) assumption that it’s better for there to be fewer leading AI projects — perhaps ideally only one.⁴ This way, it would be easier for the leading projects to coordinate if they were nearing AGI and hadn’t yet solved the alignment problem. Outside oversight, e.g. via government regulation, would also be easier if it only had to cover a small number of organizations.
• Efforts to prevent AI misuse by focusing on nonproliferation, so that fewer people have access to capital-d Dangerous technology. I critiqued this in a previous post about adaptation buffers, where I did not name Postrel’s dynamism but very much had it in mind.
• The short-lived push in 2023 to implement a licensing regime for training frontier models. The paper that introduced this approach, Frontier AI Regulation, offered it as only one of a menu of options, but inside the part of the AI policy community that focuses on existential risks, licensing was treated as a go-to policy recommendation for months.⁵
• The habit in some AI safety circles of asking what your “theory of victory” is. This is less common than asking for your p(doom) or your timelines, but still fairly common. Working backwards from a desired end state is a fine way to think about strategy. But it’s a mistake to get too fixated on driving towards one pre-specified outcome. In Postrel’s words, I hope for a “dynamic and inherently unstable” future of “emergent, complex messiness.” Thinking in terms of finite end states that constitute “victory” may not be the best way to get there.⁶

These stasist tendencies are not present in all parts of the AI safety world, and I’m not the first person to critique them. Over the past year or two, the AI safety community itself has shown growing interest in moving in a less stasist direction. Risks from “concentration of power” are increasingly included in breakdowns of different kinds of existential risks from AI; I credit OpenAI whistleblower and AI 2027 lead author Daniel Kokotajlo with being a notable advocate of ideas in this vicinity.⁷ From outside the community, Jeremy Howard stands out as someone who takes the potential power of advanced AI seriously, but argues that risks from centralized power are more severe than the risk from misaligned superintelligence. There’s increasing consensus that trying to manage AI risks by exerting massive top-down control could go very badly.

/odw-inline-subscribe-cta

Advanced AI as a threat to a dynamic future

The trouble is, though, that critiquing the AI safety community’s solutions doesn’t negate the many very real challenges we face as AI continues to advance. Indeed, I expect that many AI safety folks’ reaction to the previous section will be, “I mean, sure, dynamism is obviously what we ideally want, but dynamism doesn’t help you if all the humans are dead or totally disempowered.”

I don’t disagree. I do think it’s illuminating to reframe the challenges of transitioning to a world with advanced AI so that instead of thinking primarily about safety, we think in terms of threats to a dynamic future. For this to work, we need to make explicit that human agency is a core component of dynamism. Postrel doesn’t write directly about this, but I think it’s clear from the book that she wouldn’t celebrate a world where wealthy corporations are creating, discovering, and competing while most people can’t participate. Analogously, we can say that a world with AI calling the shots and humans either dead or disempowered would not count as real dynamism, even if there were a vibrant AI economy of invention and creation.⁸

Thinking in terms of how AI might threaten a dynamic future, one thing that becomes clear is that preventing the worst-case scenarios is only the first step. The classic AI risks — catastrophic misuse, catastrophic misalignment — could indeed be game over for dynamism, so we need to handle those. But if we handle them by massively concentrating power, we haven’t succeeded. For the future to actually go well, we will need to find our way to some new equilibrium that allows for decentralized experimentation, spontaneous adjustment, creativity, and risk-taking. Perhaps that could look similar to our current system of liberal democratic nation-states with sovereign borders and relatively unfettered markets; more likely, it will look as different from our current system as today looks from feudalism.

Perhaps aiming for “dynamism” is not much different than aiming for other feel-good terms like “human flourishing” or “democracy.” But I found Postrel’s concept a helpful way of coming at some of those same ideas from a fresh angle, and with more specificity about downsides and tradeoffs involved (e.g. that decentralization entails messiness, or that risk-taking entails failures), rather than purely positive affect.

Dynamism: An “open-ended future”

This is the part of the essay where I think I’m supposed to offer solutions. I don’t really have them.

I’m writing this anyway because I hope this is a productive lens through which we can figure this all out together. Discourse on AI safety has already shifted somewhat in this direction, which I have been glad to see. 

Postrel does describe five characteristics of “dynamist rules”:

As an overview, dynamist rules:
1. Allow individuals (including groups of individuals) to act on their own knowledge.
2. Apply to simple, generic units and allow them to combine in many different ways.
3. Permit credible, understandable, enduring, and enforceable commitments.
4. Protect criticism, competition, and feedback.
5. Establish a framework within which people can create nested, competing frameworks of more specific rules.

I see some overlap with existing ideas in AI policy:

• Transparency, everyone’s favorite consensus recommendation, fits well into a dynamist worldview. It helps with Postrel’s #1 (giving individuals access to better information that they can act on as they choose), #3 (facilitating commitments), and #4 (facilitating criticism and feedback). Ditto whistleblower protections.
• Supporting the development of a third-party audit ecosystem also fits — it helps create and enforce credible commitments, per #3, and could be considered a kind of nestable framework, per #5. 
• The value of open models in driving decentralized use, testing, and research is obvious through a dynamist lens, and jibes with #1 and #4. (I do think there should be some precautionary friction before releasing frontier models openly, but that’s a narrow exception to the broader value of open source AI resources.)

Another good bet is differential technological development, aka defensive accelerationism — proactively building technologies that help manage challenges posed by other technologies — though I can’t easily map it onto Postrel’s five characteristics. I’d be glad to hear readers’ ideas for other productive directions to push in.

The point of this post, though, is not to push specific solutions. The point is to make it easier to hold the following two things at once. 

It’s true that some of the AI safety community’s policy prescriptions — and some of its vibes — are what Postrel would call stasist, prioritizing control and stability at the expense of dynamism’s freedom and exploration and decentralized adaptation.

But it’s also true that building superhuman AI brings with it many other threats to a dynamist future, including the risks of humans using AI to take and retain power, of AI systems themselves taking over, and of humans gradually ceding our ability to steer our own destiny. We need to be able to critique solutions without being blind to the underlying problems.