🚀 **两种AI发展策略的提出：** 文章核心围绕两种截然不同的AI发展路径展开讨论——“全球停滞”（Global Shutdown）与“全球可控加速”（Global Controlled Takeoff）。“全球停滞”设想通过集中计算资源并严格限制前沿研发来阻止AI的失控发展；而“全球可控加速”则提出在国际监管下，允许AI安全研究继续进行，并利用AI自身来加速对齐（alignment）问题的解决。这两种策略都旨在应对AI技术可能带来的潜在风险，但采取了截然不同的行动方向。

⚖️ **政治可行性与策略复杂度分析：** 作者深入分析了这两种策略的政治执行难度，指出无论哪种方案，实现“集中所有GPU”这一前提都面临巨大的政治阻力，需要世界各国和企业达成前所未有的共识。作者认为，“全球停滞”在概念上比“全球可控加速”更为简单直接，后者需要国际协议能够准确区分安全与不安全的训练运行，并避免陷入官僚主义的泥潭。因此，作者并不认为“全球可控加速”在政治上比“全球停滞”更容易实现。

💡 **经济压力与AI对齐研究的挑战：** 文章强调了当前AI行业面临的经济压力，这种压力驱使企业不断追求速度，而忽略了深度的安全研究。作者认为，要有效解决AI对齐问题，可能需要超越当前主流的实证主义方法，转向更精密的“概念推理”。然而，在竞争激烈的市场环境中，企业很难有动力去放慢脚步进行这种深度的理论探索。作者指出，无论是“全球停滞”还是“全球可控加速”，关键在于能否建立一个值得信赖的第三方机制，消除经济驱动下的盲目加速。

📈 **“吓人演示”与政策制定的联动：** 作者预测，随着AI技术的发展，未来会出现更多“足够吓人的演示”（Sufficiently Scary Demos），这些演示将比抽象的理论论证更能引起公众和政策制定者的重视，从而推动对AI风险的认知和应对措施的制定。作者建议，应提前准备相关的全球性协议和框架，以便在这些关键时刻能够迅速响应，而不是手足无措。这表明，技术发展中的“事件驱动”将是影响AI治理的重要因素。

Published on September 24, 2025 5:21 PM GMT

Two somewhat different plans for buying time and improving AI outcomes are: "Global Shutdown" and "Global Controlled Takeoff."

(Some other plans some people believe in include "ad hoc semi-controlled semi-slowed takeoff" and "race, then burn the lead on either superalignment or scary demos" and "decentralized differential defensive tech world.". I mostly don't expect those to work, but am mostly not talking about them in this post.)

"Global Shutdown" and "Global Controlled Takeoff" both include an early step of "consolidate all GPUs and similar chips into locations that can be easily monitored." 

The Shut Down plan then says things like "you cannot do any frontier development with the consolidated GPUs" (maybe you can use GPUs to run existing models that seem pretty safe, depends on implimentation details). Also, maybe, any research into new algorithms needs to be approved by an international org, and frontier algorithm development is illegal. (This is maybe hard to enforce, but, it is might dramatically reduce the amount of R&D that goes into it, since you can't be a billion dollar company who straightforwardly pours tons of resources into it without going to jail)

Controlled Takeoff says instead (as I currently understand advocates to advocate) something like "Frontier research continues, slowly, carefully, leveraging frontier controlled AI to do a ton of alignment research."

I'm generally pro "Shut It Down", but I also think Global Controlled Takeoff is much better than the status quo (both because it seems better in isolation, and because achieving it makes Shut Down easier), and I see some of the appeal depending on your exact beliefs.

But, some notes on strategy here.

"What's more impossible?"

Or: "Shut it down" is simpler than "Controlled Takeoff"

A lot of AI safety arguments boil down to "what seems least impossible?". Is it more impossible to get a Global Shutdown, or to solve safe superintelligence with anything remotely like our current understanding or the understanding we're likely to get over the next 5-10 years?

I've heard a number of people say flatly "you're not going to get a global shut down", with a tone of finality that sounds like they think this is basically impossible. 

I'm not entirely sure I've correctly tracked which people are saying which things and whether I'm accidentally conflating statements from different people. But I think I've heard at least some people say "you're not getting a shut down" with that tone, who nonetheless advocate for controlled takeoff.

I certainly agree getting a global shutdown is very hard. But it's not obvious to me that getting a global controlled takeoff is much easier.

Two gears I want to make sure people are tracking:

Gear 1: "Actually consolidate the GPUs" is a huge political lift, regardless. 

By the time you've gotten various world powers and corporations to do this extremely major, expensive action, I think something has significantly changed about the political landscape. I don't see how you'd get it without world leaders taking AI more "fundamentally seriously", in a way that would make other expensive plans a lot more tractable.

Gear 2: "You need to compare the tractability of Global Shut Down vs Global Controlled Takeoff That Actually Works, as opposed to Something That Looks Close To But Not Actually A Controlled Takeoff."

Along with Gear 3: "Shut it down" is much simpler than "Controlled Takeoff."

A Global Controlled Takeoff That Works has a lot of moving parts. 

You need the international agreement to be capable of making any kind of sensible distinctions between safe and unsafe training runs, or even "marginally safer" vs "marginally less safe" training runs. 

You need the international agreement to not turn into molochian regulatory-captured horror that perversely reverses the intent of the agreement and creates a class of bureaucrats who don't know anything about AI and use the agreement to dole out favors.

These problems still exist in some versions of Shut It Down too, to be clear (if you're trying to also ban algorithmic research – a lot of versions of that seem like they leave room to argue about whether agent foundations or interpretability count). But, they at least get coupled with "no large training runs, period."

I think "guys, everyone just stop" is a way easier schelling point to coordinate around, than "everyone, we're going to slow down and try to figure out alignment as best we can using current techniques."

So, I am not currently convinced that Global Controlled Takeoff That Actually Works is any more politically tractable than Global Shut Down.

(Caveat: Insofar as your plan is "well, we will totally get a molochian moral maze horror, but, it'll generally move slower and that buys time", eh, okay, seems reasonable. But, at least be clear to yourself about what you're aiming for)

Gear 4: Removing pressure to accelerate is valuable for the epistemics of the people doing the AI-assisted alignment (if you're trying that).

One reason I think the Anthropic plan is actively bad, instead of "at-least-okay-ish," is that (given how hard they seem to actively oppose any kind of serious regulation that would slow them down), they seem intent on remaining in a world where, while they are supposedly working on aligning the next generation of AI, they have constant economic pressure to ship the next thing soon.

I believe, maybe, you can leverage AI to help you align AI.

I am pretty confident that at least some of the tools you need to navigate aligning unbounded superintelligence (or confidently avoiding creating unbounded superintelligence,) involves "precise conceptual reasoning" of a kind Anthropic-et-all seem actively allergic to. (see also behaviorism vs cognitivism. Anthropic culture seems to actively pride itself on empirics and be actively suspicious of attempts to reason ahead without empirics)

I'm not confident that you need that much precise conceptual reasoning / reasoning ahead. (MIRI has an inside view that says this is... not like impossible hard, but, is hard in a fairly deep way that nobody is showing respect for. I don't have a clear inside view about "how hard is it", but I have an inside view that it's harder than Anthropic's revealed actions thinks it is)

I think thinking through this and figuring out whether you need conceptual tools that you aren't currently good at in order to succeed, is very hard, and people are extremely biased^[1] about it. 

I think the difficult is exacerbated further if your competitor is shipping the next generation of product, and know-in-your-heart that you're reaching ASL danger levels that at least should give you some pause to think about it, but, the evidence isn't clear, and it would be extremely convenient for you and your org if your current level of control/alignment was sufficient to run the next training run.

So a lot of what I care most about with Shutdown/Controlled-Takeoff is making it no longer true that there is an economic incentive to rush ahead. (I think either Shutdown-y or Controlled Takeoff-y can both potentially work for this, if there's actually a trusted third party who is the one that makes calls about whether the next training run is allowed, who has the guns and compute).

Gear 5: Political tractability will change as demos get scarier. 

I'm not super thrilled with the "race to the edge, then burn the lead on scary demos" plan (specifically the "racing" part). But, I do think we will get much scarier demos as we approach AGI. 

Politicians maybe don't understand abstract arguments (although I think responses to If Anyone Builds It suggests they at least sometimes do). But I think there are various flavors of Sufficiently Scary Demos that will make the threat much more salient without needing to route through abstract arguments.

I think one of the most important things to be preparing for is leveraging Sufficiently Scary Demos when they arrive. I think this includes beginning to argue seriously now for global treaty shaped things and have them on hand so people go "oh, okay, I guess we do need that thing Those Guys Were Talking About After All" instead of just being bewildered.

Gears rather than Bottom Lines

I won't make a decisive claim that any of the above should be decisive in anyone's decisionmaking. I'm still processing the update that I can't really simulate the entire political disagreement yet and I'm not sure what other gears I'm missing from other people's perspective.

But, these are all individual gears that seem pretty important to me, which I think should be part of other people's overall strategizing.

I have a similar point comparing the feasibility of "Global Shut Down" vs "Decentralized Differentially Defensive Tech world that Actually Works", but, that's a fairly complex and different argument.

1. ^{^}

   to be clear this bias also applies in the MIRI-esque direction. But, they're not the one rushing ahead inventing AGI.