这封信件于2025年10月11日发布，作者向包括埃隆·马斯克、山姆·奥特曼、伊利亚·苏茨克维等在内的多位人工智能领域领袖阐述了对开发超人工智能（ASI）的担忧。作者提供了一个概率框架，估算了到2030年ASI可能带来的多种结果，包括ASI的出现、人类灭绝、永久性世界独裁，以及作者本人成为独裁者的可能性。作者强调，与其他历史上的权力斗争相比，ASI竞赛的风险极高，人类灭绝的概率远大于赢得比赛的可能性。信中还指出，社会现实不应被个人情感左右，并对比了成吉思汗和希特勒的极端行为，以说明ASI失控可能造成的灾难性后果，呼吁AI实验室自愿关闭以规避风险。

🎯 ASI的潜在诱惑与风险：作者指出，开发ASI的动机可能包括资源丰富、永生或世界统治。然而，这种追求伴随着巨大的风险，特别是人类灭绝和永久性世界独裁的可能性，这些风险的概率可能远高于个人获益的可能性。

⚖️ 概率评估与决策依据：信中提供了一个2025年到2030年AI发展结果的概率估算，如ASI出现的25%概率，以及由此推导出的“人类灭绝”7.5%和“永久世界独裁”10%的概率。作者建议AI实验室领导者应独立进行此类概率评估，并以此为依据审慎决策。

🌍 历史教训与ASI的极端性：通过对比成吉思汗和希特勒的历史事件，作者强调了ASI失控可能造成的规模远超历史任何冲突的灾难。如果是安全的ASI，可能带来前所未有的权力集中；若是不安全的ASI，则可能导致大规模毁灭，其后果是“100次世界大战或1000次大屠杀”。

💡 现实考量与呼吁：作者认为，在ASI竞赛中，个人情感或社会舆论不应影响对客观风险的判断。面对极高的潜在风险，作者呼吁AI实验室领导者不要仅仅追求技术突破，而是应考虑自愿关闭实验室，以避免可能发生的灾难性后果。

Published on October 11, 2025 7:43 AM GMT

This letter is addressed

To Elon MuskTo Shane Legg, Demis Hassabis, Mustafa SuleymanTo Sam Altman, Greg BrockmanTo Ilya SutskeverTo Dario and Daniella Amodei, Jack Clark, Jared Kaplan, Chris Olah, Tom Brown, Sam McCandlishTo anyone else I may have missed in the above list

Why you might want ASI

I understand there are many reasons you might care a lot about creating ASI.

You might want to create a future where resources are abundantYou might want to become immortalYou might want to become world dictatorMaybe something else

Probability of me becoming world dictator

I'm sure you also have other frameworks for approaching life decisions besides using probabilities like this. You can still spend 15 minutes of your time thinking through the conclusions from this framework, and then discard it.

If I were in your shoes as an AI lab head, as of 2025-10, here are my probabilities of the following outcomes by 2030.

P(ASI by 2030) = 25%
P(Human extinction by 2030) = 25% 30% = 7.5%P(Permanent world dictatorship by winner by 2030) = 25% 40% = 10%
P(I become permanent world dictator) = 25% 40% 33% = 3.3%P(Someone who is not me becomes permanent world dictator) = 6.7%
P(Unknown unknown outcome by 2030) = 25% * 30% = 7.5%
P(No ASI by 2030 but AI race is still on) = 50%
P(I become a trillionaire, and AI race is on) = 25%
P(No ASI by 2030 and AI hits a wall) = 25%
P(I become a centibillionaire, and AI hits a wall) = 15%

I would not personally accept this deal, if I was in your shoes.

The probability of human extinction is likely way higher than the probability of the AI race being winnable and you being the one who wins it. 7.5% > 3.3%The probability of someone building a world dictatorship you have no power in, is greater than the probability of you building the world dictatorship. If you lose the race, I am not optimistic you will get a seat on the coalition that wins. ASI radically centralises power. 6.7% > 3.3%There is no use of being a trillionaire if the AI race doesn't stop, you will just end up facing the same Russian roulette again in the next 5 years after that.My numbers are all non-robust guesswork. I highly recommend you plug in your own numbers. Don't just talk about plugging in the numbers, actually plug them in and see what it means.

Social reality does not matter

This is maybe obvious to you already

How much yield a nuclear bomb will output does not depend on the feelings of those deploying it. Whether you feel excited or anxious or bored or depressed, the yield is the same.People around you are probably a terrible guide for how you should think and feel about these decisions.The stakes are orders of magnitude unprecedented.
Gengis Khan wiped out ~5% of world population in order to rule over ~25% of world population. No leader in history has managed to build a world dictatorship, let alone an immortal one. If you build safe ASI, this person could be you.Adolf Hitler caused ~11 million deaths via Holocaust, and world war 2 caused around ~70 million deaths. If the race to ASI leads to human extinction, this is 100 world wars or 1000 Holocausts combined. If you build unsafe ASI, you might cause this.

Force versus persuasion

I do want your lab to be forcibly shutdown in order to prevent these outcomes, but before I do that, I wanted to make one last attempt to persuade you to voluntarily shutdown your lab.

Discuss