In order to encourage both companies to cooperate and thus be more likely to build safe AGI we want to look for interventions that transform the payout matrix of the game so that it more resembles Table \ref{tab_game_humanity} with externalities included and less resembles Table \ref{tab_game_round_2} where they are ignored. Many options are possible and some are explored elsewhere in this special issue, but one way to change the equilibrium of the AGI race game would be with an SRO for AGI which could impose externalities on the companies by various methods including:
- inspections to demonstrate to each company that the other is cooperating
- contractual financial penalties that would offset any gains from defecting
- social sanctions via public outreach that would reduce gains from defecting
- sharing discoveries between companies
- required shutdown of any uncooperatively built AGI
In this example we need penalties worth in excess of $2 trillion imposed on companies that race to make them prefer to cooperate, which in the real world would likely require the combination of several strategies to make sure the bar is cleared even if one or several sources of penalties fail. Some of these strategies may also require enforcement by state actors, which further complicates the situation since militaries may also be participating in the race, and suggests an SRO may be insufficient to prevent an AGI race unless it is partnered with an intergovernmental organization, such as the United Nations \citep{wolfsthal2007}. That said a more traditional SRO could act faster with fewer political entanglements, so there seems to be space for both an SRO focused on industrial and academic AGI research and an intergovernmental organization working in collaboration with it to adjust the incentives of state actors.
The key takeaway is that even if an SRO is not the best way to modify the equilibrium of the AGI race, there is a need for some organization to impose externalities that reduce the chance of an AGI race by making it less appealing than when externalities can be ignored. SROs provide a clear template for this sort of organization, though addressing the AGI race specifically may require innovative policy solutions outside of those normally taken by SROs. An SRO for AGI thus stands likely to be a key component in avoiding an AGI race if it is willing to evolve in ways that help it address the issue.
Conclusion
An SRO for AGI is likely valuable, and may be particularly helpful in counteracting the incentives to race to develop AGI. Although there is currently no SRO for AGI, there are several organizations that are already positioned to take on an SRO role if they so chose, although some better positioned than others. They include:
- Partnership on AI
- Centre for the Study of Existential Risk
- World Economic Forum Council on AI and Robotics
- International Telecommunications Union
- Future of Life Institute
- Future of Humanity Institute
- Leverhulme Center for the Future of Intelligence
- Machine Intelligence Research Institute
- Center for Human-Compatible AI
- Center for Safety And Reliability of Autonomous Systems
If none of these groups wish to take on the task then creating an SRO for AGI is likely a neglected cause for those concerned about the existential risks posed by AGI. It is the recommendation of the present work that either an existing organization or a new one take up the task of serving as an SRO for AGI to reduce the risk of an AGI race and otherwise foster safety in AGI research.