The Coming Wave by Mustafa Suleyman, et. al.

Just finished reading this book by the founder of Deepmind, arguably the most important AI company in the world.  He thinks the impact of LLMs will be much larger that what I thought in this post.  A wave is a major cycle of innovation across multiple domains.  In the last 250 years we have gone through 5 waves with AI and Synthetic Biology being the 6th wave of change.

Innovation Waves in Recent Human History

He argues that because AI LLMs and Synthetic Biology technologies meet four critera (a) general purpose (can affect national or global economies), (b) dual use (civilian or military use), (c) widely accessible for around $50K (Hardware for large local LLM, DNA/RNA Synthesizer), and (d) decentralized (LLMs, Biohackers) there is no way to stop people, organizations, or nation states from using them with possibly catastrophic effects (remember "to err is human").  So some person, company, or nation state could create an AGI LLM that decides humans are not needed as we are ruining the plan with our carbon emissions.  Or perhaps a biohacker who is frustrated with the slow response of humanity to climate change creates a new disease the kills 80% of the humans.  This is well within the realm of possibility.

In his mind we need to contain these technologies like we did with nuclear fission.  Even with that control, we still had many close calls. The problem is that nuclear fission while being general purpose and dual use is NOT widely accessible or decentralized.  This means that nation states were able to control its spread to some extent.  Our edge choices are to turn into dystopian societies where everyone, everything, everywhere is under surveillance all the time, some catastrophic event that wipes out a large portion of humanity, or somehow walk a narrow path between these extremes.  He then states the characteristics of that narrow path:

1. Safety: An Apollo program for technical safety - Basically a shutdown switch to turn this program off if they go havoc.  
2. Audits: Knowledge is power: Power is control - can't rely on gov and institutions that have no idea about tech. regular audits and how knowledge about AI systems empowers better control and accountability.
3. Choke Points: Buy Time - Limits sales of Chips, exports, and imports and limits the use of certain software and hardware.
4. Makers: Critics Should Build It - AI critics should participate in building and improving the technology, promoting a balanced perspective.
5. Businesses: Profit + Purpose - profits invested with a greater purpose, including ethical considerations in AI development.
6. Governments: Survive, Reform, Regulate - Govs to adapt, get with the times, and reform to effectively regulate AI technologies, ensuring their survival and responsible use.
7. Alliance: Time for Treaties - Global tech needs global treaties and alliances to establish global standards and regulations for AI, fostering cooperation among nations.
8. Culture: Respectfully Embracing Failure - Create a culture that encourages experimentation and learning from failures in AI development.
9. Movements: People Power - Let's embrace and cooperate with grassroots movements and public awareness to shape the ethical development and deployment of AI.
10. The Narrow Path: The Only Way Through - We need to understand the challenges and difficulties involved in navigating the complex landscape of AI ethics and development, emphasizing the need for a careful and considered approach.

Just step one for synthetic biology would require that every single DNA/RNA synthesizer report back to some organization as to what it is doing.  First thing that will happen, is people will share how to defeat this safety measure. 

I just cannot envision any possible future where countries, governments, businesses, individuals will accept these steps.  Can you?  So kudos to him for identifying the problem and the things that need to happy to avoid it, but if it comes to a choice between catastrophic event or dystopia, I guess I would choose dystopia.  At least then we may live.