This document from the OECD is split into two sections: principles for responsible stewardship of trustworthy AI & national policies and international co-operation for trustworthy AI. 43 governments around the world have agreed to adhere to the document. While originally written in 2019, updates were made in 2024 which are reflected in this version. Original text:  https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449 Author(s):  The Organization for Economic Cooperation and...

This report by the UK's Department for Science, Technology, and Innovation outlines a regulatory framework for UK AI policy. Per the report, "AI is helping to make our jobs safer and more satisfying, conserve our wildlife and fight climate change, and make our public services more efficient. Not only do we need to plan for the capabilities and uses of the AI systems we have today, but we must also prepare for a near future where the most powerful systems are broadly accessible and...

This statement was released by the UK Government as part of their Global AI Safety Summit from November 2023. It notes that frontier models pose unique risks and calls for international cooperation, finding that "many risks arising from AI are inherently international in nature, and so are best addressed through international cooperation." It was signed by multiple governments, including the US, EU, India, and China. Original...

This summary of UNESCO's Recommendation on the Ethics of AI outlines four core values, ten core principles, and eleven actionable policies for responsible AI governance. The full text was agreed to by all 193 member states of the United Nations. Original text:  https://unesdoc.unesco.org/ark:/48223/pf0000385082 Author(s):  The United Nations Educational, Scientific, and Cultural Organziation A podcast by BlueDot Impact. Learn more on the AI Safety Fundamentals website.

This yearly report from Stanford’s Center for Humane AI tracks AI governance actions and broader trends in policies and legislation by governments around the world in 2023. It includes a summary of major policy actions taken by different governments, as well as analyses of regulatory trends, the volume of AI legislation, and different focus areas governments are prioritizing in their interventions. Original...

This high-level overview by CISA summarizes major US policies on AI at the federal level. Important items worth further investigation include Executive Order 14110, the voluntary commitments, the AI Bill of Rights, and Executive Order 13859. Original text: https://www.cisa.gov/ai/recent-efforts Author(s):  The US Cybersecurity and Infrastructure Security Agency A podcast by BlueDot Impact. Learn more on the AI Safety Fundamentals website.

This fact sheet from The White House summarizes President Biden's AI Executive Order from October 2023. The President's AI EO represents the most aggressive approach to date from the US executive branch on AI policy. Original text: https://www.whitehouse.gov/briefing-room/statements-releases/2023/10/30/fact-sheet-president-biden-issues-executive-order-on-safe-secure-and-trustworthy-artificial-intelligence/ Author(s):  The White House A podcast by BlueDot Impact. Learn more on the AI...

This primer by the Future of Life Institute highlights core elements of the EU AI Act. It includes a high level summary alongside explanations of different restrictions on prohibited AI systems, high-risk AI systems, and general purpose AI. Original text:  https://artificialintelligenceact.eu/high-level-summary/ Author(s):  The Future of Life Institute A podcast by BlueDot Impact. Learn more on the AI Safety Fundamentals website.

This report from the Carnegie Endowment for International Peace summarizes Chinese AI policy as of mid-2023. It also provides analysis of the factors motivating Chinese AI Governance. We're providing a more structured analysis to Chinese AI policy relative to other governments because we expect learners will be less familiar with the Chinese policy process. Original text: https://carnegieendowment.org/2023/07/10/china-s-ai-regulations-and-how-they-get-made-pub-90117 Author(s):  Matt...

This report by the Center for Security and Emerging Technology first analyzes the tensions and tradeoffs between three strategic technology and national security goals: driving technological innovation, impeding adversaries’ progress, and promoting safe deployment. It then identifies different direct and enabling policy levers, assessing each based on the tradeoffs they make. While this document is designed for US policymakers, most of its findings are broadly applicable. Original...

This report from the Centre for Emerging Technology and Security and the Centre for Long-Term Resilience identifies different levers as they apply to different stages of the AI life cycle. They split the AI lifecycle into three stages: design, training, and testing; deployment and usage; and longer-term deployment and diffusion. It also introduces a risk mitigation hierarchy to rank different approaches in decreasing preference, arguing that “policy interventions will be most effective if...

This report by the Nuclear Threat Initiative primarily focuses on how AI's integration into biosciences could advance biotechnology but also poses potentially catastrophic biosecurity risks. It’s included as a core resource this week because the assigned pages offer a valuable case study into an under-discussed lever for AI risk mitigation: building resilience.  Resilience in a risk reduction context is defined by the UN as “the ability of a system, community or society exposed to hazards to...

This excerpt from CAIS’s AI Safety, Ethics, and Society textbook provides a deep dive into the CAIS resource from session three, focusing specifically on the challenges of controlling advanced AI systems. Original Text: https://www.aisafetybook.com/textbook/1-5 Author: The Center for AI Safety A podcast by BlueDot Impact. Learn more on the AI Safety Fundamentals website.

To solve rogue AIs, we’ll have to align them. In this article by Adam Jones of BlueDot Impact, Jones introduces the concept of aligning AIs. He defines alignment as “making AI systems try to do what their creators intend them to do.” Original text: https://aisafetyfundamentals.com/blog/what-is-ai-alignment/ Author: Adam Jones A podcast by BlueDot Impact. Learn more on the AI Safety Fundamentals website.

This article from the Center for AI Safety provides an overview of ways that advanced AI could cause catastrophe. It groups catastrophic risks into four categories: malicious use, AI race, organizational risk, and rogue AIs. The article is a summary of a larger paper that you can read by clicking here. Original text: https://www.safe.ai/ai-risk Authors: Dan Hendrycks, Thomas Woodside, Mantas Mazeika A podcast by BlueDot Impact. Learn more on the AI Safety Fundamentals website.

This report from the UK’s Government Office for Science envisions five potential risk scenarios from frontier AI. Each scenario includes information on the AI system’s capabilities, ownership and access, safety, level and distribution of use, and geopolitical context. It provides key policy issues for each scenario and concludes with an overview of existential risk. If you have extra time, we’d recommend you read the entire document. Original...

This resource, written by Adam Jones at BlueDot Impact, provides a comprehensive overview of the existing and anticipated risks of AI. As you're going through the reading, consider what different futures might look like should different combinations of risks materialize. Original text: https://aisafetyfundamentals.com/blog/ai-risks/ Author: Adam Jones A podcast by BlueDot Impact. Learn more on the AI Safety Fundamentals website.

This blog post from Holden Karnofsky, Open Philanthropy’s Director of AI Strategy, explains how advanced AI might overpower humanity. It summarizes superintelligent takeover arguments and provides a scenario where human-level AI disempowers humans without achieving superintelligence. As Holden summarizes: “if there's something with human-like skills, seeking to disempower humanity, with a population in the same ballpark as (or larger than) that of all humans, we've got a civilization-level...

This insight report from the World Economic Forum summarizes some positive AI outcomes. Some proposed futures include AI enabling shared economic benefit, creating more fulfilling jobs, or allowing humans to work less – giving them time to pursue more satisfying activities like volunteering, exploration, or self-improvement. It also discusses common problems that prevent people from making good predictions about the future. Note: this report was released before ChatGPT, which seems to have...

This report from McKinsey discusses the huge potential for economic growth that generative AI could bring, examining key drivers and exploring potential productivity boosts in different business functions. While reading, evaluate how realistic its claims are, and how this might affect the organization you work at (or organizations you might work at in the future). Original...

This paper by Ross Gruetzemacher and Jess Whittlestone examines the concept of transformative AI, which significantly impacts society without necessarily achieving human-level cognitive abilities. The authors propose three categories of transformation: Narrowly Transformative AI, affecting specific domains like the military; Transformative AI, causing broad changes akin to general-purpose technologies such as electricity; and Radically Transformative AI, inducing profound societal shifts...

This blog by Sam Altman, the CEO of OpenAI, provides insight into what AI company leaders are saying and thinking about their reasons for pursuing advanced AI. It lays out how Altman thinks the world will change because of AI and what policy changes he believes we will need to make. As you’re reading, consider Altman’s position and how it might affect the way he discusses this technology or his policy recommendations. Original text: https://moores.samaltman.com Author: Sam Altman A podcast...

Artificial intelligence (AI) is a potent general purpose technology. Future progress could be rapid, and experts expect that superhuman capabilities in strategic domains will be achieved in the coming decades. The opportunities are tremendous, including advances in medicine and health, transportation, energy, education, science, economic growth, and environmental sustainability. The risks, however, are also substantial and plausibly pose extreme governance challenges. These include labor...

Aim: to give a basic overview of what is going on in longtermist AI governance. Audience: people who have limited familiarity with longtermist AI governance and want to understand it better. I don’t expect this to be helpful for those who already have familiarity with the field. ETA: Some people who were already quite familiar with the field have found this helpful. This post outlines the different kinds of work happening in longtermist AI governance. For each kind of work, I’ll explain it,...