Episode 124 You may think you’re doing a priori reasoning, but actually you’re just over-generalizing from your current experience of technology. I spoke with Professor Seth Lazar about: * Why managing near-term and long-term risks isn’t always zero-sum * How to think through axioms and systems in political philosphy * Coordination problems, economic incentives, and other difficulties in developing publicly beneficial AI Seth is Professor of Philosophy at the Australian National University, an Australian Research Council (ARC) Future Fellow, and a Distinguished Research Fellow of the University of Oxford Institute for Ethics in AI. He has worked on the ethics of war, self-defense, and risk, and now leads the Machine Intelligence and Normative Theory (MINT) Lab, where he directs research projects on the moral and political philosophy of AI. Reach me at [email protected] for feedback, ideas, guest suggestions. Subscribe to The Gradient Podcast:  Apple Podcasts  | Spotify | Pocket Casts | RSSFollow The Gradient on Twitter Outline: * (00:00) Intro * (00:54) Ad read — MLOps conference * (01:32) The allocation of attention — attention, moral skill, and algorithmic recommendation * (03:53) Attention allocation as an independent good (or bad) * (08:22) Axioms in political philosophy * (11:55) Explaining judgments, multiplying entities, parsimony, intuitive disgust * (15:05) AI safety / catastrophic risk concerns * (22:10) Superintelligence arguments, reasoning about technology * (28:42) Attacking current and future harms from AI systems — does one draw resources from the other? * (35:55) GPT-2, model weights, related debates * (39:11) Power and economics—coordination problems, company incentives * (50:42) Morality tales, relationship between safety and capabilities * (55:44) Feasibility horizons, prediction uncertainty, and doing moral philosophy * (1:02:28) What is a feasibility horizon? * (1:08:36) Safety guarantees, speed of improvements, the “Pause AI” letter * (1:14:25) Sociotechnical lenses, narrowly technical solutions * (1:19:47) Experiments for responsibly integrating AI systems into society * (1:26:53) Helpful/honest/harmless and antagonistic AI systems * (1:33:35) Managing incentives conducive to developing technology in the public interest * (1:40:27) Interdisciplinary academic work, disciplinary purity, power in academia * (1:46:54) How we can help legitimize and support interdisciplinary work * (1:50:07) Outro Links: * Seth’s Linktree and Twitter * Resources * Attention, moral skill, and algorithmic recommendation * Catastrophic AI Risk slides Get full access to The Gradient at thegradientpub.substack.com/subscribe