What if the secret to successful AI governance lies in understanding the evolution of model documentation? In this episode, our hosts challenge the common belief that model cards marked the start of documentation in AI. We explore model documentation practices, from their crucial beginnings in fields like finance to their adaptation in Silicon Valley. Our discussion also highlights the important role of early modelers and statisticians in advocating for a complete approach that includes the e...

Are businesses ready for large language models as a path to AI? In this episode, the hosts reflect on the past year of what has changed and what hasn’t changed in the world of LLMs. Join us as we debunk the latest myths and emphasize the importance of robust risk management in AI integration. The good news is that many decisions about adoption have forced businesses to discuss their future and impact in the face of emerging technology. You won't want to miss this discussion.Intro and news: Th...

Our special guest, astrophysicist Rachel Losacco, explains the intricacies of galaxies, modeling, and the computational methods that unveil their mysteries. She shares stories about how advanced computational resources enable scientists to decode galaxy interactions over millions of years with true-to-life accuracy. Sid and Andrew discuss transferable practices for building resilient modeling systems. Prologue: Why it's important to bring stats back [00:00:03]Announcement from the Americ...

Can your AI models survive a big disaster? While a recent major IT incident with CrowdStrike wasn't AI related, the magnitude and reaction reminded us that no system no matter how proven is immune to failure. AI modeling systems are no different. Neglecting the best practices of building models can lead to unrecoverable failures. Discover how the three-tiered framework of robustness, resiliency, and anti-fragility can guide your approach to creating AI infrastructures that not only perform re...

Join us as we chat with Patrick Hall, Principal Scientist at Hallresearch.ai and Assistant Professor at George Washington University. He shares his insights on the current state of AI, its limitations, and the potential risks associated with it. The conversation also touched on the importance of responsible AI, the role of the National Institute of Standards and Technology (NIST) AI Risk Management Framework (RMF) in adoption, and the implications of using generative AI in decision-making.Sho...

Ready to uncover the secrets of modern systems engineering and the future of AI? Join us for an enlightening conversation with Matt Barlin, the Chief Science Officer of Valence. Matt's extensive background in systems engineering and data lineage sets the stage for a fascinating discussion. He sheds light on the historical evolution of the field, the critical role of documentation, and the early detection of defects in complex systems. This episode promises to expand your understanding of mode...

Explore the basics of differential privacy and its critical role in protecting individual anonymity. The hosts explain the latest guidelines and best practices in applying differential privacy to data for models such as AI. Learn how this method also makes sure that personal data remains confidential, even when datasets are analyzed or hacked.Show NotesIntro and AI news (00:00) Google AI search tells users to glue pizza and eat rocks Gary Marcus on break? (Maybe and X only break)Wha...

Artificial Intelligence (AI) stands at a unique intersection of technology, ethics, and regulation. The complexities of responsible AI are brought into sharp focus in this episode featuring Anthony Habayeb, CEO and co-founder of Monitaur, As responsible AI is scrutinized for its role in profitability and innovation, Anthony and our hosts discuss the imperatives of safe and unbiased modeling systems, the role of regulations, and the importance of ethics in shaping AI.Show notesPrologue: ...

Baseline modeling is a necessary part of model validation. In our expert opinion, it should be required before model deployment. There are many baseline modeling types and in this episode, we're discussing their use cases, strengths, and weaknesses. We're sure you'll appreciate a fresh take on how to improve your modeling practices. Show notes Introductions and news: why reporting and visibility is a good thing for AI 0:03 Spoiler alert: Providing visibility to AI bias audits does NOT mean...

In this episode, we explore information theory and the not-so-obvious shortcomings of its popular metrics for model monitoring; and where non-parametric statistical methods can serve as the better option.  Introduction and latest news 0:03 Gary Marcus has written an article questioning the hype around generative AI, suggesting it may not be as transformative as previously thought.This in contrast to announcements out of the NVIDIA conference during the same week.Information theory and its...

In this episode, the hosts focus on the basics of anomaly detection in machine learning and AI systems, including its importance, and how it is implemented. They also touch on the topic of large language models, the (in)accuracy of data scraping, and the importance of high-quality data when employing various detection methods. You'll even gain some techniques you can use right away to improve your training data and your models. Intro and discussion (0:03) Questions about Information Theory...

We're taking a slight detour from modeling best practices to explore questions about AI and consciousness.  With special guest Michael Herman, co-founder of Monitaur and TestDriven.io, the team discusses different philosophical perspectives on consciousness and how these apply to AI. They also discuss the potential dangers of AI in its current state and why starting fresh instead of iterating can make all the difference in achieving characteristics of AI that might resemble...

Data scientists, researchers, engineers, marketers, and risk leaders find themselves at a crossroads to expand their skills or risk obsolescence. The hosts discuss how a growth mindset and "the fundamentals" of AI can help. Our episode shines a light on this vital shift, equipping listeners with strategies to elevate their skills and integrate multidisciplinary knowledge. We share stories from the trenches on how each role affects robust AI solutions that adhere to ethical standards, and how...

Get ready for 2024 and a brand new episode! We discuss non-parametric statistics in data analysis and AI modeling. Learn more about applications in user research methods, as well as the importance of key assumptions in statistics and data modeling that must not be overlooked,  Welcome to 2024  (0:03) AI, privacy, and marketing in the tech industryOpenAI's GPT store launch. (The Verge)Google's changes to third-party cookies. (Gizmodo)Non-parametric statistics and its applications (6:49) A...

It's the end of 2023 and our first season. The hosts reflect on what's happened with the fundamentals of AI regulation, data privacy, and ethics. Spoiler alert: a lot! And we're excited to share our outlook for AI in 2024. AI regulation and its impact in 2024.Hosts reflect on AI regulation discussions from their first 10 episodes, discussing what went well and what didn't.Its potential impact on innovation. 2:36AI innovation, regulation, and best practices. 7:05AI, privacy, and data security...

Joshua Pyle joins us in a discussion about managing bias in the actuarial sciences. Together with Andrew's and Sid's perspectives from  both the economic and data science fields, they deliver an interdisciplinary conversation about bias that you'll only find here. OpenAI news plus new developments in language models. 0:03The hosts get to discuss the aftermath of OpenAI and Sam Altman's return as CEOTension between OpenAI's board and researchers on the push for slow, responsible AI development...

Episode 9. Continuing our series run about model validation. In this episode, the hosts focus on aspects of performance, why we need to do statistics correctly, and not use metrics without understanding how they work, to ensure that models are evaluated in a meaningful way. AI regulations, red team testing, and physics-based modeling. 0:03The hosts discuss the Biden administration's executive order on AI and its implications for model validation and performance.Evaluating machine learning...

Episode 8. This is the first in a series of episodes dedicated to model validation. Today, we focus on model robustness and resilience. From complex financial systems to why your gym might be overcrowded at New Year's, you've been directly affected by these aspects of model validation. AI hype and consumer trust (0:03)  FTC article highlights consumer concerns about AI's impact on lives and businesses (Oct 3, FTC)Increased public awareness of AI and the masses of data needed to train it led...

Episode 7.  To use or not to use? That is the question about digital twins that the fundamentalists explore. Many solutions continue to be proposed for making AI systems safer, but can digital twins really deliver for AI what we know they can do for physical systems? Tune in and find out. Show notes Digital twins by definition. 0:03Digital twins are one-to-one digital models of real-life products, systems, or processes, used for simulations, testing, monitoring, maintenance, or practice...

Episode 6. What does systems engineering have to do with AI fundamentals? In this episode, the team discusses what data and computer science as professions can learn from systems engineering, and how the methods and mindset of the latter can boost the quality of AI-based innovations. Show notes  News and episode commentary 0:03ChatGPT usage is down for the second straight month.The importance of understanding the data and how it affects the quality of synthetic data for non-tabular use cases...

Episode 5. This episode about synthetic data is very real. The fundamentalists uncover the pros and cons of synthetic data; as well as reliable use cases and the best techniques for safe and effective use in AI. When even SAG-AFTRA and OpenAI make synthetic data a household word, you know this is an episode you can't miss. Show notes What is synthetic data? 0:03Definition is not a succinct one-liner, which is one of the key issues with assessing synthetic data generation.Using general...

Episode 4. The AI Fundamentalists welcome Christoph Molnar to discuss the characteristics of a modeling mindset in a rapidly innovating world. He is the author of multiple data science books including Modeling Mindsets, Interpretable Machine Learning, and his latest book Introduction to Conformal Prediction with Python. We hope you enjoy this enlightening discussion from a model builder's point of view. To keep in touch with Christoph's work, subscribe to his newsletter The Mindful...

Episode 3.  Get ready because we're bringing stats back! An AI model can only learn from the data it has seen. And business problems can’t be solved without the right data. The Fundamentalists break down the basics of data from collection to regulation to bias to quality in AI.  Introduction to this episodeWhy data matters.How do big tech's LLM models stack up to the proposed EU AI Act?How major models such as Open AI and Bard stack up against current regulations.Stanford HAI - Do...