On this episode, we’re joined by Stella Biderman, Executive Director at EleutherAI and Lead Scientist - Mathematician at Booz Allen Hamilton. EleutherAI is a grassroots collective that enables open-source AI research and focuses on the development and interpretability of large language models (LLMs). We discuss: - How EleutherAI got its start and where it's headed. - The similarities and differences between various LLMs. - How to decide which model to use for your desired outcome. - The benefits and challenges of reinforcement learning from human feedback. - Details around pre-training and fine-tuning LLMs. - Which types of GPUs are best when training LLMs. - What separates EleutherAI from other companies training LLMs. - Details around mechanistic interpretability. - Why understanding what and how LLMs memorize is important. - The importance of giving researchers and the public access to LLMs. Stella Biderman - https://www.linkedin.com/in/stellabiderman/ EleutherAI - https://www.linkedin.com/company/eleutherai/ Resources: - https://www.eleuther.ai/ Thanks for listening to the Gradient Dissent podcast, brought to you by Weights & Biases. If you enjoyed this episode, please leave a review to help get the word out about the show. And be sure to subscribe so you never miss another insightful conversation. #OCR #DeepLearning #AI #Modeling #ML