Ivan Zhang, co-founder of Cohere, discusses the company's enterprise-focused AI solutions. He explains Cohere's early emphasis on embedding technology and training models for secure environments. Zhang highlights their implementation of Retrieval-Augmented Generation in healthcare, significantly reducing doctor preparation time. He explores the shift from monolithic AI models to heterogeneous systems and the importance of improving various AI system components. Zhang shares insights on using synthetic data to teach models reasoning, the democratization of software development through AI, and how his gaming skills transfer to running an AI company. He advises young developers to fully embrace AI technologies and offers perspectives on AI reliability, potential risks, and future model architectures. https://cohere.com/ https://ivanzhang.ca/ https://x.com/1vnzh TOC: 00:00:00 Intro 00:03:20 AI & Language Model Evolution 00:06:09 Future AI Apps & Development 00:09:29 Impact on Software Dev Practices 00:13:03 Philosophical & Societal Implications 00:16:30 Compute Efficiency & RAG 00:20:39 Adoption Challenges & Solutions 00:22:30 GPU Optimization & Kubernetes Limits 00:24:16 Cohere's Implementation Approach 00:28:13 Gaming's Professional Influence 00:34:45 Transformer Optimizations 00:36:45 Future Models & System-Level Focus 00:39:20 Inference-Time Computation & Reasoning 00:42:05 Capturing Human Thought in AI 00:43:15 Research, Hiring & Developer Advice REFS: 00:02:31 Cohere, https://cohere.com/ 00:02:40 The Transformer architecture, https://arxiv.org/abs/1706.03762 00:03:22 The Innovator's Dilemma, https://www.amazon.com/Innovators-Dilemma-Technologies-Management-Innovation/dp/1633691780 00:09:15 The actor model, https://en.wikipedia.org/wiki/Actor_model 00:14:35 John Searle's Chinese Room Argument, https://plato.stanford.edu/entries/chinese-room/ 00:18:00 Retrieval-Augmented Generation, https://arxiv.org/abs/2005.11401 00:18:40 Retrieval-Augmented Generation, https://docs.cohere.com/v2/docs/retrieval-augmented-generation-rag 00:35:39 Let’s Verify Step by Step, https://arxiv.org/pdf/2305.20050 00:39:20 Adaptive Inference-Time Compute, https://arxiv.org/abs/2410.02725 00:43:20 Ryan Greenblatt ARC entry, https://redwoodresearch.substack.com/p/getting-50-sota-on-arc-agi-with-gpt Disclaimer: This show is part of our Cohere partnership series