Kafka: LinkedIn's Distributed Messaging System
Listen now
Description
This episode focuses on Kafka, the distributed messaging system born at LinkedIn. Learn how Kafka was designed to tackle the massive streams of log data driving personalized recommendations, search algorithms, and real-time security. We'll explore how it outperforms traditional systems like ActiveMQ and RabbitMQ with its streamlined architecture, decentralized coordination, and focus on efficiency. Tune in to explore Kafka's unique design and how it’s becoming essential for modern data processing.
More Episodes
In this episode, we'll take a look at Meta’s ambitious approach to scaling large language models. We'll explore the shift from handling many smaller models for recommendation engines to building colossal generative AI models, and the immense challenges that come with it. From hardware and...
Published 10/24/24
In this episode, let's explore how Netflix revamped their video processing pipeline, moving from a monolithic system to a microservices architecture. What drove such a major shift? You'll hear how their original platform, Reloaded, couldn’t keep up with Netflix’s rapid pace of innovation, and why...
Published 10/23/24
Published 10/23/24