Is Flink the answer to the ETL problem? (with Robert Metzger)
Listen now
Description
Integration is probably the last, hardest, and least well thought-out part of any large software project. So anything that makes the data-streaming job easier is worth knowing about. So this week we turn our attention to Apache Flink, a flexible system for grabbing, transforming and shipping data between systems using Java, Python or good ol' SQL. So this week Robert Metzger—Apache Flink expert and PMC member—joins us to explain what problems Flink solves and how it solves them reliably. We cover the range from simple use cases to realtime aggregations & joins to its high availability strategy. If you're working on systems that include more than one database, then you're definitely going to face the kinds of problems that Flink tackles. -- Apache Flink: https://flink.apache.org/ Robert on Twitter: https://twitter.com/rmetzger_ Robert on LinkedIn: https://www.linkedin.com/in/metzgerrobert/ Kris on Twitter: https://twitter.com/krisajenkins Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/ – #software #programming #podcast #flink #apacheflink #dataintegration
More Episodes
This week on Developer Voices we're talking to Ryan Worl, whose career in big data engineering has taken him from DataDog to Co-Founding WarpStream, an Apache Kafka-compatible streaming system that uses Golang for the brains and S3 for the storage. Ryan tells us about his time at DataDog, along...
Published 05/29/24
Published 05/29/24
PostgreSQL is an incredible general-purpose database, but it can't do everything. Every design decision is a tradeoff, and inevitably some of those tradeoffs get fundamentally baked into the way it's built. Take storage for instance - Postgres tables are row-oriented; great for row-by-row access,...
Published 05/22/24