Implementing Hardware-Friendly Databases (with DuckDB co-creator, Hannes Mühleisen)
Listen now
Description
SQLite could do with a little competition, so when I invited the co-creator of DuckDB in to talk, I thought we'd be discussing the perils of trying to build a new in-process database engine. I quickly realised things went much deeper than just a tech refresh. Hannes Mühleisen joins me this week to blend his academic credentials as a database researcher with his vehement need to make that research practical. And so we dive into what modern database literature has to say on making queries faster, more parallelizable, and closer to the metal, and how it all comes together in a user-friendly package that's found its way into my day-to-day workload, and might well help out yours. If you're curious about the gory details of database queries, how they can take advantage of modern hardware, or how all that research actually turns into a useful tool, Hannes has some great answers. -- DuckDB: https://duckdb.org/ Database Systems Book: http://infolab.stanford.edu/~ullman/dscb.html Kris' first computer: https://en.wikipedia.org/wiki/File:ZX_Spectrum_Plus2_(retouched).jpg Volcano Query Evaluation System [pdf]: https://paperhub.s3.amazonaws.com/dace52a42c07f7f8348b08dc2b186061.pdf Morsel Query Engine [pdf]: https://cs.brown.edu/~kayhan/papers/morsel_cp.pdf Unnesting Arbitrary Queries [pdf]: https://cs.emis.de/LNI/Proceedings/Proceedings241/383.pdf Papers Hannes' team have published: https://duckdb.org/why_duckdb#peer-reviewed-papers-and-thesis-works DuckDB on Mastodon: https://mastodon.social/@duckdb Kris on Twitter: https://twitter.com/krisajenkins Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/ Kris on Mastodon: https://mastodon.social/@krisajenkins -- #softwaredevelopment #podcast #programming #database #duckdb #sql #sqlite
More Episodes
The actor model is a popular approach to building scalable software systems. And isn't hard to understand when you're just reading about the beginner's examples. But how do you architect a complex design using the actor model? Which patterns work well? How do you think through it? Joining me to...
Published 05/15/24
Bytewax is a curious stream processing tool that blends a Python surface with a Rust core to produce something that's in a similar vein to Kafka Streams or Apache Flink, but with a fundamentally different implementation. This week we're going to take a look at what it does, how it works in...
Published 05/08/24