Bytewax is a curious stream processing tool that blends a Python surface with a Rust core to produce something that's in a similar vein to Kafka Streams or Apache Flink, but with a fundamentally different implementation. This week we're going to take a look at what it does, how it works in theory, and how the marriage of Python and Rust works in practice… – The original Naiad Paper: https://dl.acm.org/doi/10.1145/2517349.2522738 Timely Dataflow:...

Mojo is the latest language from the creator of Swift and LLVM. It's an attempt to take some of the best techniques from CPU/GPU-level programming and package them up in a Python-compatible syntax. In this episode we explore why Mojo was created, and what it offers to Python programmers and non-Python programmers alike. How is it built for performance, and which performance features matter? What's its take on functional programming and type systems? And can it marry the high-level...

Every database has to juggle the need to process new data and to query old data. That task falls to any system that "does stuff and remembers stuff". But it's quite hard to really optimise one system for both use cases. There are different constraints on new and old data, and as a system gets larger and larger, those differences multiply to breaking point. That's something Twitter's engineers were figuring out in the 2010s. One solution that came up in those years was the Lambda...

Rust changed the discussion around memory management - this week's guest hopes to push that discussion even further. This week we're joined by Evan Ovadia, creator of the Vale programming language and collector of memory management techniques from far and wide. He takes us through his most important ones, including linear types, generation references and regions, to see what Evan hopes the future of memory management will look like. If you've been interested in Rust's borrow-check and want...

The "big data infrastructure" world is dominated by Java, but the data-analysis world is dominated by Python. So if you need to analyse and process huge amounts of data, chances are you're in for a less-than-ideal time. The impedance mismatch will probably make your life hard somehow.  So there are a lot of projects and companies trying to solve that problem. To bridge those two worlds seamlessly, and many of the popular solutions see SQL as the glue. But this week we're going to look at...

Erlang wears three hats - it's a language, it's a platform, and it's an approach to making software run reliably once it's in production. Those last two are so interesting I sometimes wonder why those ideas haven't been ported to every language going.  How much work would it be? This week we're going to dig right down into that question with Leandro Ostera. He's been working on Riot - a project to bring the best of Erlang's runtime system and philosophy to OCaml. But why OCaml? Is it...

The likes of LinkedIn and Uber use Pinot to power some astonishingly high-scale queries against realtime data. The numbers alone would make an impressive case-study. But behind the headline lies a fascinating set of architectural decisions and constraints to get there. So how does Pinot work? How does it process queries? How are the various roles split across a cluster? And equally important - what does it *not* try to achieve. Joining me to go through the nuts and bolts of how Pinot handles...

TJ DeVries is a core contributor to Neovim and several of its most interesting sub-projects, and he joins us this week to go in depth into how Neovim got started, how it's structured, and what a truly programmable editor has to offer programmers who want the perfect environment. Along the way we look at what we can learn from Neovim's successful fork of the 30-year old codebase from Vim, how it still collaborates with the original project, and what putting Lua at the heart of the system has...

Done right, a Hackathon can be a fantastic place to be a programmer - you get time and space to build and learn, in a room full of like-minded people, with swag and prizes to sweeten the deal. It's a great way to pick up new ideas and run with them. But done wrong it can be a waste of time. What's the difference between a good hackathon and a bad one? What do the good ones do right, and what can we learn from that? This week we're talking about the Joy of Hacks with Major League Hacking...

One of the most promising techniques for software reliability is property testing. The idea that, instead of writing unit tests we describe some property of our code that ought to always be true, then have the computer figure out thousands of unit tests that try to break that rule. For example, you might say, "No matter which page you visit on my website, there should always be a login button or a logout button." Then the test's job is to try to break that rule, but clicking around until it...

If you ever feel overwhelmed by the number of different programming languages, this week's episode might just offer you some solace, as we talk about an attempt to reunify many of the most popular languages by focussing on the bread & butter things that every language supports. I'm joined by Martin Johansen, who's been working on a new tool called Progsbase. With it, he's created a spec based on all the things programming languages can agree on, and is building a library that can...

A lot of programming is split into the mechanical work of writing what you know, and the creative work of figuring out what you don't know. Wouldn't it be nice to automate the mechanical stuff away? Well the good news is we're already automating a lot of it. Every time you run a refactoring tool or a pretty-printer, you're handing boring work off to the computer. But how does that magic work, and how can we do more of it? This week we're joined by one of the authors of OpenRewrite—Jonathan...

SQLite could do with a little competition, so when I invited the co-creator of DuckDB in to talk, I thought we'd be discussing the perils of trying to build a new in-process database engine. I quickly realised things went much deeper than just a tech refresh. Hannes Mühleisen joins me this week to blend his academic credentials as a database researcher with his vehement need to make that research practical. And so we dive into what modern database literature has to say on making queries...

This week we talk to Simon Peyton Jones, a veteran language designer and researcher, and key figure in the development of Haskell. Haskell. Simon has made countless contributions to advancement of functional programming, and computer programming in general, and is currently working at Epic Games, working on the foundations of their new programming language, Verse. We discuss how programming languages are made, focussing on a big design idea from both Haskell and Verse: building a large...

Benthos wants to be part of your Data Engineering toolkit - it's there as a quick and easy way to set up data pipelines and start streaming data out of A and into B. In contrast to a lot of the tools we've talked about on Developer Voices, Benthos seems focussed on cutting development time down to a minimum, so you can quickly configure a new pipeline and test it out, without making a whole sprint of the task. As quick as a quick-and-dirty shell script, without the dirt. 😉 So this week we're...

The world of game programming might seem a million miles away from 'regular' programming. But they still have to deal with the same kinds of data, scale and concurrency problems that we're all familiar with in the software world. And that makes the gaming world an interesting place for new ideas - under the hood they're solving those same problems we face, but often with some novel ideas about the solutions.  So this week we're off to the massive open world that is game development, to see...

Odin's creator, Bill Hall, makes some bold claims about the language, including that it's "programming done right". Before that starts a war on the internet, we'd best ask him to explain what that means, and how Odin tries to achieve it. And while we get deep into the details, overall his answer seems to be, "By gathering masses of feedback and then refining C until it feels joyous again. Of all the C-like languages we've looked at on Developer Voices, Odin seems to be the most at-ease with...

This week's guest describes Event Sourcing as, "all I'm going to use for the rest of my career." But what is Event Sourcing? How should we think about it, and how does it encourage us to think about writing software? In this episode we take a close look at systems designed around the idea of Events, with guest Bobby Calderwood. Bobby's been designing (and helping others design) event based architectures for many years, and enthusiastically recommends it not only as a system-design technique,...

One of our oldest languages meets one of our newest sciences in this episode, as we talk with Professor Christian Schafmeister, an award-winning nanotech researcher who's been developing a language and a design suite to help research the future molecular machines. In this episode Christian gives us a quick chemistry lesson to explain what his research is trying to achieve, then we get into the software that's doing it: A new flavour of Common Lisp. But why Lisp? What advantages does a 60...

Sometimes, what a programming language makes harder is just as important as what it makes easier. For a simple example, think of GOTO. We've been wisely avoiding it for decades because it makes confusing control flow desperately easy. Types and tests are other examples - they're as much about specifying what shouldn't work as what should. And perspective is what makes this week's topic particularly interesting: Roc is a language that's functional, fast, friendly, and extremely interested in...

One of the recurring themes in the big data & data streaming worlds at the moment is developer experience. It seems like every major tool is trying to answer this question: how do we make large-scale data processing feel trivial? In some places the answer is any library you like as long as it's Python. In other realms, a mixture of Java and SQL shows promise. But as this week's guest—Luca Pette—would say, the Unix design metaphor has plenty to give and keep on giving. So in this episode...

This week we're back on systems programming with Hare. A C-like language for the ages. We talk to its creator, Drew DeVault, about what he thinks we can learn from the past 50 years of programming, and how we can build that hindsight into a new language that will last for the next 100.  In among all that long-term ambition we talk cover everything from error handling, typed unions and linear types, to metaprogramming and Drew's microkernel operating system. It's called Ares, and it is, of...

A few months ago, Michael Drogalis quit his job and decided launch 4 viable startup business ideas in 4 months, publically documenting every step of the journey. Over here at Developer Voices it seemed fun, inspired, and just crazy enough to work. We had him on the podcast a few months back just as that journey was beginning, and since he launched his first startup things have changed,. The reception has been better than he expected and the plan has been updated to go all-in on idea number...

Integration is probably the last, hardest, and least well thought-out part of any large software project. So anything that makes the data-streaming job easier is worth knowing about. So this week we turn our attention to Apache Flink, a flexible system for grabbing, transforming and shipping data between systems using Java, Python or good ol' SQL. So this week Robert Metzger—Apache Flink expert and PMC member—joins us to explain what problems Flink solves and how it solves them reliably. We...