Dear Analyst #89: Leading high performing data teams and deciphering the data stack with David Jayatillake
Description
Most episodes I have the privilege of speaking with analysts who are in the trenches using tools and doing analyses. In this episode, we look at the role of data from a manager/director of data's perspective. David Jayatillake is currently the Chief Product & Strategy Officer at Avora, an augmented analytics solution that helps companies make better decision. Not to say David doesn't get into the weeds when he has to (his GitHub profile tells that story), but it was interesting hearing how to build high performing data teams. We get into topics relating to data strategy, bundling and unbundling of data tools, and ways to be an effective manager.
Moving from data to product
For most of David's career, he's led business intelligence and analytics teams in various industries including retail, credit, and e-commerce. That's why I thought it was interesting when he moved into a product role at Avora.
As a leader of data teams, David was using various data tools to analyze, store, and transform data. Over time, you might start to see some of the deficiencies in these tools, and even have ideas for new product features. As an end user of some of these tools in my organization, I'm quite familiar with the little quirks of these tools that just make you shrug a little bit when you have to open the tool up in your browser.
David had been advising Avora on building various product features before moving full-time into the Chief Product Officer role at the company. In terms of transitioning to a product role, David had various PMs he could observe to learn the rituals of leading a product team like leading a standup and creating a product roadmap.
Source: Product Coalition
The great bundling debate
A few months ago, David wrote this blog post about the bundling and unbundling of tools for the data engineering profession. For those new to this world in data engineering (like me), this analogy sums it up nicely from the Analytics Engineering Roundup:
Unbundling means taking a platform that aggregates many different verticals, such as Craigslist, and splitting off one or more services into stand-alone businesses that are more focused on the experience of that niche customer base. Sometimes, as was the case with Airbnb, that experience is so much better it eclipses the market cap of the original business.Anna Filippova, Analytics Engineering Roundup
Taking this concept to the modern data stack, David's blog post discusses the unbundling of Airflow into many off-the-shelf and open-source tools. I've never worked in a data engineering role and was worried that this part of the discussion would go way over my head. I hear about ETL, streaming, and automation from being in the space but have only lightly touched some of these tools in this data stack:
Source: Analytics Engineering Roundup
David's blog post also discusses the role the data engineer plays in a world of unbundled software:
For most companies, the cost of hiring and keeping data engineers working on pipelines that could be built with specialist “unbundled” tooling is unwarranted. The real value is in the next steps of data-enabled decision making, whether human or automated: Is this sneaker actually the same as this other sneaker? Should we lend to this customer?
When you think of your data warehouse, the "semantic layer" may not be the first thing that pops in your mind. Prior to reading Frances O'Rafferty's blog post on this topic, I didn't even know this was a concept that mattered in the data stack. To be honest, the concept is still a bit confusing...
Published 09/10/24
If you could only learn one programming language for the rest of your career, what would be it be? You could Google the most popular programming languages and just pick the one of the top 3 and off you go (FYI they are Python, C++, and C). Or, you could pick measly #10 and build a thriving career...
Published 08/05/24