S02 E12 Unveiling Twilio’s Data Transformation: A Journey into Modern Data Stack with Don Oriti, Head of Data Platform and Engineering at Twilio
Description
Twilio has built an open source data lake using AWS technologies and Databricks, processing billions of events daily through their Kafka environment. They aim to provide a cohesive view of data across platforms and enable other businesses to use data wherever they want. Don, the Head of Data Platform and Engineering at Twilio, shares insights into Twilio's data stack in the latest episode of the Modern Data Show. The conversation covers the Twilio data stack, which begins with data ingestion through Kafka or CDC for Aurora databases, followed by storage in S3, high-level aggregation and curation using Spark, and the use of tools such as Kudu, Reverse ETL, data governance, cataloging, and BI tools.
Prepare to be amazed in this episode as Matteo Pelati and Vivek Gudapuri, the brilliant minds behind Dozer, reveal their experience in pushing the boundaries of data management and analysis. By simplifying the process of data serving and allowing companies to create APIs quickly and efficiently,...
Published 06/06/23
Uncover the secret to turning data engineering into a superpower! As Sean Knapp, the CEO and founder of Ascend.io, joined us and discussed the value of depth and breadth in capturing the entire data value chain, emphasizing the need for an automation layer to adapt to the evolving data landscape....
Published 05/30/23