Dear Analyst #111: Master data management at AutoTrader and working with data in a merger with Korhonda Randolph
Listen now
Description
One topic that hasn't been covered on Dear Analyst is master data management (MDM). I'm surprised it took this long before someone brought it up. I've never heard of the term before and it looks like it's a core strategy for many large corporations for manager their data. Korhonda Randolph studied systems engineering at the University of Pennsylvania and started her career in engineering. She started specializing in master data management at companies like AutoTrader, Cox Automotive, and SunTrust/BB&T (merger). In this episode, Korhonda discusses what master data management is, data cleansing the CRM at AutoTrader, and the various data issues you have to work through during a merger between two banks. A "master" record in master data management The definition of master data management according to Wikipedia is pretty generic: Master data management (MDM) is a technology-enabled discipline in which business and information technology work together to ensure the uniformity, accuracy, stewardship, semantic consistency and accountability of the enterprise's official shared master data assets. After doing some quick research, MDM is closely associated with data quality and data governance. The cynical side of me says this is one of those disciplines that was created by data vendors way back when. But given the size and scope of the projects the MDM discipline is used in, it's very likely I just have never had any experience with people who have utilized this discipline. Source: Info-Tech Research Group At a high-level, the goal of MDM is very simply. Create a "master" record for a customer, product, or some other entity that doesn't change very much. Korhonda discusses working on customer data where properties like the first and last names of a customer would be an output of MDM. This data should stay consistent no matter what team or department is looking at the customer data. Data cleaning CRM data at AutoTrader AutoTrader was trendsetting in the field of data. Early on, data architects created their own MDM systems to manage customer data. If the MDM system is not created properly, then other systems would not function correctly. Korhonda's team was using Hadoop because AutoTrader works with many car dealerships who need data to help them with their businesses. Korhonda started as a project manager at AutoTrader helping coordinate all the moving parts of AutoTrader's MDM system. Eventually she became a solutions architect on the data side. I've talked about data cleaning in multiple episodes and I've discovered a few things about the process over the years: * Excel and SQL are still the main tools used for data cleansing * The same type of data problems exist at startups and large corporations alike At AutoTrader, they were trying to figure out if client A in the sales system was also client A in another system. There is missing data across systems, and AutoTrader would try to find 3rd-party data sources to fill the gaps in the customer data. They may even contact the customer directly to get the data the need. At the end of the day, this type of data problem is not unique to AutoTrader. To this day,
More Episodes
When you think of your data warehouse, the "semantic layer" may not be the first thing that pops in your mind. Prior to reading Frances O'Rafferty's blog post on this topic, I didn't even know this was a concept that mattered in the data stack. To be honest, the concept is still a bit confusing...
Published 09/10/24
Published 09/10/24
If you could only learn one programming language for the rest of your career, what would be it be? You could Google the most popular programming languages and just pick the one of the top 3 and off you go (FYI they are Python, C++, and C). Or, you could pick measly #10 and build a thriving career...
Published 08/05/24