Data Mesh Radio

Taking a needed break to focus on getting healthy. Be back in August!

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Mirela's LinkedIn: https://www.linkedin.com/in/mirelanavodaru/ In this episode, Scott interviewed Mirela Navodaru, Enterprise and Solution Architect for Data, Analytics, and AI at Swisscom. Some key takeaways/thoughts from Mirela's point of view: Specifically at Swisscom, it's not about doing data mesh. They want to make data a key part of all their major decisions - operational and strategic - and data mesh means they can put the data production and consumption in far more people's hands. Data mesh is a way to achieve their data goals, not the goal. When you are trying to get people bought in to something like data mesh, you always have to consider what is in it for them. Yes, the overall organization benefiting is great but it’s not the best selling point 😅 try to develop your approach to truly benefit everyone. Data literacy is crucial to getting the most value from data mesh. Data mesh is not about throwing away the important knowledge your data people have but it's about unlocking the value of the knowledge your business people have to be shared with the rest of the organization effectively, reliably, and scalably. ?Controversial? You really have to talk to a lot of people early in your data mesh journey to discover the broader benefits to the organization. That way you can talk to people's specific challenges to get them bought in. When designing your journey, it is important to get input from a large number of people. When talking data as a product versus data products, the first is the core concept and the second is the deliverables. Scott note: this is a really simple but powerful delineation "No value, no party." If there isn't a value proposition, there shouldn't be any action. You need to stay focused on value because there are so many potential places to focus in a data mesh implementation. You have to balance value at the use case level to the domain versus more global value to the organization. At the end of the day, everything you do should add value to the organization but sometimes use cases are much more focused at the domain and that's perfectly expected and acceptable. Data mesh, to really change the organization in the right way, needs top level buy-in. You can't only be the data team trying to head down the data mesh path. Everything in data mesh is about iterating to better. You need the space and room to learn as you go along. You can - and must - deliver value before you've got everything figured out perfectly. Relatedly, you will learn how to better iterate towards value throughout your journey. It will be tough at the start as with any learning journey. Obviously, data mesh is a large cultural change. You need to have empathy and give people the chance to grow instead of trying to move too fast. Upskilling, especially around data literacy, is crucial. There are two very valuable aspects of data mesh: the value you deliver via use cases along the way and the value you get from learning to do data better across your organization. The first is from integrating data into far more of your decisions and the second means you can react more quickly to new opportunities and build scalable and reliable approaches to data management. Something like data mesh is a big change. But it shouldn't be a shock to people. You can do it gradually and incrementally while you deliver value. One of the best ways to lose people is to thrust disruptive change on them instead of working with them through the change to prevent large-scale negative disruptions. There are so many areas where data mesh helps organizations, whether it is getting away from silos, reducing redundancy, improving quality and reliability, etc. It's not just about doing data management itself better, which has been the focus of most data approaches historically. Again, data work is not the point. The point is to make your colleagues better at their job through being more informed. That comes down to the data but it's never the actual point, it's the vehicle to delivering value. Transparency and managing expectations - and communication in general - are crucial to doing data mesh well. You need to have that space to learn and iterate. Let people know what you are doing and especially why you are doing it. Data modeling in data mesh is of course a challenge. But it's important to have some level of common language between the domains or you will have data silos. It's a balance but it's crucial to give domains flexibility but also create easy paths for people to combine data across domains. Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Alyona's LinkedIn: https://www.linkedin.com/in/alyonagalyeva/ In this episode, Scott interviewed Alyona Galyeva, Principal Data Engineer at Thoughtworks. To be clear, she was only representing her own views on the episode. Some key takeaways/thoughts from Alyona's point of view: ?Controversial? People keep coming up with simple phrasing and a few sentences about where to focus in data mesh. But if you're headed in the right direction, data mesh will be hard, it's a big change. You might want things to be simple but simplistic answers aren't really going to lead to lasting, high-value change to the way your org does data. Be prepared to put in the effort to make mesh a success at your organization, not a few magic answers. !Controversial! Stop focusing so much on the data work as the point. It's a way to derive and deliver value but the data work isn't the value itself. Relatedly, ask what are the key decisions people need to make and what is currently preventing them from making those decisions. Those are likely to be your best use cases. When it comes to Zhamak's data mesh book, it needs to be used as a source of inspiration instead of trying to use it as a manual. Large concepts like data mesh cannot be copy/paste, they must be adapted to your organization. It's really important to understand your internal data flows. Many people inside organizations - especially the data people - think they know the way data flows across the organization, especially for key use cases. But when you dig in, they don't. Those are some key places to deeply investigate first to add value. On centralization versus decentralization, it's better to think of each decision as a slider rather than one or the other. You need to find your balances and also it's okay to take your time as you shift more towards decentralization for many aspects. Change management is best done incrementally. ?Controversial? A major misunderstanding of data mesh that some long-time data people have is that it is just sticking a better self-serve consumption layer on top and we can continue to do monolithic data work under the hood. Be prepared for lots of friction in convincing some data architects that this isn't just a reskin or another layer on top of the enterprise warehouse or data lake. For data mesh, it's crucial to understand necessary changes at the technical and the business level. You can't only work on one but you also don't have to 'solve' them at the start, make progress. It's like with the four principles, you need to thin slice change across the technical and business aspects rather than only focusing on one. You can sell data engineers on data mesh by making their work more meaningful and impactful. Instead of mostly firefighting - which is the case in many organizations - they can focus on shipping new features and adding incremental value. With data mesh, you want people focusing on more than just making data valuable - what is valuable will change so how do you make your data products evolvable and maintainable? You always want to be focused on addressing people's pain points in data mesh, driving towards value. That's how you can get data people bought in as well, not just business people. !Controversial! Doing aggregated data products across domains is usually the data mesh inflection point - basically answering can data mesh work in your organization. If you can't get that cross domain collaboration going well, you should consider another model like hub and spoke. Relatedly, aggregating data across multiple domains is where there is usually the most value for an organization. But it's very hard to find good champions there because you need more vision and more hard work to collaborate across domain boundaries. Identify the people with the vision early in your journey, even if it's often better to actually only start working with them once you have more momentum and data products. Too often, there is a rush to build _something_ instead of the right thing. Don't get fooled by the idea that data work always creates value. Even if the client or business partner asks you to build something of value, always circle back to the use cases. As much as we'd like to build universal data products, they just don't exist. Relatedly though, don’t get so focused only on trying to build the consumer-aligned data products for hyper-specific use cases that you miss the forest for the trees. Sometimes the use case is something like 'we need to understand what data we even have to be able to use it to address our current problems in XYZ business line.' To kind of sum it up: stop focusing on what you can build first. Focus on what you should build and then look at the realities. What matters to the business and why? Then focus on what's possible and what will deliver sustainable and maintainable value through data work. Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Arne's LinkedIn: https://www.linkedin.com/in/arnelaponin/ Chris' LinkedIn: https://www.linkedin.com/in/ctford/ Foundations of Data Mesh O'Reilly Course: https://www.oreilly.com/videos/foundations-of-data/0636920971191/ Data Mesh Accelerate workshop article: https://martinfowler.com/articles/data-mesh-accelerate-workshop.html In this episode, Scott interviewed Arne Lapõnin, Data Engineer and Chris Ford, Technology Director, both at Thoughtworks. From here forward in this write-up, I am combining Chris and Arne's points of view rather than trying to specifically call out who said which part. Some key takeaways/thoughts from Arne and Chris' point of view: Before you start a data mesh journey, you need an idea of what you want to achieve, a bet you are making on what will drive value. It doesn't have to be all-encompassing but doing data mesh can't be the point, it's an approach for delivering on the point 😅 Relatedly, there should be a business aspiration for doing data mesh rather than simply a change to the way of doing data aspiration. What does doing data better mean for your organization? What does a "data mesh nirvana" look like for the organization? Work backwards from that to figure where to head with your journey. A common early data mesh anti-pattern is trying to skip both ownership and data as a product. There are existing data assets that leverage spaghetti code and some just rename them to data products and pretend that's moved the needle. "A data product is a data set + love." The real difference between a data product and a data set is that true ownership and care. ?Controversial?: Another common mesh anti-pattern is trying to get too specific with definitions or prescriptive advice. There isn't a copy/paste approach that will work and getting a specific definition of a data product doesn't really change much. Mindset is far more important than definitions. It can be very helpful to have some simple checklists around your data products. While there is no prescriptive way to build, checklists remove a lot of the uncertainty for teams asking 'am I doing this right?' It gives some simple reassurances that you aren't missing out on key pieces of what they're building. ?Controversial?: Most organizations probably don't need to do a ton of pre-work before starting on a data mesh implementation. They need some achievable goals, a roadmap for how they plan to achieve those goals, and a lot of willpower to push things forward and keep going when the going gets tough. You also need an enticing vision for people to buy into. THIN SLICE! Don't try to take everything on at once but also don't try to skip over any of the four pillars. There's a reason they haven't changed from Zhamak's initial blog post. Scott note: don't try to argue the governance pillar wasn't in the first blog post, it just wasn't called out separately… Three key questions to answer if you are considering data mesh: A) Do you have sufficient scale? B) Do you have a strategy that depends on deriving value from data? C) Are you prepared to take advantage of the autonomy Data Mesh will afford to your product teams? If you don't have satisfactory answers to those three questions, data mesh is probably not right/overkill for you. If people don't see the strong need to transform your business through data, it's likely to lead to troubles 6-9 months into your data mesh journey. If you aren't addressing key organizational pain points or delivering value, you will likely lose support for your data mesh implementation initiative. Doing data better has to be valued to get more budget to keep going. Another anti-pattern is focusing too much on use cases at the expense of the platform and the journey. Data mesh is designed to work at scale and that only works by finding repeatable processes. You can't treat each data product like a one-off. In order to get buy-in from the data engineers - or whoever are your data product developers - you need to invest in changing hearts and minds through the platform. If creating and managing data products is significantly harder than the old way of dealing with data, you will lose people quickly. Read about the data mesh accelerate workshop 😅 When you think about first steps with data mesh, A) build buy-in at the strategic level that you want to actually start leveraging your data for high-value purposes; B) find use cases to support those strategic initiatives; and C) make sure you are ready to actually thin slice and not try to only tackle on pillar - you have to be ready to take on a LOT of challenging work. !Controversial!: None of the four data mesh principles are all that useful on their own. Scott note: there's a figure Zhamak has that explains why all four are necessary in conjunction that's very helpful here. It's easy to want to skip bringing all your key stakeholders into alignment early in your data mesh journey. But you need matching expectations and shared understandings of what you are trying to accomplish and why. Scott note: this doesn't mean everyone has to be bought in that they are first, there's a balance to be found here. You need room to make mistakes and adjust your data mesh implementation because you will not get it all right at the start. Data mesh is as much about learning how to do data well as doing data well. It's crucial to not just ask if you are succeeding with your data mesh implementation but measure that. It can be hard to measure but consider what matters to your implementation's success and find things to measure if you're succeeding in each of those areas. Otherwise, how do you know where to focus and optimize? Subsidiarity: "everything should be decided as locally as possibly but no more so." Basically, there are many decisions that should be made in the domains but there are some that need to be made centrally. The challenge is figuring out which decisions should be made where 😅 There will be capability challenges in every organization when doing data mesh. That will impact initial decisions around how much to centralize or decentralize but as you upskill the teams, you may want to decentralize more. Find your equilibriums but equilibriums change. It's all about trade-offs! Many people are too focused on exactly if they are doing data mesh instead of are they delivering value in a scalable way through data. That happened in microservices too and it took them 10+ years to really get to best practices. Data mesh is only 5 years old and only ~3 with any number of organizations attempting it - focus on getting better instead of being worried if you're the perfect picture of data mesh yet. When talking about your data mesh success internally, you need to talk about the value from use cases AND the value of improving your data capabilities in general. You prove out you are delivering specific value along the way but also that you are getting more and more capable at doing the data work to make the organization better. Both are of valuable and you should promote the value of both aspects: use case value and capability value. When talking about data mesh, use the ADKAR method: create Awareness and Desire, give them the Knowledge about how you're doing it, upskill people so they have the Ability to do data mesh, and finally constantly Reinforce the value and that it's important. Without touting your mesh successes, you'll lose momentum. When looking for your first data mesh use case, look for something that has a customer impact - what can you do for them that you couldn't before. Personalization is a good example. Legal is potentially another place re reducing risk. Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Saba's LinkedIn: https://www.linkedin.com/in/sabaishaq/ Decide Data website: ttps://www.decidedata.com/ In this episode, Scott interviewed Saba Ishaq, CEO and Founder of her own data as a service consultancy, Decide Data, which also provides 3rd party DAaaS (Data Analytics as a Service) solutions. Some key takeaways/thoughts from Saba's point of view: "If you don't know what you want, you're going to end up with a lot of what you don't want." This is especially true in collaborating with business stakeholders when it comes to data 😅 Focus on delivering value through data instead of delivering data and assuming it has value. – “Not all data is created equal.” As a data leader, it's your role to help people figure out what they actually want by asking great questions and being a strong partner when it comes to the data/data work. Don't only focus on the data work itself but it's very easy to do data work for the sake of it instead of something that is valuable. To deliver data work that actually moves the needle, we need to start from what are the key business processes and then understand the pain points and opportunities. Then, good data work is about how do we support and improve those business processes. Relatedly, that's also the best way to drive exec alignment - talking about their business processes and how they can be improved first, data work second. They will feel seen and heard and are far more likely to lean in. At the end of the day addressing business and operational challenges is what data and analytics is all about. Deliver something valuable early in any data collaboration with a business stakeholder. You don't have to deliver an entire completed project but time to first insight is time to value and you build momentum and credibility with that stakeholder. At the beginning of a project - and delivering a data product is itself a project - you should work with stakeholders to not just define target outcomes but also define how are you going to collaborate and communicate. You can't just get requirements, go away and build. Working with data should be iterative and should have an element of continuous improvement to evolve what you deliver as you build value. Start any data work by asking someone about their business objectives, challenges, and target outcomes. You need your business stakeholders to have a clear vision of what they want to achieve, otherwise you are likely to be delivering only data work instead of business value that leverages your data work. By doing deep discovery work, you can find where are the key lynchpins and value drivers in a use case. There are points of criticality that are easy to lose in a sea of potential requirements that are really requests. Find those crucial value leverage points! Relatedly, you can use those value leverage points to keep your business execs engaged. They will - hopefully - see the importance and help you narrow in on what matters in their use case. Then it's no longer about the data work but the value to them. ?Controversial?: For data people, you have to balance career management and interesting project/technology work versus value delivery. That doesn't mean delivering value isn't interesting but it doesn't always mean getting to play with the latest and greatest. But if data people never get to have fun and play with cool tech, many will leave. It's a tough balance. Try to make the valuable work also interesting 😅. Relatedly, try understanding the data team’s learning areas of interest and see how you can build seeds to foster their skill growth while making data work valuable. Sometimes it turns out to be a win-win situation. Relatedly, be very transparent and communicate a lot to your data teams about what you are prioritizing and why. It's very easy to get lost in telling data people to do certain work rather than why they are doing that work. Keeping your data people in the loop of the why will keep them focused on what matters. For many organizations, the rate of change of their technology - application and data technologies - is growing at faster than the rate of their people change management/transformation processes. You need parallel streams to modernize both or your people will fall further behind, leading to chaos. ?Controversial?: Relatedly, your overall org and/or digital transformation strategy should be tied to your data strategy. Otherwise, they will likely be heading in different directions, creating more challenges. Scott note: Benny Benford talked a lot about this in episode #244, going far together. Data management is a very crucial element of digital transformation but it’s not the same thing as change management. The data team shouldn't be the ones leading the overall digital transformation of the organization. That's too much on a team that specializes in data rather than change management. If you are in that situation, it's a very tough spot to do well. It's very important to focus on communication to stakeholders when you think about data governance and digital transformation. For many execs, these are foreign topics so you have to work hard to engage them and keep them leaning forward on the necessary work. Data governance is beneficial for everyone, so if explained and defined well people will engage willingly after knowing what’s in it for them. As someone in the data team, you have to be well informed about digital transformation initiatives inside your organization. Otherwise, you will miss opportunities to align to those initiatives AND have all your data sources break when there is a migration you weren't told about 😅 It's easy to screw up the data steward/ownership conversation letting someone know they are responsible for the governance of their data. It's often a scary conversation for both parties. But it's necessary and you can show people why it makes sense and adds value to their work too! Relatedly, link people's pain points to current weaknesses in the data governance. Show them they are causing issues for themselves and give them an easier path to fix it without having to learn everything about data work. Data governance doesn't have to be some wholly - or holy 😅 - separate practice. It should just be part of normal work related to data. Make it less scary and more approachable for your business stakeholders. It's a team effort and it drives real, measurable benefits and value. Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Craziness of the overseas move (including a faulty office chair... long story) are to blame. Back to the normally scheduled one episode a week next week! Episode list and links to all available episode transcripts here.

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Basten's LinkedIn: https://www.linkedin.com/in/basten-carmio-2585576/ In this episode, Scott interviewed Basten Carmio, Customer Delivery Architect of Data and Analytics at AWS Professional Services. To be clear, he was only representing his own views on the episode. Some key takeaways/thoughts from Basten's point of view: Your first use case - at the core - should A) deliver value in and of itself and B) improve your capabilities to deliver on incremental use cases. That's balancing value delivery, improving capabilities, and building momentum which are all key to a successful long-term mesh implementation. When thinking about data mesh - or really any tech initiative - it's crucial to understand your starting state, not just your target end state. You need to adjust any approach to your realities and make incremental progress. ?Controversial?: Relatedly, it's very important to define what success looks like. Doing data mesh cannot be the goal. You need to consider your maturity levels and where you want to focus and what will deliver value for your organization. That is different for each organization. Scott note: this shouldn't be controversial but many companies are not defining their mesh value bet… Even aligning everyone on your organization's definition of mesh success will probably be hard. But it's important to do. For a data mesh readiness assessment, consider where you can deliver incremental value and align it to your general business strategy. If you aren't ready to build incrementally, you aren't going to do well with data mesh. A common value theme for data mesh implementations is easier collaboration across the organization through data; that leads to faster reactions to changes and opportunities in your markets. Mesh done well means it's far faster and easier for lines of business to collaborate with each other - especially in a reliable and scalable way - and there are far better standard rules/policies/ways of working around that collaboration. But organizations have to see value in that or there may be mesh resistance. As many have said, you must approach data mesh in a thin slice. Trying to focus too much on any pillar at the expense of the others leads to challenges. Scott note: Zhamak literally has a figure on this she shares often. It's easy to get unbalanced if you ignore a principle and fixing that takes more effort than thin slicing. ?Controversial?: As you build out your mesh capabilities, especially your platform, think about what you need to actually deliver on the use case(s) at hand and deliver only that. Don't get ahead of yourself. It's fun to build capabilities but it's easy to build a monstrosity of a platform instead of proper abstractions to make building and managing data products simple. Doing data mesh well is all about managing trade-offs. There are trade-offs at the use case and implementation level. And it's okay to get those wrong, just look to fail fast rather than hold on to bad decisions. Don't be precious with your decisions and build in ways to make evolving and improving easier. MVP can be minimum viable product or minimum valuable product. You need to define what value actually means for each use case. It's easy to point to pain and start solutioning but not end up addressing the pain or creating value. This will help you prioritize and deliver the most value compared to effort early. Relatedly, having a clear perspective on value in your MVP means you are more easily able to change and adapt as you learn and deliver. If you thought value was going to come from X but early indications are Y is way more important or is much different than expectations, you can more easily pivot towards Y. Data products don't magically create value. They should be bets on value delivery. So you need to have ways to gracefully retire data products when the cost exceeds the incremental value. Often times a bet was a good bet even when it didn't pay off or it is no longer paying off. Continuous communication and driving buy-in, especially with C-level execs, is unfortunately crucial to your data mesh implementation success. If they don't value better data capabilities, few people will invest their time and effort to really reinvent the way your organization does data. Relatedly, you need to tie your data mesh implementation to the overall business strategy. Execs need to know why you are doing something like data mesh and how it will deliver value. It's not an overnight success but it also can't be a 3 year project before value. Speaking to that gets more people to lean in so you can build momentum. Finding and leveraging your data success champions is always hard but it's necessary to build that momentum. Think about building champions quite early. When you're trying to get an exec to buy in to something like data mesh, always put it in terms of what's in it for them. Don't try to get them bought in to some grand vision first, why would they invest their valuable time and resources here? Find their pain points and what something like data mesh will do to address those specifically. Relatedly, trying to get someone to just take on data ownership without the understanding of what the rest of the organization is going to do to make ownership much easier - mainly through the platform and federated governance - is probably going to be a … not fun conversation. When thinking about data mesh, it should all come back to value. If you can't specifically point to value that will exceed your effort for doing something like data mesh, it's probably not worth it for your organization. What is valuable to your organization and how will data mesh help you capture that value? Relatedly, as you are on your data mesh journey, you have to be honest in assessing if data mesh is really working for your organization. It's okay to stop if it's not. The data mesh principles sound really great and helpful in the abstract. But they just might not be aligned to value with the way your company does business. ?Controversial?: It's perfectly acceptable to have hybrid approaches in a data mesh journey. There is a perception that everything should be decentralized and that just isn't always the best approach, don't get caught up in dogma. Scott note: the "decentralize everything" is a misunderstanding of Zhamak. Also, as you are on a journey, your current state won't match an ideal end state. Focus on evolving while delivering value, not trying to be perfect today instead of _getting_ to good. Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Olga's LinkedIn: https://www.linkedin.com/in/olga-maydanchik-23b3508/ Walter Shewhart - Father of Statistical Quality Control: https://en.wikipedia.org/wiki/Walter_A._Shewhart William Edwards Deming - Father of Quality Improvement/Control: https://en.wikipedia.org/wiki/W._Edwards_Deming Larry English - Information Quality Pioneer: https://www.cdomagazine.tech/opinion-analysis/article_da6de4b6-7127-11eb-970e-6bb1aee7a52f.html Tom Redman - 'The Data Doc': https://www.linkedin.com/in/tomredman/ In this episode, Scott interviewed Olga Maydanchik, an Information Management Practitioner, Educator, and Evangelist. Some key takeaways/thoughts from Olga's point of view: Learn your data quality history. There are people who have been fighting this good fight for 25+ years. Even for over a century if you look at statistical quality control. Don't needlessly reinvent some of it :) Data literacy is a very important aspect of data quality. If people don't understand the costs of bad quality, they are far less likely to care about quality. Data quality can be a tricky topic - if you let consumers know that the data quality isn't perfect, they can lose trust. But A) in general, that conversation is getting better/easier to have and B) we _have_ to be able to identify quality as a problem in order to fix it. Data quality is NOT a project - it's a continuous process. Even now, people are finding it hard to use the well-established data quality dimensions. It's a framework for considering/measuring/understanding data quality so it’s not very helpful to data stewards / data engineers in creating data quality rules. The majority of quality errors are not random, they come from faulty data mapping / bugs in pipelines. Having good quality rules will catch a large percentage of errors that can be fixed in bulk. When thinking about getting started around data quality, it doesn't have to be complex and with lots of tools. It can be people looking at the data for potential issues and talking to producers. Then you can build a business case for fixing the data to get funding. You have to roll up your sleeves and talk to people but you can get forward momentum. Data quality issues aren't inherently material to the business processes - they are only bad when they cause issues for the business. You have to find those actual business issues to get people to care and get funding for fixing it. Quality for the sake of quality is just extra cost. Do not create too many data quality rules that do not matter. Relatedly, being able to show someone a relatively basic quality indicator early is far better than asking for a lot of budget to figure out the quality levels. You can do that with something as simple as random sampling 100-200 records and an hour of 1-2 people's time. To understand which data quality challenges and use cases are the most important, data people simply have to learn more about the business. Good data quality is about fit for purpose and that means understanding the purposes :) To find your initial good data quality use cases, look to mission criticality. What dashboards or reports are actually important to the company and why? Then work backwards to see if quality is an issue for those dashboards and reports. That's how you find your early buy-in to work on a quality initiative that can scale. !Controversial!: Data contracts are not at all new, we just now have a good enough set of tools and technologies to be able to do them better at scale. ?Controversial?: Most are doing data contracts … not that well. For them, it's about the technology and not the process. There isn't a continuous approach. Scott note: Andrew Jones has said the same. It's about ensuring a process that results in quality data, not the tools. For data contracts, there MUST be a feedback loop or we aren't actually delivering to needs, especially as needs evolve. Look to the widely used customer supply model for insights into what we need to achieve and how when it comes to data contracts. Many companies are creating actual financial incentives tied to data quality in order to ensure people care about data quality. That's not right for every organization but it does send a clear message as to the importance of data quality. You have to consider your data supply chain - if your interface for data input is bad, your data is very likely to be bad. People will simply enter garbage to move forward. Doing data quality manually is not sustainable/scalable. But you don't need to start with expensive tools, you can get your arms around things initially pretty easily. It will help you identify your actual problems instead of spending time specifically on tools. ?Controversial?: Many vendors are selling their tools as the fix to data quality. But detecting data errors with the tools is only the start of the data quality improvements. Once errors are detected, root cause analysis for the errors needs to be performed and the processes / code need to be fixed. None of the data quality tools can do this. It is human’s job. Beware the snake oil. Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Michael's LinkedIn: https://www.linkedin.com/in/mjtoland/ Marta's LinkedIn: https://www.linkedin.com/in/diazmarta/ Sadie's LinkedIn: https://www.linkedin.com/in/sadie-martin-06404125/ Sean's LinkedIn: https://www.linkedin.com/in/seangustafson/ The Magic of Platforms by Gregor Hohpe: https://platformengineering.org/talks-library/the-magic-of-platforms Start with why -- how great leaders inspire action | Simon Sinek: https://www.youtube.com/watch?v=u4ZoJKF_VuA In this episode, guest host Michael Toland Senior Product Manager at Pathfinder Product Labs/Testdouble and host of the upcoming Data Product Management in Action Podcast facilitated a discussion with Sadie Martin, Product Manager at Fivetran (guest of episode #64), Sean Gustafson, Director of Engineering - Data Platform at Delivery Hero (guest of episode #274), and Marta Diaz, Product Manager Data Platform at Adevinta Spain. As per usual, all guests were only reflecting their own views. The topic for this panel was how to treat your data platform as a product. While many people in the data space are talking about data products, not nearly as many are treating the platform used for creating and managing those data products as a product itself. This is about moving beyond the IT services model for your data work. Platforms have life-cycles and need product management principles too! Also, in data mesh, it is crucial to understand that 'platform' can be plural, it doesn't have to be one monolithic platform, users don't care. Scott note: As per usual, I share my takeaways rather than trying to reflect the nuance of the panelists' views individually. Scott's Top Takeaways: You will hear "product mindset" a lot in this panel. It's important to embrace product management as a mindset and not an exact set of things to do and approaches to take. The whole point of a product mindset is to find what works and deliver reliably while focusing on value. Be ruthless. Ruthlessly prioritize and ruthlessly focus on user centricity. In data, we have a tendency to fall in love with the tools instead of the jobs to be done. But a good platform is about making it easy to get high-value work done and that's rarely about exposing tooling to users. Your platform isn't going to magically drive usage. There is change management required to get people to leverage your data platform more. Be prepared for that change management work and closely aligning with users, whether they are 'data people' or not. Platforms and products in general are about scalability. Through a platform, instead of reacting to tickets, you build services and capabilities to address the problems that caused the tickets - far more scalable than reacting to the individual tickets, addressing the disease not the symptom. By building your platform as a product, you focus on what actual capabilities are within scope so you can continue to manage and expand your data platform. As much as it would be amazing to build a data platform from scratch - think how amazing you could build it! - in many cases, you will have to build off of existing services and platforms. Don't be too dogmatic - what matters is continuing to get better not being perfect. Give yourself the space to improve the platform but products live in the real world and the real world of your organization has current/existing business needs and constraints. Products - especially in software - are as much or more about evolution as they are about their form/function at initial launch. The same should happen with your platform. And the same should happen with your vision. Don't get locked in, don't get tunnel vision. Generally speaking, most data platforms are still not serving the role a software platform - data or otherwise - should serve. They are still about the tech instead of the capabilities. You aren't alone if your platform doesn't meet the ideal vision. Your role is to make it better but it's still probably not going to be a sparkling beacon of perfection anytime soon 😅 Your data platform needs to be aligned to your company culture. So you have to meet people where they are and properly set down the easy path to where you want them to go. It's a long journey. Other Important Takeaways (many touch on similar points from different aspects): The product mindset is for the entire team, not just the platform leader and/or product manager. Your entire team will have to change their approach and mindset. Not overnight but it's hard to break old ticket-taker habits. Make sure to put your work in the context of the business. It's easy to get bogged down in the platform engineering aspects or the data tools but your data platform is there to serve business purposes. Focus on what delivers value for the business. Your platform doesn't have to address all your organization's data challenges at once, right at the start. Find where you are seeing the biggest challenges with delivering value - maybe listen back to episode #297 on the Data Value Chain - and look to focus there first. Build up to better. "Build with [your users], not for them." Don't treat your platform as a project. Yes, that's the subject of the panel but it's very important to always keep close at heart. User experience is such a crucial aspect of good software platforms. It's probably even more important when it comes to data platforms if you want to bring new users to producing and consuming data. But it's HARD and rarely discussed. What is the goal of your platform? That might sound like an obvious question but it's not really when you go to answer it. Maybe it's "make it easier to work with data" but easier for who? Really map out good outcomes of a well-made platform. Good products - at least good software products - aren't designed to be perfect at launch. Give yourself the space to not be perfect but set yourself up to understand where things can be improved and then iterate. Test, learn, iterate. What are your data platform KPIs? What will measure if you are being successful? Really consider what bets you are making and why. Usage isn't always good but it's a good first indicator as you are getting moving. To make your platform a product, you have to come back to the vision (or goal mentioned above). Otherwise, you have a collection of services. How is that vision tied to business value? If you need budget, do those holding the purse strings care about your tech decisions or business impact? A good platform helps people get what they need done with the agency to do it. It limits - and where possible eliminates - bottlenecks. A crucial aspect of product management is taking user needs and then translating that to build something to serve those needs. But when it comes to your data platform, the users and the builders (the data engineering team) usually don't speak the same language so you will probably have to spend more time translating between the users and the team building the platform than even in software. Be prepared for you data team to grumble about more meetings too 😅 Combine Simon Sinek's 'Start from the why' and user centricity. The why should be about what are you enabling your users to actually accomplish and what value that drives. Really focus on building something that drives value for the business instead of leveraging the coolest tech. Relatedly, it's likely going to be very difficult to measure the return on investment on your data platform. Be prepared for those conversations but it can be pretty squishy. At their heart, good products are about creating good user experiences and delivering/capturing value. Products need to deliver value to users and producers need to capture value as well. That is a complex topic when the value captured is internal, but it is still an appropriate mindset. If you aren't able to prove value, will you get further funding? Consider your company needs relative to data maturity. An incredibly cutting edge platform for an organization that has low data maturity is not the right fit. Even if you have a cutting edge vision for their work, your 4 year old won't be able to out sculpt Michealangelo. Be realistic and use the platform to drive towards better data capabilities but you have to meet your organization at least close to where they are. You might have difficulty getting people bought in on the vision of your data platform. If people view data as ticket-takers/a service instead of an integral part of the company's way of doing business, be prepared for the fun of lots of stakeholder alignment work. Where possible, don't only look to sell people on your vision. Try to change - or possibly nudge at most - people's behavior through your platform. It's a subtle art and hard to pull off but it will mean people do what you and they both need but without having to build a million PowerPoint decks 😅 It's important to separate your concept of a data platform and the approach of treating it as a product. Otherwise, your platform can easily fall into the trap of designing to user wants instead of what is a platform's function - to make it easier to do work in a scalable, reliable, and repeatable fashion. You need to consider the purpose of the platform instead of throw product management at a collection of tools. Leaders/execs want change. Leaders/exec do not want _to change_. Be prepared for that. Users aren't necessarily ready for the data platform to be a product. They are used to the ticket-taker model. Be prepared to shift their mindset too, not just that of the (data) engineers building the platform. It won't be a simple switch over either - very deep topic… Relatedly, you will probably need a longer run-way to transition your team to a product model for the platform. It is a mindset shift but it's also a big change in the ways of working. You will need to have some patience, you can't switch from a ticket model to only building your platform as a product overnight. A key potential area of value for the platform is making it fast to prototype and get data in consumers' hands. It can reduce a ton of back and forth and also help you quickly discover if further work on a potential data product is useful or if it should be scrapped. Research spikes are very important in data and enabling them can be very valuable - if not always valued 😅 Make sure to think about the actual scope of your data platform. Products have scope, don't try to be all things to all people and don't bite off more than you can chew. There's a reason SaaS offerings in the data space have post-sales and customer success engineers. Many of your users won't just simply be able to read the documentation and use your platform no matter how well you build and document it. Be prepared to ensure success through a bit of hand-holding. Ensuring usage that drives value is (usually) what makes software and data offerings successful. Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Carol's LinkedIn: https://www.linkedin.com/in/carol-assis/ Eduardo's LinkedIn: https://www.linkedin.com/in/eduardosan/ Continuous Integration book: https://www.amazon.com/Continuous-Integration-Improving-Software-Reducing/dp/0321336380 Measure What Matters book: https://www.amazon.com/Measure-What-Matters-Google-Foundation/dp/0525536221 Inspired by Marty Cagan: https://www.amazon.com/INSPIRED-Create-Tech-Products-Customers/dp/1119387507 Empowered by Marty Cagan: https://www.amazon.com/EMPOWERED-Ordinary-Extraordinary-Products-Silicon/dp/111969129X In this episode, Scott interviewed Carol Assis, Data Analyst/Data Product Manager and Eduardo Santos, Professor and Consultant, both at Thoughtworks. To be clear, they were only representing their own views on the episode. From here forward in this write-up, I will be generally combining both Carol and Eduardo's views into one rather than trying to specifically call out who said which part. Some key takeaways/thoughts from Eduardo and Carol's point of view: At the end of the day, the team that produces the data will get the most use out of it 9/10 times. Getting teams used to developing with data in mind isn't just useful for the organization, it is for maximizing their own team's success. Continuous integration is a crucial concept in general for learning how to automate and focus on delivering more, which leads to focusing on value. Read the book :) ?Controversial?: Data mesh is an extension of the continuous integration book/concept because of the focus on delivering value quickly and building to scale reliably. There are many methodologies for understanding value delivery in software. We just have to adapt them better to data. Don't reinvent the wheel. Far more organizations need to think about the goals of their products and then how to measure success against those goals _at product inception_. Design data into your products from the start. Data people often make the data overly complicated for non-data people to grasp. What does the data tell us and what are some simple numbers? Then people can feel like they understand without going too deep into stochastic modeling or something. Relatedly, engage data consumers' curiosity - including the producers that will consume their own data. Try to meet them where they are to get them to engage with data more. Lower the perceived bar to leveraging data. Application development teams need convincing that working with data is 1) essential to understand their own success to further improve their products and 2) much easier than it has been historically. There is a LOT of scar tissue out there… A potential good hook to get people to build their applications with data in mind is being able to show metrics and measure success from day one. The business side can get a better idea and are more likely to engage; it gives a communication bridge between developers and the business people. Having data early in the application development cycle means you have more proof points for making your decisions - assuming you side with the data 😅 that makes it easier to justify decisions instead of people making guesses. Measuring what matters is a crucial concept for the entire team to adopt. It will help people understand what data they need and why. Pressing people on what to measure and then how to measure it crystalizes what bets they are making. How are they expecting users to interact with their applications? For many business people, you may need someone playing the data translator role, translating data to business and business to data. Most organizations' data literacy is still quite low and again, you need to lower the bar to business people leveraging data. ?Controversial?: You don't need to start with data products. Yes, they are great, but teaching your teams that data matters is groundwork to head in the direction of something more scalable. A spreadsheet is a fine place to start. Focus on delivering insights that deliver value, then work towards the productized aspects :) Ask people what action they will take once they have data. Get them in the mindset that data drives action and data that won’t drive action isn't where they should focus. Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Jessika's LinkedIn: https://www.linkedin.com/in/jmilhomem/ In this episode, Scott interviewed Jessika Milhomem, Analytics Engineering Manager and Global Fraud Data Squad Leader at Nubank. To be clear, she was only representing her own views on the episode. Some key takeaways/thoughts from Jessika's point of view: There are no silver bullets in data. Be prepared to make trade-offs. And make non data folks understand that too! Far too often, people are looking only at a target end-result of leveraging data. Many execs aren't leaning in to how to actually work with the data, set themselves up to succeed through data. Data isn't a magic wand, it takes effort to drive results. Relatedly, there is a disconnect between the impact of bad quality data and what business partners need to do to ensure data is high enough quality for them. Poor data quality results in 4 potential issues that cost the company: regulatory violations/fines, higher operational costs, loss of revenue, and negative reputational impact. There's a real lack of understanding by the business execs of how the data work ties directly into their strategy and day-to-day. It's not integrated. Good data work isn't simply an output, it needs to be integrated into your general business initiatives. More business execs really need to embrace data as a product and data product thinking. Instead of a focus on only the short-term impact of data - typically answering a single question - how can we integrate data into our work to drive short, mid, and long-term value? ?Controversial?: In data mesh, within larger domains like Marketing or Credit Cards in a bank, it is absolutely okay to have a centralized data team rather than trying to have smaller data product teams in each subdomain. Scott note: this is actually a common pattern and seems to work well. Relatedly, the pattern of centralized data teams in the domains leads to easier compliance with regulators because there is one team focused on reporting one view instead of trying to have multiple teams contribute to that view. When you really start to federate data ownership, business execs can now partner far easier with other business execs in other domains leveraging data. Instead of having the central data team trying to translate, there is a focus on what needs to get done and the data work flows from that instead of the data work being the focus. It's the engine that powers their collaboration but it's no longer 'the point'. Partnering with those who "are closer to the reality" of the business, it's easier and more likely to drive good outcomes. Meaning: not the senior execs. But the senior execs often have to be on board with the work and the target results. So work on communicating up but closely collaborating at lower levels. Data for regulators often has a LOT of potential reuse for your own organization. Lean into finding those areas where you can do the data work once and get value twice :) ?Controversial?: Really consider role titles in data mesh. Data product owner might be too nebulous and quickly accumulate too many responsibilities. Data product manager is easier to understand the scope of responsibilities and the specific areas of focus. Scott note: this comes up A LOT and is generally starting with data product owner and moving to data product manager. ?Controversial?: Data leaders need to understand product management. To really scale data work, we have to start treating all aspects as a product practice. CTOs down to software engineers need to understand product management, it's time for the data org to as well. Data leaders need to have significant communication skills while maintaining their understandings of data best practices. It's all a delicate balance but the data work doesn't speak for itself. Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Marisa's LinkedIn: https://www.linkedin.com/in/marisafish/ Karolina's LinkedIn: https://www.linkedin.com/in/karolinastosio/ Tina's LinkedIn: https://www.linkedin.com/in/christina-albrecht-69a6833a/ Kinda's LinkedIn: https://www.linkedin.com/in/kindamaarry/ In this episode, guest host Marisa Fish (guest of episode #115), Senior Technical Architect at Salesforce facilitated a discussion with Kinda El Maarry, PhD, Director of Data Governance and Business Intelligence at Prima (guest of episode #246), Tina Albrecht, Senior Director Transformation at Exxeta (guest of episode #228), and Karolina Stosio, Senior Project Manager of AI at Munich Re. As per usual, all guests were only reflecting their own views. The topic for this panel was understanding and leveraging the data value chain. This is a complicated but crucial topic as so many companies struggle to understand the collection + storage, processing, and then specifically usage of data to drive value. There is way too much focus on the processing as if upstream of processing isn't a crucial aspect and as if value just happens by creating high-quality data. A note from Marisa: Our panel is comprised of a group of data professionals who study business, architecture, artificial intelligence, and data because we want to know how (direct) data adds value to the development of goods and services within a business; and how (indirect) data enables that development. Most importantly, we want to help stakeholders better understand why data is critical to their organization's business administration strategy and is a keystone in their value chain. Also, we lost Karolina for a bit there towards the end due to a spotty internet connection. Scott note: As per usual, I share my takeaways rather than trying to reflect the nuance of the panelists' views individually. Scott's Top Takeaways: If you want to dig deeper into the data value chain, consider looking into the value streams concept. What flows through your business in terms of process to generate value? Where are there points of value leakage? The same concepts are crucial in your value chain. Organizations need to really educate their entire organization on the data value chain. Part of why there are so many issues in data from upstream changes by developers breaking downstream data is they simply don't know what parts of their data are used and why. Communication is a much bigger aspect of doing data than people think. Even talking about the specific data value chain can cause people to focus too much on the data work instead of the business value delivered via data. The data value chain is crucial to understand but it's also crucial to understand data work doesn't inherently create value, it's about how it's used in the business. Dig into the value created and focus on working backwards from that to what data work needs to be done. The data value chain is crucial for companies of all sizes across all industries. At its heart, the concept is about focusing on ensuring you aren't leaking value in your business value streams/pipelines. You need to focus on what drives value and how to improve the processes there. Data value chains often cross line of business/domain boundaries. After all, a lot of the value of data is about combining information across those boundaries. That can mean cross-team handoffs, which make understanding and ensuring the success of those data value chains even harder. Who owns what isn't inherently understood/agreed to, you need to get specific. It's important to not get overly focused on a single end-point of value when it comes to data work, especially when it comes to a data product. If we want re-use, we have to focus on the processes of creating reusable value. Maintaining that larger picture focus while still ensuring each data consumer can still get value from a data product is a very hard balance. Focusing heavily on your data value chain is going to be hard. It means hard work and a lot of internal collaboration - and thus negotiation - across domain boundaries. You all have to be in it together to really get the best results - and some organizations aren't ready for that. But the hard work pays off because you are ensuring value actually gets created. As with anything in data, you have to make bets. That doesn't mean every bit of data work will create significant value or even exceed the investment. But an approach like data value chain is crucial to understand 1) what bets are you making and why and 2) who owns what aspects of the data work. That can help you really focus on the what and why rather than focusing on outputs. Other Important Takeaways (many touch on similar points from different aspects): As with many things in data, ownership is crucial to understanding your value chain. The weakest points in a value chain are the handoffs between teams. Strong ownership, including of those handoffs, prevents value leakage (from the value streams concept). To understand your data value chain, you will have to go deeper than many are willing to in the (dreaded?) operational plane. You have to understand what data you have, how it's collected, what data you can collect, etc. Some of it is working backward from what data you need/want but a lot of it is working from what data you have or can get. Relatedly, the value you can create from data is heavily reliant on what matters to the business. To think about value, you have to understand your business processes and what generates actual value. You really need to consider your approach to data collection and storage. How do you want to consider data that may have value but hasn't yet proven to have value? You don't want to have costs go out of control and most data is never back-cleaned/filled if it wasn't collected and stored for use. But you can't know all your data use cases at the launch of a new application or product. It's a balancing act. There is a question of how mature do you need to be as an organization to actually really consider using data value chain as a framework instead of merely some principles to guide your work. It can be hard to get people to understand the value and what drives that value in data when they don't understand data work in general. Relatedly, really digging into the data value chain can shine a light on underperforming activities inside and outside the data function. So you need to be prepared for some hard realizations and questions. Are you ready for transparency? What aspect of data value chains fall on the business? It's a hard question. At the end of the day, data value chains are supportive of the business value chains/streams but it depends on who has ownership over data work: the lines of business or a centralized team. Your data value chains should have explicit ownership, at least of the different 'links' of the chain. In data mesh especially but true in any data work, it's important to not see the data product as the end of the data value chain. The data product is there to make it easy for producers to reliably and scalably deliver value through data. But there is only value if that data is consumed, the value happens when someone takes action. When launching new applications/products, you have to consider what data you might want to collect even if you don't need it right at the start. Especially if that is something like hardware where you can't augment many aspects of the devices once they've been deployed. Focusing on data value chains is a mindset shift for most organizations, much like data as a product thinking. You need to get people to stop handwaving about aspects of data work and focus specifically on value and understanding that all parts of the data creation and transformation process are crucial to driving rich and sustainable value from data. Even if you do a good job at understanding your data value chains, there will still need to be rework. But it can help you prioritize data rework - you aren't going to get your data preparation perfect, especially for multiple consumers, on the first try. You have to be realistic about your data value. Your company probably won't value data and analytics that are internal facing as much as they do external-facing interactions until you prove out the value of treating those internal users with as much care. Part of that is getting specific about how much value you are generating and how :) At some point in your value chain, you aren't dealing with raw data anymore. Think about who wants what and why. Most execs want aggregated information - again, that point of driving business value instead of data work. Make sure there is clear communication to drive outcomes instead of outputs. A data value chain isn't about getting everything perfect upfront. Everything is about incremental delivery and getting better. What is the cost/benefit of that improvement? Get something out that works and is supportable/stable and then improve. Iteration is your friend. When thinking about your data value chain, it's usually best to focus again on target business outcomes/objectives. After all, that is where the value is. You can get more business people interested in data work if you are constantly talking in their language about their key objectives. Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Darren's LinkedIn: https://www.linkedin.com/in/darrenjwoodagileheadofproduct/ Darren's Big Data LDN Presentation: https://youtu.be/vUjoJrl_MEs?si=WzB0sBStVIAyqDJs In this episode, Scott interviewed Darren Wood, Head of Data Product Strategy at UK media and broadcast company ITV. To be clear, he was only representing his own views on the episode. Scott note: I use "coalition of the willing" to refer to those willing to participate early in your data mesh implementation. I wasn't aware of the historical context here, especially when it came to being used in war, e.g. the Iraq war of the early 2000s. I apologize for using a phrase like this. Some key takeaways/thoughts from Darren's point of view: Overall, when thinking about moving to product thinking in data, it's as much about behavior change as action. You have to understand how humans react to change and support that. You can't expect change to happen overnight - patience, persistence, and empathy are all crucial aspects. Transformation takes time and teamwork. ?Controversial?: In data mesh, it's crucial to think about flexibility and adaptability of your approach. Things will change, your understanding of how you deliver value will change. Your key targets will change. Be prepared or you will miss the main point of product thinking in data. When choosing your initial domains and use cases in data mesh, think about big picture benefits. You aren't looking for exact value measurements for return on investment but you also want to target a tangible impact, e.g. if we do X, we think we can increase Y part of the business revenue Z%. Zhamak defines a data product quite well in her book on data mesh. But data as a product is a much broader definition of bringing product management best practices to data. That's harder to define but quite important to get right. When thinking about product discovery - what do data consumers actually need producers to create - there is often a big difference between consumers' initial suggested requirements and what they actually want. It's the role of data product management to bridge that gap and deliver what they need instead of what they requested. ?Controversial?: There is a big difference between data product management and regular product management: in regular product management, the ability to take requirements and go away and come back with something months later works. But data is about what's happening with the business now and needs to evolve as the understanding of requirements evolves. Relatedly, data products need to be even less rigid than regular products because they are a reaction to the real world as it changes; you must focus on building something that flexible. Otherwise, you aren't going to be reflecting what matters. When building data products, get to an MVP fast. Get something in people's hands and evolve it to what they need/want. Don't try to get it perfect. When it comes to doing data products well, there is far more collaboration than most people are used to around data work. When considering what to build, data producers need to ask consumers what question they are actually trying to answer rather than what dashboard do they want. Outcomes over outputs. It's about what you are trying to do. Scott note: And then come to an agreement on the specifics - are you delivering data, the insights, or the 'so what'? ?Controversial?: Data products can - and probably should? - absolutely look to address multiple business questions. But when you are thinking about your MVP, focus on what is the most critical question and the minimum requirements to address that question. You can improve the product but getting to first valuable insights is a great initial milestone to build off. ?Controversial?: Similarly, in regular product management, you can go away for six months and come back with something new and shiny but in data, it might take that long just to change two metrics on a dashboard because it took that long to clean up the data. And at the end no one cares, no one understands what the value was, and worst still people are annoyed because they don't trust the new metrics. Relatedly, maybe consider MLP instead of MVP. No, not 'My Little Pony' but Minimum Lovable Product. What is the minimum you can deliver that someone will love - what are the need to haves instead of the nice to haves? What is at least one feature that users will love and can you deliver that early to maintain engagement as you deliver more aspects of value? Actually sit with users and see how they leverage your products. That's a crucial aspect of product management, it shouldn't be any different in data. It can be a bit harder sometimes to get to specifics but that doesn't let you off the hook. It's necessary work to do. ?Controversial?: Bring as many of the folks on the data product producer team as you can into the discussions with the initial data product consumer. That will give a more complete picture to the team. Don't treat the rest of the team as ticket takers internally. And you can probably find and address challenges earlier - e.g. flagging a low quality data source - if everyone is more informed. As with anything in product management, prioritization is crucial. Focus on delivering value, not simply data products. Again, outcomes over outputs. Initial delivery of the data product to a consumer isn't a mark of 'done', it's a great place to focus on how the consumer actually uses and interacts with the data product. Get a sense of the friction points to find places to further improve it. No more throwing things - data projects - over the wall and treating them as done. Teams that understand product thinking in general are easier to teach product thinking around data. But for those teams that don't understand product thinking at all yet, you will need to spend more time with them. It can seem obvious but each team has different educational needs to bring them to product thinking for data. You can't try to win over everyone to something like treating data as a product yourself. You need to find your champions and advocates and then provide platforms internally for them to spread the messages of value and success. It's not only about delivering value but showing that off a bit too to get others excited. Use momentum levers. When choosing your first domains to work with and finding your first data products for data mesh, look for people that are enthusiasts. You want partners early on, not someone you have to constantly convince. Also look for data products that will be reusable to find additional users to provide additional value. It's as much about building momentum as it is about delivering value early. ?Controversial?: Behavior change is a crucial aspect of implementing data mesh. You have to understand behavior change takes time. And you have to know where you will be rigid and where you will be more flexible. If you say my way or the highway, most people are just going to head for the highway. You have to work out what is non-negotiable early. Relatedly, behavior change happens at different paces for different people. That will happen inside your organization too. Look to build up the critical mass, that momentum, over time so people feel like they are joining a successful movement. But you need to do your internal PR to make sure they know about it - data success isn't self-evident. During your transition to data mesh - and is it ever really done? -, you will have data sources that aren't part of the mesh. It will potentially be hard to integrate those with data from the mesh because your data in your data mesh has legal and governance at the core. Sources outside the mesh tend to have one-off policy approaches applied. Be prepared for some tension and consternation. In most cases, people will inherently trust existing data sources over new data sources. That will be a challenge to your data mesh adoption. Even if they objectively know the quality is better with the new source, it's human nature to trust what you already know more all else being equal. Involve consumers heavily in the conversations about what is changing and why. Pushing change on people is likely to cause pushback. No one wants change to happen to them instead of with them. When doing Domain Driven Design (DDD) in data mesh, do NOT try to start with all your domains at once. Look to find ones that are capable enough and that you can learn from to make it easier to partner with other domains in the future. Relatedly, your domain map will change. That's a part of DDD. Don't try to hold onto things rigidly. Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Wendy's LinkedIn: https://www.linkedin.com/in/wendy-turner-williams-8b66039/ Culstrata website: https://www.culstrata-ai.com/ TheAssociation.AI website: https://www.theassociation.ai/ In this episode, Scott interviewed Wendy Turner-Williams, Managing Partner at both TheAssociation.AI and Culstrata and the former CDO of Tableau. TheAssociation.AI is "a global nonprofit business organization …focused on bridging the disciplines of AI, data, ethics, privacy, robotics, and security." It is focusing on things like networking and knowledge sharing to drive towards better outcomes including ethical AI. Some key takeaways/thoughts from Wendy's point of view: Right now, we try to break up the aspects of data into discrete disciplines - and then work on each completely separately - far too much. Privacy, security, compliance, performance, etc. Instead, we need to focus on the holistic picture of what we're trying to do and why. Communication is key to effective data work and driving value from data. Hire product managers and focus on the why. Break through the historical perceptions of data as a service organization. Drive to what matters - outcomes over outputs - and focus on delivering value. "What's the point of being focused on the data if you don't understand the business that the data is supposed to be used for?" ?Controversial?: "There is no transformation without automation." If you want data to play a part in transforming the business, you need to focus on automation. Data related work can't be toil work or most won't even do it. "You will never be as successful as you can be as a data organization if you're not able to influence your IT partners, your product teams, your business teams." For far too many companies, data is just an afterthought. It's not the core around how they build out initiatives. When you bolt-on the data to any aspect of the business instead of integrate it from the start and build with data in mind, it's far less impactful. You're always playing catch-up. Make data a forethought. In many respects, data has become a 'four letter word' to lots of people - meaning it has a bad connotation. There are a lot of internal politics around data. Data can mean power and it can also give people perspective on your team's performance. Try to work towards removing the politics if possible but also good luck… 😅 There is so much data in many large organizations that execs can't make sense of it. They often don't understand what data they will need to support their decisions or how to get in place the data they do know they need. There's also often a disconnect between strategy and targets/feasibility when it comes to data. There may be a strategy of grow X product with a target of 'grow X product 15%' but there isn't a good reason why 15% is the target. It becomes a dartboard instead of data feeding into creating the goals. Execution and tactical decisions are powered by data far less often than they should be. There is far too little thought or process around strategy and tactics enabled by and about creating data. Many line of business or domain leaders are simply not great at data. They may be able to leverage insights but they don't get the information cycle, especially sourcing necessary data. Data teams need to partner with them effectively - that is definitely a two-way street. ?Controversial?: Relatedly, too many data people are focused on the data work itself instead of the impact of the work. There needs to be a better understanding of what teams are trying to accomplish with the data work. It's not about the pipeline, it's about the goal of the work and the impact. You need internal processes and clear delineation of ownership or you will have multiple teams measuring the same things and getting different answers. Far too often, people are myopic in focusing on their own job instead of how they fit into the bigger picture of the organization and delivering value to customers. That leads to teams not considering how they exchange information internally, only focusing on their own usage. ?Controversial?: Data teams need to spend more time creating their own data around the impact of data work and impact of issues like data downtime. Move past the service-only/cost-center perspective. Lack of data fluency, especially among execs, causes so many issues. If people don't understand data, they don't understand how much they can trust it and thus won't rely on it. Relatedly, there is a significant lack of understanding of upstream and downstream data and business processes and needs that could be fixed by better communication. What do you need from your upstream and how are your downstream users leveraging your data? Communicate! Automating data work enables business partners to identify "business choke-points" and address them. ?Controversial?: You can't have AI without really understanding your business processes and how data supports those, how people combine data and their degree of trust and understanding of the data. Wendy started out with her perspective that in some respects, data has become "a four letter word". There's so much data and everyone is trying to use it but everyone also feels inundated. Instead of being data-driven, we are data-flooded or data-dragged. And there is a major lack of tying the data work to the actual strategy and execution. Where do we need data to support our decisions? We need a strategy to get that data in place. Relatedly, Wendy sees the major breakdown between strategy and goals when it comes to data. There may be a strategy to grow a product but how much growth is feasible, a good target? Why is that growth feasible? What does the data say about growing that product, e.g. the market dynamics and your positioning in the market? So when goals are set, it is a 'finger in the air' type guess as to how much it could grow or worse, simply how much leaders want a product to grow. And then what data do we need in place to enable the team managing that product to actually be able to grow it that much? How do we enable them to make smart tactical decisions? Basically, it's a lot of things looping back on each other. We need data to set good strategic decisions. But we need a strategy to set up our ability to capture and analyze that data. We need data to make better tactical decisions. But most companies lack the ability to make good tactical decisions to get the necessary data in place. It's top-down driven but far too often the ones who understand what needs to be done for and with data are too far down in the organization and there's a communication gap. Thus, there is a significant lack of being data-driven. We have to admit the problem first. How to fix all of that is another fun process 😅 For Wendy, far too many organizations have data as an afterthought. And that leads to subpar understanding of what's actually happening with the business and lacking the information to fine-tune their strategic decisions. There isn't a strong strategic connection flowing from the business strategy to the data work. Execs aren't spending the time to really follow-through on exactly what the tactics should be to get the right data in place. Scott note: we literally have a panel on doing that, tying the data work to the business strategy and vice versa 😎 episode #251 In many parts of many organizations, e.g. Marketing, Wendy sees there being very competent people who just don't really understand how to do data well. They need a great partner. Should the marketing leader be focused on what data sources they need and why? Or should we be able to translate their needs into the work? But first, we need to actually be able to partner and they need to understand their needs. Data people can supercharge their efforts but the business partners need to lean in. Scott note: in data mesh, part of the role is enabling them to get better. We need people to up their fluency but doing data mesh or not, everyone starts somewhere and we need to help them level up. On the flip side, Wendy also sees how often data people are stuck focusing on the data work instead of the business aspects of what that data work is tied to. Without the business context, all you are doing is pushing 1s and 0s. What do business partners need and why?! There needs to be the ability and the courage to just hammer out the understanding differences or the problems will persist. Wendy also gave some specific examples of too many cooks in the kitchen relative to certain measurements. Instead of there being one official perspective or measurement for something like usage of a cloud product, in a previous role there were many measurements across engineering, finance, marketing, sales, etc. And every single one was different because they all used slightly different methodologies and even sources. So when they tried to look at success of the product, everything told a different story. And when they tried to have a simple bill the customers could understand, it was just not possible. While single source of truth is a complicated and overloaded term, one official source of truth for a question is something you should be able to rally around. A big problem in many organizations is people are only focused on their own job and lose sight of the bigger picture and especially how they play into that bigger picture of the organization's success according to Wendy. Even if your role isn't directly improving the customer experience, your work can have a positive impact on that if you drive towards that goal. Sometimes, politics around data also gets in the way of collaboration across teams and lines of business. Wendy talked about another persistent problem in data: the service model. If your data teams are only focused on supporting other teams, you can lose sight of your big picture impact as well as the impact of bad data. She believes data teams need to spend more time creating their own data around their impact and also quantifying the costs of data issues. What are the actual impacts to the organization? And do execs outside the data team understand data well enough to understand those impacts? If they don't understand data, can they even trust it enough to rely on it? Circling back to the bigger picture, Wendy believes that teams can drive significant process improvements if they just understand the impact of their work - especially through data - upstream and downstream. What do they actually need from others? Who is consuming their data and why? What impact will changes have? How are communications set up to prevent issues and create strong understanding and trust? And then of course, try to automate as much as possible to lower the burden on everyone involved in the data flowing around :) As part of that, please just hire good product managers 😅 Wendy said, "You will never be as successful as you can be as a data organization if you're not able to influence your IT partners, your product teams, your business teams." Data is a team sport, data is about making the organization better. You need others to play with you or it won't work. When thinking about actual business transformation around data, Wendy said, "There is no transformation without automation." Historically, doing data work has required a lot of effort. The business side just wants to leverage the data, help them automate as much as possible. Otherwise many - most? - business partners won't want to engage with the data and leverage data to improve their work - it's too much effort. Also, removing the friction from data work helps people identify the friction in general business processes. So automating the data work allows them to more easily identify and then address "business choke-points". For Wendy, too many aspects of data work are treated as wholly separate disciplines instead of treating it as all part of one whole. Security, privacy, compliance/regulatory, performance, etc. We have to shift it left but also stop trying to treat them as discrete challenges to overcome instead of interoperating aspects of a working, scalable solution. Think data by design 😎 That's why she created TheAssociation.AI, "a global nonprofit business organization …focused on bridging the disciplines of AI, data, ethics, privacy, robotics, and security." She said, "there is no security, there is no privacy, there is no ethics, there is no AI without data," but we also need organizations to actually implement their policies into their data and data work. There isn't going to be ethical AI without someone leading that charge and TheAssociation.AI is looking to push that effort forward. In wrapping up, Wendy circled back to the start. What is the point of doing data work. She said, "What's the point of being focused on the data if you don't understand the business that the data is supposed to be used for?" Being a data leader, especially the CDAO, is VERY tough because you often don't own much of the infrastructure if at all and have to do your work essentially via influence. But if you build the right relationships and understanding of the business, you can still have a major impact and drive significant value for your organization. Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Amritha's LinkedIn: https://www.linkedin.com/in/amritha-arun-babu-a2273729/ In this episode, Scott interviewed Amritha Arun Babu Mysore, Manager of Technical Product Management in ML at Amazon. To be clear, she was only representing only own views on the episode. In this episode, we use the phrase 'data product management' to mean 'product management around data' rather than specific to product management for data products. It can apply to data products but also something like an ML model or pipeline which will be called 'data elements' in this write-up. Some key takeaways/thoughts from Amritha's point of view: "As a product manager, it's just part of the job that you have to work backwards from a customer pain point." If you aren't building to a customer pain, if you don't have a customer, is it even a product? Always focus on who you are building a product for, why, and what is the impact. Data product management is different from software product management in a few key ways. In software, you are focused "on solving a particular user problem." In data, you have the same goal but there are often more complications like not owning the source of your data and potentially more related problems to solve across multiple users. In data product management, start from the user journey and the user problem then work back to not only what a solution looks like but also what data you need. What are the sources and then do they exist yet? Product management is about delivering business value. Data product management is no different. Always come back to the business value from addressing the user problem. Even your data cleaning methodology can impact your data. Make sure consumers that care - usually data scientists - are aware of the decisions you've made. Bring them in as early as possible to help you make decisions that work for all. ?Controversial?: Try not to over customize your solutions but oftentimes you will still need to really consider the very specific needs of your consumers. Build for reuse but also build where your consumers are actually having their needs met. A mediocre solution for all is usually worse than a few specialized solutions. Prioritization is crucial in product management. That applies to features within the products but also the products themselves. There are many potential use cases that won't be met because there isn't enough value. That's the name of the game, return on investment; it's not about capturing all value possible. Communication and building relationships/trust are foundational in product management. It's an art as much as a science. If you can't have tough conversations and get alignment, it is FAR harder to build a product that meets customer's needs. Relatedly, establish regular communication with your customers. You shouldn't only be talking to them when things go wrong. Stay on top of what is driving value for them and look to augment your product proactively, not only reactively. Product management requires patience as much as diligence. Sometimes your data product/element violates its SLAs but it's an outlier, a one-off. Don't look to overreact and jump to changing things. But you obviously need to have serious conversations if elements aren't meeting expectations over a more extended time period. If you aren't sure what products you should create in a new area, talk to people and find the points of friction. What are the pain points and is there enough value in addressing them to justify doing the work? It's crucial to deeply converse with potential users of a data product/element to assess if it's really going to be worth the effort. There is always a chance you build something that isn't used/valuable but through deep investigation and ideation with potential customers, you can avoid that far more often. When you are building something, even before it hits 'GA', get validation. You can save yourself a ton of effort in rework as you find a better solution sooner. Product management is about collaborating to drive towards value. You are there to prioritize and coordinate. You don't have to know everything, but your job is to uncover as much understanding as possible to maximize your value creation and minimize wasted work. Always ask what value building something for your customer will drive. But also ask what happens if we don't build it. What is the cost of not acting? The only constant is change, especially in data. Leverage a "loosely dependent architecture" to be able to adapt to change. And be open and honest with customers that things will change. Emphasize you'll work with them to adapt to those changes. Amritha started the conversation on some key differences between software product management and product management around data - whether specific to 'data products' or not. One similarity is the focus on solving a particular user problem but in data, you might have to build something to address multiple users' problems. A much bigger difference is that in data, you often don't own the entire process as you might be reliant on others to source your data. In software, you are generally building the data sourcing because you own the interaction creating the data. How the data is stored and collected throughout the upstream process impacts what you can do. The user problem, the business value, and the user journey are some key guides to doing data product management well for Amritha. Keep coming back to those as you build out your solution. Focus on understanding what the user really needs and work backwards to the sources. And then of course focus on making sure you are actually addressing user needs when you deploy the solution. There are many reasons a data element may not be performing up to expectations so be prepared to deep dive; is there a problem with what you've built, what's feeding your data element - maybe sources have changed or there is a quality issue -, or is it just not performing to expectations because the hypothesis was wrong? Amritha dug a bit more into some challenges specific to product management in machine learning and AI. While data scientists want clean data, when possible they want to even be part of the process of selecting the cleaning methodologies - even that can impact the data enough to change outcomes. So really start from the process of bringing them in as a stakeholder as soon as you can and don't throw data over the wall at them. And if you already have something developed, share your methodologies and help them figure out if it's the right fit for them or if something new needs to be developed. Again, we want reuse but we also want solutions that address their problems. Always a hard set of needles to thread. "As a product manager, it's just part of the job that you have to work backwards from a customer pain point." Amritha questions if you are even building a product if you don't have a customer. What is the business value of the work? For a product manager of product without a customer, are you focused on your own thoughts and biases rather than the needs of consumers? "So the point here is that at any given point, you have to be cognizant of who are you building this for, why, and what that is the primary customer. And the secondary is: who else if I build this, what are the impacts it will have on my secondary customers, or other downstream or interacting applications?" Amritha talked about one crucial rule in product management: prioritize. There are many use cases you _could_ solve but are they actually worth the effort? Think about what will impact your organization the most. Don't try to solve every use case and don't try to make products that can serve every potential customer - focus on delivering value. Scott note: this can be a slippery slope in data mesh. You want to take on use cases you actually can tackle when you are learning. Don't only go for the biggest value but also tackle problems where the juice is worth the squeeze, where the outcome is worth the effort. In product management, Amritha believes it's absolutely crucial to understand the art and the science. The science is more about is this product specifically meeting the needs it was designed for. Basically, measuring the level of success and determining if that's good enough or especially is it _still_ good enough. But even that last bit can be a bit of art. The real art is all about communication and building relationships. If you build the world's objectively best product but no one trusts it or understands it enough to use it, it's not a valuable product. You must build strong relationships and have the tough conversations with stakeholders, earning their trust, to align on what needs to get built and why as well as when a product isn't meeting expectations. Establish regular lines of communication so it's not that the only time you talk to your customers, it's bad news or big changes. Continue to extract information from them to drive to business value. When it comes back to the science, that's when Amritha believes you should dig into the why something isn't meeting expectations from the technical perspective :) And have some patience around that. Sometimes it's a blip on the radar, not anything more. When figuring out what products/data elements you might want to build in a specific area, Amritha recommends digging into the potential workflows and user journeys. Start to really think about what you think could exist and why. But, instead of trying to ideate only yourself, go and talk to people and listen for their pain and points of friction. They may not even realize they have pain but you can find the challenges that people will want to address. Again, work backwards from the user journeys to discover what products you should build 😅 Amritha talked about how to make maximize the chance that what you're building will be used/valuable. A lot of it is simply digging in deep with potential customers in the ideation phase to make sure this will actually drive value. There are ways to do that but a lot of it is simply spending the time to really understand the likely impact of what you're building. As Alla Hale said in episode #122, "What would having this unlock for you?" Also, ask, "what if we don't do this, what is the impact of not doing this?" And make sure to get validation as you're building. It might be the value hypothesis was wrong or that you're building something that is the wrong or suboptimal way to address the challenge/opportunity. You can save yourself a lot of headaches and rework. It's all about that collaboration to drive to value. In wrapping up, Amritha talked about how changes, especially in data, are inevitable. Make sure to communicate with consumers so they have realistic expectations. Sometimes those are proactive changes but often, you don't have that much control over changes, especially coming from upstream in data. Look to build in a way that can adapt and leverage a "loosely dependent architecture". Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Nailya's LinkedIn: https://www.linkedin.com/in/nailya-sabirzyanova-5b724310b/ In this episode, Scott interviewed Nailya Sabirzyanova, Digitalization Manager at DHL and a PhD Candidate around data architecture and data driven transformation. To be clear, she was only representing her own views on the episode. Some key takeaways/thoughts from Nailya's point of view: When it came to microservices and digital transformation, we aligned our application and business architectures. Now, we have to align our application, business, and data architectures if we want to really move towards being data-driven. To do data transformation well, you must align it to your application architecture transformation. Otherwise, you have two things transforming simultaneously but not in conjunction. It's crucial to involve business counterparts in your data architectural transformation. They know the business architecture best and the data architecture is there to best serve the business. That is a prerequisite to enable continuous business value-generation from the transformation. Re a transformation, ask two simple questions to your stakeholders: What should this transformation enable? How should we enable it? It will give them a chance to share their pain points and their ideas on how to address them. The business stakeholders know their business problems better than the data people 😅 Your approach to data mesh, at the start and throughout your journey, MUST be adapted to your organization's organizational model and ways of working. Everyone starts from completely different places. Data mesh won't work if you overly decentralize. You must find your balances between centralization and decentralization yourself. ?Controversial?: Historically, teams were charged for data work and resources but with something like data mesh, they can manage their data and data costs far more efficiently. Framework processes, tools, and skills help teams to identify which data is valuable for their own or other domains and requires investment, and which data or data processing operations are redundant, and thus, a source of savings. ?Controversial?: You should consider two phases of your early data mesh implementation: “foundation” - enabling teams to own their data by building corresponding teams, processes, and tools and “operationalization and scaling” - enabling/incentivizing them to share their data well with others. They have overlap but if you don't focus on enabling to own data for themselves, you may have trouble incentivizing them to even own their own data let alone share it. To drive incentivization and prioritization well to do something like data mesh - or really most large-scale transformations - you need top-down support from the highest levels in the organization. ?Controversial?: In highly regulated industries, you will have domains that already have very strong governance practices. Focus on enabling them to safely manage their data within a new framework, rather than trying to change their ways of working. Relatedly, focus on the frameworks and guidelines as well as the tooling to enable those domains that aren't nearly as advanced in their governance. If you are looking to federate/decentralize to all domains at the same time, consider how you leverage central committees. If you don't have someone helping guide people towards some degree of consistency, you can't find repeatable scaling patterns/best practices and are likely to create data silos. The level of importance your company places on your data transformation - mesh or otherwise - should determine how you combine it with your digital transformation. It might be under it as part of digital transformation, entirely separate but at a peer level, etc. But they should get aligned no matter what to have the best impact. Everyone must understand that data mesh is journey. You will learn and adjust along the way. Getting budget for your data transformation / data mesh journey is probably more political than many expect. The easiest way to get budget is high-level management attention. Scott note: but of course, it can be hard to get that buy-in first 😅 Nailya started the conversation on the need for application, business, and data architectures to all be aligned. She gave some history about how application and business architecture were brought into alignment when we started with microservices and digital transformation. But now we have to add data into the mix which makes things even more difficult. We need both the application and data architectures to be designed to specifically support the business goals. When it comes to actually transforming your data architecture, Nailya believes the transformation should be led from the business side where possible. At the very least, the business side should be involved. They understand the business needs best so they can help direct the transformation to serve those needs. Great data work that doesn't support the business needs is often just a well-designed money pit 😅 You obviously need the strong data expertise but a transformation led exclusively by the data team is far less likely to align to business goals and priorities. In a successful past digital and data transformation, Nailya used two simple questions to the key business stakeholders: What should this transformation enable? And how should we enable it? It gave the business stakeholders a chance to fully lay out their pain points as well as some ideas how to address them. That way, the team had a very broad perspective and could come back to each of the stakeholders with solutions that worked for them somewhat tailored to their needs and thoughts. You need repeatable patterns/approaches but you also need people to feel seen and heard in order to drive buy-in where you address their specific pains and ideas. When asked about adapting data mesh to an organization's specific challenges, Nailya pointed to how every culture is so different and you need to take into account how people internally exchange information and work together as you design how you want to go forward. Overly decentralizing - so not doing any federation - or decentralizing too quickly won't work well. You have to find your balance between centralization and decentralization throughout your journey. One interesting buy-in point Nailya mentioned was cost control over data work. Because teams have traditionally been charged for data resources and work by central data teams, they were not as involved in managing costs. Data mesh empowers business teams with tools to control cost-effectiveness of their data, and thus they can identify easier which data is valuable for their business and requires investment, and which data or data processing operations are redundant, and thus, a source of savings. They can see it as a chance to do things better and align better on what work is worth doing - the central data team might have done work that the domain doesn't see as valuable when really considering it more deeply. At first, it might be only for their internal-facing to the domain data work before we can get them bought in that they are now responsible for also sharing their data with other domains. Nailya talked about her experience with a large-scale data mesh implementation. They focused on first enabling teams to own their own data. So again, giving them the chance to gain transparency to their most valuable data as well as define, align, and prioritize their data initiatives. Then, they started to work to incentivize and better enable them to share their data with the rest of the organization identifying new use cases and data customers. This may delay the biggest benefits of data mesh - high-quality, reusable data across domain boundaries - but it does mean that teams aren't struggling to own their data at the same time as learning to share it with others; this also helps with the incentivization challenge as they can take advantage of their data first for themselves before being asked to focus on sharing it with other domains. Data governance in data mesh will - unsurprisingly - be hard for every organization in Nailya's view. If there are domains that already know how to handle their data well, work to enable them to better share their information but also don't try to push them towards central ways of working. If they can safely secure and share their data in a way the rest of the organization can consume it, don't get in their way. But you should also look to create frameworks and standards for those that aren't as mature to help guide them along. Scott note: this is an adjustment for those that are already somewhat decentralized with their data work. Again, adjust for your circumstances! Nailya also recommends central committees to ensure teams are meeting some degree of conceptual consistency and also technical/architectural consistency. That way, you can really find your scalability patterns and best practices. Scott note: if you decentralize/federate all at once, this might be the best bet. But many - most? - are going domain by domain so this function is already embedded in an enabling team. In her experience, Nailya believes that aligning your data and digital transformation is very important to be able to succeed at data mesh. Again, you need to align the application and data architectures with the business architecture. Really take stock - is your data transformation part of your overall digital transformation, are they at the same level and should be partnered, etc.? Nailya again circled back to engaging with your business partners/stakeholders to have them help design your transformation efforts. They will lean in from feeling seen and heard but also, they know the most critical pain points best and can help direct where you should focus first. The conversation finished up around getting and maintaining a budget around your data mesh implementation. For Nailya, this is typically far more political than many might expect. But if you have the proper top-down support and management attention/buy-in, you should at least be able to get going. You have to show value along the way but that should be part of the prerequisite to start: what value are you trying to capture and how will you measure it? Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Jen's LinkedIn: https://www.linkedin.com/in/jentedrow/ Martina's LinkedIn: https://www.linkedin.com/in/martina-ivanicova/ Xavier's LinkedIn: https://www.linkedin.com/in/xgumara/ Xavier's blog post on data as a product versus data products: https://towardsdatascience.com/data-as-a-product-vs-data-products-what-are-the-differences-b43ddbb0f123 Results of Jen's survey 'The State of Data as a Product in the Real World' (NOT info-gated 😎👍): https://pathfinderproduct.com/wp-content/uploads/2023/12/2023-State-of-DaaP-Real-World-Study.pdf?mtm_campaign=daap-study&mtm_source=pp-blog&mtm_content=pdf-daap-study In this episode, guest host Jen Tedrow, Jen Tedrow, Director, Product Management at Pathfinder Product, a Test Double Operation (guest of episode #98) facilitated a discussion with Martina Ivaničová, Data Engineering Manager and Tech Ambassador at Kiwi.com (guest of episode #112), and Xavier Gumara Rigol, Data Engineering Manager at Oda (guest of episode #40). As per usual, all guests were only reflecting their own views. The topic for this panel was data as a product generally and especially how can we actually apply it to data in the real world. This is Scott's #1 most important aspect to get when it comes to doing data - especially data mesh - well. It's the holistic practice of applying product management approaches to data. It ends up shaping all the other data mesh principles and is a much broader topic than data mesh is in his view. But it can also be quite simple in concept when you really boil it down, it just takes patience and focus. Scott note: I wanted to share my takeaways rather than trying to reflect the nuance of the panelists' views individually. Scott's Top Takeaways: At its core, data as a product is more an organizational mindset approach than anything else. It is something you work towards. It's not an overnight change but getting the mindset right first - at least with a small core group - will help the organization figure out how to best move towards treating your data as a product. Data as a product and data product thinking must not fall into only thinking about data products - It's product thinking about data! In general product management, it isn't about creating a product, it's about creating an experience for the customer that generates value for them in some way. The best way to do that sustainably in data is a data product. But the value and experience are the point, that's your focus, the data product is merely a vehicle to deliver those. Think about good and bad product experiences. The incomprehensible manuals and unintuitive design of a bad product. The awesome tutorials and documentation around a good one. We need to create easy paths for people to not only discover our data products but discover the best ways to generate value from them. Data as a product includes considering your data sourcing strategy. If you think about a physical good, you don't just have the labor to put it together, you need to consider the materials that need to be combined to create the product. It's not just about what parts you have laying around, you manage the supply chain - hopefully reliably and scalably - to make your widgets. The same should be part of data product practices. It's not just what data you have but what data you will need. If we actually want our organizations to be data-driven (whatever that means to you 😅), people need to be able to rely on the data. If we want to embed leveraging data into the DNA, the day-to-day operations, of the company, we need to make our data creation and management processes reliable. The best way to do that is via data products because it makes it easy and reliable for consumers to leverage data. The data isn't the point, it's the mechanism to drive business value. A first step on the road to your organization treating data as a product is getting away from the data team as a service mindset. That they are ticket takers. Data shouldn't be a cost center or ticket-taking organization rather than a value generating one. A key aspect of good products is usability. We need to focus more on usability in data. That is somewhat wrapped into user experience but there are other aspects that are even more often overlooked than UX. A lot of that usability falls on the platform as well so there isn't a different user experience for each data product. Moving to a data as a product approach, instilling that data as a product mindset in your organization, will be hard. And it is quite a bit of cognitive load for people who haven't focused on data historically. Look to introduce the concept and changes needed to implement data as a product over time. This isn't a switch you flip. Other Important Takeaways (many touch on similar points from different aspects): Data as a product is a lot about the mindset and approach you bring to data. What have we learned from product management - in software and elsewhere - that we can bring to data? Most good general product experiences - at least on the consumer side - don't need a ton of hand-holding. Can we actually get there on data? Do we want it to be that easy since it's still easy to misinterpret the data? This balance will be different for every organization and probably every data product. Relatedly, fully self-service is a real question. You want to lower the amount of information requests to data owners but some of those questions can be value generating. So you want to build out the experience - especially documentation - to answer the basics but there's a question of how far you go relative to trying to document everything. There's a major question about how much change management is involved in learning to treat your data as a product. Is it just a mindset shift? Probably not. But then how do you actually change the organization to start focusing on data as a product? Jen said, "… the primary purpose of data as a product is to maximize data as utility." There isn't a single solution or approach to 'solving' product management. There won't for data as a product either 😅 prepare yourself and your teams for that. It's going to take sustained learning and evolution to get better and better. There is no 'done' but there is a ton of value to accrue along that learning journey. It's okay - if not ideal - to have multiple things in your organization called 'data products'. Not all have to meet the data mesh definition. But make it clear to people what you mean around data products - potentially call them mesh data products - or their thinking on data products will be, "okay, but just what the heck is a data product?" 😅 Software product management is to software products as data product management (the application of data as a product) is to data products. No one thinks software products and product management are the same. We shouldn't in data either. Data work and learning how to do data well both generally have a high cognitive load. Be prepared for that. Don't expect everyone to get it right away, whether that is the why of treating data as a product or the how do we actually do this :) Data as a product will be hard to instill even inside your data team. Again, this will take time and you have to let people know the why and the how. Similarly, you need to be prepared for sustained effort to communicate the benefits to those in the business. People aren't jumping up and down to own their own data and especially not to do it well 😅 Are people ready for the best product approaches in data? Doing product management well is about trying, fast failing, learning, then iterating. Are people ready for there to be more continual change in how data is served? Part of product management is stakeholder management and especially communication. In data, we need to move past requirement gathering but we also have to find better ways to communicate value and get - and then retain - buy-in. Product management, at the end of the day, is about capturing value through creating value for others. With data as a product, you need to understand what value is expected to be created from your work and where you might increase that value. But data doesn't have inherent value unless it is used so you need to stimulate data usage to create value. An important benefit of managing your data as a product is the improvements in data user experience. That means more time spent on creating value instead of wrangling data. Part of good product management is product marketing. That means discovering what data should exist but also finding champions, internally marketing successes, etc. You need people to see the value of the data work to get them to want to lean in. People aren't paying close enough attention to inherently know the impact of the data work, you have to tell them 😅 Products have owners. Treating your data as a product means your data has strong ownership. It's easy to say you are creating data products but really instilling that ownership is crucial to doing data at scale reliably. Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. Tom's LinkedIn: https://www.linkedin.com/in/tomdw/ Data Mesh Belgium: https://www.meetup.com/data-mesh-belgium/ Video by Tom: 'Platform Building for Data Mesh - Show me how it is done!': https://www.youtube.com/watch?v=wG2g67RHYyo ACA Group Data Mesh Landing Page: https://acagroup.be/en/services/data-mesh/ In this episode, Scott interviewed Tom De Wolf, Senior Architect and Innovation Lead at ACA Group and Host of the Data Mesh Belgium Meetup. Some key takeaways/thoughts from Tom's point of view: Platform engineering, at its core, is about delivering a great and reliable self-service experience to developers. That's just as true in data as in software. Focus on automation, lowering cognitive load, hiding complexity, etc. If provisioning decision specifics don't matter, why make developers deal with them? The key to a good platform is something your users _want_ to use not simply must use. That's your user experience measuring stick. When building a platform, you want to hide a lot of the things that don't matter. But when you start, especially with a platform in data mesh, there will be many things you aren't sure if they matter. That's okay, automate those decisions that don't matter as you find them but exposing them early is normal/fine. Relatedly, make that hiding easy to see through the curtain if the developer cares. Sometimes it matters to 5% of use cases but also often, engineers really want to understand the details just because they are engineers 😅 Make a platform where people can customize their experience where possible without going overboard. ?Controversial?: Few - if any - current tools in data are "aware" of the data product, they are still focused on their specific tasks instead of the target of creating an actual data product. Relatedly, the developers should be able to focus on creating and maintaining data products instead of focusing on leveraging specific tools. We need platforms that allow them to deliver value through creating and managing data products, not a focus on working with tools. ?Controversial?: Data mesh without technology is just theory. It can't only be about the people - if you focus on evangelizing without anything practical to show, it is too theoretical or abstract for people. You need a platform early to be able to show people what you mean. Scott note: you need a thin slice that has at least some aspect of all the 4 mesh principles early or your implementation becomes lopsided. Relatedly, get to something to show people in a demo as soon as possible with your mesh implementation so they can picture it and understand what you're trying to accomplish. In data mesh, you will still have data developer power users that really want to dig deep. But a key focus of your platform should be to make it easier for non-power users to still build and maintain great and valuable data products. Expand the potential number of people creating data products by lowering the bar. ?Controversial?: The platform team shouldn't be a blocker to new data products being developed. However, you should probably have certain cost guidelines/guardrails so someone doesn't develop a very expensive data product - it should only go to a central team for oversight when cost becomes an issue. That way, you prevent unnecessary friction and costs simultaneously. When there is an escalation because of a problem with a data product related to cost or governance, look to frame it as a collaborative deep-dive into an issue. Rather than a central 'you can't do this', it’s much more of a 'why is this made this way? Is this optimal or can we change it?' That collaborative discussion can keep people engaged and leaning in. You can get domains more bought in on something like data mesh/data products by showing them how this new approach won't directly tie data schema to their application schema. That way, they can still easily make changes to their application schemas and not break their data products. Good engineering is all about managing tradeoffs. Platform engineering is no different. You'll have to look at what should be specific versus generic. Orchestration is one area that should be very generic. In data mesh, you want to think about the holistic flow of data and data product work. That's why we need a platform. But the tools aren't really that well built to work together. Be ready for that frustration of having to build on top of the tools to get them to play nicely. ?Controversial?: Even if you aren't doing data mesh, your platform should focus on abstractions. What matters and why is a fundamental question. You want people solving challenges that add value. Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Please Rate and Review us on your podcast app of choice! Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here Episode list and links to all available episode transcripts here. Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn. Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here. May's LinkedIn: https://www.linkedin.com/in/may-xu-sydney/ In this episode, Scott interviewed May Xu, Head of Technology, APAC Digital Engineering at Thoughtworks. To be clear, she was only representing her own views on the episode. We will use the terms GenAI and LLMs to mean Generative AI and Large-Language Models in this write-up rather than use the entire phrase each time :) Some key takeaways/thoughts from May's point of view: Garbage-in, garbage-out: if you don't have good quality data - across many dimensions - and "solid data architecture", you won't get good results from trying to leverage LLMs on your data. Or really on most of your data initiatives 😅 There are 3 approaches to LLMs: train your own, start from pre-trained and tune them, or use existing pre-trained models. Many organizations should focus on the second. Relatedly, per a survey, most organizations understand they aren't capable of training their own LLMs from scratch at this point. It will likely take any organization around three months at least to train their own LLM from scratch. Parallel training and throwing money at the problem can only take you so far. And you need a LOT of high-quality data to train an LLM from scratch. There's a trend towards more people exploring and leveraging models that aren't so 'large', that have fewer parameters. They can often perform specific tasks better than general large parameter models. Similarly, there is a trend towards organizations exploring more domain-specific models instead of general purpose models like ChatGPT. ?Controversial?: Machines have given humanity scalability through predictability and reliability. But GenAI inherently lacks predictability. You have to treat GenAI like working with a person and that means less inherent trust in their responses. Generative AI is definitely not the right approach to all problems. As always, you have to understand your tradeoffs. If you don’t feed your GenAI the right information, it will give you bad answers. It only knows what it has been told. Always start from the problem you are trying to solve rather than the approach you are trying to use. Then evaluate if GenAI is the right approach for that problem. Simple, fundamental stuff but it's crucial to remember: start with the problem before the proposed solution. Many people are leaping to use GenAI because their past approaches to certain problems haven't worked. Dig into those pains. GenAI may or may not be the right approach but either way it can be great for surfacing persistent challenges. Leverage people's enthusiasm for GenAI to have deeper conversations about general business challenges. It can really start to highlight friction points across organizational boundaries and who is responsible for what. Scott note: But as the data team, be careful not to try to fix the entire organization, that's not what you are responsible for 😅 Right now, despite all the hype, most organizations are still at most in small-scale PoCs around GenAI. There is less of an initial focus on return on investment versus what capabilities GenAI might unlock but there is also a focus on what risks GenAI may introduce. Despite the hype, many to most organizations are doing their diligence. May started with three general approaches organizations are taking to generative AI (GenAI): 1) building their own LLMs from scratch, 2) fine tune specific, pre-trained existing LLMs, or 3) leverage pre-trained LLMs as is. Many organizations may want to do the first but it is prohibitively expensive to train your own LLMs from scratch just for the compute and you also need (very expensive) people with very specific expertise to do so. Tuning pre-trained models will likely become the standard approach for many organizations. However, being able to leverage LLMs on internal data in general requires "existing good quality data and solid data architecture." When considering training a model from scratch, May also pointed to time as an issue. Typically, it takes at least three months to properly train an LLM from scratch. Parallel training is helpful but you need to fine-tune results and retrain so you can't just throw compute at it and make the process that much faster. So again, you need high quality data - and you need a LOT of it - plus a fair amount of time plus a ton of money. Once you are in production, it also takes a lot of money and effort to keep them running and tuned properly 😅 Luckily, according to some surveys Thoughtworks did, most organizations recognize training LLMs from scratch isn't the right call for them just yet. May is seeing a trend of people moving away from the 'bigger is better' mentality. More people are starting to explore more targeted and specialized models that have fewer parameters. And often, for specific tasks, they perform better than the first L in LLMs. So we may see a trend towards more and more targeted LLMs/models. Scott note: Madhav Srinath really leaned into this in his episode, #264. Humanity in general has benefited greatly from machines through predictability and reliability according to May. Essentially, if they are made well, you essentially know what you should/will get from machines. But GenAI is designed specifically to act like humans and humans are not predictable and often not that reliable. So people have to get used to interacting with machines that may give wrong answers and are designed - in a way - to do so 😅 We can't expect predictability and reliability from GenAI. Relatedly, when thinking about where is GenAI the right choice versus like traditional machine learning/AI, May believes you really have to dig into the tradeoffs. If you really understand the problem set and what you are trying to accomplish, traditional ML/AI is probably the better approach for you. You need to really understand where the strengths of GenAI will play and feed it the data/information it needs to succeed, otherwise you'll be asking an uniformed and unpredictable entity to solve your most pressing business problems. That's probably not going to go well… May talked about going back to the basics of problem solving when it comes to Generative AI: what problem are you trying to solve instead of what way are you trying to solve a problem and then finding your way back to the problem. It can sound obvious but really, many are in such a rush to leverage these tools, it's crucial to stop and consider. Start with the problem before the solution 😅 GenAI may also surface a number of internal business challenges that aren't spoken about or people have essentially given up on tackling previously according to May. We have a new tool in the toolbox so people want to see if it will be useful to tackle something they haven't been able to address well previously. Lean into GenAI as a conversational lubricant. GenAI may not be the right tool for every one of these challenges but it means there is more internal conversation and sharing :) From what May is seeing, many to most organizations are still in the early experimenting and PoC phase with Generative AI. They are trying to figure out what opportunities GenAI brings and also what risks. Despite the hype, people are taking their time but they aren't as focused on initial return on investment, more to validate if they can actually leverage GenAI to create value. Also, there is strong trend towards domain-specific LLMs rather than general purpose ones, e.g. financial sector or media specific models. May finished on the idea that data mesh and other data management paradigms are crucial to doing something like GenAI right. There is still a strong need for quality data that is accessible, interoperable, privacy-aware, secured, etc. to be able to leverage GenAI well. Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/ If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/ If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf