Andrew Sharp, Author at Oakland

Thinking of doing Data Mesh? – Start with Data Products

Andrew Sharp — Thu, 20 Apr 2023 12:32:56 +0000

Data mesh has been the number one topic of conversation among the data crowd over the past few years. From the CDO Exchange to Big Data London, which even had a stage dedicated to the subject!

The four principles of data mesh have been debated, dissected, and diagnosed at length via books, blogs, conferences, and roundtables. But doing Data Mesh right can be expensive and undoubtedly a multi-year journey. A journey that is best undertaken in sensible steps for the most businesses.

What is the first step toward a Data Mesh implementation?

Of any of the four foundational data mesh principles, data products are, based on our current industry evidence, the most sensible place to begin.

Firstly though, it’s worth recapping what a data product means in a data mesh context. A data product is a reusable data asset built using data from trusted sources that directly solves a business problem or generates value via robust decision-making.

A data product typically corresponds to one or more business entities such as customers, orders etc. And the key feature of a data product is to make it easy to understand and use, discover and replicate and build upon (with other datasets either sourced centrally, locally, or externally).

In her seminal book ‘Data Mesh,’ Zhamak Dehghani explains the characteristics that set a data product apart from a traditional data mart or asset. A data product needs to be:

Discoverable
Addressable
Understandable
Trustworthy and truthful
Natively accessible
Interoperable and composable
Valuable on its own
Secure

She also declares the data product as the basic architectural quantum of a data mesh. To be clear, an architectural quantum is the smallest unit of architecture that can be deployed independently and still have all the structural components to do its job. So, this effectively means that a data product should encapsulate not only the data and its usability characteristics but also the data transformation logic and relevant data governance policies.

Current thinking shows that organisations considering or already implementing data products are grouping them into either analytical data products versus operational data products.

So why start with Data Products?

It’s the easiest to sell to people

Data products are the easiest of the data mesh principles to explain to those who use the data (design, build, and run) and, perhaps more importantly, those who control the data purse strings. Demonstrating that having a data product will give a scalable and repeatable solution that everyone can use and build upon is a powerful message that is more likely to achieve a commitment.

It’s the easiest to show value

Data products will show a return on investment far sooner than the implementation of the other three principles. Arguably get your end-to-end data product(s) right, and the value will be pretty immediate, which in turn will encourage you to do more deployments quicker and faster as momentum is developed.

You can do it under the radar

You can build data products without having to reconfigure your whole data estate. It can be done without needing a big data budget or data programme. You can start designing and building data products immediately if you have the right blend of technical and non-technical data teams working together.

You don’t need to worry too much about tooling

As mentioned above, a data product can be created in Excel (not something we would necessarily advocate!). Still, you don’t have to use expensive data tooling to get going in data mesh. You probably already have all the ingredients to design and build data products from a people, process, and technology point of view.

You don’t even need to worry about Data Mesh

Over recent years, the wider data mesh discussion has driven the debate on data products. But in fact, successful data product deployment can be data mesh agnostic. You can start and end your journey here. There are benefits to pursuing the other three steps. However, doing data products can be a totally discrete activity to enhance your data capability, but beware of the constraints of doing this in isolation.

Sounds Great – But there are some things to think about longer-term

Thinking you’ve done enough

While there might be good intentions to consider data products as only the first phase in a data mesh journey, it can be tempting to stop there and think, “That’s good enough.” Given the strong relationship between data products and data mesh, this might create the false belief that the company has successfully transitioned to a complete data mesh solution.

Struggling to maintain data quality at scale

Implementing data products will surely deliver immediate value and short-term satisfaction for data consumers. However, as the business scales, it will be difficult to maintain data quality without domain ownership due to the gap between data product developers and the domains where data is produced.

Bottlenecks!

If a centralised data team is responsible for maintaining your data products, requests will inevitably get backed up and must be prioritised. This will lead to frustrated data consumers. So data product developers might even be tempted to cut corners on quality or usability under pressure to reduce the backlog. This should be carefully managed, and a clear data product deployment plan should be created to minimise this from happening.

Interested in finding out more?

If you are starting your data mesh journey and wondering where to start, you could benefit from engaging with an experienced consultancy. At The Oakland Group, we help our clients see the big picture and avoid falling into traps like the ones we have talked about in this blog. We can draw on our skills and experience to help you plan your journey and implement your vision to a successful outcome.

Talk to the Oakland Group Team about how to build out your Data Products capability and how we can best assess whether Data Mesh is the right solution for you.

Andrew Sharp is a Principal Consultant at Oakland

The post Thinking of doing Data Mesh? – Start with Data Products appeared first on Oakland.

Data Mesh – Is this the evolutionary trigger to reinvigorate Data Governance?

Andrew Sharp — Tue, 01 Nov 2022 10:09:07 +0000

Just over a month on from the crowds and excitement at Big Data London (BDL). One of the key themes, which even had its own theatre was Data Mesh, and most of the two days were spent debating Data Mesh’s merits. While the first and second Data Mesh principles of “Data as a Product” and “Decentralised Data Domains” stole the limelight in London, there was little debate about the fourth, and arguably the most important, principle, that of Federated Data Governance.

It is arguably the most important and least understood Data Mesh principle. This is odd, given that it will be critical to Data Mesh success in particular, but will fundamentally change how Data Governance is delivered going forward. Why do I believe this to be the case?

Let’s consider the theory and how data governance is delivered in most organisations through the usual lens of People, Process, and Technology.

The Theory

Data Mesh assumes that the overarching Data Governance policies, procedures, and standards will be agreed upon and written at the Federated level. This is the level above the Data Domains themselves so effectively by a central function or team. These Data Governance artifacts will then be deployed into a Data Mesh solution through largely automated mechanisms to ensure that the data being used by data domain users is accessible, available, complete, accurate, and fit for purpose.

The Reality

This sounds great at a conceptual level, but when you sit that alongside what is being done in many organisations to implement Data Governance at the moment, it is a million miles away from how businesses are typically trying to deploy their data governance framework.

If we believe that Data Mesh will become the new data paradigm, then those doing Data Governance will need to change their approach and behaviours not just slightly but materially. Let’s consider this in terms of People, Process and Technology.

People

In the event of Data Governance being delivered through more automated mechanisms, it is evident that the skills needed will move into a more technically led space. Currently, many Data Governance practitioners have fallen into the discipline because they are good with numbers, have loads of common sense, and can talk to stakeholders. Successful Data Governance professionals in the future will need not just these skills but equivalent technical skills – well beyond the doing a bit of SQL that most of us have seen in the past.

We are already starting to see this shift in recruiting Data Governance professionals who need strong technical and non-technical skills.

Process

A Data Governance framework is currently designed and deployed through a blend of Website, Intranet, Training & Education type mechanisms. In Data Mesh, policies, standards, and procedures will be far more automated, so the processes underpinning good Data Governance will have to change. In the same way, those doing Data Governance will need to have a good grounding in designing and building these automated processes.

Technology

As the Technology underpinning Data Mesh can be expected to use a heavy recycling or realignment of existing technologies – with the odd addition of faster query engines such as Starburst – this is likely to have the least impact on current Data Governance best practice. So long as Data Governance practitioners understand the value and implications of their respective Data Governance technical solutions, this area is least likely to evolve.

So is Data Governance at an evolutionary tipping point? We are now starting to see changes in how data governance is delivered to organisations. Adoption of Data Mesh can and will escalate that evolution. It remains to be seen who will be the winners and who will be the losers.

Andrew Sharp is a Senior Data Governance Consultant at The Oakland Group

The post Data Mesh – Is this the evolutionary trigger to reinvigorate Data Governance? appeared first on Oakland.

Where to start with data governance tooling?

Andrew Sharp — Tue, 23 Aug 2022 15:40:34 +0000

Good Data Governance is becoming an essential component for organisational growth and resilience in an increasingly data-driven world. The challenge for Data Governance practitioners is how to support their organisation along the journey to high data maturity and effective governance. To help with this journey, Data Governance tools are often considered, but with a multitude of companies providing Data Governance tooling, selecting the right one for your organisation can be difficult. Before you start the selection process, there are some key questions you need to answer.

Why do you need a Data Governance tool?

As you build out your Data Governance framework, you will start to create resources that benefit your whole organisation, from glossaries to lineage models. A tool helps create a structured way for these resources to be shared and evolved. You might want to use the tool for some or all of the following information.

Data Catalogue – defining available data
Business Glossary – defining key terms
Data Profiling – understanding the data within systems
Data Dictionary – defining calculations and data elements
Data Lineage – understanding the journey data takes across an organisation
Document policies, standards, rules, and classifications

The fundamental part of the selection process is clearly defining the ‘why’ you need a tool. However, it is important to understand that any tool should not just be a storage solution. Clarity is needed on what the tool should be able to do for your organisation, and what problems you are trying to solve. More importantly than this is ensuring end users of the tools are engaged in providing data and how they will interact with the tool. Any tool selected needs to be easier to use than the alternatives to enrich and engage with all stakeholders and users and provide value. So that it becomes a supportive part of your resources, not a hindrance for users getting their jobs done.

At what point do you need a Data Governance tool?

It sounds obvious, but tools work best when organisational culture can support them because using a tool effectively means having people capable of interacting with it in a meaningful way. At the start of the Data Governance journey, most information created about data can be comfortably documented within readily available productivity tools. You will also find that the skills and knowledge of your key users at this stage often mean that they need significant support to use any tool.

Tools come into their own when you have significant detail at a granular level across entire data sets with multiple functions across an organisation being represented. Alongside, your users will need to have significant capability in both enterprise and data knowledge to use the tool to provide value which may involve upskilling your existing team.

Populating the tool should not be the end goal of any Data Governance tooling project but the first stage of providing meaningful, timely, accurate information about the data across your organisation which helps support the Data Governance journey. This ensures data becomes a valuable asset and not a blocker to growth.

What do you need to consider before getting your Data Governance tool?

While many tools can connect to core systems, warehouses, etc., and therefore build linage models and profile the data, the knowledge of what the data means will often be in minds of key individuals. Already having this information documented allows the tool to be used faster. Bringing in a tool at a time in which you have to spend months setting up and getting key personnel to document their knowledge will be less valuable than if the information is available to use straight away, meaning that the tool starts providing value from the start.

Data Governance tools support the Data Governance work, but they cannot replace it. Bringing in a tool will not do Data Governance for you. In fact, it might end up hindering your progress if you do not have the right capabilities and information in order to use it effectively. Any tool you use is only as useful as the knowledge of the people creating information within it and the capabilities of business users to use it. So you need a network of people with a deep understanding of data that are engaged and ready.

How can Oakland help?

Choosing the right tool is a difficult task, but here at The Oakland Group, we help you understand where your organisation is on its data journey and when a Data Governance tool might support this through:

Data Maturity assessments
Data Governance Framework creation
Data Governance Review
Building Data Governance through Lighthouse Projects
Creating an organisational Data Strategy
Driving a full Data Governance journey
Development of tool requirements leading to evaluation, selection, and implementation of appropriate tooling.

While this might start your thinking on tooling here at Oakland, we can help you bring the process to life.

The post Where to start with data governance tooling? appeared first on Oakland.

What happens when your employees think they are doing better with Data Governance than they actually are

Andrew Sharp — Mon, 25 Jul 2022 11:05:40 +0000

The classic start point in understanding your Data Governance need in an organisation is to assess your data maturity.

Once you have chosen which data maturity assessment to use, this assessment will tell you where you are doing well and not so well. Then as the theory goes, it will allow you to work out your specific data governance deliverables – for example your data ownership approach, what needs to be in your data quality toolkit and how you ensure there is a common business language for data.

It sounds simple, doesn’t it?

But what happens when you and your data team know your maturity hasn’t got past maturity level one, but the people in the organisation believe they are doing far better. So why do we need to bother with Data Governance?

Your pain point here is you have a data culture issue whereby the perception is very different from reality. But how do you fix this?

Before you do anything, you first need to tackle why there is this disconnect?

It will be due to several overlapping factors, but the most obvious suspects will be some or all of the following:-

Have employees misunderstood what data governance is actually about?

It is worth making sure that employees are not confusing data protection and security with data governance. It’s a very common problem. These activities are all interlinked, but some employees see compliance with GDPR and ensuring their data is secure as the key data governance deliverables.

If you are experiencing this, you need to work with your colleagues in Protection and Security to help dispel these myths. In addition, there will be some communication and education to be done to help your employees understand the differences.

Is there a lack of communication at the Executive level that could lead to employees concluding things are going well

If your business is not communicating with employees about certain topics, the default position for many people will be to assume everything is going well. Lack of Executive visibility is a killer for most data governance programmes as employees need to be told that things are not good enough.

An Executive (ideally the CEO) throwing in some “hand grenades” of truth about data governance will soon make people realise that they and the organisation are not in the right place. This needs to be done regularly and repeatedly to ensure employees, as a minimum, are aware that things need to change to drive improvement.

Has something happened in the past at the organisation to lead them to this conclusion?

Data Governance is not new; some organisations have been trying to implement it for years. If there have been failed attempts in the past, just repeating the same messages and approaches will definitely not give you the right cut-through.

Also, if employees are not told that past experiments have not worked, anything new will be less likely to be well received. So be open and honest about past shortcomings if you are trying to do Data Governance again because it did not work the last time.

Is your voice of Data Governance not being heard?

This is a tricky one as you cannot recruit a large data team overnight (and nor should you) and simultaneously shout about the importance and value of Data Governance. Usually, you have to grow the team organically and the downside to doing that is small teams of 1 or 2 people will often never get the cut through within the business – even if they have the ear of an Executive or Senior Leader who is championing their cause.

The solution is a combination of getting the key messages right and selling them to the key influencers within the business coupled with joining forces with your friends and allies across the business who do get Data Governance.

The old adage goes perception is reality is definitely a pain point for organisations trying to effectively implement data governance. The key will be to change that perception, but first, you need to understand what is driving it. Unless you do this, future attempts at implementing a data governance programme will be bound to fail.

The post What happens when your employees think they are doing better with Data Governance than they actually are appeared first on Oakland.

Is Starburst The Analytics Engine for Data Mesh?

Andrew Sharp — Sun, 24 Jul 2022 17:23:41 +0000

At Oakland, we’re proudly tech agnostic. What does that actually mean? We can work with whatever tech our clients already have or have set their sights on using. That doesn’t mean we can’t review or recommend tech solutions, but we don’t receive any incentives to push one solution over another. We like it that way as we can choose precisely the right tools for the job, and like anyone, we have our favourites, but these are carefully chosen as they are tried and tested by our tech team.

In the first in a series of blogs, we aim to give you some insights into just some of the tech we work with. We begin with one of the newest and most exciting new tech launches in the past few years.

Starburst: Starburst market itself as the analytics engine for all your data, but what does this mean?

What is Starburst?

Starburst is a highly scalable, distributed, in-memory SQL processing engine. So there are a lot of words in there. But hopefully, most developers will be familiar with the term SQL, which is why Starburst can be an important element. It’s distributed and scalable in a similar way to Spark. Why do we think it’s such a successful product? Because it allows considerable datasets to be computed in large, scalable ways.

Its ability to ingest vast amounts of data from disparate data sources is why it is important. Starburst is one of the biggest data integration engines. You may not have heard of it yet, but we think you will. It began life at Facebook as PrestoDB (later renamed to Trino). It was designed to let large complex organisations with domain teams in key business areas query data in a fast and decentralised manner.

So why should you care?

Starburst is one of the hottest new products to be launched in the last few years. We compare it to Spark and Databricks (whom we will cover in our next blog) five or six years ago, few of us had heard of either, and now they are multi-billion-dollar businesses.

Starburst’s strength lies in its ability to allow teams to connect the dots by sharing and ingesting data quickly and easily from wherever that data is held.

When a business needs insight, there are many reasons why generating data takes more time than producing insights. It could be time to transfer data, data residency concerns, compliance requirements, or iteration to identify the right data. To save time, data analysts take shortcuts which limit traceability and re-usability.

Starburst offers a unified way to query large amounts of data, enabling faster time to insights, which makes it incredibly useful for spinning up MVPs and POCs.

Where is it useful?

The larger the company, the more data sources it will typically rely on, which poses a massive challenge to how the data is ingested. Starburst’s ability to join data across different databases and stores is very compelling. Traditionally this is time-consuming, and we have seen many projects held up at this stage impacting an organisation’s ability to report accurately. For example:

Sensitive data or data across geographies: “What is my revenue per product category per geography.”
Frequently changing data such as segments, channels, and digital journeys “What is the performance of a segment knowing that segments are evolving every year.”
Post merging companies: “What is the consolidated performance.”

In those use cases, typical approaches lead to more data than required, implementation challenges, and maintenance efforts. Querying only the required data facilitates insights and governance.

Furthermore, where organisations have a centralised data function, teams can often be waiting for their central data engineering team, who we know are incredibly busy. You now don’t have to wait for the data to be put into your data lake or warehouse (or Lakehouse…or whatever we’re calling it these days).

It will allow you to start ingesting from source into your memory and start processing it. This lets you create connections to all your sources and use it with one SQL statement. Saving both money and memory as queries use a cost-efficient cluster. That’s where we believe the power lies in helping demonstrate value quickly.

Starburst helps connect teams, which means you are slipping data mesh under the radar. It’s a tool that opens up that connection to the other groups, the other data sources, repositories, and data lakes – without needing to provision three or four accounts for each. It can also enable you to create and maintain a user group that everyone can access, so you know who’s using it and who can access and modify particular tables showing the full lineage, which can help bring data mesh to life. It should be said; that technically it does this by integrating with tools like Ranger, Atlas, and Purview to provide the role-based access controls and other governance capabilities.

Teams can better understand the reporting logic inside out and build and own the reporting layer without waiting for the transformation program or change project to deliver your report. Individuals can provide their own reporting layer in a mesh/ data hub/data mart in their domain-specific teams, leaving this centralised data layer to be maintained by the central data team.

We think Starburst will be big simply because of its use cases, flexibility, and power. Although we have to say it isn’t the right solution for everyone. For organisations not looking to decentralise their data, or those with significant ongoing investments in tech, a low level of data maturity and lukewarm senior stakeholder buy-in. It’s probably not the right solution for you.

In our next blog, we’ll look at another US power player Databricks.

The post Is Starburst The Analytics Engine for Data Mesh? appeared first on Oakland.

How does traditional Data Governance get ready for Data Mesh?

Andrew Sharp — Thu, 26 May 2022 20:17:33 +0000

Much has been written on the technical changes arising from more and more Data Mesh deployments as organisations seek to achieve a step-change in collecting, organizing, and utilising their data assets. But far less has been written about the non-technical changes that will be needed to achieve such a change in general and Data Governance in particular.

In her latest instalment, Data Mesh Guru, Zhamak Deghani https://www.thoughtworks.com/profiles/z/zhamak-dehghani gives an overview of the significant cultural and organisational change needed to implement a Data Mesh solution effectively.

This includes the recognition that the current role of Data Governance will need to “shift”, with the caveat that the exact manner of the “shift” will be expected to vary from one organisation to another.

This undoubtedly will be the case, but we believe it will need to be more than a “shift”. It is much more likely to require a complete rethink in how Data Governance Managers and their teams organise themselves and carry out their roles.

This will be new news and not necessarily what current Data Governance professionals expect.

So the sooner the profession starts to face into the changes arising from Data Mesh, the more likely that Data Governance can be part of the solution rather than being an afterthought.

Let’s consider some of the Data Governance activities likely to change.

Data Policies & Frameworks

The effective data governance of Data Mesh will still require the appropriate Data Policy(s) and Data Framework(s). This is good news as this is one of the key tasks to good Data Governance, and this will continue with the adoption of Data Mesh. However, the bad news is that the policies and frameworks will need to significantly change to reflect that those involved will now reside in very different types of teams.

The more traditional Data Governance command and control approach from a centralised team or individual(s) will simply not work in the same way. Data Governance professionals will need to think about how a data mesh data policy will align across multiple data domains and data structures.

Data Ownership

The current best practice approach to data ownership is to create a network of Data Owners, Data Stewards, and Data Custodians. Taking data stewards as an example, currently, these individuals are in charge of managing, investigating, and resolving data issues for one or multiple domains. With data mesh, this role will move to the specific data domains and become part of the domain itself. Therefore, the stewardship role becomes more about being a data product steward.

People currently in these owner, steward, and custodian-type roles will have a choice to make. Either shift to a domain data product role or become a data platform specialist, assuming such a role exists. This role may no longer be required for data custodians who are usually hands-on and in charge of the day-to-day work with data sources and maintaining the data.

It could shift to the data mesh domains as a data product developer role, but these are very early days, so it is too soon to be sure.

Also, it is essential to note that in a Data Mesh structure,

if the roles and responsibilities have been well defined, it begs the question as to whether there will actually be a need for the traditional data ownership model.

Data Artefacts

In the same vein, within Data Mesh, the need for data definitions to ensure consistency across the data domains will be even more important. However, getting such discipline will be more challenging than at present, as Data Governance teams will have to deal with a broader number of stakeholders.

Data Committees & Councils

The structure within an organisation, which is the highest decision-making body with accountability for data, would change in a Data Mesh world. The representation at such committees and councils would be a federated model with representatives from the respective data mesh domains rather than the more traditional functional areas.

Data C-Suite

Functional executives, such as the Chief Data Officer or Chief Technology Officer, have been concerned with all things data. They are accountable for the enterprise-wide creation, utilisation, and governance of data that drives business benefit. Moving data from a specialised concern to a generalised one, with the distribution of the data responsibility moving to cross-functional domain teams and the dispersion of data expertise, will change their roles.

Leading to more of an enablement role, with close collaboration required with fellow C-Suite colleagues.

Data Literacy

Data mesh requires everyone to have the core skill sets needed to recognise, understand and use data. There will be the need to remove the barriers of organizational silos between the data specialists and everyone else to help ensure the cross-fertilisation of the appropriate data skills. Organisations will need to invest in creating and executing planned data literacy programmes for all levels of staff, including increased data training and new career development pathways to allow people to step into these new roles. It will be even more important that people share data knowledge across the organisation to achieve the common goal of data democratisation.

In Summary, Data Mesh is still at an early stage of evolution.

However, it will undoubtedly change the way in which Data Governance is designed and delivered. Data Governance professionals have a limited window of opportunity to plan how they will adapt and modify their roles as these changes happen.

If you’d like to talk to one of our data governance experts on how to integrate data mesh into your organisation then please contact us.

The post How does traditional Data Governance get ready for Data Mesh? appeared first on Oakland.

Is Data Mesh a potential Data Mess?

Andrew Sharp — Tue, 17 May 2022 09:05:17 +0000

Since we re-emerged from lockdown, the Data & Analytics industry has seen a massive boost and investment in data as businesses escalate and turbocharge their data agendas on a technical and non-technical basis to ensure they have the best digital presence. Throw in the emergence of Data Mesh as a potentially new and better way of organising, processing, and delivering data to organisations and you have all the potential ingredients for a perfect data storm.

Over the last few weeks, Data Mesh has been the hottest topic at the data events I have attended. But interestingly, awareness and understanding vary considerably even amongst data leaders. Ranging from those who know something about Data Mesh. Typically they have read and watched the excellent Zharmak Dehghani from Thought Works https://www.youtube.com/watch?v=L_-fHo0ZkAo set out her rationale and vision. To those who have heard of the term, but know very little or nothing much about it beyond that. Through to those still in blissful ignorance that a new “data kid” is now coming onto the block! Then, there are those who still wonder whether Data Mesh is the same as Data Fabric.

So it seems there is an awful lot of talk about the theory of Data Mesh, but much less visibility and debate about what is actually happening in practice in companies and the data teams themselves.

Taking the Data Mesh theory first, this is now starting to be better understood. We know that there are four high-level foundational principles – all of which make sense.

1). Domain-driven data ownership

2). Data as a product

3). Self-serve infrastructure as a Platform

4). Federated Governance.

That said, we have after all been talking about these four things, in whole or part, on and off, in different guises for the last 20 years as the suggested Data best practice(s) for an organization. Data Mesh shows how they can all be brought together.

So four very sensible principles that few who currently work in data would violently disagree with. You then overlay on top of these the view that our existing data technologies, particularly Data Warehousing and Data Lake, have not delivered on their original and planned expectations. Dehghani feels that these data solutions have failed in their objectives. Perhaps a harsh statement given the many millions spent and continuing to be spent on them across the globe. A further criticism is because they have not created a single version of the truth as intended and have become centralised and monolithic structures that incur significant costs to build and maintain rather than the agile solution for which they were intended. Also, they have created siloed ways of working and have had limited success in providing strong and fresh data insights to business users. These criticisms make it easy to make the argument for a new technical data solution. Oh, and lest we should forget, the acknowledgement that many companies have and still continue to struggle with the non-technical elements of managing data – e.g. Governance, Literacy, and Culture, then Data Mesh does seem to offer a way forward.

However, what did Alexander Pope famously say, “a little knowledge is a dangerous thing”? As a data expert you can immediately sense the same is happening in the UK data community with Data Mesh. A few of us have read the book and seen the videos but have very little practical experience of it and whether it can address all criticisms levelled at the current data landscape.

For example, when you start to scratch beneath the surface, it isn’t easy to find any UK organisation doing Data Mesh in any meaningful way. Also, any company which is doing it seems to be in the US rather than based in the UK. Then for those you can find who are claiming to do it, you find a few are calling it Data Mesh (e.g. Netflix), but it isn’t really, and there are a few who have done such a tiny element of it that it seems such a token gesture as to hardly warrant a discussion.

So if nothing else, Data Mesh is creating a lot of conversation. There is clearly much Data Mesh talk happening here and overseas within the technical data teams and less within the non-technical teams, which is understandable. The excellent Data Mesh Users Slack channel has been created – and helped to flush out some excellent discussion and debate on key topic areas. But are we still at risk of rushing the data community into focusing on a solution that, for the vast majority of companies, are not yet ready for.

Maybe we can learn from those controlling the budgets at the top table we need to hold our nerve before we rush to make strategic and tactical business cases for Data Mesh. Before we commit our limited budgets and resources, we need to be convinced of its credentials and impacts.

Data warehousing and Data Lake are far from perfect, but they have served us well thus far in collecting, hosting, processing, analysing, reporting, and storing data. Are we really sure that focusing upon Data Mesh will remove all of the ongoing data challenges we continue to face every day, like Data Governance and Data Literacy. Undoubtedly, it offers a way forward and might somewhat ease some of the problems in certain areas, but I suspect the underlying data issues will persist regardless.

For sure, Data Mesh is likely to be here to stay. Indeed, it will undoubtedly find a place in our data technology toolkit going forward at some point as its understanding and knowledge of it matures, and some companies decide to take the plunge and invest in an unproven solution.

But lets as a community of data professionals not make a mess of it before we have even started and make sure we understand and see its practical application first.

Andrew Sharp is the Oakland Group Data Governance Lead

The post Is Data Mesh a potential Data Mess? appeared first on Oakland.

The five signs your Data Governance activities are working

Andrew Sharp — Tue, 15 Mar 2022 10:34:25 +0000

Wondering how to kick start your Data Governance programme? You’re in luck then; there are an overwhelming number of books and articles on how to do exactly that. These include the classics from Rob Steiner on Non-Invasive Data Governance https://amzn.to/3q6BPCX and the thought-provoking John Ladley with the concisely titled book Data Governance https://amzn.to/3q6o3QV. The Oakland Group published an equally informative Data Governance by Stealth Guide http://bit.ly/393R86o. This tells you all the things you need to consider and helps you plan for getting up and running, and critically that you don’t need loads of investment to do it! It also sets out how to ensure you align your data governance strategy to your broader data strategy.

This guide is a great resource to get you started, but there is little information available on how you can measure the success of all of your hard work. Sure, these books tell you all about the formal stuff you need to show your stakeholders at the relevant Data Councils, Committees, and Forums how you are doing. Things like how the number of open data issues are coming down month on month, how many data quality issues have been fixed in the quarter, or how many employees have read the Data Governance policy in the e-learning module. All critical ways to formally demonstrate that your efforts are having an impact.

But since successfully landing Data Governance into any organisation is really about winning people’s hearts and minds, the success of any Data Governance programme is not just about the positive movement against an agreed set of KPIs on a dashboard. It is that emotional connection or the warm and fuzzy feeling you get from talking to the people within your organization when you know your efforts are cutting through.

This is actually the thing that will really help you judge that the Data Governance activities you have instigated are being embedded into your business. These are those soft but hard to quantify things that you sense within an organisation as it progresses on its Data Governance journey.

Coffee Conversations

Downtime at work is important, and what better way to spend it than stopping off for a coffee and having a chat with your colleagues? Generally, the stuff that people talk about in these breaks are the burning topics that are closest to their hearts. Hearing them talking about data quality, data governance, and master metadata data at these break times will be an excellent barometer of whether your efforts are paying dividends. And whether they are positive or negative comments too!

Uninvited Attendees

Meetings and workshops are part and parcel of modern working life regardless of whether you are remote or face to face. Getting Data Governance up and running requires numerous and regular meetings with stakeholders and colleagues to ensure there is an agreed, ongoing and collaborative approach to delivery. Usually, we try to limit those attendee’s meetings to avoid meetings for the meeting’s sake and ensure only those who need to be there are involved. But from my experience, another great temperature check is when people you haven’t invited ask to attend or turn up regardless. This can be challenging to manage, but treat it as a positive as it means the message about Data Governance is getting out there.

Taking Note

Much data governance relies on people reading and following the required policies and frameworks. So many measures used to determine the success of Data Governance judge this being the number of people who have read the policy or framework or percentage completing a Data Governance learning module. But simply clicking to say they have read the Data Governance policy is not the same as saying someone has actually understood what they have read and, consequently, is changing their behavior to comply with that policy. This is a difficult one to judge – and it is not specific to Data Governance – but talk to some of the people who claim to have read it and get a sense of whether they have just skimmed it and not taken it in. You will very quickly get a sense of whether the policy has been read and understood, is impacting people, and how much you are winning hearts and minds!

Unexpected Volunteers

Effective Data Governance is heavily reliant on having robust data ownership across an organisation. So much time and effort is spent on working out the right structure for data owners, stewards, and custodians. It is central to the success of any Data Governance initiative. But it is when either those individuals and or those not in those nominated roles start to offer up their time and services to help with taking the data agenda forward you will know that employee attitudes and behaviours are, in fact, changing in your business.

Is there a Buzz?

We have all walked into organisations and been able to judge a business’s atmosphere or mood very quickly – or put another way, is there a buzz about the place? Evaluating the effectiveness of your Data Governance programme is no different – can you sense a positive or negative atmosphere to what you are doing? Are people talking up the data quality issues they are working on or moaning about the amount of time and effort it takes to tick the Data Governance and Data Protection box? What sort of language are people using when you talk to them about it? Listen and observe.

In summary, effective Data Governance is all about winning the hearts and minds of your teams. We need to be careful not to fall into the trap of just using data tools and techniques that show the formal progress of an initiative on a dashboard to please the Data Council and miss those “warm and fuzzy” indicators that are all around us. Ultimately, it should be a blend of the two.

Andrew Sharp

Data Governance Lead

The post The five signs your Data Governance activities are working appeared first on Oakland.

Modernising and Monetising your Data Governance

Andrew Sharp — Wed, 19 Jan 2022 10:58:48 +0000

Covid 19 has forced many companies to rapidly accelerate their data and digital transformations. And with this rapid modernisation of systems and processes, the importance of having a robust Data Governance framework has also grown exponentially.

Which specific Data Governance tasks vary from company to company depending on their size, their nature, and the level of data maturity they already possess. So, data ownership remains a critical challenge for some, whereas it is a priority to improve data quality for others. For others, it will be a need for both data ownership and data quality, plus numerous other key data governance activities. The typical ‘one size fits all’ model can no longer offer the dynamic and agile approaches that digital-savvy companies require.

While the need for an adaptive data governance approach is one of the enablers for successful business transformation, demonstrating the value of investing in the necessary people, processes, and technology continues to be elusive. This is partly driven by a strong perception amongst those who control and make budgetary decisions that data governance is a cost rather than a benefit. Having such perceptions can and will stifle the ability of an organisation to fully and successfully implement data governance.

So along with implementing the data governance activities themselves, your data governance plan must show how value is being derived. The current buzz is around how you monetise your data, and showing value from your data governance activities is just one element of this monetisation challenge. According to the Gartner (www.gartner.com) CDO 2021 survey, while 38.7% of CDO’s are measured on data monetisation, only 27.2% report that their D&A teams are producing tangible business value to the organisation.

We would advise caution in undertaking this monetisation activity (itself being perceived to be a cost to the organisation). So, as in previous blogs, we have spoken about the implementation of Data Governance being something that organisations can do without significant up-front investment (The recent Oakland Group Lighthouse report – Data Governance by Stealth – explains how many of the benefits of Data Governance can be achieved through no or low-cost approaches) Using this same idea of doing things by “stealth” should also equally be applied to how you monetise your Data Governance.

Let’s take Data Quality as an example. How do we show that fixing a specific data quality issue makes good business sense? Rather than pick a complex data quality issue, start by focusing on an issue that almost everyone understands well. A great one is to show the cost of having an incorrect customer date of birth.

So if you know that within your call centre, 1 in every 20 dates of birth are being misreported or are incorrect, and you know the amount of new business you are generating in any given day, say 60 new customers per call operator, this means three incorrect dates of birth are being generated on average per operator every single day! Scale that up by the number of call operators, say 10 in this example, then over a typical month, then some 900 incorrect dates of birth are being generated or nearly 11,000 of your customers in a year. That is a big data quality issue that needs fixing before you even start calculating the cost to fix it.

Armed with these sorts of statistics, you can estimate the cost of correction – probably the cost of an analyst. So using an hourly or daily rate, you can start to cost out how much your data quality issue is costing the organisation.

The costs of your analysts having to correct simple things like dates of birth manually can be costing your business £££s. However, this could be money well spent if your analysts can identify systematic patterns and trends in the data quality issue. So fixing these issues may incur an initial cost, but it will derive an immediate benefit if the problem is permanently fixed. Of course, in the call centre example, a key task will be training and educating employees to be more diligent, which will have a cost in terms of training materials and training time.

The way you monetise the value of the activity will vary by business. But you will quickly be able to show the financial impact of not doing data governance correctly over a given month or a year. Focusing on these easy-to-understand issues and simple arithmetic will quickly make your stakeholders sit up and take note. In fact, from our experience, sharing such insight with your Finance Team will very quickly result in them giving you more precise values to use in your calculations.

Another easy example is people moving house. We know from research that some 4 to 5% of the population move home each year. So if you do nothing to your data to check that the customer details are accurate, your data quality is eroding at least 5% per year. This is, of course, assuming all your data about those who have not moved was complete, accurate, and fit for purpose to start with! Again, this latter point is relatively easy to validate against recognized databases such as the Postal Address File (PAF) to give you a figure of how accurate your customer address data might be. Then using this figure, you can work out the indicative cost of content being sent to wrong addresses and the costs involved when customers ring up to explain a delivery or letter has gone astray. The numbers can be mind-blowing but easy to compute.

These few examples highlight it is then easy to start showing the cost of not fixing data quality for the most basic customer details. Suppose you capture a record of your data quality issues on a register. In that case, having an indicative cost for everyone will start to reveal the true cost of inaction to those who doubt the value of data governance!

Hopefully, this blog shows that it only takes some simple metrics to monetise the costs to a business of poor data quality. These statistics are compelling in getting people to sit up and think about the overall cost of taking the appropriate action on fixing data quality or, more generally, not implementing proper data governance. As the Oakland Data Governance by Stealth report explains, ((Data Governance by Stealth Lighthouse Paper, September 2021).this monetisation can be done by stealth and under the radar of extensive transformational programmes. But how many of us are telling this data story to our business stakeholders?

Andrew Sharp is the Oakland Data Governance Lead.

The post Modernising and Monetising your Data Governance appeared first on Oakland.