Data is core to modern, effective operating models that are helping leaders to increase efficiencies, optimise resources, and drive down costs – all while maximising profit. So, what do you need to do to incorporate data into your target operating model, and does it need one of its own? 

Read on for what you need to know about target operating models, and why data is integral to their success.

What is a Target Operating Model?

A target operating model – or ‘TOM’ as we like to say – is the blueprint of how your organisation runs. It includes the overall capabilities of the business, from people and processes to assets and technology, and is used to make sure operations and processes are in line with the vision of the leadership team. 

If you’re wanting to drive meaningful, long-lasting change, a target operating model is the place to start.

Data Target Operating Model Explained

As the name suggests, a data target operating model focuses on delivering everything data, from your overriding data strategy to ongoing data management . It’s just as aligned with your overall business TOM and vision, but the elements within are more targeted towards data. 

“A data target operating model is how an organisation’s data capabilities are structured,  managed and sustained to support delivery of the data strategy, which should align with delivering your business outcomes and needs.”

Craig Lambert, Senior Consultant at Oakland

Dive deeper into the importance of data to your business in our blog: Why is data important for business?

What Should a Data Target Operating Model Include?

Your data TOM involves the different systems and frameworks you have in place to aid data-driven decision-making. When developing a data operating model, we structure it using the following framework:

• Strategy and purpose

Sets the organisational vision and mission.

• Leadership and talent

Translates the strategy and makes it meaningful to different business units, as well as demonstrating the business’s ways of working.

• Organisational structure, roles and responsibilities

The size and structure of the team, and the different roles and associated responsibilities of each colleague. The policies regarding secure data management and how issues are raised and resolved are also included in this area.

• Culture and ways of working

Embedding data as a core business asset requires the right skills and mindsets. Your data TOM is the perfect place to set out what these are and how leaders/managers can empower employees to confidently own and manage data. 

• Technology, tools, and data

Typically includes the technologies that can be scaled easily to future-proof longer-term growth or automate tasks to increase efficiency. It should include all tools used for data integration, storage, and analysis, too, so the different data components are reflected in the wider business TOM.

• Processes and feedback loops

To deliver the data operating model effectively, well-defined processes that connect technology solutions are key. Here is where you can explain how insight over data issues should be shared with other areas of the business to make for more effective functions. 

Target Operating Model Case Study

One of the best examples of the importance of a target operating model is our work for a major UK media organisation. While creating the customer data platform that delivered growth and competitive advantage, we also developed a TOM. The organisation’s headcount and skillset was growing fast, so an operating model became critical to success.

Overall, the TOM we developed has delivered: 

• Increased control and governance across the delivery cycle 
• More streamlined and insightful execution 
• Better reporting, KPIs and performance tracking 
• Higher clarity over the functional split and handoffs between teams 
• Identification of areas needing investigation

For the full details of our work with this media organisation, please read: Creating a customer data platform for long-term innovation.

Harness Data to Deliver Real Business Impact

You’ll know that data holds the key to unlocking more efficiencies and informed decision-making within your organisation. But there’s a seismic difference between how far you’ll get with average data and the opportunities you’ll discover with great data. 

To deliver sustainable, long lasting value from your data and ensure desired returns on investment in your data capabilities and products, you’ll need a target operating model that’ll safeguard your business for years to come. 

Please contact our friendly team to find out how we can support data transformation within your business.

We’ve also got plenty of resources for you to use to improve the integrity of your data TOM and overarching data strategy.

The post Why Data is Integral to Your Target Operating Model appeared first on Oakland.

Last week, I had the opportunity to attend the highly anticipated Big Data and AI World conference, an all-encompassing event that showcased the breadth and depth of the data industry. From the infrastructure backbone of data centers to the cutting-edge applications in machine learning (ML) and large language models (LLMs), the conference offered a comprehensive look at the current state and future directions of data technologies. 

Highlights 

The conference prominently featured a range of topics within the data pipeline, emphasizing well-established processes including DevOps, DevSecOps, and Cloud Security. While the ML and LLM segment of the conference was relatively modest, it provided a valuable glimpse into the evolving landscape of AI applications. 

Key Topics and Speakers 

One of the opening sessions, aimed at a broad audience, focused on how to use ChatGPT, indicating a general entry-level understanding of LLMs among attendees. Despite this, there were notable discussions and presentations that delved into more innovative applications of LLMs: 

• Pangeanic highlighted their work on leveraging LLMs over data, showcasing how AI can enhance data analysis and insights. 
• TAAP demonstrated their initiative in developing applications powered by LLMs, pointing towards practical uses of AI in software solutions. 
• A showcase involved a company that developed a full-stack LLM application, extending from software down to the hardware level, emphasizing the potential for integrated AI systems. 

Microsoft also made an appearance with their Copilot solution, underscoring the growing interest in AI-assisted development tools – as well as a few resellers of Copilot. 

Notable Exhibitors and Innovations 

Among the exhibitors, Compare the Market stood out for their advanced development work with LLMs. They shared a series of best practices that they’ve implemented, offering valuable insights into the practical challenges and solutions in deploying LLMs in a business context. 

Equally compelling was the announcement from Multiverse, a pioneering quantum computing company, which presented a revolutionary algorithm capable of significantly condensing the size of large language models (LLMs), such as LLAMA2. Their technology has achieved an astounding 85% reduction in the model’s size with a mere 5% loss in accuracy. This breakthrough paves the way for more sustainable and accessible LLM implementations, reducing computational demand and making it feasible to deploy sophisticated AI applications in more constrained environments. 

These exhibitors highlight the innovative spirit permeating the conference, showcasing not only the practical application of LLM technologies but also the forefront of AI and quantum computing research. 

Personal Takeaways 

The conference provided a fascinating overview of the data and AI industry’s current landscape, with a broad spectrum of technologies on display. However, one of my key observations was that the application of large language models (LLMs) among exhibitors was primarily at a superficial level. Despite the potential of LLMs to act as powerful reasoning engines, their utilization seemed to be confined to more basic tasks such as content generation, serving as advanced chatbots, or simplifying interactions with data. 

This suggests that there’s a significant gap between the capabilities of LLMs and their current usage in the industry. Many exhibitors are yet to explore the full maturity scale of these models, leveraging them beyond the initial layers of application to truly tap into their transformative potential. The reliance on LLMs for relatively straightforward tasks underscores a broader industry trend of cautious engagement with AI’s deeper functionalities. 

Reflecting on the conference, it’s clear that there’s a vast untapped potential for LLMs to revolutionize not just how we interact with data, but how we reason with it, use it to make decisions, and innovate. The journey of integrating LLMs more profoundly into our technological solutions is just beginning, and I’m excited to see how they will be pushed beyond their current boundaries to achieve their full transformative potential. 

Author: Mike Le Galloudec is an Innovation Lead at Oakland

The post Everything you need to know about Big Data & AI World appeared first on Oakland.

There is Something Human about Waste

Let’s start by setting the scene – an all too familiar one. Humans love waste. We throw away around 100 billion pieces of plastic every year in the UK, and around 1.9 billion tonnes of food that can be eaten is discarded on an annual basis, and even on a household level – between 9-16% of energy is wasted on standby.

This fascinating graphic from No Planet B by Mike Berners-Lee shows the world supply chain for food based on energy loss at each stage. As you can see, at every level of the picture, our food waste. The depressing part can be seen in the bottom three rungs; even if we’re lucky enough to have food we still waste around 50% of it.

The impact of this level of overconsumption is evidenced in the world around us: fires in Maui, droughts in Uruguay, and extreme flooding in Bangladesh – all directly attributed to climate change driven largely by Greenhouse gas emissions.

But what does this have to do with data?

It doesn’t come as much of a surprise that physical human behaviour also maps onto digital human behaviour. The attitude towards waste and overconsumption has transcended from the physical to the digital in the case of data – and that has led to some huge inefficiencies in the way that we currently manage data – directly resulting in some pretty stark impacts on the environment. We talk about inefficiency through a few lenses – the first being the underutilisation of resources which we can see represented by the fact that the utilisation rate of on-premises servers is only around 18%, and it doesn’t get much better when we talk about cloud storage and usage, where around 33% of paid-for services are considered waste.

Combine this with the fact that 55% of stored data is considered dark (currently unused, with no future use-case) – a frightening statistic! That means stored and processed data that will never realistically be used for operational or analytical purposes. It’s one of the darker legacies of the big data wave of 2017!

Finally let’s not forget the onslaught of Artificial Intelligence. The amount of data we are creating is out of control.

Data Centres – The Lords of Darkness

When you consider the artificially inflated needs of server racks, the ramifications of this wastage are very real – data centres are being constructed at a rate of knots and are having pretty dire consequences on the environment.

Think about all the utilities that you need to maintain a data centre, primarily you might just think about the energy costs of powering your servers. Ignoring the energy requirements for cooling and lighting, server energy usage on its own has the potential for improvement. Server configuration and distribution of workloads can often drive higher than necessary energy usage.

Due to the sheer amount of energy digital technologies power through, data centres have become the world’s second largest source of greenhouse gases (2.7%) – behind petrochemicals but ahead of the poster boy of climate change, the airline industry (2%).

With no signs of slowing down, the environmental costs of increasing energy consumption will be around 14% of the world’s share by 2030.

All of this has led to data and digital becoming a primary concern for those interested in sustainability initiatives. Most large businesses require decarbonisation initiatives to meet regulatory requirements (e.g. SECR or CSRD). As the issue becomes more and more severe, these regulations are expanding from not only being focused on direct emissions (e.g. owned vehicles, building emissions) but also including scope 3 or supply chain emissions. In the context of data, on-premises data centres can utilise as much as 25% of an organisation’s total energy expenditure, so when you add your cloud usage into the picture – you can see the scope for improvement.

So what?

So we’ve set the scene – but why should businesses care? The way Oakland views this, there are four main drivers of why you should care as a business – boiled down into direct and indirect impacts. If you are considering or are currently involved in any kind of digital transformation, these are the metrics you should consider.

Cost

Direct impacts are associated with cost and regulation. With wastage comes excess fees – you can see enormous waste in resource use across cloud estates, and there is an associated cost with this resource wastage. Waste is the key word here – it’s cost that isn’t being spent on business outcomes.

The main concern with many sustainability initiatives is that they cost a lot of money and negatively impact business processes. However, with data, many of the issues are down to businesses not operating as efficiently as possible, meaning both environmental and financial costs are spiraling for everyone! By tackling these problems, you could potentially have an environmental sustainability initiative that maintains business processes and operations and positively impacts your bottom line.

Regulatory

Regulations are coming in that expect businesses (corporate sustainability reporting directive) to report not only their emissions for scope 2 and 3 (direct emissions and supply chain emissions) but also carbon reduction initiatives. You become more compliant if you can show how you plan to reduce carbon emissions across scopes.

Reputation

Reputation is critical – businesses are now becoming more attractive to prospective employees and customers because of their sustainability credentials. Imagine a data engineer who wants to go and work for two competing organisations. Both roles are bog-standard data engineering, but one of the organisations is committed to its environmental footprint by making its data infrastructure as carbon neutral as possible. The same goes for customers; if you can offer a sustainable option against one that hasn’t even thought about it, there’s a clear winner.

Doing what is right for the planet

Arguably THE most important driver – the long-term ecological disasters that are likely to increase due to the ever-increasing number of data centres and uncontrollable data estates. We mentioned them earlier, but on a more local level, we are seeing huge impacts of data centre development in places like Ireland (18% of energy is used by data centres); and in the US, where new data centres are being constructed in sunny, arid areas to take advantage of PV, we’re seeing reliant water consumption leading to severe droughts.

So really we need to tackle this at the level that we do business and change how we do things. Head to our Water Utilities page for more information on how we work with the water sector.

How to Approach Change

We need to talk about a new way to look at data operations. Can you support the delivery of both cloud and on-premises data estates in the most efficient possible way and have a more muted impact on the environment whilst having a more positive impact on your finances?

Let’s talk about FinOps

There’s been an obsession with FinOps in the past five years – and what that tends to focus on is cloud efficiency, but we saw from the stats before that it’s not necessarily working.

Rates of adoption are pretty low, action is limited to just IT or data teams, and cost becomes once again “a data problem.” When things are just “a data problem,” we don’t see the action which is needed.

So, let’s widen the conversation – How is sustainability baked into your data strategy? – that’s one of the key drivers to create buy-in to data across the business. As we’ve seen the next levels of both regulation and CSR being rolled into key strategic goals for organisations, we can leverage our approach to data in a way that appeals to these initiatives. Cut the waste and your cost can drop whilst meeting your strategic sustainability goals.

The answer? Include Sustainability

So, we approach data with a fourth lens – a lens of sustainability.

This acknowledges there is an issue with how we do business: by handling processes more efficiently we can save energy and carbon emissions, and by driving down energy spend and resource usage, cost savings can be achieved!

By revolutionising how we look at and assess the efficiency of our data estate, as a vehicle to support organisational sustainability initiatives we can drive the value from our data and drive the business outcomes needed from the data itself! This revolution moves us from looking at FinOps, to GreenOps.

Alas this isn’t easy – although we can quite easily say “how much” we spend financially on data, it is difficult to get a full picture of what our energy actually costs.

Oakland and our partners at Interact have devised a four-step process that businesses and data teams can use to start to adopt a GreenOps approach.

There are four main stages: Display, Diagnose, Decide, Deliver.

Display

You can’t take action without first understanding where you currently are – think of this as a baselining stage, but there are some major challenges in this phase – starting with understanding the impact your cloud is having. You need to consider many things, such as replication factors and networking – things that aren’t necessarily covered by existing cloud reporting.

Diagnose

A much more straightforward process – once you have completed your assessment, the next stage is to identify the key areas that are having the biggest impact without reaping benefits. One of the critical things from a cloud perspective could be where you do the majority of your computing, on-premises you want to look at utilisation rates and potential for consolidation to provide short term wins.

An example of this would be when we talk about the statistics of average utilisation of servers; combined with the linear nature of power usage vs. utilisation, you can triple the existing “utilisation” of that server (making it 54% utilised), whilst only using around 50% more energy.

That could result in a 300% increase in productivity for just a 50% increase in energy usage. Think about the carbon impact of this change and the cost savings that can be attached to your energy bills! With the high cost of energy, this is well worth an investigation.

Decide

Take the insights and diagnosis which enable you to start making decisions, reviewing what processes are necessary and if it is possible to change the location of your servers and the times of your heaviest usage.

Consider the wider impact that this has on your business – are there applications or data that you store on-premises that would be better served by migrating to the cloud? Or alternatively, you may even find potential for saving energy and money by moving your “archival” data back to on-premises servers.

The decisions you make here can be broken down into two lenses – short-term and long-term. Short-term decisions are those largely around consolidation and immediate fixes, such as changing the time of an ETL process to when a grid is being run on more renewable energy. To make these kinds of impacts, you don’t always have to fundamentally change what you’re doing in the short term – in fact, with Oakland’s carbon efficiency tool, we have seen that by shifting your heavy compute to one day rather than another has the potential to drop associated carbon emissions by around 10% – which costs nothing!

Your longer-term view of things might be migrating on-premises data to the cloud, but to do this most efficiently you need to understand the potential environmental impact – but the most important thing is to create a plan.

You want to focus on the areas you see the most potential to support your business and data strategies. If you’re going for a big push on governance, perhaps you need to initially consider where all that pesky dark data is sat and create a plan to decommission it, if you’ve got some horribly inefficient applications that aren’t optimised for the cloud – then create a plan to refactor them.

Deliver

Migrate, consolidate, and even delete! You need to deliver to the plan. When it comes to the delivery of these larger scale migration projects, you need to consider business continuity – how are you going to maintain services whilst doing this migration? One way to approach this is through introducing new tooling – for example, our partners at Starburst provide a distributed analytics engine that enables you to continually query from both on-premises and cloud instances to maintain services during a migration process.

What challenges might you face?

A lack of transparency from cloud providers on the carbon impact their services are making (for example, when we talk about replication factors, you need to consider that AWS lambda instances need to be replicated six times across different servers – meaning your carbon impact is six-fold).

A lack of publicly available data to support on-premises data assessments. Without support and data, it will take a long time to understand the existing impact that your large on-prem data centres are having.

We face a number of challenges as an industry, and to take on a GreenOps approach we need to be equipped with the tools and capability to assess and benchmark where we are, from an on-premises perspective and a consolidated view across both cloud and on-premises.

What are the next steps?

To support businesses in taking their next steps towards GreenOps, Oakland and Interact have partnered to create a revolutionary service based on the four-step method we’ve talked about here. Interact are experts in on-premises data centre efficacy assessments and how you can save money and energy by consolidating and reconfiguring servers.

Oakland can support organisations in reporting on, understanding, and investigating their cloud estate using our carbon efficiency tool – an enhancement from the well-regarded Thoughtworks cloud carbon footprint tool to create an estimate of the carbon footprint of your data estate.

We work together to identify how you can streamline your data estate and optimise for both carbon and cost perspectives. It’s important to note that the platform itself is only a part of the battle; you need governance to ensure that retention policies are being fulfilled, which can have a huge impact on your dark data, which needs to be baked into your data strategy as a means of alignment, and from an analytics perspective, you have to start to report on data sustainability as a KPI to support carbon reduction initiatives.

As an industry, we need to do better for ourselves and the planet by working together to minimise the environmental impact of our data in a cost-efficient way.

If you’d like more information about Oakland’s revolutionary GreenOps service please get in touch by emailing Luke.sharma@theoaklandgroup.co.uk

The post What is the environmental impact of your data? appeared first on Oakland.

Over the last few years, Power BI has gone from a visualisation tool on the side to become front and centre in the business intelligence (BI) industry. A combination of including the tool in Office 365 subscriptions and adding more and more features over the years has positioned Power BI as a market leader in preparing and presenting data for analysis (5 million subscribers in 2016). Previously where Tableau or Qlik may have been one of the few key players, now Microsoft has a seat at the visualisation table. 

What is “BI”? 

Business intelligence means a few different things – from visualising, analysing to just organising and reporting upon data. All of this helps organisations make data-driven decisions. If you’re a business that can easily utilise all of your data to drive change and streamline processes, then you likely have a modern (or at least impactful) BI setup. Examples of how business intelligence can help include: 

• Identifying market trends 
• Analysing customer behaviour 
• Reporting on priority issues 
• Forecasting profit or loss 
• Simplifying and visualising complex information 

Why use a BI tool at all? 

Visualising data is a powerful thing. When done right, it provides an intuitive shop window on potential insights and data within. When done wrong it can be misleading, confusing, or difficult to maintain and interact with. Visualisation at its worst, a set of reporting and analytics costs a company time and money, and at its best, it can guide them to success. 

Data-driven businesses understand the value not just in having easy access to prepared data but in going a step further to ensure it is an easily digestible format for users – whether this is internal staff or customers outside the organisation. Having a BI tool and a capable BI development team helps build a reporting and analytics capability more than just spreadsheets or databases. The aim is to allow users from the high-level executive board to the tech-savvy analyst to be able to access the information they need quickly and accurately. 

What’s changed? 

None of this information is new, with BI tools such as QlikView, Tableau, and Spotfire offering such reporting capability for a while and evolving with the industry. Power BI is simply a younger player in this industry by comparison with a few offerings of its own. 

What has changed dramatically, however is the way in which data is utilised- especially the scale.  The volume of data being extracted, stored, and processed is ever-increasing and the cloud computing industry has grown to handle this increase. Data engineers now have a variety of data pipelines, templated connectors, APIs, and computational power to move data quickly and automatically from A to B. It’s never been easier to get data out of a system, model it and connect a BI tool for reporting. 

A further change is the way in which a user interacts to seek information. Rather than passively consuming a dashboard or report, there is a push for “self-service” reporting to provide users with the tools and visualisation package to equip them to find the answers to their own questions with ease. Interestingly, this is something Microsoft has embraced with features like the “Ask a question about your data” to auto-generate reporting and insight from natural language input. 

So how did Power BI become so popular? 

Power BI has stormed into the number one spot according to Gartner’s Magic Quadrant for Analytics and Business Intelligence Platforms. While this isn’t a definitive yes for Power BI being fit for every use case or scenario, it demonstrates the tool’s popularity. The Microsoft product has quickly become an office staple for many organisations, for multiple reasons. 

As mentioned, the licensing method is attractive to Office 365 users, meaning Power BI Desktop is free, so users can easily develop reports and complex dashboards without having to worry about building a business case for the tool. Microsoft has leveraged their ecosystem to create another product that sits firmly alongside others with which users are familiar. This results in user’s being used to some aspects of the user interface from other office products (Excel, Access) whilst also capitalising on a huge amount of integration potential. 

BI tools have been racing to ease the process of connecting to multiple data sources, creating pre-built connections to a number of different systems, and Power BI is no exception. There is always more to do, but the tool, like its competitors, is able to connect to a huge range of systems, filetypes, and repositories. 

In conclusion… 

Power BI is a tool at the forefront of data visualisation and analytics, but it is by no means the only BI software available. This blog gives a surface view of BI, and at Oakland, we always have more to share on the value of embedding such processes within organisations. For more complex examples of how we’ve leveraged Power BI and other BI suites into our projects, please get in touch by emailing hello@theoaklandgroup.co.uk or calling 0113 234 1944. 

The post The rise of Power BI (and BI as a whole)  appeared first on Oakland.

• Outlining the strategy and tooling to map SharePoint Data into a Data Governance strategy
• How to ingest SharePoint tabular data like CSV and Excel files into a Data Lake and/or Data Warehouse
• How to start classifying your SharePoint data for Data Quality, Master Data Management, and sensitive data exposure.

(Note the following applies mostly to Office 365 version of SharePoint, SharePoint Online, not the on-premises versions of SharePoint that uses different APIs, though would follow the same broad strategy.)

Building an Initial Assessment of SharePoint Structure and Data

To better regulate this space, you first need to assess your SharePoint Structure and data before planning any changes or policy decisions. In essence, you need to know what the current issues are and how much work it’s going to take to reach certain standards of process and data quality.

To be able to crawl a SharePoint site, and understand its contents, you need two things – authentication via linked Azure Active Directory and the Microsoft Graph API, to enable bulk download of data and automate custom actions. Being a web REST API, it can be called through most programming languages and special API debuggers like Postman.

A SharePoint site can be crawled for files by first finding all the base folders, known in the Graph API as Drives, and then searching through all or some of the base folders. Finding all the files does require a bit of work as there is no “get all folders and files in drive” API call we can make – but you can write some code of medium complexity to find all the folders, sub folders and files of a SharePoint Base folder.

Example architecture – note while Azure Active Directory is a requirement, the Graph API can communicate with servers in other cloud environments such AWS, GCP or other locations, if allowed to.

Ingest tabular data into a Cloud Data Lake and/or Data Warehouse

Because every file has a unique URL (web address), that can be retrieved when it is scanned by the Graph API, we can now download all files in SharePoint via the URL into the cloud and convert it to a Data Lake file or Data Warehouse table (there is also more advanced strategies of combining multiple files into one table).

There should be a large caveat applied to this – it’s likely some or even most Excel sheets cannot be ingested into a Data Warehouse if it does not match a pre-determined template or is not machine-readable (examples below on what is machine-readable or not).

This isn’t the end of the road for unreadable files – we can use the SharePoint Crawler we built in the previous step to flag which files are readable or not, with the results helping to start a process to update the data so that it is readable by databases and software programs.

Also, it is possible to read data of other types of files – Word Documents and PDFs, etc. – however, they should match a robust template so a program can read their data reliably or use a Machine Learning Algorithm, however, this approach may give inexact results.

Classifying and Mastering your SharePoint Data

Once your data is in a cloud data source, it can be an easy process to assess your data. Data Governance tools such as Azure Purview can be used to classify your data and detect Personally identifiable information (PII) information or use database tools such as SQL Server’s sensitive data detection tools.

Once classified, you can begin to shape your data mastery, where you choose one “true” version of an object such as Customer, Contract, Project, Product, etc, for other sources to compare against.

However, this can become rather complicated if you have multiple sources of the same data. As such, it’s recommended to use specialist Master Data Management (MDM) software – these often require a Data Warehouse / Data Lake sources, so SharePoint data will benefit from ingestion into the cloud as mentioned above.

But Why?

So why go to such lengths to classify and master your SharePoint data? Mainly due to the consequences of using incorrect/outdated data or unnecessarily exposing sensitive data, some of which are listed below:

• Fines for breaching GDPR
• Fines from financial authorities for incorrect financial information.
• Loss of reputation and future sales for data leaks and incorrect data
• How to improve your SharePoint Site
• Saving rework of data by finding incorrect data, out-of-date data, and removing duplicates.

Aside from this more negative viewpoint, better management of your SharePoint data can also drive more positive impacts, such as the ability to perform additional analyses and grasp a better understanding of your organisation.

As you’d expect, completing this work can be rather difficult, due to the potential amount of data and number of nested folders in a companies’ SharePoint space. However, it can be made easier through automated tooling, such as the scanner mentioned earlier, and with some guidance from those who’ve been there before. We at Oakland have decades of experience in quality improvement from various angles, including people, processes and technology, should you like additional resources in this area.

The post Gain control of your SharePoint Data appeared first on Oakland.

There are typically four basic use cases that we observe when building out a Project Analytics capability: 

1. Report Consumers 

2. Analysts 

3. Partners/Advanced Analysts 

4. Machine Learning 

Use Cases for Report Consumers: 

By this stage, report consumers are likely to be loving the analytics and reporting capabilities you have created. They should now have fast, high-quality reports and analysis related to the specialist insights they need. 

It should be relatively easy to find use cases for this group simply by exploring the information you already report on today. The reports you already produce are most likely helpful but perhaps in need of improvement. 

However, it may not always be that straightforward to perform a straight enhancement of your existing reports for several reasons: 

• Extracting data from its source (without adjustment) will invariably have issues. By automating data pipelines, you’ve removed the ability to manipulate and finesse the data manually, which places greater pressure on resolving any data quality issues further upstream. 
• Adding commentary to your reporting/analytics output becomes more challenging because the reporting tools don’t give you the same flexibility as the previous ‘hand-cranked’ reports your team may have created in a spreadsheet or Powerpoint deck.  
• Report proliferation’ can be another challenge. Now that you can report on everything, there is often a temptation to create endless dashboards and reports that slice and dice the data in an infinite variety of ways. The lesson here is to keep your outputs simple and constrain the variations at the outset. 
• Finally, be mindful of how your reports are commissioned and version-controlled to ensure quality. You need to strike a balance between allowing business users to create their own reports or waiting for the analytics team to produce reports (but potentially becoming a bottleneck). 

The moral here is that ‘reporting democratisation’ can sound like a great idea, but it can turn and bite you if left unchecked. The whole argument for ‘self-service’ reporting is more of an operational and cultural challenge than a technical one. 

Use Cases for Analysts: 

By this stage, your analysts should be delighted with the project analytics capabilities you’ve created. However, as adoption increases, you’ll need to think carefully about what controls you need to put in place for this community. 

Analysts are often keen to build complex logic with data visualisation tools (e.g. PowerBI). Packing excessive data processing into the visualisation layer can soon become a maintenance and configuration headache if left unchecked. 

One solution is to give the analysts their own project analytics ‘sandpit’ to build new reports, test different datasets, and ensure any changes don’t negatively impact the wider production environment. You must also provide awareness training and possibly additional tooling to ensure that the relevant policies are understood and followed. 

Where data is ‘blended’ or processed from non-standard sources, you need to set clear guidelines. This exercise would include activities such as tagging, or flagging, your reporting analytics so that it identifies any data that comes from an approved source as trusted, versus any ‘sandpit’ reports that may lack assurances of trusted provenance and data quality. 

Finally, you will need to consider how to provide support and help to this community. Again, this is less a technical headache and more a cultural requirement. Solutions here will include forming various communities or forums to help spread the knowledge amongst the internal analyst workers. 

Use Cases for Partners and Advanced Analysts: 

By now, you will have covered off 80% of your project analytics use cases, but some people will want more and need to go to the raw data source. There is enormous value to be had when sharing your data with partners along the data supply chain. 

It is certainly worth highlighting the Project Data Analytics Task Force, which has made considerable progress in formulating ideas and approaches around increased data sharing for project analytics. 

Ideally, it makes sense to start with a limited use case of certain suppliers providing data, then extend your partner model for data ingestion/sharing over time. The merits of bilateral information sharing are clear for all to see, but you must ensure the appropriate governance and controls are in place from the outset. 

Use Cases for Machine Learning and Artificial Intelligence: 

There is no doubt that the potential for leveraging machine learning and AI is enormous, but you need to start with a substantial pool of quality data to train your AI/ML models. 

Even if your organisation runs many projects, for effective machine learning, that is likely to be too sparse a data set compared to other industries where models are trained over thousands or millions of data points. 

You can drop down into sub-components to gather more data, for example, by taking more granular time-slices such as monthly reports. 

Whatever approach you take, your models need to run on large data sets to create statistically relevant findings. 

Another aspect to consider is the ‘black box’ nature of machine learning versus something that is more transparent and helps the users clearly understand how the model derived a result. Generally, we would recommend opting for a more transparent model to increase confidence. 

Machine learning does allow you to gauge the likely outcome of a project based on learning about past performance across various feature categories. For example, you can find correlations between how well a project is run based on the quality of information entered and the project outcome.  

It makes sense to exploit external data to help train more accurate machine learning models. If you’re in the business of building bridges for example, you may only deliver a modest number of projects each year so you’ll still need a large enough sample size of bridge-building project data to create an accurate machine learning model. 

Whatever data you source for machine learning, you will still need to invest in the appropriate skills and technology to execute correctly and deliver impactful use cases. Standard, off-the-shelf visualisation tools, are unlikely to be sufficient for machine learning, so you may need to invest in tools such as Databricks. 

Additional Use Cases: 

One side benefit of all this project data is that you effectively construct a Digital Twin of your operation. You can observe precisely what is happening across the entire project lifecycle. When you have project snapshot data, you can analyse the project over time, creating a powerful resource to improve the business. 

You can start to see how often people are updating the project data, altering the project baseline, and maintaining an accurate commentary of project updates. 

Be careful not to leap in with assertions too soon, but gathering the right type of information will be extremely helpful for improving data quality through monitoring and improving confidence in project assurance and integrity. 

Creating high-quality historical project data analytics has proven to add value far beyond machine learning and AI. As discussed, the benefits of building the Digital Twin of a major project can be far more valuable than the allure of AI and machine learning. 

Managing the Culture Change of Project Analytics:  

Just because you’ve invested heavily in the technology and tools of project analytics doesn’t necessarily mean people will share the same passion or even use them.  

Project analytics is far more than a ‘tech project’; you need to get people using the tools by highlighting how they positively impact their working lives and deliver better outcomes for all concerned. 

It would help if you increased the workforce and management maturity and resilience, particularly in relation to coping with the inevitable discovery of issues and problems that shine a light on poor performance or other ‘skeletons in the closet’. There will always be bumps along the way, so try and avoid any adversarial scenarios where each party is trying to prove the other wrong. 

Finally, you want to instil a culture that becomes progressively less tolerant towards those who refuse to use the approved tools and revert to doing things ‘their way’.   

You will need to go out of your way to validate your findings and demonstrate that the information your project analytics capability provides is trustworthy and defensible. Expect some manual hard work in the near term to provide transparency, and don’t underestimate the desire for everyone to go back to their trusty spreadsheets.  

In short, culture change takes time, planning and persistence. 

The Journey Never Ends 

We started this series in the ‘foothills’ of project analytics and climbed steadily through the various peaks of expertise required to deliver a fully operational capability. 

The reality is that even at this late stage, you’re still at the beginning of a long journey. 

What comes next will first be impacted by changes in scope around:  

• Business focus: Are you going to start reporting on additional data and other parts of the business? 
• Data sources: Will there be more data sources, or more data from the same sources? 
• Periodicity: Are you going to increase the frequency of reporting? 
• Snapshotting: Do you want to increase snapshots that allow for more granular performance analysis over time? 

There are plenty of factors and dimensions that influence where you go next, and of course, as your approach changes and matures, you’ll find that you’ve constructed a highly competent team that is capable of adapting. 

By this point, you’ll have:  

• Architected a solution 
• Engineered the data 
• Created a team of analysts 
• Built machine learning/AI models 
• Managed effective change 

Your project community will have become far more aware of project analytics, many of them having undergone new skills training.  

You will also have created more intelligent customers, so now may be the time to invest in additional functionality because you will be better informed on the options and benefits available. 

What Next? 

There are no silver bullets or ‘plug and play’ solutions for project analytics because every organisation is unique. 

However, the building blocks exist. They have been proven and are relatively straightforward to build out. 

As we’ve highlighted in this series, the solution requires a mix of technology, business expertise and user engagement, but it is actionable and within reach. 

The key is to start small, and the rest of your journey will write itself. 

The post The Use Case For Project Analytics appeared first on Oakland.