{"id":7790,"date":"2023-11-08T13:36:50","date_gmt":"2023-11-08T13:36:50","guid":{"rendered":"https:\/\/www.theoaklandgroup.co.uk\/?p=7790"},"modified":"2024-04-17T09:59:05","modified_gmt":"2024-04-17T09:59:05","slug":"should-you-build-or-buy-your-data-platform","status":"publish","type":"post","link":"https:\/\/weareoakland.com\/blog\/should-you-build-or-buy-your-data-platform\/","title":{"rendered":"Should you build or buy your data platform?"},"content":{"rendered":"

\u201cBuild Versus Buy\u201d is More a Scale of Options than a Binary Decision<\/strong><\/h4>\n

As consultants, we\u2019ve been involved in helping our clients to decide which is the best software to invest in (or not invest in some cases). This is quite a responsibility as we are putting our reputation in the hands of a vendor, and we know that tech vendors are amazing at making claims. We see it in every trade show we attend. Therefore, it can be impossible to verify all the features of every product.<\/p>\n

From our experience, this can be difficult to get right. So here are a few lessons in buying software (or not) to avoid making costly mistakes.<\/p>\n

Limiting Your Potential Options is Sensible but Dangerous<\/p>\n

One sales tactic is to present two options. This makes sense in some ways: management is time constrained and doesn\u2019t want to assess all 100+ options that are possible, but this can lock you into a limited way of thinking, especially if you only assess two extreme \u201cbuy\u201c and \u201cbuild\u201d options.<\/p>\n

\"Diagram<\/p>\n

Scale of Product Types from Build to Buy. By Jake Watson.<\/em><\/p>\n

So you want to look at a variety of options, and for most organisations, look at options across the scale. We find a managed\u00a0Platform as a Service<\/span><\/a>\u00a0(PaaS) in the middle of the scale,\u00a0fits well for most but not all!<\/p>\n

Think Beyond a Product<\/h4>\n

You may also want to consider a mix of both build\u00a0and\u00a0buy components; this can get you closer to fulfilling a set of complex requirements in a reasonable budget and timeframe.<\/p>\n

One of our favourite examples is\u00a0Fivetran<\/span><\/a>, a vendor that offers a super quick and easy way of getting source data into your Lakehouse or Warehouse. However, it can cost a lot at scale, so you may end up doing the math and finding that building your own ingestion is more cost-effective for some, but not all, sources. Or try open source (Airbyte<\/span><\/a>,\u00a0Meltano<\/span><\/a>).<\/p>\n

And yes, maintaining two or more systems is generally worse than one, but that should be a guideline, not a rule, as sometimes, you are the exception.<\/p>\n

There are Too Many Choices Though!<\/h4>\n

You can go too far the other way and spend months going through dozens of options\u00a0out of 1400+ products that exist in data and A<\/span><\/a>I.<\/p>\n

One example of the variety of options available to you is real time data processing, where you can choose from least to most managed:<\/p>\n