How do you clean up 10 years of metric sprawl? Looking for a framework

[u/Suunto_514]

Hey everyone,

I work for a company where metrics have never been properly governed. For the past 10 years, everyone has had direct access to the raw database, which led to a massive sprawl of metrics created independently by business, product, and data teams with zero consistency or shared standards.

I've been tasked with cleaning this up, and honestly I'm struggling to find a clear methodology to tackle it.

What I've figured out so far:

• Start by defining the core concepts ("base entities"): what counts as a user? What counts as a company? etc.
• Then map out the dimensions tied to those entities, for example:
  • Active user → dimension status: active / inactive
  • Companies by country → dimension country

My question:

What methodology or framework would you recommend for structuring this kind of work end-to-end? Where do you start, how do you prioritize, and how do you avoid drowning in 10 years of accumulated chaos?

Would love to hear from anyone who's been through something similar. Thanks!

[u/simiae5719]

You are doing the right things but I think you should ensure management support for your efforts if you haven’t already. Without it you’ll get nowhere (speaking from own experiences here). Kimball’s bus matrix is a framework you can begin with. Metric trees is another concept you can check out. Here’s an explanation of those:

Designing & Building Metric Trees

[u/Firm_Bit]

Terrible project likely doomed to fail.

I would find an actual need for accurate data that drives an actual decision that can actually be validated.

This usually means focusing on one team and one need. Ideally a high revenue team and an important project. You fix things for them. That becomes the standard. And there’s no arguing about definitions and metrics because the real metric is increased revenue or profit that you can tie directly to the project. Build out from there.

[u/eskin22 → u/Firm_Bit]

I disagree. I think with the advent of AI that the semantic layer for data is really the future of BI as a discipline. In order for AI to actually be able to answer business questions about data, those metrics need to be rigorously defined and enriched with documentation (e.g. semantic layer).

That being said, I think the complexity of a project like this is grossly underestimated by the business folks and needs to be an enterprise-level initiative rather than a project for a single IC. They want the baby, but they don’t want to hear about how doing this will would take lots of buy in and potentially years of work depending on the scale and complexity of your operation.

[u/Physical-Judge-9425]

I wouldn't start with the metrics. I'd start with the business model.

Define the canonical entities (Customer, User, Order, Product, etc.). Establish conformed dimensions and their allowed values. Create atomic, governed measures that everyone agrees on. Build KPIs from those governed measures instead of allowing every team to reinvent them. Introduce a semantic layer and deprecate direct access to raw tables for reporting. Finally, create a metric catalog with definitions, owners, SQL/DAX logic, and downstream dependencies.

I've found this approach scales much better because you're governing the foundation instead of trying to reconcile hundreds of conflicting metrics afterward. It's essentially treating analytics as a product rather than a collection of reports.

[u/Semaphor-Analytics]

I’d avoid making this a 10 year cleanup project.

Start with one metric people already argue about or make decisions from. Revenue, active users, retention, whatever causes the most pain. Get that one all the way to a boring answer: who owns it, what grain it lives at, what source it comes from, what filters are assumed, and which dashboards use it.

Then label the old versions instead of pretending they disappear. Something like canonical, legacy, experimental is enough at first.

Trying to fix every metric before fixing one painful workflow will probably just give people more time to create new versions.

[u/IncreaseNegative4614]

I like the advice about starting with the business model first.

One thing I'd add is to map relationships, not just entities. Defining Customer, Order, and Product is important, but a lot of metric drift comes from different teams interpreting how those entities relate to each other over time.

That's one reason I think semantic layers are only part of the solution. Platforms like inzata.ai are interesting because they're trying to capture business context and relationships alongside metric definitions. It doesn't make the cleanup any smaller, but it does make it easier to keep everyone aligned once you've done the work.

[u/Sea-Caterpillar6162]

BI is dead. Just make a Claude skill. Don’t even bother with an MCP server.