Skip to content

Data mesh

Data mesh is an organizational and architectural approach to analytics data. Instead of one central team ingesting everything into one platform, each business domain owns its analytical data and publishes it as a data product for others to use. A shared platform makes publishing easy, and shared governance keeps products interoperable.

It responds to a common failure: central data teams becoming a bottleneck between the people who understand the data and the people who need it.

flowchart TB
    subgraph orders [Orders domain]
        op[Orders data product]
    end
    subgraph pay [Payments domain]
        pp[Payments data product]
    end
    subgraph cust [Customer domain]
        cp[Customer data product]
    end
    platform[[Self-serve data platform]] --- op & pp & cp
    gov{{Federated governance:<br/>standards, security, interoperability}} -.- op & pp & cp
    op & pp & cp --> consumers[Analysts, ML teams, other domains]

Data mesh rests on four principles:

  • Domain ownership. The team that runs a business domain also owns its analytical data.
  • Data as a product. Each dataset is discoverable, documented, trustworthy and has an owner, like any product with customers.
  • Self-serve data platform. A platform team provides tooling so domains can build and publish products without becoming infrastructure experts.
  • Federated computational governance. Global rules (security, privacy, naming, interoperability) are agreed centrally and enforced automatically by the platform.
  • Removes the central bottleneck: domains publish data without waiting on one team.
  • Better data quality: the people who understand the data are responsible for it.
  • Scales with the organization: new domains add products without redesigning a central pipeline.
  • Clear accountability: every dataset has an owner.
  • Big organizational change: domains need data skills and must accept new responsibilities.
  • Requires a strong platform: without one, every domain reinvents pipelines.
  • Governance is hard: standards must be agreed and enforced across independent teams.
  • Overhead for small organizations: the coordination costs exceed the benefits.
  • Large organizations with many domains, where a central data team is clearly a bottleneck.
  • Companies with mature domain-oriented engineering teams.
  • When leadership is willing to fund a platform team and change ownership.
  • Small or medium organizations with a single, responsive data team.
  • When the goal is a technology swap. Data mesh is mostly an operating-model change.
  • Rebranding without changing ownership. Calling datasets “products” while the central team still builds them changes nothing.
  • No platform. Asking domains to own data without giving them tools leads to inconsistent, fragile pipelines.
  • Governance as an afterthought, producing products that can’t be joined because identifiers and definitions differ.

Glossary: data mesh, bounded context

  • Zhamak Dehghani, Data Mesh: Delivering Data-Driven Value at Scale
  • Zhamak Dehghani, “Data Mesh Principles and Logical Architecture” (martinfowler.com)

Written and edited by Abhinav Taduka, enterprise software architect.