Data mesh
Data mesh is an organizational and architectural approach to analytics data. Instead of one central team ingesting everything into one platform, each business domain owns its analytical data and publishes it as a data product for others to use. A shared platform makes publishing easy, and shared governance keeps products interoperable.
It responds to a common failure: central data teams becoming a bottleneck between the people who understand the data and the people who need it.
Diagram
Section titled “Diagram”flowchart TB
subgraph orders [Orders domain]
op[Orders data product]
end
subgraph pay [Payments domain]
pp[Payments data product]
end
subgraph cust [Customer domain]
cp[Customer data product]
end
platform[[Self-serve data platform]] --- op & pp & cp
gov{{Federated governance:<br/>standards, security, interoperability}} -.- op & pp & cp
op & pp & cp --> consumers[Analysts, ML teams, other domains]
How it works
Section titled “How it works”Data mesh rests on four principles:
- Domain ownership. The team that runs a business domain also owns its analytical data.
- Data as a product. Each dataset is discoverable, documented, trustworthy and has an owner, like any product with customers.
- Self-serve data platform. A platform team provides tooling so domains can build and publish products without becoming infrastructure experts.
- Federated computational governance. Global rules (security, privacy, naming, interoperability) are agreed centrally and enforced automatically by the platform.
- Removes the central bottleneck: domains publish data without waiting on one team.
- Better data quality: the people who understand the data are responsible for it.
- Scales with the organization: new domains add products without redesigning a central pipeline.
- Clear accountability: every dataset has an owner.
- Big organizational change: domains need data skills and must accept new responsibilities.
- Requires a strong platform: without one, every domain reinvents pipelines.
- Governance is hard: standards must be agreed and enforced across independent teams.
- Overhead for small organizations: the coordination costs exceed the benefits.
When to use it
Section titled “When to use it”- Large organizations with many domains, where a central data team is clearly a bottleneck.
- Companies with mature domain-oriented engineering teams.
- When leadership is willing to fund a platform team and change ownership.
When to avoid it
Section titled “When to avoid it”- Small or medium organizations with a single, responsive data team.
- When the goal is a technology swap. Data mesh is mostly an operating-model change.
Common pitfalls
Section titled “Common pitfalls”- Rebranding without changing ownership. Calling datasets “products” while the central team still builds them changes nothing.
- No platform. Asking domains to own data without giving them tools leads to inconsistent, fragile pipelines.
- Governance as an afterthought, producing products that can’t be joined because identifiers and definitions differ.
Related patterns
Section titled “Related patterns”- Lakehouse
- Data warehouse
- Domain-driven design
Glossary: data mesh, bounded context
Further reading
Section titled “Further reading”- Zhamak Dehghani, Data Mesh: Delivering Data-Driven Value at Scale
- Zhamak Dehghani, “Data Mesh Principles and Logical Architecture” (martinfowler.com)
Written and edited by Abhinav Taduka, enterprise software architect.