One of the most recent concepts that have arisen as of late is the data mesh. Simply put, a data mesh is a decentralized data architecture that organizes data by specific business domains, whether it is for marketing, sales, or customer service, among others, in order to establish ownership of the producers of a given dataset.
This allows for specific data governance policies to be set for each domain in terms of documentation, quality, and access. This, in turn, enables self-service use across an organization, eliminating multiple operational bottlenecks associated with centralized, monolithic systems.
Being relatively new, data mesh as a concept is one shaped by each enterprise’s unique ownership, organization, security, pace of change, technology, and cost management requirements. Given the enormous challenge to balance all of these requirements, these companies pick a common architecture to match most of these requirements.
For businesses looking to build their data mesh architectures, there are at least six different design models to consider, depending on the business requirements:
Fine-grained fully federated mesh
This model espouses a peer-to-peer data distribution while all governance-related metadata is logically centralized. Data in this topology is owned, managed, and shared by each individual domain, creating organizational flexibility and fewer dependencies because the interaction is many-to-many. It also promotes the intensive reuse of data since there’s a high degree of data product creation.
This topology is more suited for companies that are born on the cloud, do multi-cloud, are relatively young, and have many highly skilled software engineers. It is also suited for companies with a high degree of autonomy already established. Otherwise, it is a complicated topology to achieve.
Fine-grained and fully governed mesh
This model is somewhat similar to the fine-grained fully federated mesh but with the addition of a central layer of distribution. Each domain still has a clear boundary and autonomous ownership over its own applications and data products, but the data products are required to be distributed via a central logical entity. Such structure helps address possible data distribution, conformation, and gravity concerns that may arise.
One drawback of this model is that the centralization it employs leads to a longer time to market and could hinder domains from delivering business value when capabilities aren’t ready. As such, this model is employed mainly by financial institutions and governments, as well as other companies that value quality and compliance over agility.
Hybrid federated mesh
As the name implies, this model employs the qualities of both federated and governed meshes, though it is mainly a governed or centralized model. As such, there is a central platform instance in which data products are being maintained and created but with a higher degree of autonomy and mesh-style distribution in terms of data consumption.
One consideration to keep in mind is the increased management overhead for source-system-aligned domains, as well as the more complex guidance and principles brought by a hybrid model like this one. There also might be inconsistent rules for data distribution because consuming domains often become providing domains.
Value chain-aligned mesh
This model utilizes a value chain structure, in which a group of smaller domains closely work together at higher levels of autonomy but adhere to central standards in the case of cross-domain data products. One consideration to keep in mind is that it requires stronger guidance from architects because boundaries might not be always that explicit.
This model is prevalent among organizations that specialize in supply chain management, product development, or transportation as they require hyper-specialization or stream-alignment for bringing value to their customers.
Coarse-grained aligned mesh
This model is suited for large-scale companies since these companies tend to have domains that hold tens or hundreds of applications, also known as “coarse=grained” domains. Such is a result of the company’s organic growth, usually brought about by mergers and acquisitions, creating complex landscapes with multiple applications and systems.
This complicated setup can cause issues such as data being unaligned with the boundaries of domains and business functions and capability duplication across each coarse-grained domain. It is therefore critical to strongly guide a transition to data mesh. Because while this model requires a higher level of autonomy, it also requires stronger governance policies and self-service data platform capabilities.
Coarse-grained governed data mesh
This model is similar in many ways to the coarse-grained aligned mesh but with some additional characteristics of the governed data mesh topology, such as addressing the time-variant and non-volatile concerns. On the other hand, it allows for more relaxed controls within these larger boundaries.
Data mesh is redefining data management for organizations, allowing them to define which data should fall under a strong data governance system and data that is better suited to a self-service platform. It is critical for a business to implement the right data mesh topology model so it can manage and utilize its data more efficiently and effectively.
Comments