Storing the data · Topic 4 of 4
Data Platform Design and Federated Governance
Build a data platform with Dataplex, Dataplex Catalog, BigQuery and Cloud Storage, and design federated, data mesh governance for distributed teams.
- 5 min
- A platform combines Cloud Storage, BigQuery, Dataplex and sharing; build to the requirements.
- Dataplex Catalog uses entries, entry groups, aspect types, aspects and glossary terms.
- Federated governance: domains own products; the centre sets rules and tools.
What it is
A data platform is the shared set of storage, processing, governance and sharing services that every team in a company builds on. Federated governance splits the work: domain teams own and publish their data, while a central team sets the rules and provides the tools that apply those rules everywhere.
How it works
Building blocks
| Need | Google Cloud tool |
|---|---|
| Object storage and open-format lake | Cloud Storage |
| Warehouse and SQL engine over lake and warehouse | BigQuery, with BigLake tables |
| Metadata, search and business context | Dataplex Catalog: entries, aspects, business glossary |
| Data quality and profiling | Dataplex data quality and data profile scans |
| Where data came from | Data lineage |
| Organizing storage by domain and zone | Dataplex lakes and zones |
| Column-level classification | Policy tag taxonomies |
| Sharing between domains and with partners | BigQuery sharing (Analytics Hub) listings |
Build only what the requirements ask for. A single analytics team may need just BigQuery datasets and a few policy tags; a group of business units with separate ownership needs the full federated model.
Dataplex Catalog concepts
| Concept | Meaning |
|---|---|
| Entry | A catalogued asset, such as a BigQuery table or a Cloud Storage fileset |
| Entry group | A container of entries, often per system or domain |
| Aspect type | A template for metadata, for example "data product" with owner, tier and SLA fields |
| Aspect | Metadata of that type attached to an entry |
| Glossary term | A business definition, such as "active customer", linked to columns |
Technical metadata from BigQuery and other Google Cloud services is harvested automatically. Teams add aspects and glossary links so users can judge ownership, quality and meaning.
Federated governance model
| Responsibility | Central team | Domain team |
|---|---|---|
| Classification scheme and policy tags | Defines | Applies to its columns |
| Organization policies, VPC Service Controls, CMEK rules | Sets and enforces | Works within |
| Data quality | Sets minimum standards and tooling | Writes rules for its data and fixes failures |
| Catalog | Runs it and defines required aspects | Registers and documents its products |
| Data products | Provides templates and the sharing platform | Builds, owns, supports and publishes them |
| Access requests | Defines the process | Approves access to its own data |
This is the data mesh idea: domain ownership, data as a product, a self-service platform and federated computational governance. "Computational" means rules are applied by the platform, such as organization policies, Cloud DLP discovery and policy tags, rather than by manual review.
Applying a central classification
The central team publishes one policy tag taxonomy, for example "Restricted" and "Confidential". Domain teams attach the tags to their columns, and access is granted through the tag, not table by table.
[
{"name": "customer_id", "type": "STRING"},
{"name": "email", "type": "STRING",
"policyTags": {"names": ["projects/gov-central/locations/eu/taxonomies/1001/policyTags/2001"]}}
]bq update prod-sales:crm.customers schema.jsonWhen a stem says "each business unit owns its data, but the company needs consistent security and discoverability", the answer is federated governance: domain projects, a shared catalog and central policies, not one central team doing all data work.
Project layout for a federated platform
- One or more projects per domain, grouped in a folder per domain.
- A central governance project for taxonomies, catalog configuration and shared tools.
- Shared listings in BigQuery sharing (Analytics Hub) for products other domains consume.
- Organization policies at the organization or folder level so every domain inherits the guardrails.
A worked example
A retail group has five business units, each with its own engineers. Today a central team builds every pipeline, and the backlog is months long. Security wants the same PII rules everywhere.
The engineer gives each unit its own folder and projects and a Dataplex lake. A central taxonomy classifies PII, and Cloud DLP discovery flags untagged sensitive columns. Each unit publishes curated tables as data products with a required "data product" aspect, and consumers subscribe through BigQuery sharing listings.
How the exam asks about this
- "Business units want to own their data while the company enforces consistent governance." Federated governance with domain projects, Dataplex Catalog and central policy tags and organization policies.
- "Users cannot tell which table is the official definition of revenue." Register data products in the catalog with ownership aspects and glossary terms.
- "Apply the same column restrictions to PII across hundreds of tables owned by different teams." A central policy tag taxonomy applied to columns, with access granted on the tags.
What to remember for the exam
- A platform combines Cloud Storage, BigQuery, Dataplex and sharing; build to the requirements.
- Dataplex Catalog uses entries, entry groups, aspect types, aspects and glossary terms.
- Federated governance: domains own products; the centre sets rules and tools.
- Policy tag taxonomies apply consistent column security across teams.
- Organization policies at folder or organization level give every domain the same guardrails.
Centralizing all pipeline work in one team is the problem federated governance solves. An answer that adds more central reviewers does not scale.
Common mistakes
Confusing federation with no governance. Domains own data, but central rules still apply automatically.
Granting access table by table. With hundreds of tables, use policy tags and groups instead.
Copying data between domains. Share through listings or authorized views so one source of truth remains.
Check yourself
In federated governance, who owns a data product?
The domain team that produces it; the central team sets standards and provides the platform.
What is an aspect in Dataplex Catalog?
A piece of structured metadata, defined by an aspect type, attached to a catalog entry, such as owner and SLA for a data product.
How can one PII rule cover many tables owned by different teams?
Use a central policy tag taxonomy, have domains tag their PII columns, and grant access through the policy tags.
Topic 4 of 4 in Storing the data
Test yourself on Storing the data
The full Google Cloud Professional Data Engineer question bank has 733 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Google Cloud. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.