Managing Data · Topic 1 of 1

Certified Data, Governed Tags, Lineage and Cleaning

Databricks Data Analyst: find certified datasets, apply governed tags, read lineage and external lineage, and clean Unity Catalog tables with SQL.

5 min

In a nutshell

Certified and deprecated are values of the system governed tag system.certification_status.
Applying a governed tag needs ASSIGN on the tag plus ownership or APPLY TAG on the object.
Governed tags limit values to the tag policy and show a lock icon.

What it is

Unity Catalog is where an analyst finds, labels and traces data. A certified dataset is one its owner has marked as trusted, a governed tag is a label whose allowed values are controlled at account level, and lineage shows where data came from and where it goes.

Before a table feeds a report it often needs cleaning: removing invalid rows and handling missing values. In Databricks that is plain SQL on the Unity Catalog table.

How it works

Discovering and managing certified datasets

Use the workspace search or Catalog Explorer to find tables. Search accepts filters such as type:table certificationStatus:certified, which returns only certified tables.

Certification is the system governed tag system.certification_status with the value certified or deprecated. Certified assets show a check mark; deprecated assets show a restricted icon so users move away from them.

SET TAG ON TABLE main.sales.orders `system.certification_status` = `certified`;
SELECT region, sum(amount) FROM main.sales.orders GROUP BY region;

To set the status you need the ASSIGN permission on that governed tag, plus ownership or APPLY TAG on the object and USE CATALOG and USE SCHEMA on its parents. In the UI, open the object's kebab menu and choose Assign certification. Querying a certified table needs only the normal SELECT grants.

Governed tags vs ordinary tags

	Ordinary tag	Governed tag
Defined	Free form key and optional value	At account level with a tag policy
Allowed values	Anything	Only the values in the policy
Who can assign	Owner or holder of `APPLY TAG`	The same, plus ASSIGN on the governed tag
In the UI	Plain tag	Lock icon, listed under Governed

To tag in Catalog Explorer, open the object, and on the Overview page click Add tags, then choose the key and an allowed value. Columns can be tagged too, which is how PII columns are flagged. In SQL:

SET TAG ON TABLE main.hr.staff `sensitivity` = `confidential`;
ALTER TABLE main.hr.staff ALTER COLUMN ssn SET TAGS ('pii' = 'ssn');

System tags such as system.certification_status are governed tags whose keys and values Databricks predefines.

Viewing lineage

Unity Catalog captures lineage automatically for queries run on Databricks. In Catalog Explorer, the Lineage tab lists upstream and downstream tables, notebooks, jobs and dashboards, and See lineage graph draws them, including column level lineage.

Use lineage to answer impact questions: before changing a column, see which dashboards read it.

External lineage

Systems outside Databricks, such as Salesforce, Tableau or Power BI, are not captured automatically. You add them as external metadata objects in Catalog Explorer, then create an upstream or downstream relationship to a table.

Action	Privilege needed
Create an external metadata object	`CREATE EXTERNAL METADATA` on the metastore
Link it to a relationship	`MODIFY` on the external metadata object
Downstream link from a table	Read access, such as `SELECT`
Upstream link into a table	Write access, such as `MODIFY`

The external object then appears in the lineage graph next to Databricks objects.

Cleaning data in SQL

DELETE FROM main.sales.orders WHERE amount < 0 OR order_id IS NULL;
UPDATE main.sales.orders SET region = 'Unknown' WHERE region IS NULL;
SELECT order_id, COALESCE(discount, 0) AS discount FROM main.sales.orders;

Need	SQL
Remove invalid rows for good	`DELETE ... WHERE <condition>`
Keep the source, build a clean copy	`CREATE TABLE ... AS SELECT ... WHERE <valid>`
Replace a missing value	`COALESCE(col, default)` or `UPDATE ... WHERE col IS NULL`
Remove exact duplicate rows	`SELECT DISTINCT *`
Avoid a failed cast on bad text	`try_cast(col AS INT)` returns NULL instead of an error
Count missing values	`count(*) - count(col)`

Because the tables are Delta tables, a mistaken DELETE can be undone with time travel and RESTORE.

How the exam asks about this

"Analysts must find only tables the data team trusts." Search with certificationStatus:certified, or look for the certified mark.
"Sensitivity labels must use only approved values." Use a governed tag with a tag policy.
"Show that a Tableau workbook reads this table." Add an external metadata object and a downstream lineage relationship.

What to remember for the exam

Certified and deprecated are values of the system governed tag system.certification_status.
Applying a governed tag needs ASSIGN on the tag plus ownership or APPLY TAG on the object.
Governed tags limit values to the tag policy and show a lock icon.
Lineage is automatic for Databricks workloads; external systems are added as external metadata.
COALESCE returns the first non NULL argument; count(col) skips NULLs while count(*) does not.
try_cast turns unparseable values into NULL rather than failing the query.

Watch out

WHERE col = NULL matches no rows, so a cleaning statement written that way silently does nothing. Always test missing values with IS NULL or IS NOT NULL.

Common mistakes

Assuming certified means access. Certification marks a table as trusted; it grants nothing. Users still need SELECT plus USE CATALOG and USE SCHEMA.

Expecting external tools in lineage by default. BI tools and SaaS sources outside Databricks only appear after someone creates external metadata and a lineage relationship.

Deleting when a filtered copy was asked for. If the raw data must be kept, build a clean table or view with a WHERE filter instead of running DELETE on the source.

Check yourself

What must a user have to mark a table as certified?
The ASSIGN permission on the system.certification_status governed tag, plus ownership or APPLY TAG on the table and USE CATALOG and USE SCHEMA on its parents.
How do you show a Power BI report as a consumer of a Unity Catalog table in lineage?
Create an external metadata object for the report in Catalog Explorer and add a downstream lineage relationship from the table to it.
Which expression replaces a NULL discount with zero in a query?
COALESCE(discount, 0), which returns the first non NULL value, so missing discounts appear as 0.

Topic 1 of 1 in Managing Data

Test yourself on Managing Data

The full Databricks Data Analyst Associate question bank has 600 questions, with timed practice exams and an explanation for every answer.

Get Full Access Try Sample Exam

This study guide is independent and is not affiliated with, endorsed by, or sponsored by Databricks. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.