Data Governance · Topic 2 of 7
Lineage, Dependencies and Data Quality
Monitor Snowflake data with lineage graphs, ACCESS_HISTORY, OBJECT_DEPENDENCIES and data metric functions that run scheduled data quality checks.
- 4 min
- Lineage in Snowsight, built from recorded data movement, supports impact and root cause analysis (Enterprise).
ACCESS_HISTORYhasdirect_objects_accessed,base_objects_accessed,objects_modified.OBJECT_DEPENDENCIESlists referencing and referenced objects.
What it is
Three questions come up when a number looks wrong or a column must change. Where did this data come from and where does it go? Which objects will break if I change this one? Is the data itself still good?
Lineage shows data flow between objects. Object dependencies show which objects reference which. Data quality monitoring measures the data itself with data metric functions on a schedule.
How it works
Lineage
Lineage shows how data moves from sources through transformations to downstream objects, at table and column level. It is built from the operations that move data, such as CTAS, INSERT, MERGE and CREATE VIEW, and works for tables, views, dynamic tables, stages and similar objects.
- In Snowsight, open an object and use the Lineage tab to see the graph upstream and downstream.
- Lineage helps with impact analysis before a change and with root cause analysis after a failure.
- It needs Enterprise edition or higher, and the viewer needs access to the objects shown.
ACCESS_HISTORY
SNOWFLAKE.ACCOUNT_USAGE.ACCESS_HISTORY records one row per query with what the query read and wrote. It is also the basis for auditing who touched sensitive columns.
| Column | Contains |
|---|---|
direct_objects_accessed | Objects named in the query, such as a view |
base_objects_accessed | The underlying tables the view resolves to |
objects_modified | Objects the query wrote to, with column level lineage |
policies_referenced | Masking and row access policies that applied |
Requires Enterprise edition or higher. Because it holds both direct and base objects, a query on a view shows the view as direct and the tables below it as base.
Object dependencies
SNOWFLAKE.ACCOUNT_USAGE.OBJECT_DEPENDENCIES lists each pair of referencing and referenced objects. For example, a view that selects from a table is the referencing object, and the table is the referenced one.
SELECT referenced_object_name, referencing_object_name, referencing_object_domain
FROM SNOWFLAKE.ACCOUNT_USAGE.OBJECT_DEPENDENCIES
WHERE referenced_object_name = 'ORDERS';
| Referencing object | Referenced object |
|---|---|
| View, materialized view | Tables and other views |
| Dynamic table | Source tables |
| Function | Objects named in its body |
The dependency_type column shows BY_NAME, BY_ID or BY_NAME_AND_ID. Use it before dropping or altering an object, to find what would break.
Lineage is the visual graph built from recorded data movement, while OBJECT_DEPENDENCIES lists references defined in object definitions. Use dependencies for "what breaks if I drop this", and lineage for "where did this data come from".
Data quality monitoring
Data quality uses data metric functions (DMFs). A DMF is a function that measures one thing about a table or view, such as null counts, and Snowflake runs it on a schedule.
| DMF type | Examples |
|---|---|
System DMFs in SNOWFLAKE.CORE | NULL_COUNT, ROW_COUNT, ACCEPTED_VALUES, plus duplicate, unique, freshness and similar measures |
| Custom DMFs | Functions you write with CREATE DATA METRIC FUNCTION |
ALTER TABLE customers SET DATA_METRIC_SCHEDULE = '60 MINUTE';
ALTER TABLE customers
ADD DATA METRIC FUNCTION SNOWFLAKE.CORE.NULL_COUNT ON (email);
Key points:
- The schedule is set on the table or view and applies to all its DMFs. It can be an interval such as
'5 MINUTE', aUSING CRONexpression, orTRIGGER_ON_CHANGESto run after data changes. The default is one hour. - Setting the schedule to an empty string
''suspends all DMFs on the table.MODIFY DATA METRIC FUNCTION ... SUSPENDpauses one. - DMFs run on serverless compute, billed as credits.
- Results appear in the
DATA_QUALITY_MONITORING_RESULTSview, which you filter bymeasurement_time, and in Snowsight data quality pages. - The feature needs Enterprise edition or higher.
An expectation turns a measurement into a pass or fail check. You attach it to a DMF, for example "null count must equal 0", and a violation is reported when the value fails.
CREATE TABLE customers (id NUMBER, email VARCHAR)
WITH DATA METRIC FUNCTION (
SNOWFLAKE.CORE.NULL_COUNT ON (email) EXPECTATION no_null_email (VALUE = 0)
);
Common mistakes
Confusing lineage and dependencies. Lineage follows recorded data movement; dependencies follow object references.
Expecting DMFs to use a warehouse. They use serverless compute.
Setting a schedule per DMF. The schedule belongs to the table or view.
Dropping an object without checking dependencies. Views and dynamic tables above it may break.
What to remember for the exam
- Lineage in Snowsight, built from recorded data movement, supports impact and root cause analysis (Enterprise).
ACCESS_HISTORYhasdirect_objects_accessed,base_objects_accessed,objects_modified.OBJECT_DEPENDENCIESlists referencing and referenced objects.- DMFs: system in
SNOWFLAKE.CORE, custom withCREATE DATA METRIC FUNCTION; the schedule is set on the table, the default is one hour. - Expectations compare a DMF result with a rule.
Check yourself
Which view shows the underlying tables behind a view that a query read?
ACCESS_HISTORY, in the base_objects_accessed column.
Where do you set how often data metric functions run?
On the table or view with DATA_METRIC_SCHEDULE, which applies to all its DMFs.
What does an expectation add to a data metric function?
A pass or fail rule on the measured value, so violations are reported.
Topic 2 of 7 in Data Governance
Test yourself on Data Governance
The full Snowflake DEA-C02 question bank has 709 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Snowflake. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.