Data Governance · Topic 2 of 7

Lineage, Dependencies and Data Quality

Monitor Snowflake data with lineage graphs, ACCESS_HISTORY, OBJECT_DEPENDENCIES and data metric functions that run scheduled data quality checks.

  • 4 min
In a nutshell
  • Lineage in Snowsight, built from recorded data movement, supports impact and root cause analysis (Enterprise).
  • ACCESS_HISTORY has direct_objects_accessed, base_objects_accessed, objects_modified.
  • OBJECT_DEPENDENCIES lists referencing and referenced objects.

What it is

Three questions come up when a number looks wrong or a column must change. Where did this data come from and where does it go? Which objects will break if I change this one? Is the data itself still good?

Lineage shows data flow between objects. Object dependencies show which objects reference which. Data quality monitoring measures the data itself with data metric functions on a schedule.

Source tablerawfeedsView or tabletransformedfeedsReport tableconsumedtraced byLineage graphupstream and downstream

How it works

Lineage

Lineage shows how data moves from sources through transformations to downstream objects, at table and column level. It is built from the operations that move data, such as CTAS, INSERT, MERGE and CREATE VIEW, and works for tables, views, dynamic tables, stages and similar objects.

  • In Snowsight, open an object and use the Lineage tab to see the graph upstream and downstream.
  • Lineage helps with impact analysis before a change and with root cause analysis after a failure.
  • It needs Enterprise edition or higher, and the viewer needs access to the objects shown.

ACCESS_HISTORY

SNOWFLAKE.ACCOUNT_USAGE.ACCESS_HISTORY records one row per query with what the query read and wrote. It is also the basis for auditing who touched sensitive columns.

ColumnContains
direct_objects_accessedObjects named in the query, such as a view
base_objects_accessedThe underlying tables the view resolves to
objects_modifiedObjects the query wrote to, with column level lineage
policies_referencedMasking and row access policies that applied

Requires Enterprise edition or higher. Because it holds both direct and base objects, a query on a view shows the view as direct and the tables below it as base.

Object dependencies

SNOWFLAKE.ACCOUNT_USAGE.OBJECT_DEPENDENCIES lists each pair of referencing and referenced objects. For example, a view that selects from a table is the referencing object, and the table is the referenced one.

SELECT referenced_object_name, referencing_object_name, referencing_object_domain
FROM SNOWFLAKE.ACCOUNT_USAGE.OBJECT_DEPENDENCIES
WHERE referenced_object_name = 'ORDERS';
Referencing objectReferenced object
View, materialized viewTables and other views
Dynamic tableSource tables
FunctionObjects named in its body

The dependency_type column shows BY_NAME, BY_ID or BY_NAME_AND_ID. Use it before dropping or altering an object, to find what would break.

Exam tip

Lineage is the visual graph built from recorded data movement, while OBJECT_DEPENDENCIES lists references defined in object definitions. Use dependencies for "what breaks if I drop this", and lineage for "where did this data come from".

Data quality monitoring

Data quality uses data metric functions (DMFs). A DMF is a function that measures one thing about a table or view, such as null counts, and Snowflake runs it on a schedule.

DMF typeExamples
System DMFs in SNOWFLAKE.CORENULL_COUNT, ROW_COUNT, ACCEPTED_VALUES, plus duplicate, unique, freshness and similar measures
Custom DMFsFunctions you write with CREATE DATA METRIC FUNCTION
ALTER TABLE customers SET DATA_METRIC_SCHEDULE = '60 MINUTE';
ALTER TABLE customers
  ADD DATA METRIC FUNCTION SNOWFLAKE.CORE.NULL_COUNT ON (email);

Key points:

  • The schedule is set on the table or view and applies to all its DMFs. It can be an interval such as '5 MINUTE', a USING CRON expression, or TRIGGER_ON_CHANGES to run after data changes. The default is one hour.
  • Setting the schedule to an empty string '' suspends all DMFs on the table. MODIFY DATA METRIC FUNCTION ... SUSPEND pauses one.
  • DMFs run on serverless compute, billed as credits.
  • Results appear in the DATA_QUALITY_MONITORING_RESULTS view, which you filter by measurement_time, and in Snowsight data quality pages.
  • The feature needs Enterprise edition or higher.

An expectation turns a measurement into a pass or fail check. You attach it to a DMF, for example "null count must equal 0", and a violation is reported when the value fails.

CREATE TABLE customers (id NUMBER, email VARCHAR)
WITH DATA METRIC FUNCTION (
  SNOWFLAKE.CORE.NULL_COUNT ON (email) EXPECTATION no_null_email (VALUE = 0)
);
Tablewith DMFs attachedschedule triggersDMF runsserverlesswritesResults viewper measurementcompared withExpectationpass or violation

Common mistakes

Confusing lineage and dependencies. Lineage follows recorded data movement; dependencies follow object references.

Expecting DMFs to use a warehouse. They use serverless compute.

Setting a schedule per DMF. The schedule belongs to the table or view.

Dropping an object without checking dependencies. Views and dynamic tables above it may break.

What to remember for the exam

  • Lineage in Snowsight, built from recorded data movement, supports impact and root cause analysis (Enterprise).
  • ACCESS_HISTORY has direct_objects_accessed, base_objects_accessed, objects_modified.
  • OBJECT_DEPENDENCIES lists referencing and referenced objects.
  • DMFs: system in SNOWFLAKE.CORE, custom with CREATE DATA METRIC FUNCTION; the schedule is set on the table, the default is one hour.
  • Expectations compare a DMF result with a rule.

Check yourself

  • Which view shows the underlying tables behind a view that a query read?

    ACCESS_HISTORY, in the base_objects_accessed column.

  • Where do you set how often data metric functions run?

    On the table or view with DATA_METRIC_SCHEDULE, which applies to all its DMFs.

  • What does an expectation add to a data metric function?

    A pass or fail rule on the measured value, so violations are reported.

Topic 2 of 7 in Data Governance

Test yourself on Data Governance

The full Snowflake DEA-C02 question bank has 709 questions, with timed practice exams and an explanation for every answer.

This study guide is independent and is not affiliated with, endorsed by, or sponsored by Snowflake. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.