Knowledge Retrieval and Genie Configuration · Topic 3 of 4
Unity Catalog Metadata and Genie Space Curation
Databricks Context Engineer: find the Unity Catalog metadata gap behind wrong agent answers and pick tables, views, queries and functions for Genie.
- 5 min
- Table and column comments are the first metadata agents and Genie read; missing comments cause wrong column and table choices.
- Primary and foreign keys and table purpose comments clarify joins and grain.
- Genie spaces hold up to 50 Unity Catalog tables and views, plus instructions, sample questions, SQL expressions and trusted assets.
What it is
Agents and Genie spaces read Unity Catalog metadata to understand data. Table and column comments, clear names, key constraints and tags tell the model what a column means, how tables join and which values are valid. When that metadata is missing or vague, the model guesses, and the guess shows up as an accuracy gap.
A Genie space (newer docs may call it a Genie Agent) is a curated, natural language interface over a chosen set of Unity Catalog tables and views. What you put in the space, and what you leave out, decides how well it answers.
How it works
Finding the metadata gap
Read the failing questions and the metadata side by side. The pattern of errors points to the missing element.
| Symptom | Missing or weak metadata | Highest impact fix |
|---|---|---|
| Picks the wrong one of two similar columns | Column comments absent or identical | Write column comments that state meaning and units |
| Cryptic names such as amt_2 or flg_c misread | No comments on coded columns | Comment each column with what it holds |
| Wrong join, duplicated totals | Join path unclear | Declare primary and foreign keys, comment the relationship |
| Status values misused ("A", "C", "X") | Valid values not described | List code meanings in the column comment |
| Wrong table chosen among near duplicates | Table comments do not say purpose or grain | Comment each table with its purpose and one row meaning |
| Business term not understood | Domain vocabulary absent | Genie general instructions or a SQL expression |
The highest impact change is the one that fixes the most failing questions at their root. If ten failures all involve one undocumented revenue column, a precise comment on that column beats a long new system prompt.
Comments can be set in Catalog Explorer, where AI generated suggestions can be reviewed, or in SQL:
COMMENT ON TABLE sales.gold.orders IS 'One row per customer order line. Amounts in USD.';
ALTER TABLE sales.gold.orders ALTER COLUMN net_amt
COMMENT 'Order line value after discounts, before tax, in USD';What a Genie space is made of
| Component | Use it for |
|---|---|
| Unity Catalog tables and views (up to 50) | The data Genie may query |
| General instructions | Domain vocabulary and business rules |
| Sample questions | Showing users what the space answers |
| SQL expressions | Reusable measures, filters and dimensions |
| Trusted assets | Parameterized example SQL queries and UC SQL functions; answers carry the Trusted label |
| Benchmarks | A question set that measures accuracy |
Genie works from tables and views. It does not query volumes, and you cannot add a whole catalog as one object.
Choosing objects for a business domain
| Object | Include when |
|---|---|
| Managed tables | They hold the curated, authoritative facts for this domain at the right grain |
| Views | You want to pre join, rename, filter or hide columns so Genie sees a simpler shape |
| Parameterized example queries | A question is frequent and the exact SQL must be right every time, such as a regulatory metric |
| UC SQL functions | Logic must be governed and reused by Genie and other tools, such as a fiscal period calculation |
Keep the space focused on one domain and its users. A finance space with 12 relevant tables answers better than one with 45 tables from finance, HR and marketing, because extra tables give Genie more wrong candidates.
Leave out raw or staging tables when a cleaned gold table covers the same facts. Two tables with the same meaning invite Genie to pick the wrong one.
A small worked check
A benchmark of 40 questions scores 26 correct (65%). Nine of the 14 failures use one ambiguous column, region_cd. Commenting that column with its code meanings fixes 8 of them and lifts the score to 34 of 40 (85%). Adding trusted queries for the six remaining failures raises it further.
How the exam asks about this
- "Genie confuses gross_amt and net_amt; neither column has a comment." Add column comments that define each amount; that fixes the root cause.
- "Quarterly regulatory capital must always use one approved calculation." Add it as a trusted asset, a parameterized example query or a UC SQL function.
- "A sales space includes raw ingestion tables alongside the gold tables, and answers vary." Remove the raw tables and keep the curated gold tables or views.
What to remember for the exam
- Table and column comments are the first metadata agents and Genie read; missing comments cause wrong column and table choices.
- Primary and foreign keys and table purpose comments clarify joins and grain.
- Genie spaces hold up to 50 Unity Catalog tables and views, plus instructions, sample questions, SQL expressions and trusted assets.
- Trusted assets are parameterized example queries and UC SQL functions; their answers carry the Trusted label.
- Use views to simplify shape and hide columns; use functions for governed, reusable logic.
- Measure changes with a benchmark set, before and after.
Do not fix a metadata gap with instructions alone. If a column has no comment, every agent and Genie space over that table inherits the gap; the comment fixes it once at the source.
Common mistakes
Adding every table "just in case". Extra tables increase the chance Genie picks the wrong one. Curate the smallest set that answers the domain's questions.
Using sample questions to teach vocabulary. Sample questions show users what to ask. Domain terms and business rules belong in general instructions or SQL expressions.
Writing comments that repeat the name. A comment such as "net amount" on net_amt adds nothing. State the meaning, the unit and whether tax or discounts are included.
Check yourself
An agent misreads a status column holding codes A, P and X. What is the highest impact fix?
Add a column comment that lists what each code means, so every agent and Genie space reading the table interprets it correctly.
When should a calculation be added to a Genie space as a UC SQL function?
When the logic must be governed and reused exactly, by Genie and other tools, rather than rewritten in each generated query.
Why include a view instead of the underlying tables?
A view can pre join, rename and filter the data and hide columns, so Genie sees a simpler, safer shape for the domain.
Topic 3 of 4 in Knowledge Retrieval and Genie Configuration
Test yourself on Knowledge Retrieval and Genie Configuration
The full Databricks Context Engineer Associate question bank has 600 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Databricks. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.