Sharing and Improving Genie Agents

What it is

Once a Genie Agent is built, the author shares it in the Share dialog and decides who may ask and who may change it. The agent can reach users through links, embedded pages, external apps and chat tools.

After launch, real questions, feedback and benchmark runs show where Genie goes wrong, and the author fixes the context.

How it works

Permission levels

The creator gets CAN MANAGE automatically. Permissions are granted to users, groups or all account users in the Share dialog.

Data access still applies

Agent permission alone does not show any data. End users need the consumer access or Databricks SQL entitlement, and they need SELECT on the tables (plus USE CATALOG and USE SCHEMA) because each question is checked against the asking user's own Unity Catalog permissions.

Row filters and column masks on the tables therefore work inside Genie too.

Distributing an agent

Whatever the channel, the user's identity and Unity Catalog permissions still decide what data comes back.

Tracking questions and feedback

The Monitor tab lists the questions asked in the agent and their answers. It filters by time, rating, user and status. Users with CAN MANAGE see every user's questions here.

Users rate answers and can request a review of a response they doubt. Authors with CAN MANAGE review the flagged exchanges. Feedback on its own does not change how Genie behaves, so the author must act on it.

Updating instructions and trusted assets

Stakeholder input usually points to a missing definition or a repeated wrong answer:

• A misunderstood business term: add it to general instructions or as a SQL expression.
• A question that must always return the same figure: add a parameterized example SQL query as a trusted asset.
• A wrong column picked: improve the column description or add synonyms.

Validating accuracy with benchmarks

Benchmarks are test questions written by the author, each with an optional expected SQL Answer. Running them shows how accurately Genie answers, so the author can check that a change helped and did not break other questions.

Users with at least CAN EDIT can run benchmarks. The Evaluations tab shows an accuracy figure for each run.

Refreshing Unity Catalog metadata

Genie relies on table and column comments, so keep them current in Unity Catalog. Descriptions edited in the agent are scoped to the agent and do not change Unity Catalog; Reset on a table description restores the Unity Catalog one.

COMMENT ON TABLE sales.gold.orders IS 'One row per order line, net of returns';
ALTER TABLE sales.gold.orders
  ALTER COLUMN region COMMENT 'Sales region code, such as EMEA';

When new values appear in a column, such as a new region code, use Refresh prompt matching on that column so Genie knows the current values.

How the exam asks about this

• "A manager must change the agent's instructions but not its permissions." Grant CAN EDIT.
• "Sales staff should ask Genie inside Microsoft Teams." Use the Teams channel app.
• "Prove that new instructions improved accuracy before sharing widely." Run the benchmarks and compare accuracy.