Snowflake Gen AI Functions · Topic 20 of 26
Performance: Model Choice, Latency, Accuracy and Provisioned Throughput
Balance model size, latency, accuracy and capability in Snowflake Cortex, and know when fine-tuning, RAG or Provisioned Throughput is the right lever.
- 4 min
- Smaller model: lower latency and cost. Larger model: more capability.
- Hallucination: RAG first. Domain style or task format: fine-tuning.
- SQL functions favor throughput and batch; REST API for interactive latency.
What it is
Performance in Cortex has four dimensions that pull against each other: latency (how fast), accuracy (how right), capability (what the model can do) and throughput (how much per unit time). Exam scenarios give a symptom and ask for the lever that fixes it, so you need to map symptom to lever.
How it works
Choosing a model
Start from the task, not from the biggest name. Snowflake hosts models from several providers, and availability depends on region. SHOW CORTEX BASE MODELS lists what your account can use, and the models page shows regional availability and cross region options.
| Need | Lean toward |
|---|---|
| Simple classification, short summaries, high volume | A small model |
| Multi step reasoning, complex instructions, long documents | A larger model |
| Images or documents in the prompt | A model that lists image or document input |
| Many languages | A model with strong multilingual support |
| Structured JSON output | A model that follows response_format reliably |
Compare cost too: larger models process the same tokens at a higher credit rate, so right sizing matters for volume work.
Latency
- Model size. Smaller models respond faster. Moving a labeling job from a very large model to a small one is the usual first fix.
- Output length. Fewer output tokens means a faster answer. Set
max_tokensand ask for short answers. - Input length. Trimming the prompt, or sending retrieved chunks instead of whole documents, reduces time and tokens.
- Interface. The SQL AI functions are optimized for throughput on large tables and batch work. For latency sensitive interactive apps, Snowflake recommends the REST APIs (Complete, Embed and Agents).
- Warehouse size. The warehouse that runs the query does not speed up the LLM. The docs recommend MEDIUM or smaller, because a larger warehouse adds cost without more performance.
"An interactive chat is slow when it calls SQL functions" points to the REST API, and "label millions of rows" points to the SQL functions in batch.
Accuracy
Order the levers from cheapest to heaviest:
- Prompt engineering: clear instructions, role, format, a few examples,
temperaturenear 0 for consistent output. - RAG: retrieve real documents with Cortex Search so the model answers from facts. This is the standard fix for hallucination and for questions about private or recent data.
- Fine-tuning: adapt a base model with prompt and completion examples.
SNOWFLAKE.CORTEX.FINETUNEtakes a training query that returnspromptandcompletioncolumns, and the tuned model is called withAI_COMPLETE. The docs say it can improve latency and results for domain specific tasks. The feature page notes it is scheduled for deprecation in Q1 2027, with Cortex Training recommended for LLM fine-tuning, so check current status. - Guardrails:
guardrails: TRUEfilters unsafe output with Cortex Guard, at extra input token cost.
RAG changes what the model knows at question time. Fine-tuning changes how the model behaves. Use RAG for facts that change, fine-tuning for a consistent style or task format.
Model capability
Capability means the model supports the input and task: image input, document input, context window size, tool calling, structured output. If a prompt plus document exceeds the context window, the call fails or truncates, so check size first. AI_COUNT_TOKENS estimates tokens and does not charge for tokens, only warehouse compute. AI_AGG and AI_SUMMARIZE_AGG are not subject to context window limits, so use them to summarize across many rows.
Provisioned Throughput
On demand usage shares capacity and is subject to rate limits. Provisioned Throughput reserves dedicated capacity so end users get consistent throughput.
- You buy it in provisioned throughput units (PTUs) for a one month term, for chosen models in listed regions. Each model has a minimum and an increment of PTUs.
- It is billed for the full term whether you use it or not, and it does not renew automatically.
- Creating one needs the account level
CREATE PROVISIONED THROUGHPUTprivilege, held by ACCOUNTADMIN by default. Callers needUSAGEon it and theSNOWFLAKE.CORTEX_USERrole. - You pass the provisioned throughput ID in the REST API call. The API behavior does not change.
Common mistakes
Upsizing the warehouse to speed up an LLM call. It does not help.
Choosing fine-tuning to add fresh facts. Use RAG.
Using the largest model for every row. It raises cost and time with no gain on simple tasks.
Treating Provisioned Throughput as pay per use. It is a committed term.
High temperature makes output varied. For extraction and classification keep it at 0 or near it.
What to remember for the exam
- Smaller model: lower latency and cost. Larger model: more capability.
- Hallucination: RAG first. Domain style or task format: fine-tuning.
- SQL functions favor throughput and batch; REST API for interactive latency.
- Provisioned Throughput: PTUs, one month term, REST API, full term billing.
AI_COUNT_TOKENSchecks prompt size against the context window.
Check yourself
Which lever reduces hallucination about internal documents?
RAG with Cortex Search, which supplies retrieved facts.
A chat app calling SQL functions feels slow. What does Snowflake recommend?
Use the Complete REST API for latency sensitive interactive use.
How is Provisioned Throughput billed?
By PTU for the whole one month term, whether used or not.
Topic 20 of 26 in Snowflake Gen AI Functions
Test yourself on Snowflake Gen AI Functions
The full Snowflake GES-C02 question bank has 585 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Snowflake. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.