Application Development · Topic 1 of 4
Selecting LLMs and Models
Databricks GenAI Engineer: choose an LLM by context window, latency, cost and license, read model cards in a hub or Marketplace, compare MLflow runs.
- 5 min
- Start from the application's attributes: task, context length, latency, cost, tool calling and license.
- Model cards give intended use, context length, training data, license, size and benchmark results.
- Public benchmarks shortlist models; your own evaluation set decides.
What it is
Model selection means choosing the LLM (and any helper model) whose attributes fit what the application needs. The exam gives you a requirement such as long documents, a tight latency budget, low cost or commercial use, and asks which model fits.
You make the choice from three kinds of evidence: the application's own attributes, the model card in a hub or the Databricks Marketplace, and the metrics your experiments log in MLflow.
How it works
Matching the LLM to the application
| Application attribute | What to look for in the model |
|---|---|
| Long inputs, many retrieved chunks | A large enough context window |
| Chat agent that calls tools | Support for tool (function) calling |
| Output parsed by code | Reliable structured (JSON) output |
| Tight latency or high volume | A smaller, faster model |
| Hard reasoning or multi step planning | A larger, more capable model |
| Commercial product | A license that allows commercial use |
| Several languages | Multilingual training data |
| Data must stay inside Databricks | A Databricks hosted model, not an external provider |
Bigger is not automatically better. A smaller model that passes your quality bar is cheaper and faster, which is often the stated goal in a question.
Where the models come from
On Databricks, Foundation Model APIs serve popular open and proprietary models on ready endpoints, billed pay per token or with provisioned throughput for steady production load. External models let a serving endpoint call a third party provider such as OpenAI or Anthropic through the same interface, governed by AI Gateway.
You can also pick a model from a model hub such as Hugging Face or from the Databricks Marketplace, register it in Unity Catalog and serve it with Mosaic AI Model Serving.
Reading a model card
A model card is the metadata that tells you whether a model fits before you test it.
| Model card field | Why it matters |
|---|---|
| Intended use and task | Chat, summarization, embedding, classification, code |
| Context length | Whether your prompt and context fit |
| Training data and languages | Domain fit and language coverage |
| License | Commercial use, attribution, redistribution limits |
| Size (parameters) | Rough guide to cost and latency |
| Evaluation results | Scores on public benchmarks for the task |
| Limitations and bias notes | Known failure areas |
Use the card to narrow a shortlist. Public benchmark scores do not prove the model works on your data, so the final choice still comes from your own evaluation.
Comparing models with experiment metrics
Log each candidate as an MLflow run in the same experiment, evaluated on the same dataset, and compare runs side by side. Common metrics are:
| Metric | Used for |
|---|---|
| Judge or scorer pass rate (correctness, relevance, safety) | Overall answer quality |
| ROUGE | Summarization overlap with a reference |
| BLEU | Translation overlap with a reference |
| Exact match | Short factual answers |
| Toxicity | Harmful language |
| Latency and token counts | Speed and cost |
import mlflow
from mlflow.genai.scorers import Correctness, Safety
results = mlflow.genai.evaluate(
data=eval_df, predict_fn=answer_with_model_a,
scorers=[Correctness(), Safety()])Pick the model that meets the quality bar on the metric that matches the task, then break ties on latency and cost. A model with a slightly higher score but double the latency can be the wrong answer when the question stresses response time.
How the exam asks about this
- "Contracts of 40 pages must be summarized in one call." Choose a model whose context window fits the whole document plus the prompt.
- "The product will be sold commercially and the model card shows a non commercial license." Reject that model whatever its scores.
- "Two models tie on correctness; one has half the latency." Pick the faster model, since quality is equal.
What to remember for the exam
- Start from the application's attributes: task, context length, latency, cost, tool calling and license.
- Model cards give intended use, context length, training data, license, size and benchmark results.
- Public benchmarks shortlist models; your own evaluation set decides.
- Compare candidates as MLflow runs on the same data with the same scorers.
- Match the metric to the task: ROUGE for summaries, BLEU for translation, exact match for short facts.
- Foundation Model APIs and external models share one serving interface, so swapping models is easy.
Never choose a model on benchmark scores alone when the question mentions a license, data residency or latency requirement. A hard requirement removes a model from the list before quality is compared.
Common mistakes
Choosing the largest model by default. A large model costs more and responds more slowly. If a smaller model meets the quality bar, it is the better engineering choice.
Comparing runs on different data. Scores from different evaluation sets are not comparable. Every candidate must be scored on the same dataset with the same scorers.
Ignoring the license. A model card license that forbids commercial use rules the model out for a commercial product, however well it scores.
Check yourself
What should decide between two models that both pass the quality bar?
The remaining requirements, usually latency and cost, so the faster or cheaper model wins when quality is equal.
Which model card fields matter most for a RAG chatbot over long manuals?
The context length, so prompts and retrieved chunks fit, plus the license and intended use for chat or question answering.
Which metric fits a translation task with reference translations?
BLEU, which measures overlap between the model's output and the reference translation, logged per run in MLflow.
Topic 1 of 4 in Application Development
Test yourself on Application Development
The full Databricks Generative AI Engineer Associate question bank has 600 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Databricks. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.