Architecting low-code AI solutions · Topic 4 of 4
Building, Tuning and Optimizing Gemini Applications
Build gen AI solutions with Gemini, Imagen, Veo and models as a service, decide when to tune, and optimize Gemini apps for cost, latency and availability.
- 5 min
- Includes a diagram
- Order of effort: prompt engineering, context engineering, grounding or RAG, then tuning.
- Tuning changes style and task behaviour; RAG supplies current facts with citations.
- Imagen makes images, Veo makes video, Gemini reasons over multimodal input.
What it is
A gen AI solution is a foundation model plus everything around it: the prompt and context you send, the data you ground it on, any tuning, and the serving choices that set cost, latency and availability. On Gemini Enterprise Agent Platform these models are Gemini for text and multimodal reasoning, Imagen for images, Veo for video, and models as a service in Model Garden.
How it works
Building for a use case
| Use case | Model | Notes |
|---|---|---|
| Summarize, classify, extract, answer questions, write code | Gemini | Multimodal input: text, images, audio, video, PDFs |
| Create or edit marketing images | Imagen | Text-to-image and image editing; by default outputs carry an invisible SynthID digital watermark |
| Generate short video clips from text or an image | Veo | Text-to-video and image-to-video |
| A specific partner model the business has standardized on | Model as a service in Model Garden | Managed API, pay per use |
Improving answers: prompt, ground, then tune
Work through these in order; each step costs more than the last.
| Technique | Fixes | How |
|---|---|---|
| Prompt engineering | Vague or badly formatted output | Clear task, system instructions, few-shot examples, output format such as a JSON schema |
| Context engineering | Missing or noisy information in the prompt | Give the model only the relevant facts, history and tool results it needs |
| Grounding / RAG | Wrong or out-of-date facts, need for citations | Retrieve passages from your documents (vector search) or ground on Google Search, then answer from them |
| Supervised fine-tuning | Consistent style, format or domain task that prompts cannot reach | Train on hundreds or more of prompt and response examples |
Tuning teaches behaviour, not fresh facts. If the data changes daily, use RAG: the model reads current documents at request time.
from google import genai
from google.genai import types
client = genai.Client(vertexai=True, project="my-project", location="global")
response = client.models.generate_content(
model="gemini-3.5-flash",
contents="Classify this review as positive, neutral or negative: ...",
config=types.GenerateContentConfig(
system_instruction="You label customer reviews. Answer with one word.",
temperature=0.0,
),
)
print(response.text)Optimizing for cost
| Lever | Effect |
|---|---|
| Smaller model tier (Flash, Flash-Lite) | Lower price per token and faster |
| Shorter prompts and capped output tokens | Fewer billed tokens |
| Context caching | Reuses a large repeated prefix (a long document or system prompt) at a reduced token price |
| Batch inference | Lower price for jobs that can wait instead of answering in real time |
| Tuning a smaller model | A tuned small model can replace a large model with long few-shot prompts |
Optimizing for latency
- Use a smaller tier and limit output length.
- Stream responses so users see the first words immediately.
- Keep prompts short; cache long shared context.
- For thinking models, lower the thinking level or budget when the task is simple.
Optimizing for availability and throughput
Pay-as-you-go traffic shares capacity, and a busy model can return 429 resource exhausted errors.
| Need | Option |
|---|---|
| Fewer capacity errors with no commitment | Use the global endpoint so requests can run in any region with capacity |
| Guaranteed throughput for a critical app | Buy Provisioned Throughput |
| Resilient client | Retry 429 and 5xx errors with exponential backoff and jitter |
| Data must stay in a region | Use a regional endpoint and plan capacity there |
A worked example
A travel company's Gemini chatbot answers from a 300-page policy manual pasted into every request. It is slow and expensive, and it sometimes quotes old rules.
The engineer replaces the pasted manual with RAG: the manual is chunked and embedded, and only the top passages go into each prompt. The model moves from a large tier to Flash after it passes the same evaluation set. Responses are streamed, and Provisioned Throughput covers the peak booking season.
How the exam asks about this
- "The model must answer from product documents that change every week and cite sources." Ground the model with RAG over the documents; do not fine-tune.
- "A high-traffic app gets intermittent 429 errors at peak and needs guaranteed capacity." Purchase Provisioned Throughput and retry with exponential backoff.
- "Every request repeats the same 100-page contract before a short question." Use context caching for the shared contract content.
What to remember for the exam
- Order of effort: prompt engineering, context engineering, grounding or RAG, then tuning.
- Tuning changes style and task behaviour; RAG supplies current facts with citations.
- Imagen makes images, Veo makes video, Gemini reasons over multimodal input.
- Cost: smaller tier, fewer tokens, context caching, batch inference.
- Availability: global endpoint, Provisioned Throughput, retries with backoff.
Fine-tuning on a snapshot of company documents does not keep answers current. When the facts change, the tuned model is wrong until retrained; RAG reads the latest data.
Common mistakes
Tuning before trying prompts. Few-shot prompts and system instructions often reach the target at no training cost.
Treating 429 errors as a bug in the code. They signal capacity limits; the fix is backoff, the global endpoint, or Provisioned Throughput.
Using real-time calls for overnight jobs. Batch inference is cheaper when nobody is waiting for the answer.
Check yourself
When should you fine-tune Gemini instead of using RAG?
When you need a consistent behaviour, style or output format that prompting cannot reach; RAG is better when the model needs current facts.
What does context caching save?
The cost of re-sending a large, repeated prompt prefix such as a long document or system prompt on every request.
What gives a Gemini application guaranteed capacity?
Provisioned Throughput, which reserves throughput for the model instead of sharing pay-as-you-go capacity.
Topic 4 of 4 in Architecting low-code AI solutions
Test yourself on Architecting low-code AI solutions
The full Google Cloud Professional Machine Learning Engineer question bank has 749 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Google Cloud. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.