Architecting low-code AI solutions · Topic 3 of 4
Choosing Models in Model Garden and AI APIs
Select the right model from Model Garden and build applications with Document AI, Vision API, Translation API and other pretrained Google Cloud AI APIs.
- 5 min
- Pretrained APIs first for standard tasks: documents, images, speech, translation, sentiment.
- Document AI extracts structured fields; Vision API analyses images; Translation API supports glossaries.
- Model Garden holds Google models, partner models as a service, and open models you deploy.
What it is
Many AI problems do not need a model you train. Pretrained AI APIs such as Document AI, Vision API and Translation API solve common tasks with one call. Model Garden on Gemini Enterprise Agent Platform is the catalog of foundation models: Google models such as Gemini, Imagen and Veo, partner models offered as a service, and open models you can deploy yourself.
The exam asks you to pick the simplest option that meets the task, cost and governance requirements.
How it works
Industry-specific and task APIs
| API | What it does | Typical stem |
|---|---|---|
| Document AI | Processors that extract text, key-value pairs, tables and entities from documents: OCR, Form Parser, specialized processors (for example invoices), and custom extractors | "Extract totals and dates from scanned invoices into BigQuery" |
| Vision API | Labels, objects, text detection (OCR), faces, landmarks, logos and SafeSearch on images | "Flag inappropriate user uploads", "read text in photos" |
| Translation API | Translates text and documents, with glossaries to keep product terms consistent | "Translate support articles into 12 languages keeping brand names" |
| Speech-to-Text / Text-to-Speech | Transcribes audio; synthesizes natural speech | "Transcribe call recordings for analysis" |
| Natural Language API | Sentiment, entities and content classification of text | "Measure sentiment in product reviews" |
| Video Intelligence API | Labels, shot changes, objects and text in video | "Index a video archive by content" |
Choose an API when the task is standard and the output format is fixed. They need no training data, scale automatically and are billed per request or page.
A Document AI processor returns structured fields with confidence scores. Route low-confidence results to a human reviewer instead of training a new extraction model.
from google.cloud import documentai
client = documentai.DocumentProcessorServiceClient()
name = client.processor_path("my-project", "us", "INVOICE_PROCESSOR_ID")
with open("invoice.pdf", "rb") as f:
raw = documentai.RawDocument(content=f.read(), mime_type="application/pdf")
result = client.process_document(
request=documentai.ProcessRequest(name=name, raw_document=raw))
for entity in result.document.entities:
print(entity.type_, entity.mention_text, entity.confidence)What Model Garden offers
| Group | Examples | How you use it |
|---|---|---|
| Google first-party models | Gemini (text, code, images, audio, video in; text out), Imagen (image generation and editing), Veo (video generation), embedding models | Fully managed API; nothing to deploy |
| Partner models as a service | Third-party models offered through managed APIs | Call an API and pay per use; no infrastructure |
| Open models | Gemma and other open-weight models | Deploy to your own endpoint or GKE, or tune them; you pay for the serving hardware |
Each model has a model card describing intended use, inputs and outputs, limits and licensing. Many open models offer a one-click deploy and example notebooks.
Evaluating and selecting a model
| Criterion | Question to ask |
|---|---|
| Task and modality | Text only, or images, audio, video or documents as input? Does it need to generate images or video? |
| Quality | How does it score on your own evaluation set, not just public benchmarks? |
| Context window | Do the documents and conversation history fit? |
| Latency and cost | Is a smaller, faster model good enough? |
| Tuning support | Can it be tuned if prompting is not enough? |
| Control and data | Must weights run inside your project or network? Are there licensing or data residency rules? |
Gemini models come in tiers: larger models for complex reasoning, and smaller Flash and Flash-Lite models for high volume and low latency. Start with the smallest model that passes your evaluation.
A worked example
A bank receives 50,000 loan applications a month as PDFs. It needs applicant name, income and loan amount in BigQuery, and a short summary of each application for underwriters.
The engineer uses a Document AI custom extractor for the fields, because the output must be structured and auditable. A Gemini Flash model then writes the summary from the extracted text. No custom model is trained.
How the exam asks about this
- "Detect and blur inappropriate images uploaded by users, with no ML team." Vision API SafeSearch detection.
- "A regulated customer needs an open model that runs only on infrastructure in its own project." Deploy an open model from Model Garden to its own endpoint or GKE cluster.
- "Translate manuals so product names are never translated." Translation API with a glossary.
What to remember for the exam
- Pretrained APIs first for standard tasks: documents, images, speech, translation, sentiment.
- Document AI extracts structured fields; Vision API analyses images; Translation API supports glossaries.
- Model Garden holds Google models, partner models as a service, and open models you deploy.
- Select models on your own evaluation data, plus latency, cost, context window and control needs.
- Smaller Gemini tiers are often enough and are cheaper and faster.
Training a custom OCR or translation model is almost never the best answer when a pretrained API covers the task; it adds data labeling, cost and maintenance.
Common mistakes
Picking the largest model by default. It raises cost and latency; measure whether a smaller tier passes first.
Using Gemini for a fixed extraction task that needs audit trails. A Document AI processor gives typed fields and confidence scores that are easier to validate.
Ignoring licensing on open models. The model card states the license and intended use; check it before production.
Check yourself
Which API extracts invoice totals and dates from scanned PDFs?
Document AI, using an OCR, Form Parser, specialized invoice or custom extractor processor.
What is the difference between a partner model as a service and an open model in Model Garden?
A model as a service is called through a managed API with no infrastructure; an open model is deployed and served on hardware you provision and pay for.
How should you choose between Gemini tiers?
Evaluate on your own data and pick the smallest, fastest tier that meets the quality bar, moving up only if needed.
Topic 3 of 4 in Architecting low-code AI solutions
Test yourself on Architecting low-code AI solutions
The full Google Cloud Professional Machine Learning Engineer question bank has 749 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Google Cloud. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.