Chapter 4 of 6 · Google Cloud Professional Machine Learning Engineer
Serving and scaling models
Batch and online inference, Model Registry and scaling.
- 3topics
- 15 minfast track
- 1diagrams and exhibits
- 20%of the exam
Topics in this chapter
Take them in order, or jump to the one you need.
- 1Deploying Models for Batch and Online InferenceServe models in batch or online on Agent Platform, Model Garden, Cloud Run or GKE, package them in prebuilt or custom containers…5 min
- 2Model Registry Versions and Rollout StrategiesOrganize and version models in Gemini Enterprise Agent Platform Model Registry and compare versions with A/B tests, canary…Diagram5 min
- 3Scaling Online Serving with Endpoints and FeaturesServe features from Feature Store, choose public or private endpoints and serving hardware, autoscale on throughput and tune…5 min
About this chapter
What this chapter covers
Serving and scaling models is about 20% of the Google Cloud Professional Machine Learning Engineer exam. It covers getting trained models into use on Gemini Enterprise Agent Platform (the platform formerly called Vertex AI) and other Google Cloud services, releasing new versions safely, and keeping online serving fast and affordable as traffic grows.
It covers serving models (batch and online inference on Agent Platform, Model Garden, Cloud Run and GKE; prebuilt and custom containers for PyTorch, XGBoost and other frameworks; versioning in Gemini Enterprise Agent Platform Model Registry; A/B testing and canary rollouts; inference preprocessing and postprocessing) and scaling online serving (serving features from Agent Platform Feature Store; public and private endpoints; CPU, GPU, TPU and edge hardware; scaling with Gemini Enterprise Agent Platform Inference and containerized serving; tuning models for production).
The exam has 50 to 60 multiple choice and multiple select questions in 2 hours. Questions here usually give a latency target, a traffic pattern and a network or release constraint, then ask "What should you do?".
How the topics fit together
Every model goes through the registry, gets packaged in a container, and is served in batch or online. Online models then need safe rollouts and scaling that follows traffic.
Decide batch or online first. "Nightly", "weekly" or "for all customers" means a batch inference job with no endpoint. "Per request" or "in milliseconds" means an online endpoint.
How to use these pages
Each topic page has the same shape, so you can skim for what you need.
- What it is: the idea in plain words.
- How it works: the mechanism, with tables and code examples.
- What to remember for the exam: the facts to memorise and how questions are phrased.
- Common mistakes: the classic traps.
- Check yourself: three questions to test your understanding.
Ready for more practice?
The full Google Cloud Professional Machine Learning Engineer question bank has 749 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Google Cloud. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.