Implementation and Integration · Topic 3 of 8
Model Coordination, Routing and Cascading
Combine specialized FMs, ensembles and model selection, route requests by content or metrics, and cascade from small models to large ones to save cost.
- 6 min
- Includes a diagram
- Specialized FMs handle sub tasks they suit; ensembles combine outputs with custom aggregation such as voting or a judge model.
- Model selection is measured on your own evaluation data for quality, latency and cost.
- Smaller pre trained models cut cost and latency for narrow tasks.
What it is
No single FM is best at everything. A large model writes the best analysis but is slow and expensive; a small model classifies or extracts fields quickly and cheaply.
Model coordination uses several models together. The exam tests three ways to do it: giving each task to a specialized model, combining several model outputs, and routing each request to the cheapest model that can handle it.
How it works
Specialized models and ensembles
A complex task often splits into sub tasks that suit different models. A claims workflow might use a multimodal model to read photos, a small model to extract fields, and a large model to write the decision letter.
Ensembles send the same request to several models and combine the results with custom aggregation logic:
| Aggregation | How it works | Fits |
|---|---|---|
| Majority vote | Pick the answer most models agree on | Classification, yes or no decisions |
| Weighted vote | Weight each model by measured accuracy | Models with known different strengths |
| Judge model | A further model compares the candidates and picks or merges | Free text answers |
| Confidence threshold | Accept only if models agree, otherwise escalate | High risk outputs |
Ensembles raise quality and robustness but multiply cost and latency. Use them only where an error costs more than the extra calls.
Model selection frameworks
A model selection framework is a repeatable way to pick the model for each task. It compares candidates on quality for the task, latency, cost per request, context window and modality, using an evaluation set.
Amazon Bedrock model evaluation can score candidate models on your own prompt dataset. Store the chosen model ID per task in configuration, such as AWS AppConfig or Parameter Store, so you can change it without a code deploy.
The exam favors measured choices. "Choose the model with the best benchmark score" loses to "evaluate candidate models on a representative dataset and pick the smallest model that meets the quality bar".
Smaller models and cascading
The cheapest way to optimize a GenAI workload is often to use a smaller pre trained model for narrow tasks such as classification, extraction, summarization of short text or routing. Smaller models answer faster and cost less per token.
Model cascading tries the cheap model first and escalates only when needed:
- Routine queries stay on the small model, so most traffic is cheap.
- A check decides when to escalate: a confidence signal, a validation rule, or a classifier that predicts difficulty.
- Behind an API, the cascade is invisible to the client; this is API based cascading.
Routing options
| Routing type | Implemented with | Pick it when |
|---|---|---|
| Static routing | Application code or configuration maps a task type to a model ID | Task types are known and stable |
| Content based routing | Step Functions Choice state or a classifier reads the request and sends it to a specialized FM | Requests differ by topic, language or modality |
| Metric based routing | Router uses latency, error rate, cost or quality metrics from CloudWatch | Model choice should adapt to live conditions |
| Gateway routing | API Gateway with request transformations or a routing Lambda sets the target model | Clients call one stable API while routing changes behind it |
| Amazon Bedrock Intelligent Prompt Routing | A prompt router predicts response quality and sends each prompt to a model within one model family | You want managed quality and cost routing with no custom code |
Static routing is simplest and fully predictable. Content based routing in Step Functions adds visible branches and retries. Metric based routing shifts traffic when a model slows down or starts throwing errors.
Inference profiles and Regions
Cross Region inference profiles in Amazon Bedrock spread on demand requests across several Regions in a geography, which raises available throughput. Routing logic should call the inference profile ID rather than a single Region model ID when the scenario mentions throttling at peak load.
How the exam asks about this
- "Most customer questions are simple FAQs, but some need deep analysis; reduce cost without hurting quality" -> cascade from a small model to a large model, escalating on low confidence.
- "Send contract questions to a legal tuned model and code questions to a code model, with visible branching and retries" -> Step Functions with a Choice state for content based routing.
- "Clients must keep one endpoint while the team changes which model serves each request" -> API Gateway in front, with routing in a Lambda function or request transformation.
Common mistakes
Using the largest model for every call. It is slower and costlier, and for classification or extraction it often adds nothing.
Hard coding model IDs across services. Changing a model then needs many deploys. Keep the mapping in configuration or a central router.
Ensembling everything. Several models per request multiply cost. Reserve ensembles for high risk decisions.
Cascading without a reliable escalation check. If the small model is wrong but confident, the cascade never escalates. Validate the output, not only the confidence.
What to remember for the exam
- Specialized FMs handle sub tasks they suit; ensembles combine outputs with custom aggregation such as voting or a judge model.
- Model selection is measured on your own evaluation data for quality, latency and cost.
- Smaller pre trained models cut cost and latency for narrow tasks.
- Cascading sends routine queries to a small model and escalates hard ones.
- Static routing lives in code or configuration; dynamic content based routing uses Step Functions; metric based routing reacts to live CloudWatch data.
- Bedrock Intelligent Prompt Routing is the managed option for routing within a model family.
Intelligent Prompt Routing chooses among models in the same family. If the scenario needs routing across providers or to a custom specialized model, build the router yourself.
Check yourself
What is model cascading?
Sending each request to a cheap small model first and escalating to a larger model only when the answer fails a confidence or validation check.
Which service fits dynamic content based routing with auditable branches?
AWS Step Functions, using a Choice state that sends each request to a specialized FM.
How do you let clients keep one API while model routing changes?
Put API Gateway in front and do routing behind it with request transformations or a routing Lambda function.
Topic 3 of 8 in Implementation and Integration
Test yourself on Implementation and Integration
The full AWS AIP-C01 question bank has 827 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.