Foundation Model Integration, Data Management, and Compliance · Topic 2 of 9

Foundation Model Selection, Customization and Lifecycle

Choose FMs with benchmarks and Bedrock model evaluation, then fine tune, register, deploy, roll back and retire customized models on SageMaker AI.

  • 6 min
In a nutshell
  • Select FMs on capability, context window, latency, cost, Regional availability and results on your own data.
  • Bedrock model evaluation compares models with automatic, LLM as a judge or human evaluation.
  • LoRA trains small adapters on a frozen base model; adapters are cheap to train, store and swap.

What it is

Choosing an FM means matching a model's capabilities, limits and cost to a specific use case, on evidence rather than reputation. A model that tops a public leaderboard may still be too slow, too expensive or weak on your data.

When prompting and RAG are not enough, you customize the model. Then the model becomes a software artifact: it needs versions, approval, safe deployment, rollback and eventually retirement.

Candidate FMscapability shortlistevaluateChosen FMbest fitcustomizeFine tuned modelLoRA or fullregisterModel Registryapproved versiondeployEndpointcanary then full

How it works

Assessing and choosing an FM

Compare candidates on the dimensions the scenario cares about.

DimensionWhat to check
CapabilityText, image, embeddings, tool use, supported languages, reasoning strength
Context windowWhether the longest prompt plus retrieved context fits
PerformanceLatency, throughput, quality on your own test set
CostPrice per input and output token, or hosting cost for self hosted models
LimitationsRegional availability, modalities, maximum output length, customization support
GovernanceLicense terms, data handling, responsible AI behavior

Performance benchmarks such as public leaderboards give a first shortlist, but they measure generic tasks. Capability analysis checks that the model supports the features you need, such as tool use or image input. Limitation evaluation rules out models that are unavailable in your Region, cannot be customized or cannot handle your context length.

Final selection should come from Amazon Bedrock model evaluation on your own prompts. It supports automatic metrics, LLM as a judge scoring and human review, so you can compare candidates side by side.

Exam tip

The smallest model that meets the quality bar is usually the right answer. It is cheaper and faster, and the exam often rewards that trade.

Customization options

OptionWhat changesUse when
Prompt engineeringNothing in the modelAlways try first
RAGContext at query timeModel needs current or private facts
Fine tuningModel weights, with labeled examplesModel must follow a domain style, format or task
Continued pre-trainingModel weights, with unlabeled domain textModel must learn a domain's language broadly
DistillationA smaller student learns from a larger teacherNeed near teacher quality at lower cost and latency

Parameter efficient fine tuning (PEFT) trains only a small number of extra parameters and leaves the base weights frozen. Low-rank adaptation (LoRA) is the common form: it adds small low rank matrices, called adapters, to selected layers.

Adapters are small and quick to swap, so one base model can serve several domain adapters instead of one full model copy per domain.

Deploying customized models

On Amazon SageMaker AI, you deploy a fine tuned model to a real time endpoint. With inference components, several models or LoRA adapters can share the same endpoint and its accelerators, which improves utilization.

On Amazon Bedrock, fine tuned models are served with Provisioned Throughput, or with on demand inference where the model supports it. Bedrock Custom Model Import brings weights you customized elsewhere, such as on SageMaker AI, into Bedrock for supported model architectures, so the app uses the same Bedrock API.

Versioning, pipelines and rollback

SageMaker Model Registry stores models as versions inside a model package group. Each version carries metrics and an approval status: PendingManualApproval, Approved or Rejected.

Training jobnew datacreatesModel versionPendingreviewApprovedregistry statustriggersPipelinedeploy to stagingpromotesProductioncanary rollout

An automated pipeline, built with SageMaker Pipelines or AWS CodePipeline, trains, evaluates and registers each new model. An approval status change can trigger deployment through an Amazon EventBridge rule.

For rollout, SageMaker AI deployment guardrails use blue/green deployments with all at once, canary or linear traffic shifting. You attach Amazon CloudWatch alarms, and if an alarm fires during the baking period, the endpoint rolls back automatically to the previous fleet.

StrategyBehaviorChoose for
CanarySmall share of traffic first, then the restCatch problems with minimal user impact
LinearTraffic shifts in equal stepsGradual confidence building
All at onceFull switch after the new fleet is readyFastest, still with alarm based rollback
Shadow testNew model receives a copy of traffic, users see the old oneCompare a model on live traffic with zero user risk

Retiring and replacing models

Models age out. Amazon Bedrock marks model versions as Active, Legacy or End-of-Life (EOL), and calls to a model fail after its EOL date. Track the dates, evaluate the replacement on your test set and migrate before the deadline.

For your own models, retire old registry versions, delete unused endpoints to stop cost, and keep the approved previous version available as a rollback target.

How the exam asks about this

  • "Five business units need domain behavior from one base model at low hosting cost" -> LoRA adapters served from one SageMaker AI endpoint with inference components.
  • "A new model version must roll back automatically if latency degrades" -> SageMaker AI deployment guardrails with a CloudWatch alarm.

Common mistakes

Choosing by leaderboard rank only. Public benchmarks do not measure your data, cost or latency. Validate on your own dataset.

Fine tuning when RAG is needed. Weights do not stay current with changing facts. Fine tune for behavior, retrieve for knowledge.

Deploying without an approval gate. Pushing every trained model straight to production skips evaluation. Use Model Registry approval status as the gate.

Watch out

A model past its EOL date stops answering. Ignoring the Legacy status notice is an outage waiting to happen.

What to remember for the exam

  • Select FMs on capability, context window, latency, cost, Regional availability and results on your own data.
  • Bedrock model evaluation compares models with automatic, LLM as a judge or human evaluation.
  • LoRA trains small adapters on a frozen base model; adapters are cheap to train, store and swap.
  • SageMaker inference components let several models or adapters share one endpoint.
  • Custom Model Import brings externally customized weights of supported architectures into Bedrock.
  • Model Registry: model package groups, versions, approval status as the deployment gate.
  • Deployment guardrails: canary or linear shifting with automatic rollback on CloudWatch alarms.
  • Bedrock model lifecycle: Active, Legacy, EOL.

Check yourself

  • Why is LoRA cheaper to deploy for many domains than full fine tuning?

    Each domain needs only a small adapter on a shared frozen base model, so you do not host a full model copy per domain.

  • What controls which model version a pipeline may deploy?

    The approval status in SageMaker Model Registry, where only Approved versions should trigger deployment.

  • How do you make a new endpoint version revert on high error rates?

    Use deployment guardrails with canary or linear traffic shifting and a CloudWatch alarm that triggers automatic rollback.

Topic 2 of 9 in Foundation Model Integration, Data Management, and Compliance

Test yourself on Foundation Model Integration, Data Management, and Compliance

The full AWS AIP-C01 question bank has 827 questions, with timed practice exams and an explanation for every answer.

This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.