Architecture Design and Implementation Built for the Cloud · Topic 11 of 11
Generative AI, Agents, RAG and ML Inference
Integrate Amazon Bedrock across accounts, build RAG with knowledge bases and vector stores, run agents on AgentCore and pick SageMaker AI endpoints.
- 5 min
- Use PrivateLink, guardrails and invocation logging to make Bedrock enterprise-ready.
- Cross-Region inference raises throughput; Provisioned Throughput reserves capacity for steady load.
- Knowledge bases handle parsing, chunking, embedding and storage; tune chunking and metadata to improve retrieval.
What it is
Generative AI features on AWS usually call foundation models through Amazon Bedrock. Retrieval augmented generation (RAG) grounds the answers in company data, and agents let a model take actions by calling tools.
Models you train or host yourself run on Amazon SageMaker AI endpoints. The exam asks you to place these pieces in a multi-account estate with the right controls, cost model and latency.
How it works
Bedrock integration across accounts
- Private access: call Bedrock through VPC interface endpoints (AWS PrivateLink) so traffic stays off the internet.
- Guardrails: Amazon Bedrock Guardrails filter harmful content, block denied topics, redact sensitive information and check grounding. IAM conditions can require a specific guardrail on model calls.
- Throughput and Regions: cross-Region inference profiles spread requests across Regions for higher throughput. Provisioned Throughput reserves capacity for steady, high-volume use.
- Cost tracking: application inference profiles carry tags, so model spend can be allocated per team or application.
- Governance: SCPs can limit which models or Regions accounts may use, and model invocation logging records prompts and responses to CloudWatch Logs or S3.
| Multi-account pattern | How it works | Trade-off |
|---|---|---|
| Central AI platform account | A shared API (for example API Gateway with Lambda) fronts Bedrock for all teams | One place for guardrails, logging, quotas and chargeback; the platform becomes a dependency |
| Bedrock in each workload account | Teams call Bedrock directly, governed by SCPs and IAM | More autonomy; governance relies on organization policies and per-account logging |
Amazon Bedrock Evaluations compare models and RAG setups against datasets with automatic metrics or a model acting as judge. Use it to choose a model or tune retrieval before production.
RAG: ingestion and retrieval
A Bedrock knowledge base connects a data source, such as S3, to an embeddings model and a vector store. A sync job parses documents, splits them into chunks, embeds each chunk and writes the vectors.
| Ingestion choice | Options |
|---|---|
| Chunking | Default, fixed size, hierarchical, semantic, or no chunking |
| Vector store | Amazon OpenSearch Serverless, OpenSearch Service managed clusters, Amazon S3 Vectors, Aurora PostgreSQL (pgvector), Neptune Analytics (GraphRAG), Pinecone, Redis Enterprise Cloud, MongoDB Atlas |
| Metadata | Attributes per document for filtering at query time |
Hierarchical chunking retrieves small precise chunks but returns their larger parent for context. Semantic chunking splits on meaning, which suits long documents with varied topics.
At query time, Retrieve returns the relevant chunks for your own prompt logic, while RetrieveAndGenerate retrieves and calls the model in one step. Metadata filters, reranking and hybrid search on supported stores improve relevance.
Choose the vector store by need: S3 Vectors for low-cost storage of large, infrequently queried vector sets, OpenSearch for low-latency search with hybrid queries, and Aurora PostgreSQL when the team already runs it.
Agents and tools with Bedrock AgentCore
Amazon Bedrock AgentCore is a set of services to run and operate agents built with any framework and model.
| Service | Role |
|---|---|
| Runtime | Serverless hosting for agents with session isolation |
| Gateway | Turns APIs, Lambda functions and services into MCP-compatible tools |
| Memory | Short-term and long-term memory across sessions |
| Identity | Agent identity and access to third-party credentials through existing identity providers |
| Policy | Deterministic rules on which tools an agent may call, enforced at the Gateway |
| Observability and Evaluations | Traces of each agent step and automated quality scoring |
Combine an agent with a knowledge base when it must both look up facts and take actions. Keep tool permissions narrow, because the model decides when to call them.
ML inference on SageMaker AI
| Endpoint type | Fits |
|---|---|
| Real-time | Interactive, low-latency, steady traffic |
| Serverless | Intermittent traffic that tolerates cold starts, no instances to manage |
| Asynchronous | Large payloads (up to 1 GB) and long processing (up to one hour), queued requests |
| Batch transform | Offline predictions over a whole dataset, no endpoint kept running |
Use endpoint auto scaling for variable load. Multi-model endpoints and inference components host several models on shared instances to raise utilization.
How the exam asks about this
- "Forty teams use generative AI and the company needs one place to enforce guardrails, log prompts and charge back cost" -> a central AI platform account fronting Bedrock, with guardrails, invocation logging and application inference profiles.
- "A chatbot must answer from internal PDFs with citations and minimal custom code" -> Bedrock knowledge base over S3 with RetrieveAndGenerate.
- "Image inferences take twenty minutes and traffic arrives in bursts" -> SageMaker AI asynchronous inference.
What to remember for the exam
- Use PrivateLink, guardrails and invocation logging to make Bedrock enterprise-ready.
- Cross-Region inference raises throughput; Provisioned Throughput reserves capacity for steady load.
- Knowledge bases handle parsing, chunking, embedding and storage; tune chunking and metadata to improve retrieval.
- AgentCore Gateway exposes tools over MCP, and AgentCore Policy restricts which tools an agent may use.
- Real-time for steady low latency, serverless for intermittent, asynchronous for large or long jobs, batch transform for offline sets.
- Bedrock Evaluations helps pick models and tune RAG with measured results.
Fine-tuning a model is not the answer when the requirement is answers from frequently changing company documents. RAG with a knowledge base keeps answers current by syncing the source, without retraining.
Common mistakes
Calling models over the public endpoint from private subnets. Without a VPC interface endpoint the traffic needs a NAT path to the internet. PrivateLink keeps it private and simplifies network controls.
Using a real-time endpoint for overnight scoring. A running endpoint costs money all day. Batch transform processes the dataset and stops.
Giving an agent broad IAM permissions. The model chooses which tool to call, so a broad role widens the impact of a bad decision. Scope each tool and enforce rules with AgentCore Policy or narrow roles.
Check yourself
Which RAG chunking strategy searches small chunks but returns larger context?
Hierarchical chunking, which matches child chunks and returns their parent chunk to the model.
Which SageMaker AI option suits large payloads that take many minutes to process?
Asynchronous inference, which queues requests and supports large payloads and long processing times.
What does AgentCore Gateway do?
It converts APIs, Lambda functions and existing services into MCP-compatible tools that agents can call through a gateway endpoint.
Topic 11 of 11 in Architecture Design and Implementation Built for the Cloud
Test yourself on Architecture Design and Implementation Built for the Cloud
The full AWS SAP-C03 question bank has 850 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.