Testing, Validation, and Troubleshooting · Topic 5 of 6
Content Handling and FM API Troubleshooting
Fix context window overflow and truncated answers, then diagnose Amazon Bedrock API errors with invocation logs, request validation and stop reason checks.
- 5 min
- Measure tokens with
usage,CountTokensand theInputTokenCountmetric. - Fix overflow with history summarization, fewer or smaller chunks, reranking and dynamic chunking.
stopReasonmax_tokensmeans truncated output;model_context_window_exceededmeans the context window was exceeded.
What it is
Many generative AI bugs are not quality problems at all. The model never saw part of the input, the answer was cut off, or the API call failed and the application hid the error.
Content handling troubleshooting makes sure all the needed information fits into the request and the full answer comes back. Integration troubleshooting finds and fixes failed or malformed calls to Amazon Bedrock and related services.
How it works
Context window overflow
Every model has a maximum context window: system prompt, conversation history, retrieved chunks, tool definitions and the reserved output all share it. When the input grows past the limit, the request fails or the model cannot finish.
Diagnose it with numbers, not guesses:
- The
usagefield in theConverseresponse returnsinputTokensandoutputTokens. - The
CountTokensAPI returns the input token count for a request before you send it, using the model's own tokenizer. - The
InputTokenCountCloudWatch metric shows the trend; a steady climb in long chats points at unbounded history.
| Cause | Fix |
|---|---|
| Whole conversation history resent every turn | Keep a sliding window of recent turns and summarize older ones |
| Too many or too large retrieved chunks | Fewer results, smaller chunks, reranking so only the best chunks go in |
| Huge documents pasted into one prompt | Dynamic chunking: split, process each part, then combine (map and reduce) |
| Verbose system prompt and examples | Tighten instructions, keep only the few-shot examples that change behavior |
Dynamic chunking sizes chunks to the space left in the context window after the prompt and expected output, instead of a fixed size for every request.
Truncation-related errors
Truncated output is silent unless you check for it. Always read stopReason in the Converse response.
| stopReason | Meaning | Action |
|---|---|---|
end_turn | Model finished normally | None |
max_tokens | Output hit the maxTokens limit | Raise maxTokens, ask for shorter output, or continue in a follow up call |
stop_sequence | A stop sequence was generated | Check the stop sequence is not too broad |
tool_use | Model wants a tool call | Run the tool and return the result |
guardrail_intervened | A guardrail blocked or masked content | Review the guardrail trace |
model_context_window_exceeded | The model context window was exceeded | Reduce input size or history |
A JSON answer that fails to parse is often a max_tokens stop, not a formatting problem. Check the stop reason before rewriting the prompt.
Raising maxTokens hides the symptom if the real issue is a prompt that asks for too much. Prefer splitting the task when outputs regularly reach the limit.
API error diagnosis
Bedrock Runtime errors fall into a few groups. The fix depends on whether the client or the service is at fault.
| Error | HTTP | Typical cause | Fix |
|---|---|---|---|
ValidationException | 400 | Malformed body, wrong parameter for the model, input too long | Validate the request against the API reference before sending |
AccessDeniedException | 403 | Missing IAM permission, such as bedrock:InvokeModel, or model access not set up | Fix the role policy and model access |
ResourceNotFoundException | 404 | Wrong model ID, Region or inference profile | Check the identifier with ListFoundationModels |
ModelTimeoutException | 408 | Processing took too long | Shorter requests, streaming, retry |
ThrottlingException | 429 | Account quota exceeded | Retry with exponential backoff and jitter, request a quota increase, cross-Region inference, Provisioned Throughput |
ModelErrorException | 424 | The model failed while processing | Inspect input, retry |
ServiceUnavailableException | 503 | Temporary capacity issue | Backoff with jitter, cross-Region inference |
Client errors such as ValidationException, AccessDeniedException and ResourceNotFoundException will not succeed on retry; throttling, timeouts and model errors can. Retrying a ValidationException only adds cost and noise.
Error logging and response analysis
- Model invocation logging is off by default. When enabled it records the full request, response and metadata to CloudWatch Logs, Amazon S3 or both. Bodies over 100 KB go to S3.
- CloudWatch metrics
InvocationClientErrors,InvocationServerErrorsandInvocationThrottlesseparate client mistakes from service issues and throttling. - AWS CloudTrail records who called which Bedrock API, which helps with permission problems.
requestMetadataonConverseadds key value tags that appear in invocation logs, so you can filter logs by tenant or feature.
Response analysis means checking every successful response too: stop reason, token counts, schema validity of structured output, and guardrail actions.
Connection problems
Long running or streaming calls that pass through a NAT gateway, interface VPC endpoint or Network Load Balancer can be dropped after 350 seconds idle. AWS recommends enabling TCP keep-alive in the SDK and setting the OS keep-alive interval below that idle timeout.
Common mistakes
Retrying every error. Retry only throttling, timeouts and server errors, with exponential backoff and jitter.
Ignoring stopReason. Treating a max_tokens stop as a finished answer ships truncated JSON and half sentences.
Resending the whole chat history. Token use and latency grow with every turn until the context window overflows.
Expecting invocation logs to exist already. Model invocation logging must be enabled before the incident you want to investigate.
What to remember for the exam
- Measure tokens with
usage,CountTokensand theInputTokenCountmetric. - Fix overflow with history summarization, fewer or smaller chunks, reranking and dynamic chunking.
stopReasonmax_tokensmeans truncated output;model_context_window_exceededmeans the context window was exceeded.ThrottlingExceptionis 429: backoff with jitter, quota increase, cross-Region inference or Provisioned Throughput.ValidationExceptionis 400: fix the request, do not retry.- Model invocation logging is disabled by default and writes to CloudWatch Logs, S3 or both.
Check yourself
Structured JSON output from a model fails to parse about 10 percent of the time. What should you check first?
The
stopReasonon the failing responses; if it ismax_tokens, the output is being truncated.Which Bedrock errors should the client retry with exponential backoff?
Throttling, timeouts and server side errors such as
ThrottlingExceptionandServiceUnavailableException, not validation or access errors.Token usage climbs steadily in long chat sessions until requests fail. What is the fix?
Stop resending the full history; keep recent turns and summarize older ones to stay inside the context window.
Topic 5 of 6 in Testing, Validation, and Troubleshooting
Test yourself on Testing, Validation, and Troubleshooting
The full AWS AIP-C01 question bank has 827 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.