Testing, Validation, and Troubleshooting · Topic 5 of 6

Content Handling and FM API Troubleshooting

Fix context window overflow and truncated answers, then diagnose Amazon Bedrock API errors with invocation logs, request validation and stop reason checks.

  • 5 min
In a nutshell
  • Measure tokens with usage, CountTokens and the InputTokenCount metric.
  • Fix overflow with history summarization, fewer or smaller chunks, reranking and dynamic chunking.
  • stopReason max_tokens means truncated output; model_context_window_exceeded means the context window was exceeded.

What it is

Many generative AI bugs are not quality problems at all. The model never saw part of the input, the answer was cut off, or the API call failed and the application hid the error.

Content handling troubleshooting makes sure all the needed information fits into the request and the full answer comes back. Integration troubleshooting finds and fixes failed or malformed calls to Amazon Bedrock and related services.

ApprequestvalidateBedrock RuntimeConverseresponseAppcheck stopReasonlogCloudWatch Logsinvocation logs

How it works

Context window overflow

Every model has a maximum context window: system prompt, conversation history, retrieved chunks, tool definitions and the reserved output all share it. When the input grows past the limit, the request fails or the model cannot finish.

Diagnose it with numbers, not guesses:

  • The usage field in the Converse response returns inputTokens and outputTokens.
  • The CountTokens API returns the input token count for a request before you send it, using the model's own tokenizer.
  • The InputTokenCount CloudWatch metric shows the trend; a steady climb in long chats points at unbounded history.
CauseFix
Whole conversation history resent every turnKeep a sliding window of recent turns and summarize older ones
Too many or too large retrieved chunksFewer results, smaller chunks, reranking so only the best chunks go in
Huge documents pasted into one promptDynamic chunking: split, process each part, then combine (map and reduce)
Verbose system prompt and examplesTighten instructions, keep only the few-shot examples that change behavior

Dynamic chunking sizes chunks to the space left in the context window after the prompt and expected output, instead of a fixed size for every request.

Truncation-related errors

Truncated output is silent unless you check for it. Always read stopReason in the Converse response.

stopReasonMeaningAction
end_turnModel finished normallyNone
max_tokensOutput hit the maxTokens limitRaise maxTokens, ask for shorter output, or continue in a follow up call
stop_sequenceA stop sequence was generatedCheck the stop sequence is not too broad
tool_useModel wants a tool callRun the tool and return the result
guardrail_intervenedA guardrail blocked or masked contentReview the guardrail trace
model_context_window_exceededThe model context window was exceededReduce input size or history

A JSON answer that fails to parse is often a max_tokens stop, not a formatting problem. Check the stop reason before rewriting the prompt.

Watch out

Raising maxTokens hides the symptom if the real issue is a prompt that asks for too much. Prefer splitting the task when outputs regularly reach the limit.

API error diagnosis

Bedrock Runtime errors fall into a few groups. The fix depends on whether the client or the service is at fault.

ErrorHTTPTypical causeFix
ValidationException400Malformed body, wrong parameter for the model, input too longValidate the request against the API reference before sending
AccessDeniedException403Missing IAM permission, such as bedrock:InvokeModel, or model access not set upFix the role policy and model access
ResourceNotFoundException404Wrong model ID, Region or inference profileCheck the identifier with ListFoundationModels
ModelTimeoutException408Processing took too longShorter requests, streaming, retry
ThrottlingException429Account quota exceededRetry with exponential backoff and jitter, request a quota increase, cross-Region inference, Provisioned Throughput
ModelErrorException424The model failed while processingInspect input, retry
ServiceUnavailableException503Temporary capacity issueBackoff with jitter, cross-Region inference

Client errors such as ValidationException, AccessDeniedException and ResourceNotFoundException will not succeed on retry; throttling, timeouts and model errors can. Retrying a ValidationException only adds cost and noise.

Error logging and response analysis

AppSDK callerror or successStructured logrequest ID, model ID, errormetricsCloudWatchalarms on error rateBedrockinvocation loggingfull request and responseCloudWatch Logs or S3analysis
  • Model invocation logging is off by default. When enabled it records the full request, response and metadata to CloudWatch Logs, Amazon S3 or both. Bodies over 100 KB go to S3.
  • CloudWatch metrics InvocationClientErrors, InvocationServerErrors and InvocationThrottles separate client mistakes from service issues and throttling.
  • AWS CloudTrail records who called which Bedrock API, which helps with permission problems.
  • requestMetadata on Converse adds key value tags that appear in invocation logs, so you can filter logs by tenant or feature.

Response analysis means checking every successful response too: stop reason, token counts, schema validity of structured output, and guardrail actions.

Connection problems

Long running or streaming calls that pass through a NAT gateway, interface VPC endpoint or Network Load Balancer can be dropped after 350 seconds idle. AWS recommends enabling TCP keep-alive in the SDK and setting the OS keep-alive interval below that idle timeout.

Common mistakes

Retrying every error. Retry only throttling, timeouts and server errors, with exponential backoff and jitter.

Ignoring stopReason. Treating a max_tokens stop as a finished answer ships truncated JSON and half sentences.

Resending the whole chat history. Token use and latency grow with every turn until the context window overflows.

Expecting invocation logs to exist already. Model invocation logging must be enabled before the incident you want to investigate.

What to remember for the exam

  • Measure tokens with usage, CountTokens and the InputTokenCount metric.
  • Fix overflow with history summarization, fewer or smaller chunks, reranking and dynamic chunking.
  • stopReason max_tokens means truncated output; model_context_window_exceeded means the context window was exceeded.
  • ThrottlingException is 429: backoff with jitter, quota increase, cross-Region inference or Provisioned Throughput.
  • ValidationException is 400: fix the request, do not retry.
  • Model invocation logging is disabled by default and writes to CloudWatch Logs, S3 or both.

Check yourself

  • Structured JSON output from a model fails to parse about 10 percent of the time. What should you check first?

    The stopReason on the failing responses; if it is max_tokens, the output is being truncated.

  • Which Bedrock errors should the client retry with exponential backoff?

    Throttling, timeouts and server side errors such as ThrottlingException and ServiceUnavailableException, not validation or access errors.

  • Token usage climbs steadily in long chat sessions until requests fail. What is the fix?

    Stop resending the full history; keep recent turns and summarize older ones to stay inside the context window.

Topic 5 of 6 in Testing, Validation, and Troubleshooting

Test yourself on Testing, Validation, and Troubleshooting

The full AWS AIP-C01 question bank has 827 questions, with timed practice exams and an explanation for every answer.

This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.