Four Amazon Bedrock Request Settings That Drive Inference Costs
Summary:AWS breaks down four request-level controls that materially change Amazon Bedrock inference bills: cache TTL, service tier, reasoning effort, and maxTokens. Longer cache lifetimes can reduce repeated-context costs, although their write premium makes them uneconomical for low-reuse workloads. Flex tiers can lower prices for latency-tolerant jobs, while lower reasoning effort can substantially reduce output tokens without necessarily hurting every task. Output ceilings constrain runaway generation
A Longer Cache TTL Is Not Automatically Cheaper
Amazon Bedrock bills input tokens, output tokens, cache writes, and cache reads separately, while reasoning tokens count as output tokens. The goal of caching is to reuse stable prefixes containing system instructions, tool definitions, and long reference context. Any change to the prefix causes a cache miss and a new write, so dynamic content should follow stable content and frequently changing prompts should be versioned.
For the published Claude Sonnet 5 and Claude Opus 5 rates cited by AWS, a five-minute cache write costs 1.25 times the normal input rate, a one-hour write costs twice that rate, and a cache read costs 0.1 times the input rate. If the normal input cost is (C), the cache-write cost is (W), and the read cost is (R), the cache cost for (n) requests is:
[ C_{cache}(n) = W + (n-1)R ]
Assuming every request reuses the exact prefix and each read refreshes the TTL, a five-minute cache pays back on the first reuse within five minutes. If requests arrive more than five minutes apart, a one-hour cache becomes cheaper from the second request within that hour. Conversely, beating no caching generally requires at least three one-hour-cache requests per hour. Actual break-even points depend on the model, Region, minimum cacheable prefix, and current rates.
Multi-tenant applications also need an explicit data-isolation strategy. Cache entries are scoped to an AWS account and Region, not to individual tenants. A tenant-specific SHA-256 hash in the cacheable prefix can partition entries, but it does not replace authorization checks. Regulated workloads that require hard boundaries should use separate AWS accounts. Prompt caching also does not work with batch inference.
Flex Service Tiers Suit Background Workloads
serviceTier: {"type": "flex"} requests a tier intended for latency-tolerant workloads. default requests Standard service. Sending Flex to an unsupported model produces a ValidationException rather than silently falling back to Standard.
AWS tested Nova 2 Lite with 200 Standard and 200 Flex requests, interleaving the two groups. Both processed the same input and output token counts, all 400 requests resolved to the requested tier, and the latency difference was too small to distinguish from noise. At the time of the test, the published Flex rate in us-west-2 was half the Standard rate, and the bill matched that rate.
These results should not be generalized across models, Regions, or time windows. Production applications need their own deadlines, should record the resolved tier in the response and CloudWatch ResolvedServiceTier, and may retry once with default when policy permits. Retrying every ValidationException is unsafe because malformed requests can raise the same error.
Reasoning Effort Is a Quality-and-Budget Dial
Claude models with adaptive thinking emit separate reasoning tokens, which are billed as output tokens. Claude Sonnet 5 and Opus 5 reportedly default to high effort, so migrating from an earlier model can increase bills even when the request omits the thinking field. Their maxTokens limit covers both reasoning and visible response tokens.
In a Claude Opus 5 tool-calling agent test, all 24 runs at every effort level produced the correct result, but median output-token consumption varied sharply:
| Effort | Correct runs | Median output tokens | Change from low |
|---|---:|---:|---:|
| low | 24/24 | 936 | Baseline |
| medium | 24/24 | 1,484 | +59% |
| high | 24/24 | 2,521 | +169% |
| xhigh | 24/24 | 2,964 | +217% |
| max | 24/24 | 3,762 | +302% |
The absence of observed failures in 24 runs does not establish that all effort levels are equivalent. In a separate code-generation workload, median output was only about 10% higher at high than at low, and the generated programs passed at every level. The appropriate setting is therefore task-dependent.
Teams should jointly track OutputTokenCount, invocation logs, task pass rate, latency, and actual cost. Routine, well-bounded tasks can start below high. Reducing tokens is not a real saving if quality declines near a task's capability boundary and causes retries or additional inference calls.
maxTokens Is Both a Spend Limit and a Concurrency Constraint
maxTokens caps the number of output tokens a request may generate. Amazon Bedrock deducts the input tokens plus the full maxTokens allowance when a request starts, so an oversized ceiling reduces request concurrency under a shared quota and can conceal prompt regressions or runaway loops.
In AWS's illustrative test, a 2,048-token ceiling was reduced to 155. The same verbose prompt produced 2,048 tokens with the former and 155 with the latter, saving 1,893 tokens, but both responses stopped at max_tokens and were truncated. An output ceiling therefore cannot replace Guardrails as a responsible-AI control. In a multi-turn or agentic chain, one truncated turn can cause downstream steps to fail.
Production calibration should derive P95 and P99 output-token distributions for each task class. P95 limits cost exposure more aggressively but accepts more truncation risk; P99 preserves more valid long-tail responses at the cost of a higher ceiling. With adaptive thinking enabled, the distribution must be measured with thinking active because reasoning and visible text share the limit. Monitor stopReason == max_tokens before increasing a task's limit.
A Practical Deployment Loop
Maintain a capability record for every model version, Region, and API, including supported cache TTLs, service tiers, reasoning levels, token-counting behavior, and output ceilings. Evaluate each control independently:
- Confirm model and parameter support.
- Measure tokens, latency, cost, cache hits, and quality on representative traffic.
- Cross-check response metadata, CloudWatch metrics, invocation logs, and billing data.
- Constrain cost reductions with task pass and completion rates rather than token counts alone.
- Monitor cache misses,
ResolvedServiceTier, abnormal token growth, andmax_tokensstop reasons after rollout.
Caching fits high-reuse stable prefixes, Flex fits latency-tolerant workloads on supported models, lower reasoning effort fits tasks that pass quality regression, and maxTokens should reflect task-level output distributions. All four controls affect unit cost, latency, and concurrency, so each optimization must be evaluated against the complete service-level objective.
— END —Source: AWS Machine Learning Blog (2026-11-12)
Tags: #Amazon Bedrock #Inference Cost Optimization #Prompt Caching #Adaptive Thinking #LLMOps
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments