LLM Cost Anomaly Explainer

Diagnose unexpected API cost spikes with a guided checklist. Find the root cause of sudden spikes, gradual increases, and bills higher than your estimate.

Cost Anomaly Explainer
Anomaly Type
Costs
Cost multiplier
4.0×
Extra cost
$300
⚠️ Significant anomaly — Check loop conditions and system prompt growth first.
Diagnostic checklist — Sudden cost spike
1.
New feature or code change?
Most spikes follow a deploy. A new feature that calls the API more often, or an agent loop with a missing exit condition, can multiply token usage instantly.
2.
Loop without termination?
Agent loops that fail to detect a "done" signal can run until they hit a rate limit, spending thousands of tokens in seconds. Add explicit loop counters and max-step guards.
3.
System prompt grew?
A system prompt that references a large document, or was accidentally duplicated, multiplies cost by every request. Check the token count in your prompt vs last week.
4.
Output tokens exploded?
Models can produce very long responses if the instruction changed (e.g., "write a detailed explanation" instead of "summarize"). Check average output token count in your logs.
5.
Model changed?
A code change from gpt-5-mini to gpt-5-4 increases cost 10–40×. Check what model your code is calling.
6.
Caching broke?
If you depend on prompt caching and the system prompt changed (even whitespace), the cache is invalidated and you pay full input prices again.

Diagnostic guidance based on common production incidents. For provider billing disputes, contact support with your usage logs. Dataset verified 2026-08-20.

How to Use

Describe the anomaly

Is it a sudden spike, gradual increase, or just higher than expected? Each has different likely causes.

Enter expected vs actual cost

This helps quantify the anomaly — a 4× spike narrows possible causes much more than 1.2×.

Work through the checklist

Each check has a specific thing to verify. Start from the top — most spikes are caused by the first 2 checks.

Fix and monitor

After fixing, set up cost alerts in your provider dashboard to catch future anomalies early.

Cost spike root cause hierarchy

80% of sudden cost spikes are caused by: (1) A deploy that changed the model or system prompt. (2) An agent loop lacking a termination condition. (3) A feature that added conversation history without windowing. Start with these before investigating more obscure causes. Log input_tokens, output_tokens, and model per request — three fields that make most anomalies self-diagnosing.

How to Cut This Cost

Root cause 1

Context bloat: check if your agent is accumulating conversation history without a sliding window or truncation strategy.

Root cause 2

Cache TTL expiry: prompt caching that costs more than it saves happens when calls arrive slower than the cache TTL.

Root cause 3

Retry storms: exponential backoff retry logic can 2–10× costs when a provider has degraded throughput. Add max retry limits.

Related Tools & Guides

FAQ

Have more questions? Contact us

Last verified: 2026-08-20 · methodology · data sources