Self-Hosting vs API — When Is the Break-Even Point?
Self-hosting open-weight models eliminates per-token costs but adds infrastructure complexity and fixed GPU costs. This guide calculates the break-even point and covers the hidden costs of self-hosting that calculators often miss.
The basic calculation
API cost = tokens × price/token. Self-host cost = GPU hours × GPU price/hour (24/7 for production). Break-even = GPU cost / (savings per token). For a RunPod A100 at $2.49/hr vs GPT-5 mini at $0.25/M: at 100M tokens/month, API = $25; A100 24/7 = $1,793. Break-even is at ~7.2B tokens/month — far above most developers.
Model quality: the hidden trade-off
Self-hosted open models (Llama 4, Qwen 2.5, DeepSeek) are competitive with GPT-5 mini on most tasks, but trail GPT-5.4 and Claude Opus 5 on complex reasoning. Before calculating break-even, verify the open model actually meets your quality bar. A 30% quality drop that increases human review time may cost more than the API savings.
Hidden self-hosting costs
1. Engineering time: initial setup, monitoring, updates, restarts — 4–8 hours/month minimum. 2. Ops risk: downtime for maintenance, GPU failures, cold starts. 3. No SLA: no contractual uptime guarantee. 4. Storage: model weights (70B at Q4 = 43GB), system image. 5. Scaling: serving spiky traffic requires over-provisioning or complex autoscaling.
The hybrid approach
Most cost-optimal: self-host a cheap small model (Llama 3.1 8B) for high-volume simple tasks; route complex tasks to API. This eliminates 70% of API cost for teams where 70% of requests are simple. The Model Routing Calculator models this precisely.
Related calculators
Related guides
Frequently Asked Questions
What is the minimum token volume where self-hosting makes sense?
Rule of thumb: 1B+ tokens/month for a medium model on A100. Below that, API is almost always cheaper when you factor in engineering time and infrastructure complexity. The Self-Host vs API calculator gives exact break-even for your specific workload.
Guide updated 2026-08-20. Pricing data verified weekly. See methodology · sources.