Practical guide
LLM API Pricing Comparison by Cost per Successful Task
Compare GPT-5, Claude Sonnet 4, and Gemini 2.5 Pro using token cost, retry rate, output length, and task success—not list price alone.
Updated · Sources checked
Compare the same evaluated task set
An LLM API pricing comparison is useful only when each model performs the same job. Run a representative evaluation set through GPT-5, Claude Sonnet 4, and Gemini 2.5 Pro. Record uncached input, cached input, output, retries, and the number of acceptable results for each model.
Imagine 10,000 document extractions. Model A averages 2,200 input and 180 output tokens and succeeds 96% of the time. Model B averages 1,900 input and 350 output tokens and succeeds 88%, requiring 1,364 retries to obtain roughly 10,000 successful results. Use the LLM API Cost Calculator for each model’s total tokens after retries, then divide spend by accepted outputs. A lower first-pass token price can lose once longer answers and rework are included.
Normalize the inputs before deciding
Keep system prompts, retrieval context, output limits, and quality thresholds consistent. Separate cached input because all three listed models expose a cached-input rate, but caching behavior and eligibility can differ. Compare latency and rate limits alongside cost if missed deadlines have business consequences.
The calculator’s dropdown contains a verified snapshot of three specific model tiers. It does not select a model, test quality, or include batch modes, long-context bands, cache writes, tools, regional processing, commitments, taxes, or negotiated discounts. Re-run the comparison when a model version, prompt, or rate changes.
FAQ
Is the cheapest token rate the cheapest model?
Not necessarily. Output mix, cache use, retries, and success rate determine cost per completed task.
Can I compare providers with different context limits?
Yes, if every candidate can complete the representative task. If a model requires truncation or a different retrieval strategy, document that architecture change rather than treating the token totals as equivalent.
Official sources
- https://developers.openai.com/api/docs/models/gpt-5
- https://platform.claude.com/docs/en/about-claude/pricing
- https://ai.google.dev/gemini-api/docs/pricing