Practical guide
GPU vs API Break-Even: Calculate the Required Throughput
Find the workloads per GPU-hour needed to beat API pricing using fully loaded GPU cost, API cost per job, and measured throughput.
Updated · Sources checked
Use one comparable workload
Define a workload before comparing prices: for example, one 1,000-token summary at a fixed quality and latency target. Benchmark how many of those jobs the proposed GPU can complete in an hour, then calculate the API price for the same job.
If the fully loaded GPU costs $2.40 per hour and the API costs $0.018 per summary, break-even is 2.40 ÷ 0.018, or 134 completed summaries per hour after rounding up. Enter the hourly GPU cost, API cost per workload, and measured GPU throughput in the GPU vs API Break-even Calculator. At 160 jobs per hour, the modeled API alternative costs $2.88 per hour, so the GPU/API cost ratio is about 0.83.
Build a fully loaded GPU rate
For rented hardware, include the instance or host required to use the accelerator, attached storage, and unavoidable networking. For owned hardware, convert purchase cost over its useful service hours and add power, hosting, and support. Use productive hours in the denominator; a GPU reserved but idle still costs money.
Benchmark end-to-end throughput with realistic batching, prompt lengths, output lengths, and concurrency. Peak tokens per second from a model card is not completed-workload throughput.
The calculator accepts user-supplied rates because GPU offers and API workloads are not interchangeable catalog items. It excludes engineering, idle capacity, redundancy, model quality, latency, networking, and contract terms unless you include them in the hourly input.
FAQ
What does a ratio above 1 mean?
At the entered throughput, GPU hourly cost is higher than API spend for the same number of workloads.
Why does the calculator also show break-even workloads per hour?
That threshold tells you the sustained completed-job rate the GPU must reach before its entered hourly cost equals the API alternative.
Official sources
- https://aws.amazon.com/ec2/instance-types/accelerated-computing/
- https://cloud.google.com/compute/gpus-pricing
- https://developers.openai.com/api/docs/models/gpt-5