The Real Cost of Running AI Yourself vs. APIs
By Jonathan Kvarfordt · Published · Updated
A $5,099 box on your desk (retailer price) looks cheap next to a stack of AI bills. Then you add power, the person who keeps it running, and the quality gap. Here is the math, with every number sourced and dated in the data file.
We compared four options for revenue teams in insurance, finance, manufacturing, healthcare and beyond:
- Own hardware (Mac Studio, DGX Spark, RTX PRO 6000 workstations, servers)
- Rented GPUs (H100 and B200 cloud instances, managed open-model hosting)
- Your own model on Nvidia's stack (NeMo, Nemotron, NIM, DGX Cloud)
- APIs from the big model labs (OpenAI, Anthropic, Google, xAI, Mistral, plus Azure OpenAI, AWS Bedrock and Google Vertex)
The baseline is a $200 per month top-tier plan per heavy user (Anthropic lists Claude Max 20x at $200 per month on its help center).
Five findings
- Desk hardware pays back in about 2.5 years against one $200 plan. A $5,099 Mac Studio with 128GB (retailer price), minus about $27/mo of power, breaks even in 29 months. A DGX Spark at Nvidia's $4,699 takes 27 months; on Amazon, where third-party prices have ranged from $3,949.99 to $9,499.99, it takes 23 to 55 months.
- Hardware prices are moving fast. DGX Spark launched at $3,999 (Nvidia, Oct 2025). Nvidia's store now lists $4,699. Amazon third-party prices have swung between $3,949.99 and $9,499.99 (camelcamelcamel tracker). RTX PRO 6000 cards list at $16,000 at Nvidia and $19,999.99 on one Amazon listing.
- Open models closed much of the quality gap on coding tasks. Qwen3.6-27B reports 59.3% on Terminal-Bench 2.0, the same score as Claude Opus 4.5 (both vendor-reported). The top leaderboard entry is 79.8%, so the frontier keeps a lead.
- API cost scales with volume and model choice, from $35 to $360 a month at small-team volume. At 50M tokens a month, Mistral Large costs about $35 and Claude Opus 5.5 about $360. At 20B tokens a month (enterprise), the same spread is $14,000 to $144,000.
- A rented GPU beats APIs only at high, steady volume. One H100 running 24/7 on Lambda costs $2,913/mo. That matches a Sonnet-class API bill at about 809M tokens a month, which means that GPU has to serve about 308 tokens per second, every second, all month.
Option 1: Own hardware
Up-front cost (checked Oct 12, 2026)
| Machine | Memory | Price | Label |
|---|---|---|---|
| Mac Studio M5 Max | 128GB | $5,099 | retailer price (Apple page not confirmed) |
| Mac Studio M5 Ultra | 256GB | £10,799 UK; US price not confirmed | Apple UK store |
| Mac Studio M5 Ultra | 512GB | not orderable; "late October" | Apple |
| DGX Spark | 128GB | $4,699 Nvidia (anchor); Amazon third-party $3,949.99 to $9,499.99, volatile | vendor; price tracker |
| RTX PRO 6000 Blackwell (card only) | 96GB | $14,900 to $19,999.99 | retail listings |
The 256GB M5 Ultra ships in 16 to 18 weeks in the UK, Canada and Singapore Apple stores.
Monthly run cost. Power at the June 2026 US residential average of 18.34 cents per kWh (EIA):
- 200W average, 24/7: 0.2 kW x 730 h = 146 kWh x $0.1834 = $26.78/mo (estimate)
- 385W (Apple's max for the 512GB Mac Studio), 24/7: 281 kWh x $0.1834 = $51.54/mo (estimate)
Break-even vs one $200/mo plan (hardware only, no staff time, no resale):
- Mac Studio 128GB: $5,099 / ($200 - $26.78) = 29.4 months
- DGX Spark at Nvidia price: $4,699 / $173.22 = 27.1 months
- DGX Spark on Amazon, tracker range: $3,949.99 / $173.22 = 22.8 months to $9,499.99 / $173.22 = 54.8 months
One box can serve a few people, so if it replaces three $200 plans the payback drops to about a third. If it only replaces part of one plan, it never pays back.
Speed. MacStories ran Qwen3-235B (4-bit) on an M3 Ultra Mac Studio with 512GB at 24 tokens per second with MLX and 16 with GGUF, using about 124GB of memory. That is one user at a time, good for chat, slow for batch jobs.
People. One person who owns updates, model swaps and security. For a small team that is part of someone's week. Nothing here needs an ML engineer.
Quality. In the same MacStories test, the smaller Qwen2.5-VL-32B called a photo of computer mice "a collection of seashells." The 72B model got it right. Smaller local models fail on simple things.
Option 2: Rented GPUs
Up-front cost. None. Pay by the minute.
Monthly run cost (on-demand list prices)
| Provider | H100 | B200 |
|---|---|---|
| Lambda | $3.99 per GPU-hour | $6.69 per GPU-hour |
| CoreWeave | $49.24/hr per 8× H100 node (about $6.16 per GPU-hour) | $68.80/hr per 8× B200 node (about $8.60 per GPU-hour) |
| AWS (p5.48xlarge) | $6.88 per H100-hour, on-demand, us-east-1 | not priced here |
Lambda and CoreWeave are vendor-stated. CoreWeave's per-GPU prices apply only to its inference platform. The AWS rate is from price trackers ($55.04/hr per 8-GPU instance); confirm on aws.amazon.com/ec2/pricing/on-demand before you budget.
- One H100, 24/7, Lambda: $3.99 x 730 = $2,912.70/mo
- One 8x B200 node, 24/7, Lambda: $6.69 x 8 x 730 = $39,069.60/mo
Break-even vs APIs. We do not publish a tokens-per-second figure for an H100 because it depends on the model, batch size and software. So we flip the math. One 24/7 H100 costs the same as:
- a Sonnet-class API ($3.60 per 1M blended tokens) at 809M tokens/mo, which needs about 308 tokens/sec sustained
- Mistral Large ($0.70 blended) at 4.16B tokens/mo, about 1,583 tokens/sec sustained
If your traffic is spiky, the GPU sits idle and you still pay.
People. At least one engineer who can deploy and monitor an inference server, handle scaling and patch it. This is a real ML-ops job.
Privacy. Data stays in your cloud account. Your cloud contract governs it.
Option 3: Your own model on Nvidia's stack
What it is. NeMo is Nvidia's toolkit to train and fine-tune. Nemotron is Nvidia's family of open models. NIM packages models as ready-to-run services. DGX Cloud is Nvidia-managed GPU capacity.
Cost.
- NVIDIA AI Enterprise (includes NIM): $4,500 per GPU per year list, or $1 per GPU-hour through cloud marketplaces (vendor-stated).
- Compute: rented GPUs as in Option 2, or owned servers.
- DGX Cloud pricing is not public. We did not find a list price.
Example: 8 GPUs licensed for a year = 8 x $4,500 = $36,000 in software before compute or staff.
People. Fine-tuning needs ML engineers, clean labeled data and an evaluation set. Plan on a small team.
Quality. Fine-tuning makes a model better at your narrow task. It does not make a mid-size open model better at everything. Nvidia's Nemotron 3 Super reports about 31% on Terminal-Bench 2.0 (vendor-reported, via an aggregator).
Option 4: APIs from the big labs
Up-front cost. None.
List prices per 1M tokens (input / output), checked Oct 12, 2026
| Provider | Model | Input | Output |
|---|---|---|---|
| Anthropic | Claude Opus 5.5 | $4 | $20 |
| Anthropic | Claude Sonnet 5.5 | $2 | $10 |
| OpenAI | GPT-6.1 Sol | $2 | $10 |
| OpenAI | GPT-6 Astra | $10 | $50 |
| Gemini 3.1 Pro Preview | $2 | $12 | |
| Gemini 3.8 Flash | $0.75 (to Dec 31) / $1.50 (2027) | $3.75 / $7.50 | |
| xAI | Grok 4.7 | $2 | $6 |
| Mistral | Mistral Large | $0.50 | $1.50 |
OpenAI also lists GPT-6 Astra, its top model, at $10 / $50; we priced GPT-6.1 Sol as the mid-tier comparison. Anthropic and xAI charge 1.1x for US-only processing; OpenAI adds 10% for regional processing.
Monthly cost by size (estimate). Assumed volumes, 80% input and 20% output:
- Small team (5 to 20 people): 50M tokens/mo
- Mid-market: 1B tokens/mo
- Enterprise: 20B tokens/mo
Formula: (input M tokens x input price) + (output M tokens x output price). Small team on Sonnet 5.5: 40 x $2 + 10 x $10 = $180.
| Model | Small | Mid-market | Enterprise |
|---|---|---|---|
| Claude Opus 5.5 | $360 | $7,200 | $144,000 |
| Claude Sonnet 5.5 / GPT-6.1 Sol | $180 | $3,600 | $72,000 |
| Gemini 3.1 Pro | $200 | $4,000 | $80,000 |
| Grok 4.7 | $140 | $2,800 | $56,000 |
| Gemini 3.8 Flash (2027 price) | $135 | $2,700 | $54,000 |
| Mistral Large | $35 | $700 | $14,000 |
Caching and batch modes cut these further; we left them out to stay conservative.
Rate limits. xAI lists 50M tokens per minute and 150 requests per second for Grok 4.7. OpenAI lists 1M, 4M and 40M tokens per minute for GPT-6.1 Sol across its Build, Launch and Grow tiers. Small teams will not hit these. Enterprise batch jobs can.
Data terms (vendor-stated)
| Route | Trains on your data? | Default retention | Zero retention | Private network |
|---|---|---|---|---|
| OpenAI API | No, since Mar 2023 | Abuse logs up to 30 days | By approval | Azure Private Link |
| Anthropic API | No, without permission | Not retained by default; 30 days for some models | On request, per org | Not checked |
| Azure OpenAI | No | Only flagged prompts are stored, for human review (no period published) | Modified abuse monitoring, Limited Access approval | Azure Private Link |
| AWS Bedrock | No; providers do not see prompts | Setting-based | `data_retention_mode=none` | AWS PrivateLink |
| Google Vertex | No, without permission | Abuse-monitoring prompt logging; 24-hour in-memory cache (can be disabled) | By exception request; Search grounding logs are kept up to 3 days and Maps grounding logs 30 days, and neither can be turned off | VPC Service Controls and Private Service Connect |
Bedrock is HIPAA eligible and FedRAMP High in GovCloud (US-West), per AWS.
People. One builder who can wire an API into your systems. No hardware staff.
Quality and speed. Frontier models top the leaderboards. Speed depends on the vendor's load, not your hardware.
Decision table by company size
| Small team (5 to 20) | Mid-market | Enterprise | |
|---|---|---|---|
| Default | APIs or $200 plans | APIs through Bedrock, Azure or Vertex | APIs with zero retention, plus a self-hosted lane |
| Own hardware | One box if data cannot leave the building or 3+ heavy users share it | Pilot for sensitive docs | Server fleet for steady, regulated loads |
| Rented GPUs | Skip | Only if steady volume tops ~800M tokens/mo | Yes for high, steady volume |
| Own model (Nvidia stack) | Skip | Skip unless one narrow, high-value task | Yes with an ML team and labeled data |
| Up-front | $0 to ~$9,000 | $0 to ~$40,000 | Six figures and up |
| Monthly (API math above) | $35 to $360 | $700 to $7,200 | $14,000 to $144,000 |
| Staff | Part of one person | 1 to 2 engineers | ML and platform team |
When local or on-prem wins
The case for running it yourself is real.
- Regulated data. If contracts or regulators say data cannot leave your network, local is the clean answer. API zero-retention options exist, but most need vendor approval (OpenAI, Azure) or a sales request (Anthropic).
- High steady volume. Our math shows a 24/7 GPU matches a Sonnet-class API bill at about 809M tokens a month. Above that, with steady traffic, owning or renting wins.
- Latency. A model on your network has no internet hop and no shared queue at peak hours.
- Sovereignty. Some buyers need processing in one country. Anthropic and xAI charge 1.1x for US-only processing; your own hardware has no surcharge.
- The broader trend. In Barclays' 1H24 CIO survey, 83% of CIOs planned to move at least some workloads from public cloud back to private cloud or on-prem over the next 12 months. IDC's Server and Storage Workloads Survey found only 8 to 9% of companies plan full workload repatriation. Most companies move some workloads and keep the rest in the cloud.
- The quality gap shrank. A 27B open model reports the same Terminal-Bench 2.0 score as Claude Opus 4.5. That model fits on a 128GB desk machine.
Want the next one in your inbox? Subscribe.
How to cite this research
Written by Jonathan Kvarfordt, Founder and Principal Analyst, The Revenue AI Report. Published under CC BY 4.0.
APA
Kvarfordt, J. (2026). The Real Cost of Running AI Yourself vs. APIs. The Revenue AI Report. Retrieved from https://www.therevenueaireport.com/research/local-vs-cloud-ai-cost
MLA
Kvarfordt, Jonathan. "The Real Cost of Running AI Yourself vs. APIs." The Revenue AI Report, 12 Oct. 2026, www.therevenueaireport.com/research/local-vs-cloud-ai-cost.
BibTeX
@misc{kvarfordt2026localvscloudaicost,
author = {Kvarfordt, Jonathan},
title = {The Real Cost of Running AI Yourself vs. APIs},
year = {2026},
publisher = {The Revenue AI Report},
url = {https://www.therevenueaireport.com/research/local-vs-cloud-ai-cost}
}