Here's a thought experiment. Tomorrow morning your AI provider (pick any of them) pulls the model your team depends on. Not hypothetical: it happened twice this year. What would it cost to have a fallback that nobody can switch off, running inside the Azure tenant you already govern?
Most people guess a number with a lot of zeros. The actual answer, priced from Azure spot list on 5 July 2026, starts at $47 a month.
The two levers that make it cheap
Everything below rests on two boring, compounding levers:
- Spot pricing. Spot VMs run ~80% below on-demand. Inference is stateless: if the box is evicted, you respin and carry on. That makes AI serving one of the few workloads where spot is a natural fit rather than a gamble.
- Scheduling. A business runs ~160 hours a month, not 730. Power the box off out of hours and you pay for a fifth of the month.
Stacked, that's roughly 95%+ below the on-demand-24×7 baseline most cost estimates silently assume. This is why the numbers below look wrong. They aren't.
The fit rule
One principle decides everything: a model only runs at full speed if it fits entirely in the box's fast memory. GPU VRAM or system RAM, whatever the box has: the model must live in it. A model that spills runs at a fraction of its potential. So you don't pick a VM by its spec sheet; you pick it by whether your chosen model fits. That single rule generates the ladder.
The ladder
Spot prices, 160 hours/month, cheapest region with 0–5% eviction rates, snapshot 2026-07-05:
| Tier | Box | Fast memory | $/mo | What fits, fully resident | For |
|---|---|---|---|---|---|
| 0 | D64ads_v7 (CPU only) | 256 GB RAM | $109 | GLM-5.2 IQ2 (222 GB), the #1 open model, no GPU, ~8 t/s | Sovereign batch lane: overnight jobs, data that can't leave |
| 1 | NV18ads (1× A10) | 24 GB | $47 | Qwen3.6-27B, Qwen coder models | Private coding assistant, one dev to a small team |
| 2 | NV72ads (2× A10) | 48 GB | $231 | 70B-class Q4, or two coder replicas | Small team, or two models side by side |
| 3 | NC40 (1× H100) | 94 GB | $224 | gpt-oss-120b resident, vLLM batching + big KV cache | Many users on one box, RAG, long context |
| 4 | NC80 (2× H100) | 188 GB | $449 | DeepSeek-V4-Flash Q4 + batching | High-throughput frontier-open serving |
| 5 | NC48 (2× A100) | 160 GB | $282 | V4-Flash Q4 (tight), Germany region | Frontier-open with EU residency |
| 6 | NC96 (4× A100) | 320 GB | $543 | GLM-5.2 IQ2 fully resident, 27.7 t/s, measured — or Qwen3-235B FP8 under vLLM | Measured under real team load: 10–12 users on the top open model (~4–8 s first token), or 20+ devs on Qwen-235B/vLLM at ~0.4 s first token, zero errors. UK residency |
| 7 | ND96 (8× A100) | 640 GB | $1,704 | GLM-5.2 Q4 (435 GB) at max quality; anything | No compromise |
Note the fit rule biting in the middle of the table: GLM-5.2's 222 GB spills on the 188 GB and 160 GB boxes, so on those you run DeepSeek-V4-Flash instead, or step up to the 4× A100. The spec sheet won't tell you that. The memory arithmetic does.
The tier-6 numbers are measured, not estimated — we ran a 30-minute sustained load test (20 simulated developers, Poisson arrivals, zero cache hits: the worst case) on that exact box. One finding worth the whole test: the giant 2-bit model walls out on prompt-processing at 10–12 users, while the smaller-but-near-frontier model under vLLM served all 20 with seven-fold headroom — on identical hardware. Seat counts are an architecture property, not a spec-sheet property. Other rows are engineering estimates, which is exactly what a proof-of-concept exists to confirm on your workload. Compute only; a persistent disk holding the model weights between sessions adds ~$15–135/month by model size.
"But Foundry already does this"
Azure AI Foundry's managed endpoints are genuinely good: zero-ops, per-token, pay for what you use. Two things it can't do:
- It doesn't host the top open model at all. GLM-5.2, the #1 on our 50-task live-deploy benchmark among open weights, is simply not on the Foundry catalogue. If you want the best open model on Azure, self-hosting is the only route.
- Per-token pricing inverts at volume. A flat-cost box doesn't care how hard your team hits it. For around 20 agentic power users (15–40M tokens each per month), the flat box typically lands 2–5× cheaper than per-token, and every token stays inside your VNET. For low or spiky usage, Foundry is cheaper, and we'll tell you so. Size to your actual volume, not to the story.
The honest caveats
- These are best-case spot numbers. Spot can be evicted on ~30 seconds' notice, and prices move. In your specific residency region the price may be higher, or spot unavailable, and on-demand with scheduling still cuts ~78% from the 24×7 baseline.
- Spot suits dev, batch, and business-hours work. Guaranteed always-on means on-demand or reservations: a different, still-quotable number.
- This is not the cheapest GPU compute on the internet. Specialist GPU clouds beat Azure per GPU-hour. It's the cheapest sovereign compute: frontier AI inside the tenant you already govern, audit, and hold agreements for. That's the thing being bought.
The thought experiment ends the usual way: the barrier to owning your AI isn't cost anymore. It's knowing the fit rule, the eviction data, and which two levers to pull.
We deploy exactly this, hardened, in your own Azure tenant, in one day. Private AI Jumpstart →