You don't need $10,000 of hardware to run DeepSeek's new model

DeepSeek's V4-Flash-0731 is the open-weights release making the noise this week: 284 billion parameters, 13 billion active per token, 155GB on disk at MXFP4. The advice circulating alongside it is that running it yourself means a five-figure workstation or a rack of GPUs.

Here is a measured counterexample: all 284 billion parameters at 16.2 tokens a second, faster than most people read, on a rented cloud box with no GPU in it, for $0.4424 an hour on Azure spot. And here is the more useful version of that sentence: we ran the same model, on the same engine build, across all five machine tiers we deploy, and measured the three numbers that actually decide which box you should rent. Most comparisons publish one of them.

The three numbers

How fast it writes. Decode speed, tokens per second, the number everyone quotes. It is bound by memory bandwidth: every generated token streams the 13 billion active parameters out of memory.

How fast it reads your prompt. Prefill. It is bound by compute rather than bandwidth, it varies between these machines by a factor of forty-five, and almost nobody publishes it. It decides whether the box feels instant or appears to have hung, and it is where the honest caveats live.

How hard it works the memory it pays for. We measured each machine's maximum bandwidth with STREAM, then compared it against what generation actually moves. The gap between those two bars is the most predictive thing in this post.

Core counts are physical cores. On x86 the 64-vCPU box has 32 real cores, and using the hyperthreads made it slower, so every figure pins threads to physical cores. GPU box maximums are measured per-card bandwidth multiplied by card count (861 GB/s per A100, 1,791 per H100, and the per-card figure matched across the two and four card boxes). "Used" bandwidth is decode multiplied by 7.1 GB per token (the 13B active parameters at MXFP4); it is a floor, since it excludes KV-cache traffic and activations, and headroom is what is left. Prefill was measured at roughly 2k tokens on the CPU machines and 8.8k on the GPU machines, so treat that column as an order of magnitude rather than a head to head.

Which box for which job

The table collapses into a simple map once you plot reading speed against writing speed.

A team, agents, or anything with long prompts: the two-H100 box, $2.64/hr. It reads an 8,800-token prompt in about six and a half seconds and writes at 55 tokens a second. But the number that actually matters for a team is in the third column: a single request uses about a tenth of this machine's measured memory bandwidth. The rest is not waste, it is headroom, and it is why this box can absorb concurrent users in a way no CPU box can.

The same model on a smaller budget: two A100s, $1.63/hr. Model plus 32k of context fits inside the 160GB of VRAM with room to spare, at 60% of the H100 price and roughly 70% of its speed on both axes.

Not four A100s, $3.39/hr. The four-card box cost 108% more than the two-card box and measured 1.5% slower. On the scatter chart they are the same dot. Adding two cards doubled the box's memory bandwidth and halved its utilisation, and changed nothing else: at a batch size of one there is nothing for the extra cards to do except add hops between them. Unless you are buying VRAM for a bigger model or throughput for heavy batching, the smallest box that fits the model wins, and this is now a measured rule rather than folklore.

That chart is deliberately free of derived metrics: observed hourly price on one axis, measured speed on the other, region on every point, and the division left to you and your duty cycle.

One person, sustained sessions, the tightest budget: the Arm box, $0.44/hr. It writes at reading speed, it costs less per eight-hour day than a coffee, and it carries this post's most important caveat. It reads prompts at 30 tokens a second, so a 2,400-token first turn takes about 80 seconds before the first word appears. Prefix caching changes the experience completely: we measured a real agentic session at 34 seconds for turn one and 0.4 seconds for every turn that built on it. So: right for a person in a sustained session, right for overnight batch work, wrong for anything that rebuilds its prompt from scratch on every call. Anyone recommending CPU inference without volunteering that number has not run the workload.

Prompt-heavier work that still wants CPU pricing: the x86 box, $0.68/hr. It writes 13% slower than the Arm box but reads 2.8 times faster (83 against 30 tokens a second), which halves that 80-second first turn to under 30. The likely cause is vector width, 512-bit units against 128-bit; we have not isolated that variable, so treat it as the probable explanation rather than a proven one.

The bandwidth column explains everything else

Microsoft publishes vCPU counts and RAM sizes. It does not publish achieved memory bandwidth, which for this workload is the specification that matters. So we measured it, and then measured how much of it the model actually uses.

Two things fall out.

The cheap boxes are already working hard; the expensive boxes are barely trying. The CPU machines convert 28 to 41% of their measured bandwidth into tokens. The GPU machines convert 8 to 16%. Same model, same build. A single stream simply cannot keep thousands of GB/s busy.

That one fact carries both of the post's practical conclusions. It is why the third and fourth A100 bought nothing: the bus was never the constraint at batch 1, so doubling it just halved utilisation, 16% down to 8%, with decode unmoved. And it is why the GPU tiers are the right answer the moment users share the box: the idle 84 to 92% of the bus is exactly the capacity that concurrency consumes. The CPU boxes have half that reserve at best, which is why they are single-user machines.

The Arm box is the anomaly worth knowing about. At 406 GB/s measured, a $0.44 an hour cloud CPU lands in the same memory-bandwidth neighbourhood as a high-end Apple desktop part. That is the entire reason a 284B-parameter model is usable on it. Its weakness is not the memory system, it is prompt ingestion, which is a compute problem, not a bandwidth one.

Region beats SKU

Everything above is engineering. The biggest lever in the whole exercise is not.

The same NC80adis_H100_v5, the same two H100 cards, in the same 0 to 5% eviction band, was $2.6444 an hour in East US and $5.3518 in Sweden Central on the same day. Identical hardware, 102% apart, roughly $2,000 a month at 24/7. No benchmark can surface this because nothing about the machine differs.

The pattern is not uniform across tiers, and the eviction data matters as much as the price:

• The A100 boxes punish the wrong region twice. NC48 in Sweden Central was $2.9290 against $1.6290 in South Central US, 80% more, and sat in a 15 to 20% eviction band against 0 to 5%. Dearer and materially less stable.
• The Arm box is cheapest in UK South at $0.3593, 19% below our quoted Sweden Central price, but UK South publishes no eviction data at all. A real saving traded against zero visibility of interruption risk. We quote the Sweden Central price because it comes with a stated 0 to 5% band on both the 7 and 30 day windows.
• Every price in this post names its region and its observation date, 5 August 2026, because a spot price without both attached is not a number, it is a rumour.

Practical notes if you run this yourself

The model loads in 7 seconds from a memory-backed filesystem, on every machine we tried. For an ephemeral spot box, that is the difference between a viable pattern and an unusable one.

Two upstream bugs, both current. On Arm CPUs the server loads all 155GB, reports ready, then aborts on the first request unless you pass --no-repack (llama.cpp issue 26537, which we found and filed). And quantising the KV cache produces garbage on this architecture (issue 26423), so keep it at f16. You do not need the saving anyway: 128k of context costs about 22GB here, thanks to the model's compressed attention.

It is a reasoning model. It fills its reasoning stream before its answer, so a test request with a small token limit returns an apparently empty response. Give it 300 tokens or more before concluding anything is broken.

What we refused to publish

Four claims from this work died in re-testing. They were the interesting ones, which is exactly why they are listed.

"Arm is 1.67 times faster than x86." It is 1.15 times. The x86 figure came from an older engine build. Like for like, most of the gap closes.

"Decode tracks memory bandwidth one for one." It collapses on the same mismatched builds, and the utilisation column above shows why it could never have been true: these machines convert anywhere from 8% to 41% of their bandwidth into tokens.

"A 128k context costs 0.7GB." Read from memory usage at startup, before the cache had been touched. The real figure is around 22GB.

Cost per million tokens. This one was in an earlier draft of this very post. The naive calculation, hourly price divided by decode speed, prices prompts at zero, and prefill is precisely where the cheap machines are weakest, so the metric systematically flattered the boxes it was recommending. An honest version has to price your prompt lengths, your idle time and your concurrency, at which point it is a property of your workload, not of the hardware. That is a post of its own. Until then the observed prices are in the table and the division is yours.

The rule that caught all four: same box, same build, same arguments, or it is not a comparison.

Method

Model: ggml-org/DeepSeek-V4-Flash-0731-GGUF, MXFP4, 155GB on disk, 284B total parameters, 13B active. Engine: llama.cpp at commit 571d0d540, one build across all five machines, same flags per architecture. Serving: single VM, no clustering, --parallel 1, f16 KV cache, 16k context on CPU and 32k on GPU, threads pinned to physical cores, --no-repack on Arm.

Decode and prefill are the median of three runs of a fixed prompt. Maximum bandwidth is STREAM copy on the same machine in the same session. Used bandwidth is decode multiplied by 7.1 GB per token (13B active parameters at MXFP4), a floor that excludes KV traffic and activations. Prices are Azure Linux spot from a 30-day census observed 5 August 2026, cheapest region holding a 0 to 5% eviction band on both the 7 and 30 day windows, except where the region section says otherwise.

Everything here was measured on our own machines. No third-party benchmark figures, and no quality claims: whether this model is any good is a separate question we have not answered yet.

If you want to run this yourself

We package this as an Azure Marketplace appliance, and the model catalogue with the measured serve configuration for each tier lives in the appliance guide, but everything in this post is reproducible without it.