Same Bandwidth, Very Different Speed: DeepSeek-V4-Flash on 30 Azure CPU VMs

One of the biggest CPU VMs on Azure, E372is_v7, measures 813 GB/s of memory bandwidth, the most of any VM we tested. It wrote DeepSeek-V4-Flash at 7.1 tokens a second. An 80-core Turin VM with half the bandwidth wrote the same model at 22.4.

If the plan for faster self-hosted AI is "buy the bigger VM", this post is the data against it. We measured 30 Azure CPU sizes on the same model, the same test and two engines. Every speed below is measured; the time-to-answer chart is calculated from those speeds. The method is at the end.

Why bandwidth is the first number we check, and why it isn't the answer

On a CPU, a model writes its answer one token at a time, and every token reads the model's active weights out of RAM. DeepSeek-V4-Flash is a mixture-of-experts model: the file is 145 GB, but each token only touches a slice of it. So the speed limit is how fast the cores can pull that slice out of memory. That makes memory bandwidth the ceiling.

It isn't the speed. The chart shows the gap. Bandwidth sorts the VMs one way; writing speed sorts them another.

Five things the data says

1. More threads on the same memory: no faster. gpt-oss-120b on a 64-core Turin VM (E128ads_v7, ik_llama.cpp): 32 threads wrote at 69.2 tokens a second, 64 threads at 70.0. Reading the prompt went from 839 to 1,110. The extra threads help reading. They don't help writing.

2. More cores of the same chip: it helps, then flattens. DeepSeek-V4-Flash on AMD Turin:

2.5x the cores for 50% more writing speed on ik_llama.cpp, 35% on llama.cpp.

3. A second socket goes backwards. E372is_v7 is two Xeon 6973P-C sockets, 186 cores and six memory domains.

Stock llama.cpp on the same box the same day: 6.33 at 24 threads, 3.65 at all 186. Every core working is the slowest setting on both engines. The AMD box with the most bandwidth, HB176rs_v4 (Genoa-X, two sockets, 670 GB/s), says the same: 9.3 tokens a second on a quarter of its cores, 8.3 on all 176 (ik_llama.cpp).

4. One socket beat two, with half the bandwidth. Intel Emerald Rapids, 32 cores either way. The one-socket VM (D64ds_v6, 230 GB/s) wrote at 11.8 tokens a second. The two-socket VM (E64ds_v6, 439 GB/s) wrote at 7.8. On llama.cpp, 10.0 against 8.7.

5. On Intel's newest chip, more cores made it slower. Granite Rapids, one socket: 32 cores 7.3, 48 cores 8.6, 64 cores 7.0, 96 cores 7.2 (ik_llama.cpp). On llama.cpp: 7.8, 9.0, 8.5, 7.8. Three times the cores, the same speed.

Our best explanation

Every token is built in many small steps, and all the threads wait for each other after each one. The more cores and memory domains those threads are spread over, the longer each wait. Across 186 cores and two sockets, the waiting costs more than the extra memory lanes give back.

So for one conversation, buy bandwidth on one chip that your engine can actually reach. Extra cores pay off when reading the prompt, not when writing the answer. If you run a two-domain machine, we covered the thread setting that works in our September post on DeepSeek on Arm: write on one domain's cores, read the prompt on all of them.

The full table: 30 sizes

Sorted by writing speed on ik_llama.cpp. Bandwidth is STREAM copy. Cores are physical cores.

A few things worth pulling out of the table:

• Two families reach 15 tokens a second: Turin and Cobalt 100. Nothing else reaches 14, whatever its bandwidth, core count or memory size.
• D and E sizes are the same speed within each family (Turin 160: 22.8 and 22.8; Genoa 96: 12.5 and 12.3). The extra memory on the E size buys nothing for this model.
• F80ads_v7 matches the 160-vCPU Turin sizes. All three have 80 physical cores. The F size counts one vCPU per core; D and E count two.
• ik_llama.cpp reads the prompt faster on every VM: 1.8x to 6x. It writes 12 to 35% faster on one-socket AMD, Cobalt and the older Intel chips. On two-socket boxes (apart from M96s_1_v3) and on Granite Rapids it writes no faster than llama.cpp, and partly that's our method (below).

What you pay against how fast it writes

Price doesn't buy speed either. The four cheapest VMs we tested are all Cobalt 100, and all four write faster than every Intel VM on the list. The most expensive, E372is_v7, writes at about the same speed as Intel VMs costing a fifth of the price. The fastest Turin VMs sit in the middle of the price range, and F80ads_v7 is the cheapest of the three 80-core Turin sizes at the same speed.

Prices are the median Linux spot price across Azure regions on 9 October 2026, from Azure's public retail prices. Spot prices move daily and vary by region, so check your own region before you size. The FX and M sizes had no spot price in our snapshot, so they're left off this chart.

How long before the answer starts

DeepSeek-V4-Flash thinks before it answers. In our 24 August test it wrote about 1,350 tokens of thinking before the first word of the answer. This chart adds that thinking time to the time it takes to read a 2,048-token prompt, using each VM's measured speeds.

Two things show up straight away. Reading the prompt (the grey part) is a few seconds on every VM with ik_llama.cpp. The wait is almost all thinking, and thinking runs at writing speed. So even the fastest VM takes just over a minute before the answer starts, and the slowest take more than three. This is a calculation, not a stopwatch: real questions think for longer or shorter, and the thinking length was measured on one question.

The two-socket caveat, stated plainly

Our ik_llama.cpp test loads the model from a RAM disk, so the weights sit in whichever memory domain the download landed on. On E372is_v7, 95% of the model sat on one of its six domains. llama.cpp copies the model and spreads it. Spreading the model under ik_llama.cpp added 3 to 20% on the two-socket boxes where we tried it (E372is_v7: 7.13 to 7.33). It never lifted a two-socket box above the one-socket VMs of the same chip family. Every finding above holds on both engines.

What this means for a private deployment

• For one user, pick the chip, not the size. An 80-core Turin or a 64 to 96-core Cobalt VM writes this model at 18 to 23 tokens a second. Bigger x86 boxes don't get there.
• Use ik_llama.cpp on CPU. Reading speed is where it pays, and reading speed is what decides how long you wait after pasting a long document.
• DeepSeek-V4-Flash on CPU is a batch model, not a chat model. With its thinking, the answer starts after a minute on the fastest VMs (chart above). Smaller mixture-of-experts models such as gpt-oss-120b write three times faster on the same hardware.

Limits

• One conversation at a time. Serving many users is a different measurement.
• DeepSeek-V4-Flash only. Smaller models that fit in one memory domain can rank differently.
• A gap under about 10% between two VMs is noise, so treat the order within each family as a tie.
• No NUMA pinning in any figure above. Pinning the biggest VM to one socket is our next test.

Method

• Model: DeepSeek-V4-Flash-0731, MXFP4 GGUF, 145 GB.
• Engines: ik_llama.cpp 3bb386e (Cobalt: fdb4599, the build with our Arm fix); llama.cpp b10582 (e85caa81e). On AMD Genoa, llama.cpp was the same commit built with runtime CPU detection.
• ik_llama.cpp: -c 8192 --jinja --threads N, model memory-mapped from a RAM disk. The Cobalt rows use the near-identical method from our September post (-c 32768, writing threads on one memory domain, three runs).
• llama.cpp: --jinja -ngl 0 -fa 1 --parallel 1 --ctx-size 8192 --no-mmap --numa distribute.
• Test: through the server API, a 2,048-token prompt and a 128-token answer, two or three runs, each run passing a 3/3 correctness check first. Thread sweep on every VM (all cores, half, one memory domain); the table shows the best.
• Bandwidth: McCalpin STREAM copy, arrays at least 4x total L3 cache, peak of a thread sweep.
• When and where: Azure spot, 24 August to 12 October 2026, mostly Sweden Central; E372is_v7, HB176rs_v4, FX and M sizes in East US; Genoa in Italy North.

Projects: ik_llama.cpp and llama.cpp.

Caleta.io sells an Azure Marketplace appliance that packages this. Everything above is reproducible without it.