A private AI server for every person on the team, from $16 a month

The bet

A Copilot seat costs $19 a month for the GitHub flavour and $30 for the Microsoft 365 one. The bet in this post: from $16 a month per person in compute, every person on your team gets an entire private server, running an open model of your choosing, inside your own Azure tenancy. No prompts, no content, no work ever leaves your boundary. Not a shared endpoint with rate limits. A server each, with the entry tier priced under the lowest seat. And because the server switches itself off when idle, the real number should come in under $10; the route to that is below.

I was not sure this would work when I started. The box cost-effective enough to make the maths interesting has 16 CPU cores and no GPU. This post is the measured answer: where it is genuinely good, and where it is not, in that order.

What the 9-cent box delivers

The test model is Qwen3-Coder-30B: 30 billion parameters total, 3.3 billion active per token (mixture of experts, which is why it is quick on modest hardware). I used a coder because it is the hardest case to satisfy; the same box runs a general assistant or a careful reasoner for non-engineers, and choosing per person is part of the design. One rule makes that choice inexpensive, and it is the most useful sentence in this post: speed follows active parameters, capability follows the full network. You can buy brains without buying slowness. It gets measured further down. The machine is Azure's Cobalt 100 Arm family on spot pricing; the entry box is the 16-core D-series with 64GB of RAM. Everything below is measured on the same pinned engine build.

• Generation: 34 tokens per second, faster than you read.
• A brand-new 500-token question: about 10 seconds to first token.
• Anything already in the session (the conversation, documents it has read): a few hundredths of a second to first token.
• Memory: the model takes 18GB of the 64GB; a full 32k-token session cache costs ~3GB. RAM is never the constraint on this box.

For calibration against hardware you might buy instead: I benchmarked the same model, at the same quantisation, on the same engine build, on the mini PC on my own desk, an AMD Ryzen AI 9 HX 370 with its Radeon 890M integrated GPU and 32GB of DDR5. I paid a little over $1,000 for it, and it would likely cost more today. The mini PC: 35 tokens per second using the integrated GPU. The 9-cent rental: 34. Owning the box is fine; the point is that renting matches it for pennies, one per person.

Two asymmetries the desk box cannot answer. Bandwidth: on these Azure boxes I can pull a terabyte of model weights in a few minutes (measured at 20 to 30 gigabits per second, depending on the size of box). On my home gigabit connection the same terabyte is a few hours. And lifecycle: because the model is baked into the boot image, a box goes from switched off (costing nothing) to serving in about two minutes, which is what makes the clock-out economics below possible at all.

The measurement that makes it work: the cache

The first time the model reads a big document or a long conversation it pays real time: 40 seconds for a ~2,000-token read on this box. That looks like a dealbreaker, and on a shared server it would be, because other people's work evicts your cache and you keep paying it.

On your own server it happens once. Every request after that, on the same session, starts in a few hundredths of a second: three orders of magnitude faster. A working day is one cold read followed by hundreds of instant requests. The precision that matters: the cache covers the UNCHANGED context. The ongoing conversation and documents already read are instant; genuinely new content (a fresh file, a pasted report, an agent's tool output) is read at the cold rate. Chat-first users barely notice. Agent-heavy users should move up a tier, below.

The server that clocks out

A loaded model waiting for input uses very little CPU. Generating lights the machine up. That gap is a gift, because it means the box can manage its own hours: it watches its own activity, and when nobody has used it for a while it deallocates itself. A quiet afternoon switches it off; waking it is one click and about two minutes (the model is baked into the boot image).

Azure Monitor does this natively; no agent, no code on the box. Start it at 8am on a schedule or on first demand. The consequence is the pricing model this whole post is about: you pay for presence, not provision. The $16 headline assumes 8 hours every working day; a normal month of meetings, leave and variable days should land under $10. Projection, not promise: the measured bill comes later in this series. The clock-out is the route to $10, and it applies to everyone.

Lower cost still, for light users

Two rungs below the headline, for completeness. There is an 8-core box at about $8 a month flat: I tested it, and it generates fine (20 tokens per second) but reads new documents at a crawl, over a minute for a big file, so treat it as tested, not recommended: right for batch jobs and trying the pattern, wrong for daily interactive use. And light users can share: ten simulated users on my bursty-load harness drew 22 tokens a second each from one 32-core box with zero failed requests. To be plain about it: that is simulated load, not a real team, and the harness is published with the receipts so you can judge it. Both are try-before-you-buy routes. The headline design stays a box each, with the clock-out doing the saving.

Moving up: cores first, then the silly RAM

The ladder above the entry box has exactly two rungs, and the second one costs four dollars.

First move up: the 32-core D-series, about $32 a month. Same family, double the cores: new content reads twice as fast (a fresh 500-token question in about six seconds) and generation jumps to 52 tokens per second. This is the box for agent-heavy users, whose tools pour new documents into the model every turn.

Four dollars more: the 32-core E-series, about $36 a month, with 251GB of RAM. Same cores, same speed (56.6 tokens per second and a 0.26 second first token measured on warm context). You pay the extra four dollars for one reason: memory, and what memory lets you do. So the real decision is 16 cores or 32; once you are at 32, the RAM upgrade is almost free.

These models are small next to 251GB. The coder is 18GB. gpt-oss-120b, a 120 billion parameter model, is 63GB. A careful 35B reasoner is 22GB. The big box holds our entire test menu, seven different models, 194GB, resident at once, each live on its own port. A loaded, idle model costs nothing but RAM. This is not just fast switching: two people, or one person and their background agent, can be talking to two DIFFERENT models on the same box in the same moment; they share the cores exactly like two requests to one model do, and the box takes it in stride. One machine, a whole menu of specialists.

And here is the rule from the top of the post, measured: the 120B model generates at 53 tokens per second on this box, essentially the same speed as the 30B coder, because it activates only 5 billion parameters per token:

The architecture, as a thought experiment

Here is the design this pricing implies, stated as intent: the next post in this series lays out the complete pattern, end to end, so you can build it yourself. Breaking it on purpose comes after that. Every person gets a named box: their model, their cache, their hours, their line on the bill. No load balancer, no shared anything: each server has a private DNS name on the VNET and their tools point at it. The fleet is identical single-instance scale sets stamped out by a for-loop, which buys three things: consistent lifecycle automation, per-person cost tags (you can answer "what did Dave's AI cost last month" to the penny), and Azure's spot-restore machinery for bringing an evicted box back automatically.

Eviction is the elephant, so it gets its own post. The short version: these boxes keep their disks when Azure reclaims the compute, the model and the session cache live on that disk, and Azure fires a 30-second warning first, which is enough time to save state. In a later post in this series I evict my own servers on purpose and publish what survives.

One more thing, because it changes the ceiling of the idea: nothing in this pattern is small-box-specific. A named server per person, the idle clock-out, the 30-second eviction save: it all runs identically on any size of machine, including GPU boxes. An 80GB GPU server rents for about 88 cents an hour on the same spot market. What your team could do with a few of those, and what it costs against the equivalent API bill, is a later post in this series.

What this is not

Not sub-second inline autocomplete; the first token on genuinely new content takes seconds, not milliseconds. Not a frontier model; it does not need to be, because it is private, per-person, always warm, and the endgame of this series is fine-tuning it on your own data, which no seat will ever do for you. And not a managed product, in the open-core sense: the pattern is free and fully documented here, and any platform engineer can stand it up in an afternoon. What you would pay for, if you chose to, is the maintained image, the curated model catalogue and the eviction machinery kept current, so you do not spend that afternoon, or the twelve after it as engines and formats move. One disclosure for completeness: the maintained image makes a single outbound call for catalogue and version checks, documented in our privacy policy. The self-built pattern makes none.

Appendix: cost per million tokens

For readers who price against API bills, the same economics in the other unit, compute only, single stream: the 16-core box works out at roughly $0.73 per million generated tokens, the 32-core boxes at $0.95 to $1.00. Shared under load, the big box drops to about $0.36 per million (ten simulated users, 157 tokens per second aggregate). The per-person monthly number carries this post because that is how seats are priced; this appendix restates the same economics in the unit API price lists use, for direct comparison against a metered API bill.