Kaufman Lab · AI architecture
Articles · AI assistants

Is a local AI model cheaper than an API?

A local AI model beats an API when the volume is large and steady, when the cloud model is expensive, or when the data is private, because the price of a graphics card is spread over its months of service, while a million tokens from a cheap cloud model costs cents. According to a calculation by researchers Pan et al., your own hardware is justified mostly from 50 million tokens a month. First count your tokens and check whether your computer can run the model.

Roman Kaufman, founder of Kaufman Lab · Data as of: October 2026

A monthly cost window: the RTX 4090 power row in a red frame, next to it a cheap API price tag. Text: Local AI or API? Local wins at large volume
Local AI on an RTX 4090: the card is $50 a month plus power, a cheap API is $6

A student who builds their own website is advised to run a local model with the words "You only pay for electricity and hardware." The other path is called an API. There the model runs in someone else's cloud, and you pay for the amount of text.

Since August 2026, the model of our personal AI assistant has also lived in the cloud. It needs five to ten calls a day. Why not on our own computer, if electricity costs almost nothing? The answer is in the electricity bill.

A home model lags behind paid ones by six months to a year

A model that fits on one home graphics card catches up with the best paid models in 6–12 months. Epoch AI researchers Venkat Somala and Luke Emberson got this figure in their analysis from September 9, 2025 across four benchmarks. They count as a home model one with up to 28 billion parameters for an RTX 4090 card.

The authors themselves warn that the best open model is more often tuned to the benchmarks, so in real work the gap is most likely bigger. By their rough estimate, the open gpt-oss-20b may be comparable to the paid o1-mini, and the gap between them is 11 months.

Your own hardware pays off against expensive APIs or at large volume

Your own graphics card earns its money back if you compare it with an expensive cloud model or run a very large amount of text through it. Researchers Guanzhong Pan et al. published their payback calculation for your own hardware on November 11, 2025. Their threshold is mostly from 50 million tokens a month (a token is a small piece of text).

For small companies with less than 10 million tokens a month, a small model on one RTX 5090 (about $2,000) can pay off in 0.3–3 months. In Pan's work the payback time depends on which paid model you compare with, for example Claude-4 Opus or GPT-5. Salaries are not included in the calculation, even though someone has to install drivers and update the models. The other lines of an owner's budget are collected in the article how much an AI agent costs.

Pan's second condition is about data that cannot be sent to someone else's server. We covered where the line is for company files in the article what data to upload to an AI.

Our assistant is at the other end of the scale. As of September 2026, the cloud provider's free plan allowed up to fifty calls a day, and the model cost us 0 rubles.

Power for the card costs more than a cheap API at 500,000 tokens a day

Even a graphics card you already own can burn more on electricity than a cheap API costs. Developer Flavio Copes updated his card vs API calculation on August 3, 2026. In his view, with a card you already own, local wins "at almost any volume."

His own numbers say something else. At 500,000 tokens a day, an RTX 4090 draws about 400 W, and at $0.16 per kWh the power comes to $8.60 a month. The same volume through a cheap API costs $6, through a mid-range one $30.

The purchase also has to be spread over months. A $1,800 card over three years of service costs $50 a month before the first token, and about $59 with power. Against a cheap API at this volume the card never pays off. Against a flagship plan at $150 a month it pays off in about 20 months. Price lists change; what matters is the order of the numbers.

That is why our assistant does not need its own card: the free cloud is cheaper than the power alone.

Count your volume first, then choose a graphics card

The decision about a local AI model starts with the number of tokens a day. You choose the graphics card after that.

  1. Calculator, before you buy. In Copes's calculator, enter your answer tokens per day in "Output tokens / day" and the API price per million in "$/M output tokens". It will show the monthly cost of your own card and of the API, and the payback time.
  2. Computer check, before you buy. Install Ollama, a program for running a model on your own machine, start a model and type `ollama ps`. In the Processor column, "100% GPU" means the model is in the graphics card. "100% CPU" means the model went entirely into the computer's regular memory, not into the graphics card.
  3. Turn off the cloud before any real data. In the file `~/.ollama/server.json`, set `"disable_ollama_cloud": true` and restart Ollama. The log will show the line "Ollama cloud disabled: true", as the Ollama help page says, checked on October 31, 2026.

A student needs only the first step. If you did install Ollama and see "100% CPU", go with the API. Otherwise, try the model on your own computer. For a marketer, step 1 will show where the API's advantage ends, and an owner with private data needs steps 2 and 3.

Frequently asked questions

Which AI model works without the internet?

For example, gpt-oss-20b can work without the internet. According to OpenAI in August 2025, it needs 16 GB of memory. In Ollama the model runs on your computer, but first the program downloads it from the internet. Ollama's own cloud models are processed on its servers; the setting from step 3 turns them off.

How do I run an AI model on my computer?

Install Ollama and type `ollama run gemma4` in the terminal. The program will download the model and run it on your machine.

What is a local AI model?

A local AI model is one that runs on your computer. Ollama says it does not see the requests you send to it.

A question about this article?

Message me on Telegram: I answer myself, look at your task and tell you which of this fits it.

Message on Telegram→
More articles
↑ top