Tokens are like chips in a casino—you burn through them really fast, and they're expensive! So you start thinking, why not host them locally? Plus, that way we'll have full control.
But what resources are needed, how much VRAM (video memory/GPU), and how do you calculate and budget for it?
It's very simple—just go to this calculator: https://apxml.com/tools/vram-calculator
Choose your category: inference / fine-tuning
Sound model, specifically the latest DeepSeek V4.1 Flash
The quantization, that is, model compression. We reduce the model's precision to save space. Generally, we choose INT4, which provides excellent compression (~70%) while maintaining acceptable quality.
Visit KV Cache Quantization, this is input cache compression. For each incoming request, it must be encoded and stored, which takes up space.
Next, we configure the operational variables for our model—that is, how we’re going to use it.
BatchSize: The number of requests processed at the same time. If there are 32, our model is capable of processing 32 requests at the same time.
Sequence Length: The token length of our input and output. The longer it is, the more time it takes us KV Cache. This depends on the expected queries. If you're processing large documents, you'll need a large sequence (> 32,000); if you're processing comments, you'll need a smaller one (8,000, for example).
And Concurrent Users : The number of users we expect to be online at the same time.
And then, Here you go, we have the necessary VRAM (GPU) to run our model locally, depending on our system configuration.
As you can see here, the bill is steep. 282.7 GB of VRAM.
At a minimum, I'll need 6x NVIDIA RTX 6000 Ada cards with 48 GB of memory (~60,000$) or a Mac Studio M3 Ultra with 512 GB of unified memory (~15,000$).
Interestingly, you can also see the Generation Speed, The number of tokens generated per second, which is very important for the speed of our system.
We can also compare the cost to that of the cloud. If I were to rent this same infrastructure in the cloud, it would cost me roughly $8.40 per hour. That is, 200$ per day, 6000$ per month, 72000$ per year. It’s clear, then, that investing in a Mac Studio M3 Ultra (~15,000$) can be Payback in less than 3 months.











