Models are open source, but fewer and fewer people can run

There is a quite divisive phenomenon in the current open source model: the weight is given more and more generously, but fewer and fewer people can run on their own.

Take DeepSeek-V4-Pro as an example. Its specifications are 1.6 trillion total parameters, 49 billion per token activation, native support for 1 million token contexts, MIT licensing, and completely open weights. This is quite thorough open source, with no applications, no commercial restrictions, and no regional restrictions.

Then you go and do some calculations and you will find that this "openness" cannot be fulfilled for the vast majority of people.

!

calculates the account of the video memory first

The algorithm is very simple:

Parameter quantity × several bytes per parameter.

At BF16 or FP16 precision, each parameter is 2 bytes. So:

1.6T × 2 = 3.2TB。

Just putting in the weights requires 3.2 TB of video memory.

Compare the current stand-alone configuration:

An 8-card H100, an 80GB card,a total of 640GB. An 8-card H200, 141GB each,a total of 1128GB. An 8-card B200, 192GB,a total of 1536GB.

Divided by 3.2TB by 141GB,approximately 23 H200s are needed. No single 8-card machine can be installed, and it must have multiple nodes.

Even if we do radical quantification, with INT4 precision, 0.5 bytes per parameter, 1.6T × 0.5 = 800GB, 640GB of 8 cards H100 is still not enough, so we have to match H200.

Moreover, this account is only weighted. KV cache is not counted, activation values are not counted, and the overhead of the reasoning framework is not counted. For the 1 million tokens context, KV cache alone takes up a large part. So the actual demand is higher than 3.2TB.

The V4-Flash of the same series is better: 284B total parameters, BF16 is about 568GB, and the 640GB of the 8-card H100 is just enough. This is why most people actually encounter Flash.

is a common confusing place

Here is a technical detail that I find that many people understand the wrong way.

DeepSeek-V4-Pro is a sparse MoE architecture. Among 1.6 trillion parameters, only 49 billion are activated per token, with an activation ratio of 3.06%(49 divided by 1600).

The efficiency data given by the official technical report is also very beautiful: compared with the previous generation V3.2, in 1 million contexts, only 27% of FLOPs for single token inference and only 10% for KV cache are required.

Looking at this set of numbers, it is easy to draw a conclusion: Since only 3% of the parameters are used, deployment should only require 3% of the resources.

That's not true.

MoE routing is dynamically selected based on token. This token is routed to experts 7, 23, and 104, and the next token may be routed to completely different ones. You can't predict, and you can't just load part of it.

Therefore, standard deployment requires all experts to store video memory, not just the activated 3%.

Technically, it is possible to uninstall experts, put infrequently used experts into CPU memory or even disk, and replace them when using them. However, the bandwidth difference between video memory and memory is an order of magnitude. Every time you switch in, you have to wait, and the latency will rise unsightly. For online services that pursue throughput, this path is basically not feasible.

Therefore,"fewer activation parameters" saves computing power and not video memory.

The official revenue of 27% FLOPs and 10% KV cache is all in the two dimensions of computing and caching. In terms of storage dimensions, should you install 1.6T or have to install 1.6T.

!

This explains a seemingly contradictory phenomenon:

Model reasoning is getting cheaper and cheaper, but fewer and fewer people can deploy it themselves.

These two things were established at the same time, and both were brought by MoE. Sparseness greatly reduces the computing cost per token, so the API price can be reduced all the way; but the number of total components is expanding simultaneously, and the memory threshold follows the total amount, so the self-deployed hardware requirements are rising all the way.

open source beneficiary structure has changed

The opening of weights itself is real, MIT licensed, completely open, and can be downloaded by anyone. According to public reports, V4-Pro received more than 170,000 downloads in its first week.

But being able to download does not mean being able to run.

I divided the people who can really use this weight into three categories:

The first category is those with multi-node clusters. Cloud manufacturers, infrastructure teams of major manufacturers, and a few companies with reserves of computing power. They can deploy fully, make deep customization, and continue training based on integrity rights. This open source is open source in the true sense of the word for them.

The second category can run quantitative versions and small versions. INT4 quantifies 8 cards of H200, or simply use a 284B Flash file. This kind of team gets the ability to get a discount, but at least the thing is in their own hands.

The third category can only be called API. The vast majority of people are in this category. For these people, whether this model is open source or not makes no difference in terms of physical feeling. Anyway, they all adjust the interface and pay by token.

The actual role of open source here has shifted: it is no longer mainly about "allowing everyone to run on their own" and more about "allowing the first type of players to not be locked in by a single vendor."

This role remains important, and may even be more important. With more high-level weight that can be deployed by yourself, there will be more bargaining space among cloud vendors, and the risk of monopoly of the model layer will be reduced. But it is not what many people think of open source.

Ten years ago, you cloned an open source project, and you could compile it and run it on your notebook. Now you clone an open source weight and need a computer room.

!

What should small and medium-sized teams do

There's no point in complaining, say something practical.

Priority is given to small and distilled versions. The same series of Flash files and distilled small models can often cover 80% of actual needs. As the flagship of 1.6T, most business scenarios simply don't use its upper limit.

Don't use "open source" as a synonym for "being able to deploy yourself." When selecting a model, first calculate the video memory account, multiply the parameter quantity by the number of precision bytes, and then check the card in your hand. This arithmetic can be done in two minutes, saving a lot of trouble later.

Quantification is not free. INT4 can compress 3.2TB to 800GB, but the loss of accuracy varies widely among different tasks, and the loss is more obvious in long text and complex reasoning scenarios. If you want to use it, you have to measure it, not just look at other people's benchmarks.

If you end up with API, focus on something else. For example, the price and stability differences of the same model among different suppliers, such as whether it can be switched to another one at any time. The existence of open source weight is itself a bargaining chip for you to negotiate with suppliers, even if you have never deployed it yourself once.

Looking back

The number of parameters will also increase, and the activation ratio will be pressed down. This direction is good for API makers, and the cost per token will continue to drop. It's bad news for those who want to self-deploy, and the memory threshold follows the total amount.

Intermediate gear will become more important. The 200B to 300B gear is just stuck on the boundary where a single machine 8 card can fit. It is the most practical size for enterprise self-deployment. I guess families will invest more in this range.

The value of quantification and offloading techniques is increasing. When the model is too big to fit on a stand-alone machine, the technology that can press the threshold back on the stand-alone machine is valuable. This piece is still relatively rough now.

few sentences boundary

Model specifications come from DeepSeek's official release and model card caliber. The efficiency data (27% FLOPs, 10% KV cache) is quoted from its technical report, which is official and has not been replicated and verified by a third party.

I calculate the account for the video memory myself according to the public parameters. The formulas are in the body and can be checked by myself. Only the weight part is calculated, and KV cache, activation value and framework overhead are not included. The actual deployment requirement is higher than this number.

The downloads in the first week came from public reports and were not officially confirmed.

The actual video memory consumption will vary under different reasoning frameworks and different parallel strategies. The single-machine configuration comparison in this paper is only a magnitude reference and cannot be used as a deployment plan.