1. Huawei Built Two Chips Where There Used to Be One
In 2026 Huawei’s flagship AI accelerator is not one chip. It is two, sharing the same core compute architecture, separated by nothing but the memory packaged alongside it.
The Ascend 950PR carries 128 gigabytes of Huawei’s own high-bandwidth memory at 1.6 terabytes per second. The Ascend 950DT carries 144 gigabytes at 4 terabytes per second. Same DaVinci cores, same interconnect, same instruction set. The difference the company chose to build a second product line around is memory bandwidth, and a factor of two and a half separates the two.
A chipmaker does not split its flagship in two without a reason that costs it money to act on. The reason is printed in Huawei’s own architecture whitepaper, and it is worth reading in the company’s own words.
This is a story about silicon. It does not stay there. The seam Huawei cut through its own product line is the same seam that runs through the income statement of every Chinese company selling inference, and by the end of this piece the 1.6 and the 4 will be numbers about gross margin.
2. Why Inference Splits Into Two Halves That Want Opposite Things
Serving a large model happens in two phases, and they place opposite demands on the hardware.
The first phase is prefill. The model reads your prompt, the whole thing at once, and builds its internal representation of it. This is a wide, parallel, compute-heavy operation. Many numbers multiplied at the same time. It leans on arithmetic throughput and it is comparatively relaxed about memory.
The second phase is decode. The model writes its answer one token at a time, and each new token has to see every token before it. At each step the accelerator streams the model’s weights and the entire accumulated context out of memory and into the arithmetic units, does a comparatively small amount of maths, and writes one token back. Then it does the whole thing again for the next token. The arithmetic units are mostly idle. What they are waiting for is memory.
Prefill is compute-bound. Decode is memory-bound. This is not an opinion, and it is not this publication’s framing imported onto Huawei’s hardware. It is Huawei’s framing.
The whitepaper states, in its opening, that agent workloads with long context and multi-turn interaction drive KV Cache storage up exponentially, and that on-chip memory alone can no longer support the business. The KV Cache is the model’s working memory of the conversation so far, the stored representation of every token already seen, and it is what decode has to stream through memory at each step. It grows with context length, which is why longer prompts and longer sessions push the memory demand up rather than the compute demand. Huawei is telling you, in its own document, that the binding constraint on modern inference is memory.
3. The Names Are the Argument
Read what Huawei called the two chips.
The 950PR stands for Prefill and Recommendation. The 950DT stands for Decode and Training. The company did not split its chip by customer, or by price tier, or by market segment. It split it by which half of inference the chip is for, and it put the high-bandwidth memory on the decode side.
A product line is the most honest thing a company publishes, because it is the one claim it has spent capital to stand behind. Marketing can say anything. A second tapeout and a second memory supply chain cannot. When Huawei put 4 terabytes per second behind decode and 1.6 behind prefill, it was pricing the exact proposition this publication argued from the silicon side in its sixth issue and from the income-statement side in its eighth: that the cost of serving a token is set by memory bandwidth, because decode is where the tokens are made and decode is memory-bound.
You do not have to take my word that decode is the expensive half. You can take Huawei’s capital allocation, which says the same thing and cost more to say.
4. Self-Sufficient and Two Generations Back, at the Same Time
Here is where the balance has to hold, because the story breaks if you drop either half.
The high-bandwidth memory in both chips is Huawei’s own. HiBL 1.0 on the 950PR, HiZQ 2.0 on the 950DT. The whitepaper calls it high-speed on-chip memory and describes it as the DRAM that holds the model’s global data, which is the same role HBM plays on an Nvidia part, so the bandwidth figures compare on a like axis rather than against a cache. For a company cut off from SK Hynix, Samsung and Micron by export controls, designing and producing its own such memory is a real achievement, and it removes the single most effective chokepoint the controls were built around. This is not a company that failed to get memory. It is a company that built its own.
And on that one axis, bandwidth, the memory it built is behind. The 950DT’s 4 terabytes per second sits at the level of Nvidia’s H200 in its HBM3e configuration, roughly 4.8, a chip that reached volume in 2024. Nvidia’s current B300 runs at 8. I am comparing memory bandwidth and nothing else here: interconnect, software ecosystem and total chip capability are separate questions on which the two companies sit very differently, and Huawei’s interconnect in particular is a genuine strength. But on the decode axis that sets inference cost per token, Huawei’s best available memory is about half of Nvidia’s current part and level with Nvidia’s part from two years earlier.
Both of these are true and neither cancels the other. Self-sufficiency solves whether you can buy the chip at all. It does not solve what the chip costs to run per token. The first is a survival question and Huawei has answered it. The second is a margin question and the answer is a ceiling.
5. The Chip That Relieves the Bottleneck Ships in Q4
Now the timing, which is the whole point and the part the roadmap coverage skipped.
The 950PR, the prefill chip with 1.6 terabytes per second, reached mass production in the first quarter of 2026. The 950DT, the decode chip with 4, is scheduled for the fourth quarter. For most of 2026, the only Ascend 950 a Chinese company can actually buy in volume is the half built for prefill.
That is the chip whose memory is slowest, deployed into the phase that needs memory most. A company standing up domestic inference capacity in mid-2026 is either running decode on a 1.6-terabyte-per-second part built for prefill, or still buying restricted Nvidia, or renting from a cloud that does one of those two things on its behalf.
This is the sentence the roadmap turns into once you read the memory column and the ship dates together. The ceiling on Chinese inference margin in 2026 is not that the country cannot build a high-bandwidth decode chip. It has designed one. The ceiling is that the decode chip is not here yet, and until it is, the economics of serving a token run through a part that was built for the other half of the job.
Which is where the silicon rejoins the income statement.
6. Put the Bandwidth on Zhipu’s Gross Margin
The eighth issue of this publication took apart the valuation gap between Zhipu and MiniMax and found it sitting inside one number: Zhipu earns an 18.9 percent gross margin on its cloud and API line, the business of selling inference by the token. Call it 19. That number was a mystery on the income statement. It is not a mystery on the silicon.
An 18.9 percent gross margin means the cost of serving is 81 percent of the revenue from serving. The largest movable part of that cost is the number of tokens a given accelerator can produce per second, because that sets how much hardware and power it takes to meet demand. Decode throughput per chip is, to first order, memory bandwidth divided by the size of the model’s active parameters and context per token. Bandwidth is the numerator. Raise it and cost per token falls. Cap it and cost per token has a floor.
Zhipu runs its inference substantially on domestic silicon, which its filings describe as co-design and which its addition to the US Entity List, effective 16 January 2025, makes a requirement rather than a preference. In 2026 domestic silicon at volume means the 950PR and its 1.6 terabytes per second, or Huawei’s prior 910 generation, or Cambricon, an independent Chinese AI chip designer. None of those is the 950DT. The memory bandwidth that caps Zhipu’s decode throughput, and therefore floors its cost per token, and therefore ceilings its cloud gross margin, is a number set in Shenzhen and Huawei’s HBM line, not in Zhipu’s model.
The 18.9 percent is not a verdict on GLM. It is the reading, on an income statement, of a memory-bandwidth number the company does not control.
7. Whether the Split Even Holds
The honest risk to this thesis is that the prefill-decode split does not survive contact with deployment.
Nvidia explored a version of disaggregated inference with its Rubin CPX, a prefill-oriented part using cheaper GDDR7 memory, and that approach has reportedly been revised since, which is a caution against assuming a clean prefill-decode hardware split is where the industry settles. Running prefill and decode on physically different chips adds scheduling complexity, network traffic between the two pools, and utilisation risk if the mix of the two phases does not match the mix of hardware you bought. It is entirely possible that Huawei’s customers run both phases on whichever 950 they can get and never deploy the clean split the product names imply.
But notice that this does not rescue the margin. The demand that decode places on memory bandwidth does not disappear because the workload runs on a prefill chip. It reappears as idle arithmetic units and low throughput, which is to say as higher cost per token. The split is Huawei’s attempt to price the two phases efficiently. If the split fails to deploy, the memory bottleneck does not go away. It just stops being labelled, and shows up unpriced in the cost of sales instead.
There is a second caution, and it cuts the other way, so it belongs here. If memory bandwidth alone set gross margin, every company on the same domestic silicon would earn the same margin. They do not, and DeepSeek is the reason to believe it. Its V4 models, released in April 2026 under an open licence, are built around a hybrid sparse-attention architecture that compresses the KV cache, the exact thing decode has to stream through memory, to roughly a tenth of the previous generation’s, and cuts single-token inference to about 27 percent of the FLOPs. Those figures are from DeepSeek’s own technical report. The point is not the specific number. It is that decode’s memory load is not a fixed tax. It is an engineering variable, and a model architected to move less data per token gets closer to the bandwidth ceiling than one that is not. Bandwidth sets the ceiling. Architecture decides how close you get. So the claim is not that Zhipu’s 18.9 percent is fixed by Huawei alone. It is that the ceiling above it is, set by the memory Zhipu can buy, and that no amount of engineering lifts that ceiling. It only closes the distance to it, and that distance is the part Zhipu still owns.
8. A Layer’s Margin, Written on a Roadmap
Step back to the machine.
The sixth issue argued that China’s compute ceiling is a memory ceiling. The eighth found that ceiling resting on a model company’s gross margin. This issue is the seam between them, and Huawei drew it in public: two chips, one bottleneck, and the relief scheduled for Q4.
The forward marker is specific and it is dated. The 950DT enters mass production in the fourth quarter of 2026. When it does, the bandwidth available to Chinese decode roughly doubles at the leading edge, from the 950PR’s 1.6 to the 950DT’s 4, and the cost-per-token floor drops with it. That is the moment the ceiling on Chinese inference margin lifts, if it lifts. Watch the ship date, watch whether the 4 terabytes per second is real in volume rather than on a slide, and watch whether Zhipu’s next disclosure shows a cloud gross margin climbing off 18.9 as the better memory arrives.
If the 950DT ships on time and Zhipu’s cloud margin is still stuck a year from now, the problem was never the memory and this thesis is wrong. If the 950DT slips, or ships thin, the ceiling stays where the 950PR put it, and the 18.9 percent is not a transitional number but a structural one.
A model company’s gross margin is not set inside its model. It is set on a wafer it does not own, in a fab it does not run, under a memory bandwidth it did not choose and cannot lift, on a schedule published by the company that does. Huawei cut the seam through its own product line and told you what it was called. It runs through everyone else’s income statement too.
Sources and Claim Types
Primary, from Huawei’s own document. The Ascend 950PR and 950DT memory capacities and bandwidths, the prefill and decode designations, the third-generation DaVinci architecture, the KV Cache statement, and the Q1 and Q4 2026 production timing come from Huawei’s Ascend 950 NPU Architecture Whitepaper, copyright 2026, and from Xu Zhijun’s roadmap presentation at Huawei Connect 2025 on 18 September 2025. The 950PR memory is 128/112 GB at 1.6/1.4 TB/s and the 950DT is 144/96 GB at 4 TB/s per the whitepaper’s specification table.
Primary, Nvidia. The B300 and GB300 figures of 288 GB HBM3e and 8 TB/s, and the H200 comparison at roughly 4.8 TB/s in its HBM3e configuration, are Nvidia’s published specifications.
Primary, US government. Zhipu’s addition to the Entity List, effective 16 January 2025, is from the Federal Register notice by the Bureau of Industry and Security. The Rubin CPX reference is Nvidia roadmap coverage and is treated as reported, not filed.
Confirmed, from filings. Zhipu’s 18.9 percent cloud gross margin and 48.8 percent on-premise gross margin are from its 2025 annual results announcement, reconciled to the audited gross profit and cross-checked against the prospectus cost-of-sales disclosure, as set out in full in the eighth issue. The blended figure is 40.96 percent.
Estimated, secondary. 950PR shipment and order figures, and named early customers, come from Chinese technology press citing supply-chain sources and are not company-confirmed. DeepSeek’s reported inference gross margin above 50 percent comes from secondary coverage of a third-party report, not a filing, and is used only as a directional counterweight, not as a load-bearing number. Both are flagged where they appear.
Inside China’s Machine is research, not investment advice. Nothing here is a recommendation to buy or sell any security.