Inside China’s Machine · Tokenomics, Episode 5 of 10
The Answer Price = price per token × tokens per attempt ÷ success rate
This episode: what counts as success
In May 2025, China’s National Data Administration published a set of case studies from companies in the data-labelling industry. One of them, submitted by 昆玉市协盈数字科技有限公司 and recommended by the data bureau of the Xinjiang Production and Construction Corps, describes the company’s data-labelling work.
The company’s work focuses on annotation in less-common languages, autonomous driving and unmanned retail. It built an image-annotation tool that automatically draws boxes around target objects, and a text-annotation tool that extracts keywords in less-common languages. It also wrote its own standards for speech transcription for less-common-language corpora, data labelling for autonomous driving, and visual dynamic review for unmanned retail.
The case sets out how that work is checked before a customer accepts it:
“质检员全检→质检员二审→组长抽审→客户验收”
A quality inspector reviews everything. A quality inspector reviews it again. A team leader samples the result and reviews it. Then the customer reviews the work and accepts it. Four layers of checking stand between a piece of work and the moment it counts as done.
The same case reports that the company’s rework rate fell “由17%降至5%”, from 17 percent to 5 percent, and that on its separate new-retail video-review platform, human time per video-review task fell “8小时降低至6.3小时”, from 8 hours to 6.3. These are the company’s own figures, not an independent audit, and the case does not say how much of the change came from AI tools. But they describe a cost the token bill leaves outside the frame. Generating work is one cost. Making sure the work is right is another.
The token bill itself keeps falling. In November 2024, Andreessen Horowitz’s Guido Appenzeller wrote that the cost of an LLM of equivalent performance was falling 10x every year. If that bill were the cost of AI work, AI work would be ten times cheaper each year.
Read that way, falling token prices flow through to the cost of AI work, to customers as lower prices or to vendors as wider margins. The cost of AI work has two parts. One is generation, what the tokens pay for when a model produces the work. The other is review, the person’s time spent accepting it. These are the two terms of the cost, and the token bill covers only the first.
That reading misses the larger term. When a person’s review is part of accepting the work, the cost of accepted work moves with review time far more than with the token price. How much of a token cut passes through depends on generation’s share of the total cost. In one software-fix illustration built from published inputs, that share is under 5%: a 100x cheaper model makes a reviewed fix about 4% cheaper, and halving the review cost makes it about 48% cheaper.
Episodes 1 to 4 priced AI work as the cost of a successful attempt, the Answer Price: the cost of all attempts divided by the number that succeeded. The attempt still has to be accepted by someone.
The labelling company’s checks have a counterpart in software. A developer reading a patch before merging it is the same kind of cost: a person accepting work.
One fix, two costs: review is 96% of it
Start with the condition. A token cut can remove only the part of the cost that tokens make up. If generation were half the cost, a 10x cut would take out 45% of it. If generation is 4%, the same cut takes out under 4%.
The example has two terms, from two studies.
Generation comes from Agentless, a software-repair system. It was tested on SWE-bench Lite, a set of 300 real software problems, mostly bug fixes, each with tests that check whether a submitted patch works. Agentless tried all 300 at an average inference cost of $0.34 per issue, at 2024 prices, so its inference bill was about $102. It solved 82 of them, which means each solved issue carries about $1.24 of inference. “Solved” means only that the patch passes the benchmark’s tests. No person has accepted a patch yet.
Review comes from METR’s 2025 trial, in which 16 experienced developers completed 246 tasks averaging 2.0 hours, each randomly assigned to allow AI or not. In a subset of 44 AI-allowed issues with labelled screen recordings, developers spent about 9% of their time reviewing and cleaning AI-generated output. Applied to a 2.0-hour task, 9% is 10.8 minutes. METR paid its participants $150 an hour, so that comes to $27.00.
At $150 an hour, review would have to fall to about 30 seconds before it cost as much as generation.
METR paid $50 an hour in a later study, and at that rate review is still about 88% of the cost.
The field gains are real, and they do not say which term moved
Two findings point the other way. Cui, Demirer, Jaffe, Musolff, Peng and Salz pooled three field experiments at Microsoft, Accenture and a Fortune 100 company, with 4,867 developers, and estimated that using an AI coding assistant increased completed tasks by 26.08%, with a standard error of 10.3%. These are counts of completed tasks, and the estimate is noisy. In February 2026, METR said developers are likely more sped up now than in its early-2025 estimates, while warning that selection effects make its new data an “unreliable signal”.
Both are gains in real work. Neither says which term produced them. A developer who completes more tasks may have generated faster, reviewed less per task, or both. A buyer shown a gain like this should ask the vendor which term it cut. A gain that comes from cheaper tokens is worth at most generation’s share of the bill.
Machine acceptance cuts review only as far as its rule can be trusted
Four cases show review moving from a person to a rule. Each moves the uncertainty from a person’s minutes to the rule.
Tests as the rule. In a 2022 experiment reported in 2023, participants who completed a JavaScript HTTP-server task took 55.8% less time on average with GitHub Copilot than without. Completion meant the first commit that passed all 12 tests.
Tests as a filter. Agentless used the repository’s tests to select among candidate patches, which raised solved issues from 78 to 82 of 300 at the same $0.34 average inference cost.
A model as the judge. On 100 expert-annotated Gherkin test scripts, GPT-4o Mini’s retry-adjusted cost was $1.01 per 1,000 evaluations, and its scores differed from the experts’ by 6.07 points on average, on a scale of 0 to 100. The judge is cheap, and the uncertainty now sits in its scores.
A person with an assistant. In a 2023 study of 47 participants, 36% of those with an AI assistant wrote SQL-injection-vulnerable solutions, against 7% of those without. The participants could edit the assistant’s output before submitting it, and 87% of the secure responses needed significant edits. Being free to edit the output did not keep more of them from submitting vulnerable code.
A vendor that sells machine acceptance has not removed the cost of being sure. It has moved that cost into the rule’s error rate, and the buyer should price the rule as carefully as the minutes it replaces.
Under these terms, the user carries the error that gets through
Whatever checking misses becomes someone’s cost. Two sets of published terms show how they allocate it.
DeepSeek’s Chinese terms of use, dated September 5, 2025, state that the company does not bear liability for harm caused to the user by using the service, with exceptions for statements in the service interface, separate agreements and legal requirements. Anthropic’s commercial terms, effective June 17, 2025, cap ordinary liability at the fees paid in the previous 12 months. They separately indemnify customers against third-party intellectual property claims, which fall outside the cap, with exclusions.
One set is consumer terms and the other commercial. Both leave the user holding the loss.
Regulation sets narrow duties. China’s 2023 interim measures require providers that annotate data during generative-AI development to sample-check the accuracy of the annotation. The EU AI Act requires high-risk systems to be designed so that natural persons can effectively oversee them.
Where the user carries the error, the buyer pays for the review that catches it.
China is building the checkers
A January 2025 Chinese policy aims for the data-labelling industry to grow at a compound annual rate above 20% to 2027. In the case from the opening, the checking is people’s work: inspectors, a team leader, then the customer. A Shijiazhuang human-resources bureau item from April 2025 says 65% of one programme’s graduates entered large-model data-service companies at an average monthly salary of 5,500 yuan. Where that industry grows, the checking it sells grows as people’s time. The 5,500 yuan is one city’s trainees’ monthly salary, not a cost per checked item.
For work a person must accept, watch review minutes per accepted unit
A buyer who budgets from the price list sees the token line fall while the total barely moves. In the example, the first 10x saves $1.12 on a $28.24 fix, and the next 10x saves another 11 cents. Halving the review saves $13.50.
Cutting review time lowers the cost of accepted work for whoever bears the review, but only if review is cut at the same quality, with no more errors getting through. Whether the buyer or the vendor keeps a real saving depends on pricing, competition and who controls the workflow. A vendor whose case rests on a token discount is cutting the smaller term. One that shortens its buyer’s review at the same quality is cutting the larger.
One question separates the claims. How many minutes of whose review does one accepted item take?
The labelling company runs four checks before its customer accepts the work. A 100x cut in the token price touches none of them.