Model Distillation Explained: Your Prompts Train Rivals
Anthropic's Sep 10 report names Alibaba, Moonshot, and DeepSeek in industrial scale distillation. How it works in plain words, plus three defenses to enable.
On this page
Your API calls may be tutoring a competitor. On September 10 Anthropic published a 154 page threat report naming Chinese labs that allegedly routed millions of user requests through Claude, harvested the answers, and served them as their own models' work. Here is how distillation works in plain words, what the receipts show, and three defenses worth enabling.
Takeaways
Distillation means learning from outputs: legitimate on your own data, illicit at industrial scale through fake accounts. Alibaba scale: nearly 3 million exchanges per day from 3,500 plus accounts, aimed at Opus 4.6 and 4.7 reasoning transcripts. Moonshot and DeepSeek: 23 million plus exchanges over May to July, and 12.1 million in a 14 day July window, some served to users as Kimi output. Defense trio: preserved thinking, zero data retention where offered, and routing hygiene so your prompts never pass through models you did not choose.
How does distillation work in plain words?
Imagine hiring a brilliant tutor, recording every explanation, then selling the recordings as your own course. In AI terms: send thousands of hard problems to a frontier model, collect the step by step reasoning, and train your own model on those transcripts. Done openly with permission it is standard practice. Done through 5,380 fraudulent accounts relaying nearly 300,000 customer requests in ten days, with answers displayed as though they came from your own Kimi model, it is what Anthropic calls illicit distillation: capability extraction without the safeguards built into the original.
legit: your prompts -> frontier model -> your product, with permission
illicit: fake accounts -> frontier model -> harvested transcripts -> rival model
signal: maxed out new accounts, agentic plus coding plus long horizon queries
risk: your users names, emails, and company data ride along in the promptsWhat counts as evidence here
Anthropic says operators affiliated with Alibaba targeted chain of thought transcripts for agentic tasks, software engineering, kernel work, and long horizon jobs. Moonshot allegedly evolved from text distillation to extracting logical frameworks for tool use and computer vision. CISA advisory AA26-251A adds StepFun and Z.AI distillation of Claude plus GPT series models into its guidance.
Which defenses actually help?
Turn on preserved thinking
Anthropic's anti-distillation safeguard stops API users from editing Claude's prior context to extract reasoning. It ships on Fable 5.1 and Opus 5.5 for accounts created on or after August 31: check the preserved thinking docs and update integrations.
Use zero data retention where offered
Opus 5.5 is available with zero retention like prior Opus models. If your prompts never persist, they cannot become training material later through any path.
Watch your own usage shape
CISA flags the tells: anomalous prompts, maxed out new accounts, enterprise scale throughput from fresh subscriptions. Monitor subscription to usage ratios on your own keys the same way.
Ask vendors where prompts travel
If you build on Moonshot, DeepSeek, or any aggregator, ask whether requests route through third party frontier models and where that data lands. The CAC probe centers on exactly this question for police, military, and state linked data.
Should you stop using the named models?
That is a risk call, not a moral one. The CAC investigation could reshape cross border data flows for Chinese labs, and Moonshot confidentially filed for a Hong Kong IPO September 3 targeting $3 billion at a $50 billion valuation, so any disclosed probe lands in its prospectus. If your prompts carry customer PII or company secrets, prefer vendors with zero retention and clear routing statements until the dust settles. If you run evals, pin model IDs and re run after any routing change.
Does distillation hurt small builders?
I only make thousands of calls: am I exposed?
Your risk is inbound, not outbound: someone else distilling does not bill you. The exposure is your prompt contents traveling further than you agreed. Strip PII before sending, keep secrets in local tools, and treat every third party model as a surface your data touches.
Why do labs distill instead of training from scratch?
Cost and time. Harvesting reasoning transcripts buys months of capability for the price of API calls, minus the safety work. Anthropic notes none of the misuse cases ran on Fable or Mythos class models except one distillation case: attackers chase the newest reasoning because that is where the value concentrates.
As of September 28, 2026: the valuable layer is model behavior, and actors route around brand boundaries to get it. Enable preserved thinking, demand routing clarity, and strip secrets before sending. Next, read why AI got cheap and fast to see how legitimate cost pressure differs from extraction.