New Delhi [India], October 13 (ANI): Artificial intelligence research company Anthropic has introduced advanced security measures to curb illicit model distillation attacks against its Claude models, reporting an estimated 90 to 99 per cent drop in data harvested by unauthorized entities.
The developments, detailed in an Anthropic blog post, address growing concerns over large-scale industrial campaigns designed to siphon proprietary reasoning capabilities from frontier systems to train competitor models without authorization.
“Distillation attacks are covert, industrial-scale campaigns to extract the capabilities of one model and replicate them in another, without the authorization of the original model’s developers,” Anthropic stated in its blog post.
“They are often conducted using networks of fraudulent accounts. These attacks allow companies to mimic frontier capabilities for a fraction of the development time and cost,” the company noted.
The report highlights that such practices pose severe biosecurity, cybersecurity, and national security risks. Because distilled variants often enter circulation without proportional safety guardrails, bad actors can exploit them for malicious purposes.
The blog cites Chinese laboratory Zhipu, which extracted cyber capabilities from leading American models to develop GLM-5.3, a model that the US Center for Advancing Innovation and Standards for Super Intelligence assessed as the most cyber-capable open-weights model to date, capable of building end-to-end exploits.
In another instance, Anthropic noted that queries to Moonshot’s Kimi models were routed to Claude, including an inquiry linked to China’s People’s Liberation Army seeking analysis of closed-circuit television surveillance.
To counter these efforts, Anthropic focuses its technical defences primarily on shielding Claude’s internal reasoning transcripts, which distillers prize above final text outputs.
“Distillers aim to generate millions of responses from a larger, more capable model (in this case, Claude), then use those responses to teach a smaller model to mimic the more capable model’s behavior,” the company explained.
“The most valuable kind of data for this purpose are transcripts of Claude’s reasoning—the thinking process that Claude writes out before it responds,” Anthropic added.
The company’s defensive framework operates across three key fronts: controlling the movement of reasoning data across accounts, deploying a specialized two-stage classifier to block extraction attempts, and restricting persistent abusers through account bans, usage caps, and billing for blocked queries. The classifier operates with a reported false-positive rate below 0.1 per cent, matching the rigor applied to bioweapon defences.
Implementing these safeguards carries commercial and technical trade-offs, with Anthropic acknowledging that it has turned away significant revenue and trillions of tokens in traffic from malicious networks. The company emphasized that mitigating unauthorized extraction requires industry-wide collaboration among model creators and cloud inference platforms alike. (ANI)
Related reading
