Definition
A model extraction attack, also called model stealing, recreates or closely approximates a proprietary machine learning model by repeatedly querying it and analyzing the outputs. The attacker does not need the original architecture, weights, or training data. Treating the target as a black box, they submit carefully chosen inputs, record the responses, and use those input-output pairs to train a substitute model that imitates the original's accuracy and decision boundaries.
Where the risk lives
Machine-learning-as-a-service is the natural target, because commercial models are exposed through prediction APIs. The richer the API response, the cheaper the theft: high-precision confidence scores and full probability vectors leak far more about the model than a bare label. Google and security researchers have repeatedly warned that as more capable models go behind public APIs, extraction becomes a growing intellectual-property and security threat.
Why a stolen model is dangerous
Beyond the obvious theft of a costly asset, a cloned model lets an attacker probe offline for weaknesses, craft adversarial inputs, and design evasion attacks that transfer back to the original system. Extraction therefore acts as a force multiplier for other attacks rather than an endpoint in itself.
Defensive measures include rate limiting, returning coarser outputs, watermarking model responses, and monitoring for the dense, systematic query patterns that extraction requires. For sovereign operators who run weights locally instead of behind a metered API, the extraction surface largely disappears. Compare with our entries on model inversion and adversarial examples.
In Simple Terms
A model extraction attack, also called model stealing, recreates or closely approximates a proprietary machine learning model by repeatedly querying it and analyzing the outputs.…