Emergent Abilities

Emergent abilities are capabilities that are absent in smaller models but appear in larger ones, such that they cannot be predicted by simply extrapolating the performance of smaller models. As described by Wei and colleagues in 2022, what makes them striking is their apparent sharpness — a task the model essentially cannot do suddenly becomes one it can do once scale crosses some threshold — and their unpredictability, since the threshold is not known in advance.

The mirage critique

The picture is contested. A 2023 Stanford analysis argued that many reported emergent abilities are an artifact of the chosen evaluation metric rather than a genuine discontinuity in the model. When a task is scored with an all-or-nothing metric (exact match), progress looks like a sudden jump; when the same outputs are scored with a smooth, continuous metric, the improvement appears gradual and predictable. Under this view, the model is improving steadily all along — the "emergence" is in how we measure, not in the model.

Why it matters for builders

The debate is not merely academic. If genuine emergence is real, larger models may acquire dangerous or surprising capabilities without warning, which is central to AI safety arguments. If emergence is largely a metric artifact, capability growth is more forecastable and easier to govern. For someone choosing which open-weight model to self-host, the practical lesson is to evaluate models on the specific tasks you care about with metrics that reflect real use, rather than trusting headline benchmark cliffs.

Emergence is most discussed in the context of frontier models and is one of the phenomena that makes scaling laws harder to interpret at the task level.