Emergent Abilities

Emergent abilities describe cases where task performance appears to change from weak or near-random to clearly useful as model scale increases. What counts as a sudden change depends on metrics and experimental conditions.

Original definition

Wei and colleagues' 2022 paper calls an ability emergent when it is absent in smaller models, appears in larger ones, and cannot be simply extrapolated from smaller-model performance. These are observations in particular model families and tasks, not a promise that every increase in size adds a new capability.

The metric debate

Schaeffer and colleagues argued in 2023 that nonlinear or discontinuous metrics can make smooth improvements look abrupt. Alternative metrics produced smoother curves for some studied abilities. This challenges interpretations of some observations; it does not establish that every scale-related capability is imaginary.

An invented illustration

Suppose five steps must all be correct to receive credit. Under an independence assumption, increasing per-step accuracy from 0.5 to 0.8 raises full-task success from about 3.1% to 32.8%. That steep change alone does not demonstrate an internal discrete transition.

Reading claims

Our suggested review checks model families, training data, prompts, sample sizes, uncertainty, and agreement between exact-match and continuous metrics. One plot is insufficient to equate emergence with AGI.

Sources

• Emergent Abilities of Large Language Models — original definition and observations.
• Are Emergent Abilities of Large Language Models a Mirage? — metric-dependent interpretations.