Emergent Behavior

As models grow, some abilities improve smoothly and others appear to switch on. A model that could not do multi-step arithmetic at one size can do it at the next, with no change to the training objective.

How much of this is genuine emergence and how much is an artefact of using pass-or-fail metrics is actively disputed. Either way it is why capability evaluation is now a research discipline of its own rather than a checkbox at the end of training.