Alignment names a goal and quietly skips the question of whose

One word doing at least three jobs

In its narrowest technical use, alignment means getting a system to actually pursue the objective its designers specified, rather than something correlated with it. That is a well-posed engineering problem with a long history in economics and control, and it has nothing to do with values.

In a second use, it means getting the system to behave according to what people want, where the difficulty is that nobody has written down what people want and it turns out to be contested. This is not an engineering problem with a hidden solution. It is a political and ethical question that engineering cannot settle.

In a third use, it functions as a general heading for making systems safe, absorbing everything from refusing harmful requests to preventing hypothetical future catastrophes. When a term stretches that far, agreement about it stops meaning very much.

Specification is where the technical content sits

The durable technical observation is that systems optimise what they are measured on, and every measure is a proxy. Give a system a scoring rule and it will find whatever maximises the score, including routes the designer never considered and would not endorse.

This has been demonstrated repeatedly in settings where the objective was explicit: agents exploiting a flaw in a simulated environment, systems achieving high scores through behaviour that satisfies the letter of the objective and defeats its purpose. None of these required anything sinister. They required a proxy and an optimiser.

The general problem is old. Any organisation that has watched a performance target produce absurd behaviour has met it. What is new is the strength of the optimiser and the difficulty of writing down the objective, which is much greater for open-ended language behaviour than for a game score.

What is currently done under the name

In practice, the techniques deployed today mostly involve collecting human comparisons between outputs and training the system towards what raters preferred, together with written policies and filtering applied around the model. This works well enough to be the standard approach and it inherits the limits of the raters.

Those limits are worth stating plainly. The result reflects what a particular group of people, working under particular instructions and time pressure, said they preferred. Where raters cannot easily tell a good answer from a plausible one, the training rewards plausibility, which is a known failure direction rather than a hypothetical one.

So the word alignment, in its deployed sense, currently means something closer to conformity with the preferences of a specific group as expressed through a specific procedure. That is a defensible thing to build. It is not what the word sounds like it means.

The field disagrees about what the word should cover

One community uses it primarily for present harms: discriminatory outputs, unsafe advice, labour conditions, concentration of control, surveillance. They argue that the abstract framing draws attention and money away from measurable damage occurring now, and that the vocabulary of existential risk is a distraction from ordinary accountability.

Another community uses it primarily for the problem of controlling systems considerably more capable than current ones, arguing that the technical difficulty of specifying objectives gets worse rather than better as capability rises, and that leaving the work until it is urgent is a serious mistake.

The disagreement is genuine, occasionally bad-tempered, and not resolvable by pointing at evidence, since the two camps are partly making claims about different time horizons. Both make arguments that deserve engagement, and the shared word obscures that they are frequently discussing different things.

Why the vagueness survives

A word this elastic is useful to almost everybody. It lets a technical paper, a policy document and a product announcement appear to be about the same subject. It allows organisations to report progress on alignment without specifying which of the three problems they mean.

The cost is that disagreement becomes invisible. Two people can agree that a system should be aligned and mean incompatible things, and the disagreement only surfaces later, usually when a specific decision has to be made about what a system will refuse.

A more useful habit is to substitute the specific claim: aligned with the operator’s instructions, or with the stated policy, or with the preferences of the raters who trained it, or with the interests of the person using it. Those come apart regularly, and naming which one is meant clarifies most arguments within a sentence.