Two communities worry about this technology and they have been arguing for a decade

Worry about thinking machines is older than the machines

Anxiety about automated systems making consequential decisions predates any working example by a long way. It appears in the earliest writing about computing, including from people building the first machines, and in the postwar cybernetics movement, whose participants wrote about automation displacing labour and about the consequences of delegating control.

That early strand was mostly speculative and mostly individual, since there was little to study. It established some of the vocabulary and none of the institutions, and it was pursued alongside the technical work rather than as a separate activity.

What changed was deployment. Once systems were making decisions about actual people, concern acquired something to examine, and examination is what produced organised research rather than commentary.

One track formed around systems already in use

From roughly the middle of the last decade, a body of work developed studying deployed systems and finding disparities in how they treated different groups, opacity in how decisions were made, and an absence of anybody to appeal to. The methods were empirical: audit a system, measure outcomes across populations, publish.

This work drew on existing traditions in law, sociology and civil rights research, and it produced its own conferences, its own terminology and its own connections to regulators. Its characteristic question is who is being harmed now and by what mechanism.

Its findings have been the most directly consequential, in that they informed the first serious regulatory attention and changed practice in several industries. They also concern relatively unglamorous systems — scoring, ranking, matching — rather than the ones attracting attention.

A recurring theme in that literature is that a technical fix is frequently the wrong shape of answer. Where a system is unfair because the institution around it was, adjusting the model addresses a symptom, and several of the most cited papers in this tradition are arguments about where the problem is actually located.

The other track formed around systems that do not exist yet

A separate community, developing over a similar period from earlier philosophical work, concerned itself with whether much more capable future systems could be made to pursue intended objectives at all. Its characteristic question is what happens if a system optimises something imperfectly specified and is capable enough for that to matter.

Its methods were initially theoretical and have become substantially empirical, producing work on specification, oversight of systems more capable than their overseers, and analysis of what trained models represent internally. Some of that research is technically deep and it is now well funded.

Its premise remains a claim about the future, which is the core of the objection to it. Reasoning carefully about a hypothetical is a legitimate activity and its conclusions are only as good as the premise, and the premise cannot be tested.

The friction between them is substantive, not merely tribal

The near-term community has argued that attention and money directed at speculative future risk is drawn away from documented present harm, and that dramatic framing serves the interests of organisations that would rather be regulated on hypotheticals than on current practice. That’s a serious argument and not obviously wrong.

The longer-term community has argued that present harms, while real, are being addressed by existing legal and regulatory machinery, and that a risk which cannot be corrected after the fact warrants attention before it arrives. That’s also a serious argument.

Both have made less serious versions of their case, characterising the other as a distraction or as unserious, and the public argument has often been conducted in that register. The underlying disagreement is about which risks are tractable and how to allocate limited attention.

The tracks have converged more than the rhetoric suggests

Work on making systems behave as intended, on evaluating what they can do, and on understanding their internals is cited by both communities, because a system that reliably does what was asked is a precondition for either set of concerns. Evaluation methodology in particular has become common ground.

Regulation has also forced convergence, since a law must cover both categories or explain why it does not, and drafters have generally attempted both. The resulting texts satisfy neither community fully and are the practical outcome the argument has produced.

A reader encountering an article about the risks of this technology is usually reading from one tradition without being told which. Knowing that there are two, with different premises and different evidentiary standards, makes a great deal of otherwise puzzling disagreement legible.