The statistical approach won by outgrowing the argument rather than settling it

The dispute was philosophical before it was practical

One tradition held that language, vision and reasoning have structure, that the structure can be described, and that a system should embody the description. The other held that structure can be left implicit, and that a system should instead estimate the probability of things from large quantities of examples.

The objection to the second position was not stupid. Counting how often things co-occur seemed obviously insufficient for phenomena that are productive and rule-governed, and there were principled arguments that no amount of observed data determines the underlying system. Serious people made those arguments and some still do.

The objection to the first was equally reasonable. Nobody had succeeded in writing down the rules for anything as messy as ordinary language, decades of trying had produced systems that broke on real input, and the exceptions always outnumbered the regularities.

Speech recognition was the first decisive battleground

Speech is a good test case because the rules approach is intuitive — phonemes, syllables, words, grammar — and because performance can be measured objectively against a transcript. Through the nineteen seventies and eighties, groups pursuing statistical modelling of the acoustic signal steadily outperformed groups encoding linguistic knowledge.

The pattern that emerged became familiar. A team would add more linguistic structure and gain a little. Another team would add more data and more parameters and gain more. Over successive evaluations the second strategy won consistently enough that the field reorganised around it.

This is where a recurring maxim of the discipline comes from: that methods leveraging computation and data tend, over long periods, to overtake methods encoding human insight about a domain. It is stated as an observation about history rather than a theorem, and how far it generalises is debated.

Three material things changed and none was an idea

The first was data. Digitised text and images accumulated in quantities that would have been unimaginable to the earlier generation, and much of it came with usable labels attached as a by-product of how people organise information.

The second was hardware. Processors built for rendering graphics turned out to be extremely good at the dense matrix arithmetic that neural networks require, and they were mass-produced for an unrelated consumer market, which made them comparatively cheap. A great deal of what followed rests on that coincidence.

The third was a set of unglamorous engineering improvements — better initialisation, better activation functions, techniques that keep training stable in deep networks — that individually look minor and collectively made it possible to train networks that had previously refused to converge. None of these is a philosophical answer to anything.

What the winning side gave up

The trade was explicit and it is worth stating without euphemism. Symbolic systems offer inspectability, guarantees, and the ability to prove that certain outputs cannot occur. Statistical systems offer coverage, graceful behaviour on unanticipated input, and performance that improves with more data rather than more labour.

Everything in the current list of complaints about modern systems is on the ledger of that trade. Unpredictability, the difficulty of explaining a decision, the impossibility of guaranteeing a behaviour, the dependence on data whose contents nobody has fully surveyed — these are not incidental defects. They are the price that was paid, knowingly, for the capabilities.

Whether it was a good trade depends entirely on the application, which is why the interesting arguments now happen at the level of specific deployments rather than at the level of approaches.

The argument is not actually over

It is easy to read the current landscape as a settled victory, and some researchers do. Others point out that the winning approach has not delivered the reliability, verifiability or sample efficiency that the earlier tradition offered, and that these remain requirements in many domains rather than preferences.

The strongest version of the sceptical position is not that statistics cannot work. It is that the observed successes come with a data appetite so far beyond what a human learner requires that something important must be missing from the account, and that the missing thing may be closer to structure than to scale.

Nobody can settle this from where we stand, and the honest thing is to note that the last two confident consensus positions in this field were both overturned. The material explanation for the current arrangement — cheap parallel hardware, abundant data, workable training methods — is solid. Any explanation that treats it as an intellectual verdict is doing more than the evidence permits.