Skip to content
What the technology actually is
AI Worth KnowingWhat the technology actually is

Proving a recording is genuine is harder than producing a convincing fake

The asymmetry between making synthetic media and verifying real media is structural, and it is reshaping evidence, journalism and everyday trust faster than any of those institutions can adapt.

By Zoya Rahman3 min read

Detailed close-up of a robot's mechanical components, emphasized by moody studio lighting.
Photograph by Pavel Danilyuk via Pexels
Editorial note. Independent reporting and analysis. Nothing here is sponsored or paid for. How we work.

The cost of production collapsed and the cost of verification did not

Producing a convincing photograph, voice or video of something that did not happen used to require skill, time and often money. That barrier is largely gone. What has not appeared is any correspondingly cheap way to establish that a given recording is authentic.

This is an asymmetry rather than a race, which is why it will not be resolved by better tools alone. Generation and detection are not symmetric problems: a generator only needs to fool the detectors that exist, while a detector needs to catch generators that have not been built yet.

The consequence is already visible in a form less dramatic than the one people anticipated. The dominant effect so far seems to be less about fabricated evidence being believed and more about genuine evidence being doubted, which is a slower and more corrosive change than a single convincing forgery would produce. It also spreads the damage across every recording rather than concentrating it on the faked one.

Detection works and then stops working

Detectors are typically classifiers trained to spot the statistical residue that generation processes leave — inconsistencies in noise, lighting, compression, or the fine structure of a signal. They can be quite accurate against the generators they were trained on and they degrade sharply against new ones.

They degrade for ordinary reasons too. Re-encoding, screenshotting, compression by a messaging platform and re-uploading all strip exactly the fine detail a detector depends on, and by the time a clip has travelled across three platforms much of the forensic signal is simply gone.

That produces a genuinely dangerous property: a detector that outputs a confident percentage while being poorly calibrated for the material at hand. A misleading verdict delivered with a number attached is worse than no verdict, because it launders uncertainty into apparent fact.

Provenance turns the problem the other way round

The more promising approach abandons detection and attempts authentication instead. Rather than examining a file for signs of fakery, record signed information at the moment of capture and carry it with the file through subsequent edits, so that a viewer can check the chain rather than the pixels.

The mechanism is sound and the obstacles are considerable. It requires adoption by camera manufacturers, editing tools and distribution platforms simultaneously; it must survive ordinary handling that strips metadata; and it must not become a mechanism for tracking the people doing the recording, which in many contexts would be dangerous.

There is also a deeper limitation nobody has solved. Provenance can establish that a camera recorded a scene at a time and place. It cannot establish that the scene was not staged, and a genuine recording of a staged event carries perfect provenance.

The liar’s dividend is the effect that actually landed

The anticipated harm was widespread deception by fabricated material. The observed harm includes something else: the mere existence of convincing fakes supplies anyone caught on record with a ready defence, and the plausibility of that defence does not depend on the recording being fake.

This corrodes something more basic than any individual falsehood. A shared assumption that recordings are broadly reliable underpins journalism, courts, insurance claims and ordinary personal accountability, and it does so silently. Once the assumption weakens, disputes that used to be settled by evidence become contests of credibility.

Courts have handled contested and manipulated evidence for a very long time, with authentication procedures, expert testimony and chains of custody. Those procedures are slow and expensive, which is precisely why they do not help with the enormous volume of ordinary material circulating outside any institution.

Where reasonable people disagree about the response

One position holds that technical standards for provenance, combined with platform requirements to preserve and display them, is the only durable answer, since detection is structurally losing. Another holds that this concentrates authority over what counts as authentic in a small number of platforms and manufacturers, which carries its own risks.

A third position argues the whole framing is too pessimistic, noting that societies have absorbed previous shocks to the credibility of media — retouched photographs, edited audio, staged newsreels — by developing institutional and social habits rather than technical fixes, and that this adjustment is under way.

All three have something to them. What is fairly clear is that the technical problem of separating real from generated material is not going to be solved by making better detectors, and any plan that depends on that happening should be treated sceptically.

Common questions

Are watermarks a solution?

They help in a limited way. Watermarks applied by cooperative generators can mark output as synthetic, but they only cover generators that choose to apply them, and many can be weakened by editing. They are a useful signal that something is synthetic, not evidence that something is genuine.

Can a detector be trusted enough to use as evidence?

Rarely on its own. Detectors are usually validated on clean material from known generators, and real disputed material is neither. Treating a detector score as decisive is a misuse of it, though it may reasonably form part of a broader investigation alongside contextual and corroborating evidence.

Has this already changed how courts handle recordings?

Authentication requirements for recorded evidence existed long before this technology and are being revisited in various jurisdictions, with different approaches and no settled consensus. The practical difficulty is less about the law than about cost, since rigorous authentication is expensive and most disputes cannot bear it.

In The Worldsynthetic mediadetectionprovenanceevidence
Zoya Rahman
Deputy editor, AI Worth Knowing

Zoya joined to cover how it works, in the world, limits & risks and stayed for the awkward questions and would rather show the working than assert the conclusion.