Limits & Risks
What the technology cannot do, and the failures that follow from assuming it can.
20 articles · page 1 of 2
Limits & Risks
Being confidently wrong is the default behaviour, not a malfunction
Nothing in the way these systems are built distinguishes a supported statement from an unsupported one, so the surprising thing is not the false answers but how many true ones there are.
By Manish Trivedi · · 4 min read
Limits & Risks
A system’s account of its own reasoning is a reconstruction, not a record
· 4 min read
Limits & Risks
These systems have no memory, and every workaround is something else wearing the name
· 3 min read
Limits & Risks
A model cannot be made to forget, and that breaks how deletion is supposed to work
· 3 min read
Limits & Risks
A deployed model decays because the world moves and the parameters do not
· 3 min read
Limits & Risks
The human reviewing the output is doing a harder job than the design assumes
· 3 min read
Limits & Risks
A system that can be fooled on purpose is a different problem from one that errs
· 3 min read