A computer said it was 93 percent sure. It was wrong, and a guy ended up in jail anyway.
Security Boulevard's breakdown of facial recognition's track record in policing reads less like an edge case and more like a pattern.
Two Cases
Robert Dillon, a 52-year-old commercial crabber from Fort Myers, Florida, was arrested after Florida's FACES system returned a "93 percent match." He lived more than 300 miles from the scene and had never been there.
Porcha Woodruff was eight months pregnant, getting her kids ready for school, when Detroit police arrested her for carjacking and robbery based partly on a facial-recognition lead. Charges were later dismissed.
It's Not Evenly Distributed, Either
NIST's 2019 demographic-effects report found some facial-recognition algorithms had false-positive rates for Asian and African American faces 10 to 100 times higher than for Caucasian faces.
The Real Pattern
The pattern the piece identifies: departments increasingly treat an algorithm's confidence score as a shortcut to probable cause, rather than as one unverified lead among many. When the match is wrong, the accountability gets diffused -- "the computer picked him," as the piece puts it, isn't supposed to be a defense.
If your local police department uses facial recognition, this is worth knowing: exactly how it's being used, and what independent verification (if any) happens before someone gets arrested on the strength of a percentage.