Imagine asking a map-maker to draw a city they have never visited. They are locked in a windowless room. All they have are maps that are fifty years old.
Anyone following the result would stop at walls knocked down years ago and look for roads that no longer exist. The map-maker has not made anything new. They have copied the old maps.
That analogy, from an award-winning essay by the young British science writer Adrian Dredge — winner of the 2026 Young Science Writer Award from the Association of British Science Writers — is the clearest short statement of why researchers are warning that AI in policing risks deepening discrimination against minority groups.
How the loop closes
Now hand the map-maker a list of past crimes and where they took place, and ask them to shade in where the next ones are likely to happen. Police then patrol the shaded areas.
More crime is recorded there — because more officers are there to see it. Areas outside the shading appear quieter, because fewer people are looking. When the next map is drawn, the same areas are shaded darker still.
The crime figures, in other words, are not a full list of crimes. They are partly a record of where police have chosen to look.
What was found inside a real force
Sarah Brayne, a sociologist at the University of Texas at Austin, spent time inside the Los Angeles Police Department studying how it used big data. Her paper in the American Sociological Review found the force increasingly using data to predict crime rather than only to respond to it — and found the data feeding those systems skewed.
Crimes in public places were more likely to be seen and recorded. Police focus was disproportionately on black neighbourhoods. Once that enters the system, she writes, “the predictions appear impartial”.
The models, in her words, create “a feedback loop in which they not only predict events such as crime or police contact, but also contribute to their future occurrence.”
The data problem is worse than the model problem
Researchers at New York University gave the underlying material a name: dirty data. These are police records shaped by corrupt, biased or unlawful practices — rigged crime figures, arrests of innocent people.
In nine of the thirteen US areas they examined, police data from periods of unlawful or biased policing was available to train these systems. The system cannot tell that the data is flawed. It treats it as a pattern to repeat.
Their conclusion is pointed: technical fixes offered by the companies selling these systems are not enough unless someone independently checks how the data was produced in the first place.
The counter-argument, which is worth hearing
Not everyone accepts the old-map framing. Dr Dimitrios Kollias, associate professor in multimodal artificial intelligence at Queen Mary University of London, told the BBC the technology has moved on.
“The old-map analogy is useful for explaining how historical bias can be inherited, but it has its limits,” he said. “Modern AI systems do not simply retrieve or copy their training examples; they can combine learned information in new ways and generate outputs and solutions that were not explicitly present in those examples.”
But he immediately names the risk that replaces it. “As AI systems become more capable of generating novel decisions and acting with greater autonomy, we need to ensure that their behaviour remains safe, reliable and under meaningful human control.”
So the choice is not between a biased system and a clean one. It is between a system that repeats old prejudice and a system that invents new decisions nobody has audited.
It is not only policing
A 2019 study in the journal Science examined an AI system used in US healthcare to decide which patients should receive extra support. It worked by predicting whose care would cost the most.
Less was already being spent on black patients than on equally ill white patients, partly because of poorer access to care. By treating higher cost as a sign of greater need, the system read a symptom of unequal access as evidence of good health.
What to ask before any of this is bought
Dr Carolyn McKay, associate professor and researcher in digital justice at the University of Sydney Law School, puts the principle plainly: “The public functions of state policing ought to be subject to human oversight and scrutiny.” Her remedy is to have someone independent check the map before anyone uses it, and rub out the lines that are wrong.
For any institution here considering analytics of this kind — a police force, a bank scoring loan applicants, a platform ranking sellers — the question that matters is not how accurate the vendor says the model is. It is where the training records came from, who produced them, and under what conditions. A model cannot see past its data, and accuracy measured against biased records only tells you the model has learned the bias faithfully.
