Proxy discrimination is the AI industry's elephant in the room, and it's shockingly simple: machine learning systems discover patterns in data that correlate with protected characteristics like race, age, gender, or socioeconomic status then use those patterns to make decisions, effectively discriminating without ever explicitly considering the protected trait itself. An AI doesn't need to know someone's race if it knows their ZIP code, shopping habits, and speech patterns. The algorithm finds the shortcuts, the proxies, and exploits them ruthlessly. Here's how it works in practice. Train an AI on historical crime data to predict future offenses, and it will absolutely learn that certain neighborhoods, certain names, certain patterns of movement correlate with arrests. But arrests aren't crime they're policing patterns, shaped by decades of discriminatory enforcement. The AI learns the bias baked into the data, then amplifies it. A 2019 study published in Science Advances found that commercial facial analysis systems had error rates of up to 34.7% for darker skinned women compared to 0.8% for lighter skinned men. The algorithms weren't explicitly trained to be racist they just learned from biased datasets where lighter skin was overrepresented and better lit. The UK's enthusiasm for AI powered knife crime surveillance is racing toward this exact disaster. Multiple police forces have deployed or tested systems that claim to identify potential knife carriers through CCTV analysis, gait recognition, and behavioral pattern matching. The Metropolitan Police Service has experimented with predictive analytics for serious violence. South Wales Police ran facial recognition trials. The logic sounds appealing: use AI to spot suspicious behavior, predict hotspots, identify individuals likely to carry weapons. The reality is mathematical racism at scale. These systems inevitably learn proxies for race, class, and age because those characteristics correlate with who gets stopped, searched, and arrested under current policing practices. Young Black men in London are statistically more likely to be stopped and searched not because they commit more knife crime, but because they're targeted more aggressively. Feed that data to an AI, and it learns: flag young Black men. The algorithm doesn't understand systemic racism or police bias. It just sees patterns and optimizes. Research by the Ada Lovelace Institute in 2022 found that predictive policing tools in the UK consistently overpoliced areas with higher proportions of ethnic minorities, creating feedback loops where biased predictions led to biased enforcement, which generated biased data, which reinforced biased predictions. The technical problem is insidious because the proxies are often legitimate predictive features in isolation. Clothing style, time of day, location, movement patterns these things do correlate with various behaviors. But when combined, they create discriminatory outcomes because they're standing in for protected characteristics. An AI might learn that people wearing certain brands of clothing in certain neighborhoods at certain times are worth flagging. It's not explicitly racist, but the pattern it's learned maps almost perfectly onto race and class. Remove one proxy, and the algorithm finds another. Researchers at MIT found that even when race was explicitly excluded from criminal justice algorithms, the systems reconstructed racial bias through proxies like ZIP codes, first names, and prior contact with police. Now here's where it gets uncomfortable: what if new data showed a spike in knife crime among young white males, and the AI started targeting them instead? Would that still be racist? Absolutely yes and this is the critical point that exposes why these systems are fundamentally broken. Discrimination law protects all races, not just minorities. If an AI system started disproportionately flagging young white men based on demographic patterns rather than individual behavior, that would be textbook proxy discrimination, just pointed in a different direction. The Equality Act 2010 in the UK explicitly protects against discrimination based on race (including white), age, and sex. A system that treats any racial or age group as inherently suspicious violates those protections, regardless of which group it's targeting. But the deeper issue is that such a system would still be learning from biased enforcement patterns, not underlying crime rates. If police suddenly started aggressively stopping and searching young white males in affluent areas, the arrest data would spike, the AI would learn the pattern, and the feedback loop would begin just with a different demographic. The fundamental flaw isn't which group gets targeted; it's that the system conflates police activity with criminal activity. Crime happens everywhere. Policing is selective. Any AI trained on selective policing data will learn to be selectively discriminatory. Whether it's discriminating against young Black men in Brixton or young white men in Cornwall, it's still using demographic proxies instead of individualized evidence, and that's illegal under UK law. The surveillance dream is collapsing in real time. In 2023, the UK's Information Commissioner's Office and the Equality and Human Rights Commission jointly warned that many AI systems being deployed by police forces likely violated data protection and equality laws. Amsterdam banned facial recognition in public spaces after concluding the discrimination risks were unmanageable. San Francisco, Boston, and Portland banned government use of facial recognition entirely. These aren't anti technology decisions they're admissions that the technology fundamentally doesn't work without embedding and amplifying existing inequalities. The fundamental issue is this: AI systems are prediction machines trained on historical data to forecast future outcomes. But if your historical data reflects a discriminatory society and it does your predictions will perpetuate and amplify that discrimination. There's no algorithmic fix for this. You can't engineer your way out of biased training data when the bias is society itself. The knife crime surveillance systems being proposed will absolutely identify patterns. They'll just be patterns of who gets policed, not who commits crimes. And when those systems deploy, they'll concentrate surveillance, stops, and arrests even more heavily on the communities already over policed, creating the exact feedback loop that makes the problem unsolvable.
💻 technology
AI's Dirty Secret: Proxy Discrimination Will Kill Surveillance Dreams
Artificial intelligence looks brilliant in demos but catastrophically fails when deployed in the real world. The culprit? Proxy discrimination - where AI learns to be racist, classist, and ageist without anyone explicitly teaching it to be. And it's about to blow up spectacularly in surveillance projects.
My Take
The AI industry is selling police forces a product that's mathematically guaranteed to be racist, and nobody wants to say it out loud. These aren't bugs they're features of how machine learning works. You cannot train an algorithm on biased data and expect unbiased outcomes. It's like trying to build a level house on a tilted foundation. Every sophisticated technical solution just finds more creative ways to discriminate. What enrages me is the willful ignorance. Police forces and government officials are buying these systems knowing or choosing not to know that they'll disproportionately target Black and brown communities. They're hiding behind the objectivity myth of AI, pretending that because a computer made the decision, it's somehow fair. But algorithmic discrimination is still discrimination, even if it's laundered through mathematics. And when these systems inevitably produce scandals wrongful arrests, discriminatory stop and search patterns, civil rights lawsuits the same officials will act shocked, blame the vendor, and promise better oversight. The pattern is depressingly predictable. The knife crime surveillance projects will fail. Not because the technology is immature it's mature enough to be dangerous. They'll fail because they're trying to solve a social problem with a technical solution that encodes and amplifies the social problem. When the lawsuits start rolling in, when the discrimination becomes undeniable, when the public backlash hits, these programs will be quietly shelved. But not before they've done real harm to real communities. That's the cost of our AI enthusiasm: experiments run on the bodies of the already marginalized.
What Happens Next
The first major discrimination lawsuit against a UK police force using AI surveillance will land within 18 months. It won't come from a high profile case it'll be a teenager from Hackney or Brixton, stopped repeatedly by officers acting on algorithmic risk scores, who sues under the Equality Act 2010. The case will reveal internal police documents showing the system flagged him based on location, clothing, and social media patterns that perfectly correlate with race. The force will settle quietly, but the internal data will leak. Meanwhile, insurance companies and private security firms are already buying similar technology, and they'll face even less scrutiny. By 2027, we'll see proxy discrimination scandals in hiring algorithms, credit scoring, and health insurance risk assessment all using the same flawed logic as the policing systems. The pattern is already emerging: AI systems deployed in the US for bail decisions, parole recommendations, and recidivism prediction have all been found to discriminate against Black defendants through proxy variables. The UK is simply running a few years behind on the same disastrous timeline. The real question is whether regulators will act preemptively or wait for the body count of ruined lives to mount. The European Union's AI Act, which came into force in 2024, classifies certain AI systems as 'high risk' and bans some uses entirely but the UK, post Brexit, is pursuing a lighter touch regulatory approach. That's going to cost lives and liberty. By the time the scandals force action, thousands of people will have been subjected to discriminatory surveillance, stops, searches, and arrests based on algorithmic bias. The technology will be banned or heavily restricted, but not before it's done its damage. The only uncertainty is how many communities will be guinea pigs in this failed experiment.
What History Tells Us
This isn't the first time new technology has promised to make policing 'objective' while actually encoding discrimination. In the 1930s, criminal anthropology and phrenology claimed to scientifically identify criminal tendencies through skull measurements and facial features. These pseudosciences were used to justify racial discrimination and eugenics policies. They were eventually discredited, but not before causing immense harm. More recently, the 1994 Crime Bill in the United States introduced 'evidence based' risk assessment tools that were supposed to reduce bias in sentencing. Instead, these tools systematically recommended longer sentences for Black defendants because they were based on historical data reflecting discriminatory enforcement and socioeconomic inequality. A 2016 ProPublica investigation found that COMPAS, one of the most widely used criminal justice algorithms, was twice as likely to falsely flag Black defendants as high risk compared to white defendants. The UK is now repeating this exact mistake with more sophisticated technology but the same flawed logic: using biased historical data to predict the future, then acting surprised when the predictions perpetuate bias.
Market Impact
AI surveillance companies are heading for a regulatory cliff that will devastate their valuations. Palantir Technologies (PLTR), currently trading around $22, generates significant revenue from government contracts including UK policing and intelligence agencies. As discrimination scandals mount and regulations tighten, these contracts will face cancellation and legal challenges. The stock could see 20 30% downside in the next two years as the proxy discrimination crisis becomes undeniable. Bearish on Clearview AI and similar facial recognition startups most are private, but investors should watch for any IPO announcements and avoid. Bullish on traditional security and safety companies that focus on physical infrastructure rather than AI prediction. Companies like Johnson Controls International (JCI) and Honeywell (HON) that provide physical security systems without the algorithmic discrimination risk look relatively safer. The broader AI sector faces a credibility crisis. While companies like Microsoft (MSFT) and Google (GOOGL) have diverse revenue streams, their AI ethics reputations are already damaged by internal conflicts over bias and discrimination. Expect increased volatility and regulatory risk premium across the AI sector as the limitations of current machine learning become undeniable. The 2025 2027 period will likely see the first major wave of AI discrimination lawsuits, forcing companies to either abandon high risk applications or face massive legal exposure.