The AI Sentencing Machine Goes to Market

How a generation of proprietary recidivism-prediction tools spawned a compliance market, a regulatory reckoning, and an impossibility theorem no vendor has solved.

By Carry and Conquer Publications

Governments are not abandoning algorithmic justice. They are rebuilding it from scratch.

The old system is crumbling on two fronts. COMPAS, the dominant commercial recidivism-prediction tool used in more than a million U.S. court hearings across California, New York, Wisconsin, and beyond, has become the defining case study in what can go wrong when a proprietary black box acquires the force of law. A 2016 ProPublica investigation found that Black defendants were incorrectly labeled high-risk at nearly twice the rate of white defendants, while white defendants were more frequently mislabeled low-risk. Equivant, the renamed successor to the tool's original developer Northpointe, refuted the claims, arguing that the algorithm was well-calibrated across racial groups. Both sides were right in a precise, disqualifying sense: the two definitions of fairness they were invoking are mathematically incompatible. A criminal justice AI that satisfies one cannot satisfy the other. That contradiction is now the foundational problem for every government trying to design the next generation of these systems.

A Tool That Outran Its Oversight

COMPAS was developed in the 1990s by Tim Brennan, a professor of statistics, and Dave Wells, a corrections industry professional, under the Northpointe banner. Wisconsin formally integrated it into presentence investigation reports in 2012, giving its risk scores an institutional foothold in sentencing decisions. By 2016, the tool had been applied to over a million offender profiles. Its algorithm draws on 137 input features, collects data through a lengthy questionnaire, and outputs a score from 1 to 10 mapping to low, medium, and high risk. What the algorithm does not publish is how it weighs each of those 137 factors. The model is closed-source, proprietary, and beyond the reach of third-party audit.

That opacity became a legal problem in State v. Loomis, decided by the Wisconsin Supreme Court in 2016. The defendant challenged his COMPAS-influenced sentence on three due process grounds: the algorithm was proprietary and could not be examined; it violated the right to an individualized sentence; and it incorporated gender in its predictions. The court upheld the use of COMPAS, but required that judges be warned not to rely on it exclusively. The warning was not the same as a fix. The algorithm's internal logic remained inaccessible, and no mechanism existed for a defendant to contest a score whose derivation was a trade secret.

A subsequent study published in Science Advances made the situation more uncomfortable: COMPAS, despite collecting 137 features, was no more accurate than predictions made by people with no criminal justice experience. The same predictive accuracy could be achieved with two variables. The complexity and proprietary nature of COMPAS had been purchasing the appearance of rigor rather than the substance of it.

The Regulation Arrives

The European Union moved first at legislative scale. The EU AI Act, which entered into force in August 2024, classifies recidivism prediction and judicial decision-support systems as high-risk AI under Annex III, point 8. Full compliance obligations for those systems take effect in August 2026. The requirements are substantial: providers must conduct conformity assessments, maintain detailed technical documentation, implement bias monitoring, ensure human oversight, and make systems explainable to the affected individuals. The prohibition tier, effective since February 2025, bans AI that assesses criminal offense risk based solely on profiling or personality traits. The line between banned and heavily regulated runs through the concept of human augmentation: a system that replaces human judgment is banned; one that informs it while remaining accountable and contestable is regulated.

The distinction matters commercially. A generation of European govtech vendors and legal AI firms now faces the choice between rebuilding their products to satisfy Annex III requirements or exiting the market. Transparency documentation, bias auditing, and explainability infrastructure are not features these companies historically budgeted for. Building them is expensive, slow, and technically difficult. The firms that figure it out first will hold a durable advantage in a market where government procurement, once won, is slow to reverse.

The European Experiment

In parallel with the regulatory framework, the European Commission is funding proof-of-concept alternatives to the COMPAS model. FAIR-PReSONS, a two-year project coordinated by the University of the Aegean and funded by DG Justice, ran from June 2024 through May 2026. The project assembled partners across Greece, Portugal, and Bulgaria, collected and digitized prison and offender management data from each country, built those datasets into knowledge graphs, and used artificial neural networks combined with bias-mitigation algorithms to generate a recidivism assessment tool that aims at explainability rather than opacity.

The FAIR-PReSONS tool generates a risk score accompanied by guidance for interpretation. It tracks categories including social, educational, and national background alongside criminal history, and explicitly flags factors associated with protective outcomes, not just risk factors. The system was designed with a target of convincing at least 80 judges to use and validate it. On April 29, 2026, IPS Innovative Prison Systems introduced the tool during a training event in Lisbon, bringing together justice professionals, legal researchers, and institutional stakeholders for demonstrations and case studies. The project's explicit constraint distinguishes it from COMPAS: the tool is not intended for individual deterministic decisions. It is built to explore patterns and support reflection, not to issue verdicts.

That design choice reflects the mathematical reality that scholars have been documenting since 2016. The impossibility theorem of algorithmic fairness, formalized by researchers including Jon Kleinberg and Alexandra Chouldechova, establishes that under realistic conditions, equal calibration across demographic groups and equal false positive rates cannot be simultaneously achieved. Any deployed system encodes a value judgment about which form of fairness to prioritize. FAIR-PReSONS sidesteps some of this by positioning itself as exploratory rather than decisional, but that posture also limits its commercial and operational utility. A justice system cannot run on recommendations that it has been told not to act on.

The American Tangle

The U.S. picture is messier. All 50 states have implemented some form of risk assessment in criminal justice, deployed at various points across the pretrial, probation, parole, and sentencing pipeline. The Arnold Ventures Public Safety Assessment, built on data from 1.5 million cases and validated against 500,000 cases in held-out jurisdictions, has been deployed in over 200 jurisdictions and presents its factors transparently, a deliberate contrast to COMPAS. But transparency of factors does not resolve the underlying problem. A study of the PSA's implementation in Kentucky found that judges continued to apply racial disparities in bail decisions even after the algorithm's introduction, imposing cash bail on Black defendants with moderate risk scores more frequently than white defendants with identical scores.

State-level regulation of criminal justice AI expanded sharply in 2025, with 38 states passing more than 100 AI-related laws. That momentum now faces a collision with federal authority. On December 11, 2025, President Trump signed Executive Order 14257, asserting broad federal authority over AI regulation and establishing a Department of Justice task force, formally constituted on January 10, 2026 under Attorney General Pam Bondi, to challenge state AI laws in federal court. The order targeted state-mandated bias mitigation requirements, which it framed as potentially forcing AI systems to produce inaccurate outputs. Brookings Institution researchers who advise policymakers on criminal justice AI argued in April 2026 that the executive order, lacking legislative preemption authority, does not actually override state statutes, and that states retain the power to regulate their own law enforcement and justice agencies' use of AI.

The legal resolution is unclear. What is clear is that vendors selling into the U.S. criminal justice market now face regulatory bifurcation: EU requirements moving toward stringent transparency and contestability standards in August 2026, and a U.S. environment where state-level requirements are under litigation pressure and federal standards remain undefined.

What the Market Is Actually Buying

Beneath the political and regulatory noise is a technology procurement cycle. Criminal justice agencies across the U.S. and Europe are under pressure to replace or upgrade first-generation risk assessment tools with systems that can survive legal challenge and regulatory scrutiny. That replacement market encompasses bias auditing services, explainability platforms, transparency documentation, human oversight architecture, and the training and change management infrastructure required to make judges and parole officers actually use these tools as intended rather than relying on their own priors.

The business case is complicated by a structural constraint that no vendor has fully resolved: the speed of recidivism datasets creates a reinforcement problem. If higher-risk scores lead to longer incarceration, and longer incarceration changes recidivism outcomes, the ground truth the algorithm trains on is not ground truth at all. It is a previous algorithm's output. COMPAS and its competitors may be measuring their own effects rather than an independent underlying risk. FAIR-PReSONS's decision to work from datasets in multiple countries and to flag this limitation explicitly is a step toward honesty. It does not solve the problem.

The market signal, however, is unmistakable. Governments are investing in algorithmic justice infrastructure even as they argue about how to regulate it. The EU AI Act creates mandatory compliance spending for every tool operating in the justice administration category. U.S. states that survive the preemption battle will create their own procurement requirements. The audit and governance layer of the criminal justice AI market is being built whether vendors are ready or not.

The impossibility theorem does not disappear. Fairness to individuals and fairness across groups, accuracy and explainability, deterrence and due process: these objectives are in irreducible tension, and no software system can dissolve them. What the next generation of criminal justice AI is actually selling is a better-documented, more legally defensible version of the same underlying problem. That may be the best available product. It is not a solved one.