Problem-solving courts do not work
This is the first of a multi-part series criticizing problem-solving courts. The first installment questions the efficacy of problem-solving courts. The second installment will address separation of powers issues. The third piece will raise ethical and moral concerns about the true reasons problem-solving courts exist, and the final column will point out the financial conflicts of interest inherent in the system.
The first problem-solving court, a drug court, was established thirty years ago. Since that time, and particularly over the last ten years, problem-solving courts have vastly proliferated. Today, there are over 4,000 problem-solving courts throughout the country. Indiana, which is amid a problem-solving court expansion, has over one hundred and fifty. Despite the enthusiasm for problem-solving courts, there is no good evidence that they work any better than traditional courts.
A quick google search will find many proclamations that problem-solving courts are effective. Untold numbers of people will give anecdotal accounts of success. Real, empirical studies, however, do not support these claims. University of Richmond School of Law professor, Erin Collins, published a 2021 article in the University of California, Davis Law Review entitled, The Problem with Problem-Solving Courts. In her article, Collins points out the many issues with the data, or lack thereof, on problem-solving courts:
“the data regarding drug courts does not actually tell the unmitigated success story their proponents recite. For example, a recent analysis of drug court evaluations found mixed results. Some studies showed drug courts reduced recidivism (at “modest levels”), while others indicated they had no impact on recidivism, and one even found they increased recidivism.”
“Thus, much of the available data about problem-solving court performance undermines or at least tempers the unqualified claims that this is an effective and efficient reform mechanism.”
“And retrospective studies often reveal that preliminary, court-created reports observing drastic reduced recidivism are drastically overstated and unfounded. Moreover, despite the fact that the problem-solving court approach has existed for thirty years, the data leaves much to be desired. Notably, with the exception of drug courts, it is widely accepted that problem-solving courts have not been analyzed with rigor sufficient to form a conclusion about their impact.”
“While some of the earliest studies of drug courts demonstrated recidivism reductions among their participants, it is now widely acknowledged that these studies were marred by methodological flaws that undermine their findings, such as sample size, lack of a meaningful comparison group, and selection bias.”
“In sum, drug court evaluations seem to demonstrate that some drug courts modestly reduce recidivism for some individuals, some of the time. And even for those studies that indicate recidivism reductions, very little is known about which aspect of drug courts lead to these reductions. Moreover, conclusions of efficacy on other metrics—such as cost savings or reducing substance use—are even more tentative.”
Comprehensive reviews of the research literature on problem-solving courts, such as was performed by Collins, reveal that the data on performance is unclear, and cannot be relied upon to support claims that they reduce recidivism. Proponents of problem-solving courts cannot, in good faith, claim that research supports their efficacy.
One of the reasons the research fails to yield reliable conclusions is that problem-solving courts are resistant to empirical study. To meaningfully research problem-solving courts, one must push through the voluminous anecdotal accounts that make up much of the argument for the courts’ existence. While anecdotal stories may sound good, they do not provide actual data. Further, the anecdotal accounts come from “stakeholders” and participants, each of whom have an interest in saying good things about the courts. When asked to show evidence that they work, problem-solving court proponents most commonly resort to feel good stories.
Even when drug courts have been more systematically studied, however, there are no consistent criteria. The first step in studying any problem is defining the terms. There are no standardized definitions in this area. For instance, the big question is always recidivism rates. However, there is no agreed upon definition of recidivism. Some places define recidivism as re-arrest. Some define it as new convictions. Some use the amorphous term “undesirable outcomes.” There is also no consensus on the relevant length of time following graduation from the program for one to track recidivism. Does an undesirable outcome three years after graduation count as recidivism? Five years?
There is also no consistent definition of success or failure. If a participant fails every drug screen but the problem-solving court team votes to let him graduate anyway, is that success? As we will see in later installments, incentives align to show “success” despite poor performance, further marring study.
Finally, there are no studies that follow participants for the length of time necessary to demonstrate real results. Logistical concerns and costs prevent such longitudinal studies, which are necessary to properly evaluate outcomes.
Problem-solving courts have multiplied despite a lack of evidence that they work. Those arguing for them point to stories of success rather than data. There are no solid studies showing efficacy, and the studies that exist are plagued by methodological flaws. As public intellectual, Thomas Sowell, wrote in his book, A Vision of the Anointed, “Much of the social history of the Western world over the past three decades has involved replacing what worked with what sounded good.” That is exactly what has happened with the problem-solving court movement. Absent good research, or some other valid reason to be creating these courts, policymakers should pause their expansion in favor of our time-honored traditional methods.