Access to The Champion archive is one of many exclusive member benefits. It’s normally restricted to just NACDL members. However, this content, and others like it, is available to everyone in order to educate the public on why criminal justice reform is a necessity.
Back in August 2014, in remarks delivered at NACDL’s Annual Meeting, then-Attorney General Eric Holder sounded an alarm bell about the use of risk assessment tools. While noting that intensive analysis and data-driven approaches can be useful at various phases of the criminal legal process, he cautioned that especially when employed to drive front-end decision-making, such as sentence determination, “basing sentencing decisions on static factors and immutable characteristics — like the defendant’s education level, socioeconomic background, or neighborhood — may exacerbate unwarranted and unjust disparities that are already far too common in our criminal justice system and in our society.”{1} 1 https://www.justice.gov/opa/speech/attorney-general-eric-holder-speaks-national-association-criminal-defense-lawyers-57th. Of course, other static factors, such as arrest and conviction records, which are often driven by disparate policing practices, also perpetuate racial and ethnic disparity.
Despite these concerns, risk assessment tools are now used pervasively at almost every stage of the criminal legal process, including pretrial release, sentencing, conditions of probation and parole, suitability for early release, as well as determinations related to sexual offender registration, civil commitment, and revocation of parole or probation. Indeed, the First Step Act, which provides opportunities for early release for various categories of federal prisoners, bases eligibility on the use of a risk assessment tool known as PATTERN.{2} 2 18 U.S.C. § 3632. The First Step Act authorized the Department of Justice (“DOJ”) to create a new risk and needs assessment to be used by the Bureau of Prisons -- endnote
The ubiquity with which these tools are employed has a direct impact on defense advocacy and by extension has been an area of evolving concern for NACDL. In May 2017, at the behest of the Pretrial Justice Institute, which at the time was an ardent proponent of the use of risk assessment tools as an alternative to money bail, NACDL joined with the American Council of Chief Defenders, Gideon’s Promise, the National Legal Aid and Defender Service, and the National Association for Public Defense in endorsing the use of validated pretrial risk assessment instruments with a set of necessary checks and balances.{3} 3 The Pretrial Justice Institute has since reversed course and now opposes the use of pretrial risk assessment instruments. See https://university.pretrial.org/viewdocument/the-case-against-pretrial-risk-asse. The statement was motivated both by a joint concern about the harm and injustice of wealth-based detention and by a recognition that in some jurisdictions these tools reduced unjust pretrial practices. Thereafter, in March 2019, amid increasing concern that the tools may exacerbate racial and ethnic disparity, the groups modified the statement of support to condition the use of a validated pretrial risk assessment tool upon “input from the local criminal defense stakeholders and impacted communities who understand the unique circumstances surrounding pretrial release in their jurisdiction, that it will serve to reduce unnecessary detention and help to eliminate racial and ethnic bias in the outcome of the pretrial decisions.” The statement went on to provide a comprehensive list of checks and balances that must be in place as a precondition for use of these tools.{4} 4 https://www.nacdl.org/Document/JointStatementRiskAssessmentnstruments.
Meanwhile, recognizing the multifaceted nature of the conundrum presented by the potential for big data to either help or harm individual clients, in 2017 NACDL’s president, Rick Jones, appointed a task force comprised of a cadre of outstanding defense attorneys from throughout the nation to study the use of risk assessment tools.{5} 5 The task force was chaired by Marvin Schechter and Vicki H. Young. Members included Franklin Draper, Nina J. Ginsberg, Stephen Ross Johnson, Matthew Knecht, John Philipsborn, Gail Shifman, Colette Tvedt, and CeCelia Valentine-Andrews. The stated goal of the task force was to “guide the defense bar in developing policies and strategies to ensure that their use comports with fundamental constitutional principles.” When it embarked upon its work, there was an expectation that a major objective would be to articulate a position either favoring or opposing the use of these tools. As the work unfolded, however, the task force reviewed a plethora of materials and sought input from various stakeholders, which led it in a very different direction.
The sheer pervasiveness of these tools and the complexity of the issues surrounding how they are developed and deployed made it abundantly clear that far more important than any policy that NACDL might ultimately adopt was the need to comprehensively educate the defense bar. As experienced practitioners themselves, the deeper the members of the task force became enmeshed in the details of how risk assessments operate, the clearer it became that the most important contribution that NACDL could make would be to produce a resource for the defense bar to help practitioners effectively safeguard the interests of clients whose lives are impacted by risk assessment tools. Accordingly, the NACDL Task Force on Risk Assessment Tools commissioned Dr. Melissa Hamilton to author a comprehensive report with specific recommendations to guide the field.{6} 6 Dr. Melissa Hamilton is Reader of Law and Criminal Justice at the University of Surrey School of Law in England.
On Nov. 16, 2020, NACDL published Risk Assessment Tools in the Criminal Legal System — Theory and Practice: A Resource Guide.{7} 7 http://www.nacdl.org/riskassessmentreport. The report provides an in-depth analysis of risk assessment instruments that demystifies the process by which they are developed and applied in practice and provides a road map that defense counsel can employ for critical analysis. As Dr. Hamilton notes, “[t]he guise of science has sheltered these practices from significant challenges from forensic scientists, criminal justice officials, practitioners, and other stakeholders. Consequently, risk assessment tools appear to be given too much deference when a critical eye is more befitting.”
The Risk Assessment Tools Resource Guide is that critical eye. It traces the evolution of these tools and explains the science underlying algorithmic risk tools. It comprehensively addresses the critically important issues of accuracy and validation, error rates, reliability, user bias, systemic bias, and conflicts of interest. It also includes an analysis of legal and ethical issues as well as identifies a litany of due process and equitable considerations. Additionally, the Guide identifies dozens of policy considerations relevant to the specific risk assessment tool issue addressed throughout the report. These considerations provide counsel with invaluable insight to understand and combat a weakness, deficiency, or defect in the application of a risk assessment tool.
As stated in the Guide, it “does not take a unitary position as to whether risk assessment should be used in criminal justice. Context matters. Risk assessment may be more or less acceptable as a legal, policy, and/or ethical question depending on such matters as the nature of the decision point, the relevant rights of defendants, legal precedent, and the availability of expertise and resources. Counsel may reasonably contend for a variety of reasons that risk assessment is not suitable in a given context. Many of the points raised herein are intended to inform counsel if, despite arguments to the contrary, a risk tool is used anyway.”
Thus, while the Guide does not take a position pro or con on the question first presented to NACDL by Eric Holder in 2014, it is an indispensable weapon in a defense lawyer’s arsenal on behalf of a client facing a harmful outcome when a big data tool substitutes for individualized justice. Policies are important. But when liberty is at stake for countless thousands, informed practices must take precedence.
About the Author
Norman L. Reimer is NACDL’s Executive Director and Publisher of The Champion.
Norman L. Reimer
NACDL
Washington, DC
202-465-7623
nreimer@nacdl.org
www.nacdl.org
@NACDLExecDir
Endnote
- 18 U.S.C. § 3632. The First Step Act authorized the Department of Justice (“DOJ”) to create a new risk and needs assessment to be used by the Bureau of Prisons (“BOP”) to periodically assess prisoners’ risk of recidivism and categorize them as having a minimum, low, medium, or high risk of recidivism. First Step Act, Pub. L. 115-391 § 101-102, 132 Stat. 5194, 5195 (Dec. 21, 2018). On July 19, 2019, the DOJ released its tool titled Prisoner Assessment Tool Targeting Estimated Risk and Needs (“PATTERN”). Department of Justice, The First Step Act of 2018: Risk and Needs Assessment, July 2019, https://nij.ojp.gov/sites/g/files/xyckuh171/files/media/document/the-first-step-act-of-2018-risk-and-needs-assessment-system_1.pdf. An updated version was released in January 2020. Department of Justice, The First Step Act of 2018: Risk and Needs Assessment-UPDATE, January 2020, https://www.bop.gov/inmates/fsa/docs/the-first-step-act-of-2018-risk-and-needs-assessment-system-updated.pdf. (^ return)
