Inmate risk assessment tool still in use 16 years after report raises concerns about bias against women

Open this photo in gallery:

A report commissioned by Correctional Service Canada found serious flaws with its most important risk assessment tool and recommended the agency design a new one, an internal document obtained by The Globe and Mail reveals. Sixteen years on, the tool remains unchanged.

The report, dated September, 2004, was prepared by Public Safety Canada, the agency that oversees the federal prison service, after watchdogs and academics raised concerns about the risk tool’s fairness toward women.

Researchers were asked to examine the Custody Rating Scale, a risk assessment used to set an inmate’s initial security classification. This score affects everything from where they serve their sentence to their access to treatment programs, use of solitary confinement and their likelihood of being recommended for parole.

For Indigenous women, systemic racial bias in prison leaves many worse off than men

Bias behind bars: A Globe investigation finds a prison system stacked against Black and Indigenous inmates

During the examination, Public Safety Canada analysts peer-reviewed the studies prepared by CSC’s own research branch when the agency was developing its tool. They found it might be biased against women, particularly Indigenous ones, and that there wasn’t clear evidence the scale worked for men, either.

“For Aboriginal women placement decisions are being made that are not based on sound, empirical evidence,” the report reads. “The [Custody Rating Scale] was originally developed for use with male offenders. Although there is some evidence for its application with men, this evidence is not overly convincing.”

“With female offenders,” it continues, “the evidence is much less convincing and opens the potential for systemic bias.”

The report makes a single recommendation: that CSC update its security classification tool. It could do that either by looking at tools used in other jurisdictions, or by developing a new scale from scratch, researchers wrote.

The full report

A Globe investigation found that Black men and Indigenous women were, respectively, roughly 24 per cent and 64 per cent more likely than their white counterparts to receive the highest security classification at admission after accounting for variables such as age and severity of offence. Since then, the House of Commons public safety committee has announced a study into systemic racism in CSC’s risk assessments.

Concerns about the Custody Rating Scale have existed since at least 2001, when Status of Women Canada recommended CSC’s risk tools be built from the ground up with women’s contexts in mind. In 2003, the Office of the Auditor-General found the reliability of the scale’s classifications for female inmates hadn’t been established. Later that year, the Canadian Human Rights Commission said the tool had “serious shortcomings,” and that it wasn’t designed to account for the circumstances of women or racialized groups. Then, in the summer of 2004, an academic study of the scale’s classifications for women found many of the tool’s questions had no predictive power and might be biased against Indigenous people.

Public Safety Canada issued its report on the heels of these critiques, and notes that CSC’s commissioner, the head of the agency, requested an independent review of their initial security classification tool.

Public Safety researchers James Bonta, Karl Hanson and Annie Yessine were tasked with examining the validity of the Custody Rating Scale. Mr. Bonta and Mr. Hanson are widely seen as global leaders in the field of risk assessment, and Ms. Yessine had just finished three years working at CSC. Each worked independently, and all came to the same conclusions.

“In our opinion,” the document reads, “continued research on the scale with women is likely to produce only minor improvements that may be insufficient to justify the use of the [Custody Rating Scale] for such a serious task as security placement.”

They concluded that “the present custody placement instrument used by CSC does not meet the highest standards for custody placement.”

Developed in the 1980s and in wide use since the 1990s, the 12-question test, which grades inmates on things such as their offence severity and their history of substance abuse, has been used on every federal inmate for more than two decades, and has only been revised once, in 2001. Parole officers use the tool to assign inmates an initial minimum, medium or maximum security classification, though they can “override” that decision if they so choose.

What a CRS looks like

In past statements, CSC has repeatedly said the agency’s research has found the Custody Rating Scale to be valid. When asked specifically about the 2004 review, spokesperson Marie Pier Lécuyer said CSC was “not able to find the relevant records” and would confer with Public Safety Canada. Despite several follow-up requests, CSC did not provide additional clarification.

In a statement, Public Safety Canada spokesperson Mary-Liz Power said the government’s work in corrections “must be guided by evidence,” and that “practices or evaluations that systemically discriminate or disadvantage certain groups must be eliminated,” but did not specifically address the 2004 report.

Mary Campbell, a long-time Public Safety official who retired in 2013, told The Globe the report was sent directly to CSC’s senior leadership. At the time of the report, she served as director-general of Public Safety Canada’s Corrections and Criminal Justice Directorate and worked closely with the researchers who authored the report.

CSC’s request for a review was highly unusual, Ms. Campbell recalled. Though the report came together almost two decades ago, she remembers the events clearly, in part owing to CSC’s unofficial response – the agency was “furious.”

“It has stuck with me all these years,” she said. “There was real anger that, frankly, they didn’t get the answer that they wanted.”

That researchers found flaws with the Custody Rating Scale wasn’t a problem in itself, Ms. Campbell said; it simply indicated the tool had to be updated. “That’s not a negative,” she said. “What’s a negative is if that finding is made and the people responsible don’t do anything in response.”

“To me, the tragedy of that report, or rather of CSC’s response to it, was just the ongoing display that CSC does not like to hear from people outside of CSC,” Ms. Campbell said. “If they didn’t think of it, then forget it.”

CSC has its own research department, which has published hundreds of research pieces on everything from substance abuse to risk assessment since 2004. According to their website, the most recent major examination of the Custody Rating Scale came in 2014.

Following the Canadian Human Rights Commission’s 2003 report, CSC agreed to develop a new security classification tool for women. A prototype was eventually delivered by a contract researcher in 2010, but the prison agency shelved it a year later, citing in a brief memo its similarities to the current Custody Rating Scale.

Jason Gratl, a civil rights lawyer who represented an Indigenous inmate at a 2018 Supreme Court case looking at the cultural bias in CSC’s psychological risk tools, said Public Safety Canada’s researchers’ warning about the Custody Rating Scale is “unequivocal,” especially about the adverse effect on Indigenous women.

“I think this document, authored in 2004, marks a turning point for CSC,” he said. “At some point, systemic, unconscious effects-based racism turns into conscious, retrograde, deliberate racism.”

Once senior leadership had been made aware of this report, Mr. Gratl said, the need for action should have been clear – that the same tool is still in use is inexcusable.

“The deliberate refusal to respond constitutes conscious racism,” he said. “No other conclusion is possible.”

Fourteen years after the Office of the Auditor-General first raised issues with women’s security classifications, in 2017 the agency once again noted the Custody Rating Scale was still being used on women.

CSC’s official response was an echo of the one it had offered in 2003. It promised a new, “comprehensive analysis” that would “identify risk factors relevant to women offenders.”

Our Morning Update and Evening Update newsletters are written by Globe editors, giving you a concise summary of the day’s most important headlines. Sign up today.