(Reuters Health) - A widely used healthcare algorithm that flags patients at high risk of severe illness and targets them for extra attention has an unintentional built-in bias against black patients, a new study finds.
After examining the records of 6,079 black and 43,539 white patients through the lens of the software tasked with identifying those at highest risk of serious illness, researchers determined that the algorithm was more likely to flag white patients for extra medical attention than blacks who were just as sick.
The study results were published in Science.
“This family of algorithms operates behind the scenes at nearly every health system in the U.S., said lead author Dr. Ziad Obermeyer of the University of California, Berkeley, School of Public Health. “They are used to screen millions of patients for important decisions - like who gets extra help with managing their chronic illnesses.”
In concrete terms, “the top 3% of patients in terms of algorithm risk score are auto-identified for enrollment in high-risk care management programs - this doesn’t guarantee they get in, but it’s a bit like a fast track,” Obermeyer explained.
But the problem with the algorithm examined by Obermeyer and his colleagues is its use of healthcare expenditures, rather than actual medical data, to identify the sickest patients.
As it turns out, although healthcare providers are spending about the same amount of money caring for black and white patients, African-Americans are generally less likely to seek care in the earlier stages of illness - when costs of providing care would be relatively low - and more likely to run up a big bill near the end of life, when chronic conditions have resulted in very sick patients.
“Black patients generate very different kinds of costs,” the researchers write. “For example, fewer inpatient and outpatient specialist costs, and more costs related to emergency visits and dialysis.”
The reasons for that disparity are complicated and include a distrust of the medical system by African-Americans, the study team notes.
Obermeyer and his colleagues aren’t disputing the need for software to discover the sickest patients. They are concerned that the algorithms currently in use are emphasizing the wrong factors in their calculations.
“This is obviously a critical activity for our health system to be doing - we want algorithms to help predict who gets sick, and to help us prevent illness before it happens,” Obermeyer said in an email. “But we want the algorithms to do that in a fair way.”
When they were done with their analysis, Obermeyer and his colleagues approached the company that created the algorithm and suggested changes be made to remove the inadvertent bias and better identify patients at risk of becoming severely ill.
“The manufacturer independently replicated our analyses on its national dataset of 3,695,943 commercially insured patients,” the researchers write. Since then, the company and the researchers have been working together to root out biases in the software.
“Our results show that, while there is enormous scope for harm, we can also fix bias: by paying close attention to the technical choices we make when building algorithms, choices that are grounded in awareness of the deep social and historical inequalities that shape the data,” Obermeyer said.
The researchers have highlighted the error of “using cost as a surrogate marker for who has poor health and who needs most to have healthcare services wrapped around them,” said Dr. Cardinale Smith, an associate professor of medicine at the Icahn School of Medicine at Mount Sinai in New York City and director of quality for cancer services at Mount Sinai Health System.
“When you use this surrogate marker, you are creating a bias against racial and ethnic minorities who don’t always get care for the diseases they have and so tend to present later in the illness,” Smith said. “If you take the example of cancer, we know minority patients (are less likely to) get the standard of care compared to their non-minority counterparts.”
And that means minority patients may cost the system less for years, but there is a big price tag at the end when the patients are very sick, Smith explained.