Artificial Intelligence and Discrimination in Health Care
Sharona Hoffman & Andy Podgurski
ABSTRACT. Artificial intelligence (AI) holds great promise for improved health-care outcomes. It has been used to analyze tumor images, to help doctors choose among different treatment options, and to combat the COVID-19 pandemic. But AI also poses substantial new hazards. This Article focuses on a particular type of health-care harm that has thus far evaded significant legal scrutiny. The harm is algorithmic discrimination.
Algorithmic discrimination in health care occurs with surprising frequency. A well-known example is an algorithm used to identify candidates for "high risk care management" programs that routinely failed to refer racial minorities for these beneficial services. Furthermore, some algorithms deliberately adjust for race in ways that hurt minority patients. For example, according to a 2020 New England Journal of Medicine article, algorithms have regularly underestimated African Americans' risks of kidney stones, death from heart failure, and other medical problems.
This Article argues that algorithmic discrimination in medicine can violate civil rights laws such as Title VI and Section 1557 of the Affordable Care Act when it exacerbates health disparities or perpetuates inequities. It urges that algorithmic fairness constitute a key element in designing, implementing, and validating AI and that both legal and technical tools be deployed to promote fairness. To that end, we call for the reintroduction of the disparate impact theory as a robust litigation tool in the health-care arena and for the passage of an algorithmic accountability act. We also detail technical measures that AI developers and users should implement.
AUTHORS. Sharona Hoffman, Edgar A. Hahn Professor of Law and Professor of Bioethics, Co-Director of Law-Medicine Center, Case Western Reserve University School of Law; B.A., Wellesley College; J.D., Harvard Law School; LL.M. in Health Law, University of Houston; S.J.D. in Health Law, Case Western Reserve University. Author of ELECTRONIC HEALTH RECORDS AND MEDICAL BIG DATA: LAW AND POLICY (Cambridge University Press 2016). For more information, see https://sharonahoffman.com. Andy Podgurski, Professor of Computer and Data Sciences, Case Western Reserve University; B.S., M.S., Ph.D., University of Massachusetts. The authors thank Peter Gerhart, Jessie Hill, Katharine Van Tassel, and all participants in the Case Western Reserve University School of Law summer faculty workshop for their very helpful comments on earlier drafts. We also thank Mariah Dick for her invaluable research assistance.
RECOMMENDED CITATION. Sharona Hoffman & Andy Podgurski, Artificial Intelligence and Discrimination in Health Care, 19 Yale J. Health Pol'y L. & Ethics (2020).