(How) Does Accountability Require Understanding Machine Learning Models?
Dr. Huzeyfe Demirtas
Harvard University Postdoctoral Fellow in Embedded EthiCS
November 11th
4:15 p.m. – 5:15 p.m.
Pardee Hall Room 321

Concerns about artificial intelligence—particularly systems based on machine learning models whose internal operations are opaque to human understanding—are frequently framed in terms of accountability. But these concerns are often left underspecified, encompassing a heterogeneous set of issues that call for distinct responses. This paper focuses on a specific subset of accountability concerns: the “who” questions.
Dr. Demirtas distinguishes between two questions: When an AI system causes harm, who is to blame, and who bears the obligation to provide compensation? He argues that the answers to these two questions require different information and don’t entail one another. Then he investigates whether the opacity of contemporary machine learning models undermines our capacity to hold the relevant actors accountable in these two senses of accountability.
What is required, he argues, is not understanding of the internal processes of the particular machine learning model deployed, but rather, a general understanding of how upstream decisions in the development process shape downstream model behavior and internal processes. He concludes with various practical implications of our discussion. Among others – for a pro tanto moral obligation to document contextual and historical information in the process of developing machine learning models.