Requirements and Attitudes towards Explainable AI in Law Enforcement

Decision-making aided by Artifcial Intelligence in high-stakes domains such as law enforcement must be informed and accountable. Thus, designing explainable artifcial intelligence (XAI) for such settings is a key social concern. Yet, explanations are often misunderstood by end-users due to being overly technical or abstract. To address this, our study engaged with police employees in the Netherlands, who are users of a text classifer. We found that for them, usability and usefulness are of great importance in explanation design, whereas interpretability and understandability are less valued. Further, our work reports on how design elements included in machine learning model explanations are interpreted. Drawing from these insights, we contribute recommendations that guide XAI system designers to cater to the specifc needs of specialized users in high-stakes domains and suggest design considerations for machine learning model explanations aimed at domain experts.