Explaining Model Behavior with Global Causal Analysis

Marcel Robeer; Floris Bex; Ad Feelders; Henry Prakken

doi:10.1007/978-3-031-44064-9_17

Explaining Model Behavior with Global Causal Analysis

Marcel Robeer^*
, Floris Bex
, Ad Feelders
, Henry Prakken

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

We present GLOBAL CAUSAL ANALYSIS (GCA) for text classification. GCA is a technique for global model-agnostic explainability drawing from well-established observational causal structure learning algorithms. GCA generates an explanatory graph from high-level human-interpretable features, revealing how these features affect each other and the black-box output. We show how these high-level features do not always have to be human-annotated, but can also be computationally inferred. Moreover, we discuss how the explanatory graph can be used for global model analysis in natural language processing (NLP): the graph shows the effect of different types of features on model behavior, whether these effects are causal effects or mere (spurious) correlations, and if and how different features interact. We then propose a three-step method for (semi-)automatically evaluating the quality, fidelity and stability of the GCA explanatory graph without requiring a ground truth. Finally, we provide a detailed GCA of a state-of-the-art NLP model, showing how setting a global one-versus-rest contrast can improve explanatory relevance, and demonstrating the utility of our three-step evaluation method.

Original language	English
Title of host publication	Explainable Artificial Intelligence
Subtitle of host publication	First World Conference, xAI 2023, Lisbon, Portugal, July 26–28, 2023, Proceedings, Part I
Editors	Luca Longo
Place of Publication	Cham
Publisher	Springer
Pages	299–323
Number of pages	25
Edition	1
ISBN (Electronic)	978-3-031-44064-9
ISBN (Print)	978-3-031-44063-2
DOIs	https://doi.org/10.1007/978-3-031-44064-9_17
Publication status	Published - 30 Oct 2023

Publication series

Name	Communications in Computer and Information Science
Publisher	Springer
Volume	1901
ISSN (Print)	1865-0929
ISSN (Electronic)	1865-0937

Bibliographical note

Publisher Copyright:
© 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Funding

This study has been partially supported by the Dutch National Police. The authors would like to thank Elize Herrewijnen and Gizem Sogancioglu for their valuable feedback on earlier versions of this work.

Funders
Dutch National Police

Keywords

Causal explanation
Explainable Machine Learning (XML)
Model-agnostic explanation
Natural Language Processing (NLP)

Access to Document

10.1007/978-3-031-44064-9_17

978-3-031-44064-9_17Final published version, 1.04 MBLicence: Taverne

Cite this

Robeer, M., Bex, F., Feelders, A., & Prakken, H. (2023). Explaining Model Behavior with Global Causal Analysis. In L. Longo (Ed.), Explainable Artificial Intelligence: First World Conference, xAI 2023, Lisbon, Portugal, July 26–28, 2023, Proceedings, Part I (1 ed., pp. 299–323). (Communications in Computer and Information Science; Vol. 1901). Springer. https://doi.org/10.1007/978-3-031-44064-9_17

@inproceedings{cc04aee55cea4208a848d129ccd64ecd,

title = "Explaining Model Behavior with Global Causal Analysis",

abstract = "We present GLOBAL CAUSAL ANALYSIS (GCA) for text classification. GCA is a technique for global model-agnostic explainability drawing from well-established observational causal structure learning algorithms. GCA generates an explanatory graph from high-level human-interpretable features, revealing how these features affect each other and the black-box output. We show how these high-level features do not always have to be human-annotated, but can also be computationally inferred. Moreover, we discuss how the explanatory graph can be used for global model analysis in natural language processing (NLP): the graph shows the effect of different types of features on model behavior, whether these effects are causal effects or mere (spurious) correlations, and if and how different features interact. We then propose a three-step method for (semi-)automatically evaluating the quality, fidelity and stability of the GCA explanatory graph without requiring a ground truth. Finally, we provide a detailed GCA of a state-of-the-art NLP model, showing how setting a global one-versus-rest contrast can improve explanatory relevance, and demonstrating the utility of our three-step evaluation method.",

keywords = "Causal explanation, Explainable Machine Learning (XML), Model-agnostic explanation, Natural Language Processing (NLP)",

author = "Marcel Robeer and Floris Bex and Ad Feelders and Henry Prakken",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.",

year = "2023",

month = oct,

day = "30",

doi = "10.1007/978-3-031-44064-9\_17",

language = "English",

isbn = "978-3-031-44063-2",

series = "Communications in Computer and Information Science",

publisher = "Springer",

pages = "299–323",

editor = "Luca Longo",

booktitle = "Explainable Artificial Intelligence",

edition = "1",

}

Robeer, M, Bex, F , Feelders, A & Prakken, H 2023, Explaining Model Behavior with Global Causal Analysis. in L Longo (ed.), Explainable Artificial Intelligence: First World Conference, xAI 2023, Lisbon, Portugal, July 26–28, 2023, Proceedings, Part I. 1 edn, Communications in Computer and Information Science, vol. 1901, Springer, Cham, pp. 299–323. https://doi.org/10.1007/978-3-031-44064-9_17

Explaining Model Behavior with Global Causal Analysis. / Robeer, Marcel; Bex, Floris ; Feelders, Ad et al.
Explainable Artificial Intelligence: First World Conference, xAI 2023, Lisbon, Portugal, July 26–28, 2023, Proceedings, Part I. ed. / Luca Longo. 1. ed. Cham: Springer, 2023. p. 299–323 (Communications in Computer and Information Science; Vol. 1901).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Explaining Model Behavior with Global Causal Analysis

AU - Robeer, Marcel

AU - Bex, Floris

AU - Feelders, Ad

AU - Prakken, Henry

PY - 2023/10/30

Y1 - 2023/10/30

N2 - We present GLOBAL CAUSAL ANALYSIS (GCA) for text classification. GCA is a technique for global model-agnostic explainability drawing from well-established observational causal structure learning algorithms. GCA generates an explanatory graph from high-level human-interpretable features, revealing how these features affect each other and the black-box output. We show how these high-level features do not always have to be human-annotated, but can also be computationally inferred. Moreover, we discuss how the explanatory graph can be used for global model analysis in natural language processing (NLP): the graph shows the effect of different types of features on model behavior, whether these effects are causal effects or mere (spurious) correlations, and if and how different features interact. We then propose a three-step method for (semi-)automatically evaluating the quality, fidelity and stability of the GCA explanatory graph without requiring a ground truth. Finally, we provide a detailed GCA of a state-of-the-art NLP model, showing how setting a global one-versus-rest contrast can improve explanatory relevance, and demonstrating the utility of our three-step evaluation method.

AB - We present GLOBAL CAUSAL ANALYSIS (GCA) for text classification. GCA is a technique for global model-agnostic explainability drawing from well-established observational causal structure learning algorithms. GCA generates an explanatory graph from high-level human-interpretable features, revealing how these features affect each other and the black-box output. We show how these high-level features do not always have to be human-annotated, but can also be computationally inferred. Moreover, we discuss how the explanatory graph can be used for global model analysis in natural language processing (NLP): the graph shows the effect of different types of features on model behavior, whether these effects are causal effects or mere (spurious) correlations, and if and how different features interact. We then propose a three-step method for (semi-)automatically evaluating the quality, fidelity and stability of the GCA explanatory graph without requiring a ground truth. Finally, we provide a detailed GCA of a state-of-the-art NLP model, showing how setting a global one-versus-rest contrast can improve explanatory relevance, and demonstrating the utility of our three-step evaluation method.

KW - Causal explanation

KW - Explainable Machine Learning (XML)

KW - Model-agnostic explanation

KW - Natural Language Processing (NLP)

UR - https://www.scopus.com/pages/publications/85176947124

U2 - 10.1007/978-3-031-44064-9_17

DO - 10.1007/978-3-031-44064-9_17

M3 - Conference contribution

SN - 978-3-031-44063-2

T3 - Communications in Computer and Information Science

SP - 299

EP - 323

BT - Explainable Artificial Intelligence

A2 - Longo, Luca

PB - Springer

CY - Cham

ER -

Robeer M, Bex F , Feelders A , Prakken H. Explaining Model Behavior with Global Causal Analysis. In Longo L, editor, Explainable Artificial Intelligence: First World Conference, xAI 2023, Lisbon, Portugal, July 26–28, 2023, Proceedings, Part I. 1 ed. Cham: Springer. 2023. p. 299–323. (Communications in Computer and Information Science). doi: 10.1007/978-3-031-44064-9_17