Trust in Prediction Models: a Mixed-Methods Pilot Study on the Impact of Domain Expertise

Jeroen Ooge; Katrien Verbert

doi:10.1109/TREX53765.2021.00007

Trust in Prediction Models: a Mixed-Methods Pilot Study on the Impact of Domain Expertise

Jeroen Ooge^*
, Katrien Verbert

^*Corresponding author for this work

KU Leuven

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

People's trust in prediction models can be affected by many factors, including domain expertise like knowledge about the application domain and experience with predictive modelling. However, to what extent and why domain expertise impacts people's trust is not entirely clear. In addition, accurately measuring people's trust remains challenging. We share our results and experiences of an exploratory pilot study in which four people experienced with predictive modelling systematically explore a visual analytics system with an unknown prediction model. Through a mixed-methods approach involving Likert-type questions and a semi-structured interview, we investigate how people's trust evolves during their exploration, and we distil six themes that affect their trust in the prediction model. Our results underline the multi-faceted nature of trust, and suggest that domain expertise alone cannot fully predict people's trust perceptions.

Original language	English
Title of host publication	Proceedings - 2021 IEEE Workshop on TRust and EXpertise in Visual Analytics, TREX 2021
Publisher	IEEE
Pages	8-13
Number of pages	6
ISBN (Electronic)	9781665418171
DOIs	https://doi.org/10.1109/TREX53765.2021.00007
Publication status	Published - 2021
Externally published	Yes
Event	2021 IEEE Workshop on TRust and EXpertise in Visual Analytics, TREX 2021 - Virtual, Online, United States Duration: 24 Oct 2021 → …

Publication series

Name	Proceedings - 2021 IEEE Workshop on TRust and EXpertise in Visual Analytics, TREX 2021

Conference

Conference	2021 IEEE Workshop on TRust and EXpertise in Visual Analytics, TREX 2021
Country/Territory	United States
City	Virtual, Online
Period	24/10/21 → …

Bibliographical note

Publisher Copyright:
© 2021 IEEE.

Funding

This work was supported by the Research Foundation-Flanders (FWO, grant G0A3319N), the Slovenian Research Agency (grant ARRS-N2-0101), and the European Union’s Horizon 2020 research and innovation program: BigDataGrapes Project (grant 780751). We thank all four participants for their insightful feedback.

Funders	Funder number
Research Foundation Flanders
Horizon 2020 Framework Programme	780751
Fonds Wetenschappelijk Onderzoek	G0A3319N
Javna Agencija za Raziskovalno Dejavnost RS	ARRS-N2-0101

Access to Document

10.1109/TREX53765.2021.00007

Cite this

@inproceedings{fdcb4aaaf7694e0b9079fffa7ebe5ccd,

title = "Trust in Prediction Models: a Mixed-Methods Pilot Study on the Impact of Domain Expertise",

abstract = "People's trust in prediction models can be affected by many factors, including domain expertise like knowledge about the application domain and experience with predictive modelling. However, to what extent and why domain expertise impacts people's trust is not entirely clear. In addition, accurately measuring people's trust remains challenging. We share our results and experiences of an exploratory pilot study in which four people experienced with predictive modelling systematically explore a visual analytics system with an unknown prediction model. Through a mixed-methods approach involving Likert-type questions and a semi-structured interview, we investigate how people's trust evolves during their exploration, and we distil six themes that affect their trust in the prediction model. Our results underline the multi-faceted nature of trust, and suggest that domain expertise alone cannot fully predict people's trust perceptions.",

author = "Jeroen Ooge and Katrien Verbert",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE.; 2021 IEEE Workshop on TRust and EXpertise in Visual Analytics, TREX 2021 ; Conference date: 24-10-2021",

year = "2021",

doi = "10.1109/TREX53765.2021.00007",

language = "English",

series = "Proceedings - 2021 IEEE Workshop on TRust and EXpertise in Visual Analytics, TREX 2021",

publisher = "IEEE",

pages = "8--13",

booktitle = "Proceedings - 2021 IEEE Workshop on TRust and EXpertise in Visual Analytics, TREX 2021",

address = "United States",

}

Ooge, J & Verbert, K 2021, Trust in Prediction Models: a Mixed-Methods Pilot Study on the Impact of Domain Expertise. in Proceedings - 2021 IEEE Workshop on TRust and EXpertise in Visual Analytics, TREX 2021. Proceedings - 2021 IEEE Workshop on TRust and EXpertise in Visual Analytics, TREX 2021, IEEE, pp. 8-13, 2021 IEEE Workshop on TRust and EXpertise in Visual Analytics, TREX 2021, Virtual, Online, United States, 24/10/21. https://doi.org/10.1109/TREX53765.2021.00007

Trust in Prediction Models: a Mixed-Methods Pilot Study on the Impact of Domain Expertise. / Ooge, Jeroen; Verbert, Katrien.
Proceedings - 2021 IEEE Workshop on TRust and EXpertise in Visual Analytics, TREX 2021. IEEE, 2021. p. 8-13 (Proceedings - 2021 IEEE Workshop on TRust and EXpertise in Visual Analytics, TREX 2021).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Trust in Prediction Models

T2 - 2021 IEEE Workshop on TRust and EXpertise in Visual Analytics, TREX 2021

AU - Ooge, Jeroen

AU - Verbert, Katrien

PY - 2021

Y1 - 2021

N2 - People's trust in prediction models can be affected by many factors, including domain expertise like knowledge about the application domain and experience with predictive modelling. However, to what extent and why domain expertise impacts people's trust is not entirely clear. In addition, accurately measuring people's trust remains challenging. We share our results and experiences of an exploratory pilot study in which four people experienced with predictive modelling systematically explore a visual analytics system with an unknown prediction model. Through a mixed-methods approach involving Likert-type questions and a semi-structured interview, we investigate how people's trust evolves during their exploration, and we distil six themes that affect their trust in the prediction model. Our results underline the multi-faceted nature of trust, and suggest that domain expertise alone cannot fully predict people's trust perceptions.

AB - People's trust in prediction models can be affected by many factors, including domain expertise like knowledge about the application domain and experience with predictive modelling. However, to what extent and why domain expertise impacts people's trust is not entirely clear. In addition, accurately measuring people's trust remains challenging. We share our results and experiences of an exploratory pilot study in which four people experienced with predictive modelling systematically explore a visual analytics system with an unknown prediction model. Through a mixed-methods approach involving Likert-type questions and a semi-structured interview, we investigate how people's trust evolves during their exploration, and we distil six themes that affect their trust in the prediction model. Our results underline the multi-faceted nature of trust, and suggest that domain expertise alone cannot fully predict people's trust perceptions.

UR - https://www.scopus.com/pages/publications/85123757634

U2 - 10.1109/TREX53765.2021.00007

DO - 10.1109/TREX53765.2021.00007

M3 - Conference contribution

AN - SCOPUS:85123757634

T3 - Proceedings - 2021 IEEE Workshop on TRust and EXpertise in Visual Analytics, TREX 2021

SP - 8

EP - 13

BT - Proceedings - 2021 IEEE Workshop on TRust and EXpertise in Visual Analytics, TREX 2021

PB - IEEE

Y2 - 24 October 2021

ER -

Trust in Prediction Models: a Mixed-Methods Pilot Study on the Impact of Domain Expertise

Abstract

Publication series

Conference

Bibliographical note

Funding

Access to Document

Other files and links

Fingerprint

Cite this