Artificial intelligence-assisted decision-making in long-term care: a qualitative study on opportunities and prerequisites for responsible innovation (Preprint)

Dirk R.M. Lukkien, Nathalie E. Stolwijk, Sima Ipakchian Askari, Bob M. Hofstede, Henk Herman Nap, Wouter P.C. Boon, Alexander Peine, Ellen H.M. Moors, Mirella M.N. Minkman

Research output: Working paperPreprintAcademic

Abstract

Background:
While use of artificial intelligence (AI)-based technologies such as decision-support systems (AI-DSSs) could help sustaining and improving the quality and efficiency of care, their deployment also creates ethical and social challenges. In recent years, there has been a growing prevalence of high-level guidelines and frameworks to provide guidance on responsible AI innovation. However, few studies specify how AI-based technologies such as AI-DSSs can be responsibly embedded in specific contexts such as the nursing process in the long-term care (LTC) for older adults.

Objective:
Opportunities and prerequisites for responsible AI-assisted decision-making in the nursing process were explored from the perspectives of nurses and other professional stakeholders in LTC.

Methods:
Semi-structured interviews were conducted with 24 care professionals in Dutch LTC, including nurses, care coordinators, data specialists and care centralists. Two imaginary scenarios about the future use of AI-DSSs were developed beforehand and used to enable participants to articulate their expectations regarding the opportunities and risks of AI-assisted decision-making. After first openly discussing opportunities and possible risks associated with both scenarios, six high-level principles for responsible AI were used as probing themes to evoke further consideration on risks of using AI-DSSs in LTC. Further, participants were asked to brainstorm about possible strategies and actions in the design, implementation and use of AI-DSSs to address or mitigate the mentioned risks. A thematic analysis was carried out to identify opportunities and prerequisites for responsible innovation in this area.

Results:
Professionals’ stance towards the use of AI-DSSs is not a matter of purely positive or negative expectations, but rather a nuanced interplay of positive and negative elements that lead to a weighed perception of opportunities and prerequisites for responsible AI-assisted decision-making. Both opportunities and risks were identified in relation to early identification of care needs, guidance in devising care strategies, shared decision-making, and caregivers’ workload and work experience. To optimally balance opportunities and risks of AI-assisted decision-making, seven categories of prerequisites for responsible AI-assisted decision-making in the nursing process were identified: (1) regular deliberation on data collection, (2) a balanced proactive nature of AI-DSSs, (3) incremental advancements aligned with trust and experience, (4) customization for all user groups including clients and caregivers, (5) measures to counteract bias and narrow perspectives, (6) human-centric learning loops, and (7) routinization of using AI-DSSs.

Conclusions:
Opportunities of AI-assisted decision-making in the nursing process could turn into drawbacks, depending on the specific shaping of the design and the deployment of AI-DSSs. Therefore, we recommend viewing the responsible use of AI-DSSs as a balancing act. Moreover, given the interrelatedness of the identified prerequisites, we call for various actors, including developers and users of AI-DSSs, to cohesively address different factors important to the responsible embedding of AI-DSSs in practice.
Original languageEnglish
PublisherJMIR Human Factors
DOIs
Publication statusPublished - 1 Jan 2024

Keywords

  • responsible innovation
  • ethics
  • stakeholder perspectives
  • decision-support systems
  • long-term care

Fingerprint

Dive into the research topics of 'Artificial intelligence-assisted decision-making in long-term care: a qualitative study on opportunities and prerequisites for responsible innovation (Preprint)'. Together they form a unique fingerprint.

Cite this