Improving Confidence in the Estimation of Values and Norms

Luciano Cavalcante Siebert, Rijk Mercuur, Virginia Dignum, Jeroen van den Hoven, Catholijn M. Jonker

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

Autonomous agents (AA) will increasingly be interacting with us in our daily lives. While we want the benefits attached to AAs, it is essential that their behavior is aligned with our values and norms. Hence, an AA will need to estimate the values and norms of the humans it interacts with, which is not a straightforward task when solely observing an agent's behavior. This paper analyses to what extent an AA is able to estimate the values and norms of a simulated human agent (SHA) based on its actions in the ultimatum game. We present two methods to reduce ambiguity in profiling the SHAs: one based on search space exploration and another based on counterfactual analysis. We found that both methods are able to increase the confidence in estimating human values and norms, but differ in their applicability, the latter being more efficient when the number of interactions with the agent is to be minimized. These insights are useful to improve the alignment of AAs with human values and norms.
Original languageEnglish
Title of host publicationCoordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII. COIN COINE 2017 2020
PublisherSpringer
Pages98-113
ISBN (Electronic)978-3-030-72376-7
ISBN (Print)978-3-030-72375-0
DOIs
Publication statusPublished - 2021

Keywords

  • cs.AI
  • cs.MA

Fingerprint

Dive into the research topics of 'Improving Confidence in the Estimation of Values and Norms'. Together they form a unique fingerprint.

Cite this