LLM Agents in Interaction: Measuring Personality Consistency and Linguistic Alignment in Interacting Populations of Large Language Models

Ivar Frisch, Mario Giulianelli

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

While both agent interaction and personalisation are vibrant topics in research on large language models (LLMs), there has been limited focus on the effect of language interaction on the behaviour of persona-conditioned LLM agents. Such an endeavour is important to ensure that agents remain consistent to their assigned traits yet are able to engage in open, naturalistic dialogues. In our experiments, we condition GPT-3.5 on personality profiles through prompting and create a twogroup population of LLM agents using a simple variability-inducing sampling algorithm. We then administer personality tests and submit the agents to a collaborative writing task, finding that different profiles exhibit different degrees of personality consistency and linguistic alignment to their conversational partners. Our study seeks to lay the groundwork for better understanding of dialogue-based interaction between LLMs and highlights the need for new approaches to crafting robust, more human-like LLM personas for interactive environments.

Original languageEnglish
Title of host publicationProceedings of the 1st Workshop on Personalization of Generative AI Systems (PERSONALIZE 2024)
EditorsAmeet Deshpande, EunJeong Hwang, Vishvak Murahari, Joon Sung Park, Diyi Yang, Ashish Sabharwal, Karthik Narasimhan, Ashwin Kalyan
PublisherAssociation for Computational Linguistics (ACL)
Pages102-111
Number of pages10
ISBN (Electronic)9798891760721
Publication statusPublished - 2024
Externally publishedYes
Event1st Workshop on Personalization of Generative AI Systems, PERSONALIZE 2024 - St. Julian's, Malta
Duration: 22 Mar 2024 → …

Conference

Conference1st Workshop on Personalization of Generative AI Systems, PERSONALIZE 2024
Country/TerritoryMalta
CitySt. Julian's
Period22/03/24 → …

Fingerprint

Dive into the research topics of 'LLM Agents in Interaction: Measuring Personality Consistency and Linguistic Alignment in Interacting Populations of Large Language Models'. Together they form a unique fingerprint.

Cite this