RLBOA: A Modular Reinforcement Learning Framework for Autonomous Negotiating Agents

Jasper Bakker, Aron Hammond, Daan Bloembergen, Tim Baarslag

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    Abstract

    With the increasing potential for the use of autonomous negotiating agents in real world applications, there is also an increasing interest to create negotiating agents that are generally applicable in different negotiation settings. The variety of settings and opponents that may be encountered prohibits the use of a single predefined negotiation strategy, and hence the agent should be able to learn such a strategy autonomously. To this end we propose RLBOA, a modular framework that facilitates the creation of autonomous negotiation agents using reinforcement learning. The framework allows for the creation of agents that are capable of negotiating effectively in many different scenarios. A big challenge in using reinforcement learning for autonomous negotiation is the diversity of settings as well as the size of the state and action spaces. To overcome this we leverage the modular BOA-framework, which decouples the negotiation strategy into a Bidding strategy, an Opponent model and an Acceptance condition. Furthermore, we project the multidimensional contract space onto the one dimensional utility axis which enables a compact and generic state and action description. We demonstrate the value of the RLBOA framework by implementing an agent that uses tabular Q-learning on the compressed state and action space to learn a bidding strategy. We show that the resulting agent is able to learn well-performing bidding strategies in a range of negotiation settings and is able to generalize across opponents and domains.
    Original languageUndefined/Unknown
    Title of host publicationProceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems
    Place of PublicationRichland, SC
    PublisherInternational Foundation for Autonomous Agents and Multiagent Systems
    Pages260-268
    Number of pages9
    ISBN (Print)978-1-4503-6309-9
    Publication statusPublished - 2019

    Publication series

    NameAAMAS '19
    PublisherInternational Foundation for Autonomous Agents and Multiagent Systems

    Keywords

    • Automated negotiation, Negotiation agent, Bargaining and negotiation, Learning agent-to-agent interactions, Reinforcement Learning

    Cite this