The dark side of event sourcing: Managing data conversion

Michiel Overeem, Marten Spoor, Slinger Jansen

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    Abstract

    Evolving software systems includes data schema changes, and because of those schema changes data has to be converted. Converting data between two different schemas while continuing the operation of the system is a challenge when that system is expected to be available always. Data conversion in event sourced systems introduces new challenges, because of the relative novelty of the event sourcing architectural pattern, because of the lack of standardized tools for data conversion, and because of the large amount of data that is stored in typical event stores. This paper addresses the challenge of schema evolution and the resulting data conversion for event sourced systems. First of all a set of event store upgrade operations is proposed that can be used to convert data between two versions of a data schema. Second, a set of techniques and strategies that execute the data conversion while continuing the operation of the system is discussed. The final contribution is an event store upgrade framework that identifies which techniques and strategies can be combined to execute the event store upgrade operations while continuing operation of the system. Two utilizations of the framework are given, the first being as decision support in upfront design of an upgrade system for event sourced systems. The framework can also be utilized as the description of an automated upgrade system that can be used for continuous deployment. The event store upgrade framework is evaluated in interviews with three renowned experts in the domain and has been found to be a comprehensive overview that can be utilized in the design and implementation of an upgrade system. The automated upgrade system has been implemented partially and applied in experiments.

    Original languageEnglish
    Title of host publicationSANER 2017 - 24th IEEE International Conference on Software Analysis, Evolution, and Reengineering
    EditorsGabriele Bavota, Martin Pinzger, Andrian Marcus
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages193-204
    Number of pages12
    ISBN (Electronic)9781509055012
    DOIs
    Publication statusPublished - 21 Mar 2017
    Event24th IEEE International Conference on Software Analysis, Evolution, and Reengineering, SANER 2017 - Klagenfurt, Austria
    Duration: 21 Feb 201724 Feb 2017

    Publication series

    NameSANER 2017 - 24th IEEE International Conference on Software Analysis, Evolution, and Reengineering

    Conference

    Conference24th IEEE International Conference on Software Analysis, Evolution, and Reengineering, SANER 2017
    Country/TerritoryAustria
    CityKlagenfurt
    Period21/02/1724/02/17

    Keywords

    • CQRS
    • Data Conversion
    • Data Transformation
    • Deployment Strategy
    • Event Driven Architecture
    • Event Sourcing
    • Schema Evolution
    • Schema Versioning
    • Software Evolution

    Fingerprint

    Dive into the research topics of 'The dark side of event sourcing: Managing data conversion'. Together they form a unique fingerprint.

    Cite this