Meta-Dominance Analysis – A Tool for the Assessment of the Quality of Digital Behavioural Data

Andreas Schneck, Wojtek Przepiorka*

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

We propose a simple yet comprehensive conceptual framework for the identification of different sources of error in research with digital behavioural data. We use our framework to map potential sources of error in 25 years of research on reputation effects in peer-to-peer online market platforms. Using a meta-dataset comprising 346 effect sizes extracted from 109 articles, we apply meta-dominance analysis to quantify the relative importance of different error components. Our results indicate that 85% of explained effect size heterogeneity can be attributed to the measurement process, which comprises the choice of platform, data collection mode, construct operationalisation and variable transformation. Error components attributable to the sampling process or publication bias capture relatively small parts of the explained effect size heterogeneity. This approach reveals at which stages of the research process researcher decisions may affect data quality most. This approach can be used to identify potential sources of error in established strands of research beyond the literature of behavioural data from online platforms.

Original languageEnglish
JournalSocial Science Computer Review
DOIs
Publication statusE-pub ahead of print - 17 Jun 2024

Bibliographical note

Publisher Copyright:
© The Author(s) 2024.

Keywords

  • data quality
  • digital behavioural data
  • meta-dominance analysis
  • reputation effects
  • total error framework

Fingerprint

Dive into the research topics of 'Meta-Dominance Analysis – A Tool for the Assessment of the Quality of Digital Behavioural Data'. Together they form a unique fingerprint.

Cite this