Reconciliation of inconsistent data sources by correction for measurement error: The feasibility of parameter re-use

Paulina Pankowska*, Bart Bakker, Daniel L. Oberski, Dimitris Pavlopoulos

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

National Statistical Institutes (NSIs) often obtain information about a single variable from separate data sources. Administrative registers and surveys, in particular, often provide overlapping information on a range of phenomena of interest to official statistics. However, even though the two sources overlap, they both contain measurement error that prevents identical units from yielding identical values. Reconciling such separate data sources and providing accurate statistics, which is an important challenge for NSIs, is typically achieved through macro-integration. In this study we investigate the feasibility of an alternative method based on the application of previously obtained results from a recently introduced extension of the Hidden Markov Model (HMM) to newer data. The method allows a reconciliation of separate error-prone data sources without having to repeat the full HMM analysis, provided the estimated measurement error processes are stable over time. As we find that these processes are indeed stable over time, the proposed method can be used effectively for macro-integration, to reconciliate both first-order statistics-e.g. the size of temporary employment in the Netherlands-and second-order statistics-e.g. the amount of mobility from temporary to permanent employment.

Original languageEnglish
Pages (from-to)317-329
Number of pages13
JournalStatistical Journal of the IAOS
Volume34
Issue number3
DOIs
Publication statusPublished - 1 Jan 2018

Keywords

  • administrative data
  • data quality
  • Hidden Markov Model
  • labour market transitions
  • measurement error
  • register data
  • survey data

Fingerprint

Dive into the research topics of 'Reconciliation of inconsistent data sources by correction for measurement error: The feasibility of parameter re-use'. Together they form a unique fingerprint.

Cite this