TY - JOUR
T1 - Reconciliation of inconsistent data sources by correction for measurement error
T2 - The feasibility of parameter re-use
AU - Pankowska, Paulina
AU - Bakker, Bart
AU - Oberski, Daniel L.
AU - Pavlopoulos, Dimitris
PY - 2018/1/1
Y1 - 2018/1/1
N2 - National Statistical Institutes (NSIs) often obtain information about a single variable from separate data sources. Administrative registers and surveys, in particular, often provide overlapping information on a range of phenomena of interest to official statistics. However, even though the two sources overlap, they both contain measurement error that prevents identical units from yielding identical values. Reconciling such separate data sources and providing accurate statistics, which is an important challenge for NSIs, is typically achieved through macro-integration. In this study we investigate the feasibility of an alternative method based on the application of previously obtained results from a recently introduced extension of the Hidden Markov Model (HMM) to newer data. The method allows a reconciliation of separate error-prone data sources without having to repeat the full HMM analysis, provided the estimated measurement error processes are stable over time. As we find that these processes are indeed stable over time, the proposed method can be used effectively for macro-integration, to reconciliate both first-order statistics-e.g. the size of temporary employment in the Netherlands-and second-order statistics-e.g. the amount of mobility from temporary to permanent employment.
AB - National Statistical Institutes (NSIs) often obtain information about a single variable from separate data sources. Administrative registers and surveys, in particular, often provide overlapping information on a range of phenomena of interest to official statistics. However, even though the two sources overlap, they both contain measurement error that prevents identical units from yielding identical values. Reconciling such separate data sources and providing accurate statistics, which is an important challenge for NSIs, is typically achieved through macro-integration. In this study we investigate the feasibility of an alternative method based on the application of previously obtained results from a recently introduced extension of the Hidden Markov Model (HMM) to newer data. The method allows a reconciliation of separate error-prone data sources without having to repeat the full HMM analysis, provided the estimated measurement error processes are stable over time. As we find that these processes are indeed stable over time, the proposed method can be used effectively for macro-integration, to reconciliate both first-order statistics-e.g. the size of temporary employment in the Netherlands-and second-order statistics-e.g. the amount of mobility from temporary to permanent employment.
KW - administrative data
KW - data quality
KW - Hidden Markov Model
KW - labour market transitions
KW - measurement error
KW - register data
KW - survey data
UR - http://www.scopus.com/inward/record.url?scp=85052646970&partnerID=8YFLogxK
U2 - 10.3233/SJI-170368
DO - 10.3233/SJI-170368
M3 - Article
AN - SCOPUS:85052646970
SN - 1874-7655
VL - 34
SP - 317
EP - 329
JO - Statistical Journal of the IAOS
JF - Statistical Journal of the IAOS
IS - 3
ER -