Quality Control, Data Cleaning, Imputation

Dawei Liu; Hanne I. Oberman; Johanna Muñoz; Jeroen Hoogland; Thomas P. A. Debray

doi:10.1007/978-3-031-36678-9_2

Quality Control, Data Cleaning, Imputation

Dawei Liu, Hanne I. Oberman, Johanna Muñoz, Jeroen Hoogland, Thomas P. A. Debray^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Chapter › Academic › peer-review

Abstract

This chapter addresses important steps during the quality assurance and control of RWD, with particular emphasis on the identification and handling of missing values. A gentle introduction is provided on common statistical and machine learning methods for imputation. We discuss the main strengths and weaknesses of each method, and compare their performance in a literature review. We motivate why the imputation of RWD may require additional efforts to avoid bias, and highlight recent advances that account for informative missingness and repeated observations. Finally, we introduce alternative methods to address incomplete data without the need for imputation.

Original language	English
Title of host publication	Clinical Applications of Artificial Intelligence in Real-World Data
Pages	7-36
Number of pages	30
DOIs	https://doi.org/10.1007/978-3-031-36678-9_2
Publication status	Published - 4 Nov 2023

Keywords

Missing data
Imputation
Missing atrandom
Missing not at random
Missingcompletely at rando
Informativemissingness
Sporadically missing
Systematically missing
Joint modelling imputation
Conditional modelling imputation
Machine learning imputation
Nearest neighbor
Matrix completion
Tree-based ensembles
Support vectormachines
Neural networks
Rubin’s rules
Pattern submodels
Surrogate splits
Missingindicator
Heckman selection model

Access to Document

10.1007/978-3-031-36678-9_2

978-3-031-36678-9_2Final published version, 1.28 MBLicence: Taverne

Cite this

@inbook{09f57c7e63b34269a23a4cb176d0a085,

title = "Quality Control, Data Cleaning, Imputation",

abstract = "This chapter addresses important steps during the quality assurance and control of RWD, with particular emphasis on the identification and handling of missing values. A gentle introduction is provided on common statistical and machine learning methods for imputation. We discuss the main strengths and weaknesses of each method, and compare their performance in a literature review. We motivate why the imputation of RWD may require additional efforts to avoid bias, and highlight recent advances that account for informative missingness and repeated observations. Finally, we introduce alternative methods to address incomplete data without the need for imputation.",

keywords = "Missing data, Imputation, Missing atrandom, Missing not at random, Missingcompletely at rando, Informativemissingness, Sporadically missing, Systematically missing, Joint modelling imputation, Conditional modelling imputation, Machine learning imputation, Nearest neighbor, Matrix completion, Tree-based ensembles, Support vectormachines, Neural networks, Rubin{\textquoteright}s rules, Pattern submodels, Surrogate splits, Missingindicator, Heckman selection model",

author = "Dawei Liu and Oberman, {Hanne I.} and Johanna Mu{\~n}oz and Jeroen Hoogland and Debray, {Thomas P. A.}",

year = "2023",

month = nov,

day = "4",

doi = "10.1007/978-3-031-36678-9_2",

language = "English",

isbn = "9783031366772",

pages = "7--36",

booktitle = "Clinical Applications of Artificial Intelligence in Real-World Data",

}

TY - CHAP

T1 - Quality Control, Data Cleaning, Imputation

AU - Liu, Dawei

AU - Oberman, Hanne I.

AU - Muñoz, Johanna

AU - Hoogland, Jeroen

AU - Debray, Thomas P. A.

PY - 2023/11/4

Y1 - 2023/11/4

N2 - This chapter addresses important steps during the quality assurance and control of RWD, with particular emphasis on the identification and handling of missing values. A gentle introduction is provided on common statistical and machine learning methods for imputation. We discuss the main strengths and weaknesses of each method, and compare their performance in a literature review. We motivate why the imputation of RWD may require additional efforts to avoid bias, and highlight recent advances that account for informative missingness and repeated observations. Finally, we introduce alternative methods to address incomplete data without the need for imputation.

AB - This chapter addresses important steps during the quality assurance and control of RWD, with particular emphasis on the identification and handling of missing values. A gentle introduction is provided on common statistical and machine learning methods for imputation. We discuss the main strengths and weaknesses of each method, and compare their performance in a literature review. We motivate why the imputation of RWD may require additional efforts to avoid bias, and highlight recent advances that account for informative missingness and repeated observations. Finally, we introduce alternative methods to address incomplete data without the need for imputation.

KW - Missing data

KW - Imputation

KW - Missing atrandom

KW - Missing not at random

KW - Missingcompletely at rando

KW - Informativemissingness

KW - Sporadically missing

KW - Systematically missing

KW - Joint modelling imputation

KW - Conditional modelling imputation

KW - Machine learning imputation

KW - Nearest neighbor

KW - Matrix completion

KW - Tree-based ensembles

KW - Support vectormachines

KW - Neural networks

KW - Rubin’s rules

KW - Pattern submodels

KW - Surrogate splits

KW - Missingindicator

KW - Heckman selection model

U2 - 10.1007/978-3-031-36678-9_2

DO - 10.1007/978-3-031-36678-9_2

M3 - Chapter

SN - 9783031366772

SN - 9783031366789

SP - 7

EP - 36

BT - Clinical Applications of Artificial Intelligence in Real-World Data

ER -

Quality Control, Data Cleaning, Imputation

Abstract

Keywords

Access to Document

Fingerprint

Cite this