Balance measures for determining optimal caliper width in propensity score matching: A simulation study

Background: When estimating the effects of exposure in observational data, propensity score (PS) methods can be used to control for confounding. When PS matching is used, often a pre-specified caliper width is applied. A crucial part of this matching approach is assessment of how close the co-variate distributions are in the two treatment groups, i.e. balance. The choice of the caliper may influence the balance of covariates between treatment groups and, therefore, the bias and precision of the PS adjusted effect estimate. Although several balance measures have been described as tools for PS model selection, their role in choosing optimal caliper widths is not well studied. Objectives: To explore the usefulness of balance measures in selecting the optimal caliper width for propensity score matching. Methods: We conducted Monte Carlo simulations to assess the usefulness of balance measures (standardized difference) to select optimal caliper width and PS models that yielded the least biased estimates. In different simulations with binary covariates, exposure and outcome status, different sample sizes (n = 500, 1,000, 3,000) and strength of exposure-outcome association (OR = 1, 2) were considered. Caliper widths were varied between 0.05 and 0.6 (steps of 0.05) of the standard deviation of the PS. The balance of covariates between PS matched groups was assessed using the standardized difference (SDif) for each PS model-caliper width combination. PS model with the lowest value of SDif (i.e. most optimal balance) was selected and treatment effects were estimated using conditional logistic regression. Results: The PS models selected using various caliper widths were closely related and these models often included interaction and squared terms. When using balance measures to select a certain PS model, the choice for a certain PS model seems to have much more impact on bias and precision of exposure effects than the caliper width used. Conclusions: Balance measures are useful tools for selecting the optimal PS model and the PS model selected has more impact on bias and precision than the caliper width that is used in PS matching.