Comparing effectiveness estimates from randomized and nonrandomized studies, using subgroup analyses and individual patient data

Background: A possible explanation for differences in results from randomized clinical trials (RCTs) and those from nonrandomized studies is confounding. Besides this, it is well know that different patients are included by RCTs and nonrandomized studies. When there is effect modification the inclusion of different patient types may also cause differences between results from RCTs and nonrandomized studies. Objectives: To show how comparability of results across different designs is affected by the inclusion of different patient types and how to adjust for this. Methods: The effect of beta-blocker vs. diuretics use on the risk of non-fatal myocardial infarction (MI) was compared using data from 2 RCTs (the antihypertensive MRC trials), a case-control study, and a cohort study. Age by treatment interaction was assessed and confounding was adjusted for using restriction and (timevarying) regression. Effects were estimated as hazard ratios (HRs) or odd ratios (ORs), with 95% confidence intervals (95% CI), for beta-blocker vs. diuretic use. Results: After adjusting for confounding, both the RCT and case control estimates were similar, HR 1.09 (95% CI: 0.78;1.52) and OR 1.25 (95% CI: 0.91;1.72), but different from the cohort study HR 2.55 (95% CI: 1.90;3.42). The age by treatment interaction was nonsignificant. However, restriction to subjects aged > 65 years, showed comparable effect estimates: RCT HR 2.28 (95% CI: 1.16;4.52), case-control OR 1.61 (95% CI: 0.91;2.85) cohort study, HR 2.30 (95% CI: 1.53;3.45). Conclusions: In the presence of interaction, differences between results from RCT and nonrandomized studies can be due to the inclusion of different patients groups.