journal article·2024-04-25Open access

A Comprehensive Review and Tutorial on Confounding Adjustment Methods for Estimating Treatment Effects Using Observational Data

Amy X. Shi, Paul N. Zivich, Haitao Chu

Applied Sciences· Volume 14· Issue 9

Abstract

Controlling for confounding bias is crucial in causal inference. Causal inference using data from observational studies (e.g., electronic health records) or imperfectly randomized trials (e.g., imperfect randomization or compliance) requires accounting for confounding variables. Many different methods are currently employed to mitigate bias due to confounding. This paper provides a comprehensive review and tutorial of common estimands and confounding adjustment approaches, including outcome regression, g-computation, propensity score, and doubly robust methods. We discuss bias and precision, advantages and disadvantages, and software implementation for each method. Moreover, approaches are illustrated empirically with a reproducible case study. We conclude that different scientific questions are better addressed by certain estimands. No estimand is uniformly more appropriate. Upon selecting an estimand, decisions on which estimator can be driven by performance and available background knowledge.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Causal Inference Techniques · Health Systems, Economic Evaluations, Quality of Life · Statistical Methods and Inference

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Late Stage Development, Cardiovascular, Renal and Metabolism (CVRM), Biopharmaceuticals R&D, AstraZeneca, Durham, NC 27703, USA

Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA

Statistical Research and Data Science Center, Pfizer Inc., New York, NY 10001, USA

Division of Biostatistics and Health Data Science, University of Minnesota at Twin Cities, Minneapolis, MN 55455, USA

References (80)

Show source-provided references
1. Friedman, L.M., Furberg, C.D., DeMets, D.L., Reboussin, D.M., and Granger, C.B. (2015). Fundamentals of Clinical Trials, Springer.10.1007/978-3-319-18539-2 ↗
2. Yang, H., and Yu, B. (2021). Real-World Evidence in Drug Development and Evaluation, CRC Press.10.1201/9780429398674 ↗
3. ICH (2001). E10 Choice of Control Group and Related Issues in Clinical Trials, ICH.
4. Generalizing the per-protocol treatment effect: The case of ACTG A509510.1177/1740774518806311 ↗
5. Reducing Bias in Estimates of Per Protocol Treatment Effects: A Secondary Analysis of a Randomized Clinical Trial10.1001/jamanetworkopen.2023.25907 ↗
6. FDA (2019). Rare Diseases: Natural History Studies for Drug Development, Guidance for Industry.
7. FDA (2021). Considerations for the Use of Real-World Data and Real-World Evidence to Support Regulatory Decision-Making for Drug and Biological Products.
8. Using multivariate matched sampling and regression adjustment to control bias in observational studies
9. The central role of the propensity score in observational studies for causal effects10.1093/biomet/70.1.41 ↗
10. Optimal matching for observational studies10.1080/01621459.1989.10478868 ↗
11. Hernan, M.A., and Robins, J.M. (2023). Causal Inference: What If, CRC Press.
12. Causal inference and effect estimation using observational data10.1136/jech-2022-219267 ↗
13. Ding, P. (2023). A first course in causal inference. arXiv.
14. Introduction to computational causal inference using reproducible Stata, R, and Python code: A tutorial10.1002/sim.9234 ↗
15. Causal inference: A missing data perspective10.1214/18-STS645 ↗
16. Dealing with limited overlap in estimation of average treatment effects10.1093/biomet/asn055 ↗
17. Balancing covariates via propensity score weighting10.1080/01621459.2016.1260466 ↗
18. Addressing extreme propensity scores via the overlap weights
19. Estimating causal effects from epidemiological data10.1136/jech.2004.029496 ↗
20. The consistency statement in causal inference: A definition or an assumption?10.1097/EDE.0b013e31818ef366 ↗
21. Zivich, P.N., Cole, S.R., and Westreich, D. (2022). Positivity: Identifiability and estimability. arXiv.
22. Planning a method for covariate adjustment in individually randomised trials: A practical guide10.1186/s13063-022-06097-z ↗
23. On the role of the propensity score in efficient semiparametric estimation of average treatment effects10.2307/2998560 ↗
24. Matching as an econometric evaluation estimator10.1111/1467-937X.00044 ↗
25. A new approach to causal inference in mortality studies with a sustained exposure period—Application to control of the healthy worker survivor effect10.1016/0270-0255(86)90088-6 ↗
26. Identifiability, exchangeability, and epidemiological confounding10.1093/ije/15.3.413 ↗
27. Large sample properties of matching estimators for average treatment effects10.1111/j.1468-0262.2006.00655.x ↗
28. Matching methods for causal inference: A review and a look forward10.1214/09-STS313 ↗
29. Guo, S., and Fraser, M.W. (2014). Propensity Score Analysis: Statistical Methods and Applications, SAGE Publications.
30. An introduction to propensity score methods for reducing the effects of confounding in observational studies10.1080/00273171.2011.568786 ↗
31. Semiparametric efficiency in multivariate regression models with missing data10.1080/01621459.1995.10476494 ↗
32. Marginal structural models and causal inference in epidemiology10.1097/00001648-200009000-00011 ↗
33. Estimation of causal effects using propensity score weighting: An application to data on right heart catheterization10.1023/A:1020371312283 ↗
34. Efficient estimation of average treatment effects using the estimated propensity score10.1111/1468-0262.00442 ↗
35. Comparison of propensity score methods and covariate adjustment: Evaluation in 4 cardiovascular studies10.1016/j.jacc.2016.10.060 ↗
36. Relaxing the rule of ten events per variable in logistic and Cox regression10.1093/aje/kwk052 ↗
37. Propensity score weighting for covariate adjustment in randomized clinical trials10.1002/sim.8805 ↗
38. FDA (2023). Adjusting for Covariates in Randomized Clinical Trials for Drugs and Biological Products.
39. Implementation of G-computation on a simulated data set: Demonstration of a causal inference technique10.1093/aje/kwq472 ↗
40. Randomization does not justify logistic regression10.1214/08-STS262 ↗
41. Improving propensity score weighting using machine learning10.1002/sim.3782 ↗
42. Stratification for the propensity score compared with linear regression techniques to assess the effect of treatment or exposure10.1002/sim.3133 ↗
43. Comparison of multivariate matching methods: Structures, distances, and algorithms10.1080/10618600.1993.10474623 ↗
44. Propensity score weighting under limited overlap and model misspecification10.1177/0962280220940334 ↗
45. Doubly robust estimation in missing data and causal inference models10.1111/j.1541-0420.2005.00377.x ↗
46. Lamm, M., and Yung, Y.-F. (2017, January 2–5). Estimating Causal Effects from Observational Data with the CAUSALTRT Procedure. Proceedings of the SAS Global Forum 2017 Conference, Orlando, FL, USA. Available online: http://support.sas.com/resources/papers/proceedings17/SAS0374-2017.pdf.
47. Targeted maximum likelihood estimation for causal inference in observational studies10.1093/aje/kww165 ↗
48. Balance diagnostics after propensity score matching10.21037/atm.2018.12.10 ↗
49. Assessing balance in measured baseline covariates when using many-to-one matching on the propensity-score10.1002/pds.1674 ↗
50. Yang, D., and Dalton, J.E. (2012). SAS Global Forum, Available online: https://support.sas.com/resources/papers/proceedings12/335-2012.pdf.
51. Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies10.1002/sim.6607 ↗
52. Prognostic score–based balance measures can be a useful diagnostic for propensity score methods in comparative effectiveness research10.1016/j.jclinepi.2013.01.013 ↗
53. Ridgeway, G., McCaffrey, D.F., Morral, A.R., Cefalu, M., Burgette, L.F., Pane, J.D., and Griffin, B.A. (2022). Toolkit for Weighting and Analysis of Nonequivalent Groups: A Tutorial for the R TWANG Package, RAND Corporation.
54. Fong, C., Ratkovic, M., Imai, K., and Hazlett, C. (2024, April 21). Package ‘cbps’. Available online: https://cran.r-project.org/web/packages/CBPS/CBPS.pdf.
55. Propensity score weighting analysis and treatment effect discovery10.1177/0962280218781171 ↗
56. Haris, A., and Chan, G. (2015). ATE: Inference for average treatment effects using covariate balancing. R Package Version 0.2.0.
57. Greifer, N., and Greifer, M.N. (2024, April 21). Package ‘WeightIt’. CRAN. Available online: https://cran.r-project.org/web/packages/WeightIt/index.html.
58. Bodory, H., and Huber, M. (2024, April 21). The Causal Weight Package for Causal Inference in R. Available online: https://cran.r-project.org/web/packages/causalweight/causalweight.pdf.10.1016/j.softx.2022.101294 ↗
59. Zubizarreta, J.R., Li, Y., Kim, K., Allouah, A., and Greifer, N. (2024, April 21). Package ‘sbw’. Available online: https://cran.r-project.org/web/packages/sbw/sbw.pdf.
60. Zhou, T., Tong, G., Li, F., and Thomas, L.E. (2020). PSweight: An R package for propensity score weighting analysis. arXiv.
61. AIPW: An r package for augmented inverse probability–weighted estimation of average causal effects10.1093/aje/kwab207 ↗
62. Glynn, A., Quinn, K., Quinn, M.K., Quinn, K.M., and Estimator, P.W. (2024, April 21). Package ‘CausalGAM’. Available online: https://cran.r-project.org/web/packages/CausalGAM/CausalGAM.pdf.
63. Nonparametric causal effects based on incremental propensity score interventions10.1080/01621459.2017.1422737 ↗
64. tmle: An R package for targeted maximum likelihood estimation10.18637/jss.v051.i13 ↗
65. Mayer, I., Zhao, P., Greifer, N., Huntington-Klein, N., and Josse, J. (2024, April 21). Cran Task View: Causal Inference. Available online: https://misscausal.gitlabpages.inria.fr/misscausal.gitlab.io/files/ctv/CausalInference.html.
66. Zivich, P.N., Davidson-Pilon, C., Diong, J., and Reger, D. (2024, April 21). Pzivich/zEpid: v0.9.1 (v0.9.1). Available online: https://zenodo.org/records/7242696.
67. Zivich, P.N., Klose, M., Cole, S.R., Edwards, J.K., and Shook-Sa, B.E. (2022). Delicatessen: M-estimation in Python. arXiv.
68. Sharma, A., and Kiciman, E. (2024, April 21). DoWhy: A Python Package for Causal Inference. Available online: https://github.com/py-why/dowhy.10.1016/j.softx.2022.101294 ↗
69. SAS (2023). The CAUSALTRT Procedure in in SAS/STAT® 15.3 User’s Guide, SAS Institute Inc.
70. The effectiveness of right heart catheterization in the initial care of critically III patients10.1001/jama.1996.03540110043030 ↗
71. The relative ability of different propensity score methods to balance measured covariates between treated and untreated subjects in observational studies10.1177/0272989X09341755 ↗
72. Ren, J., Cislo, P., Cappelleri, J.C., Hlavacek, P., and DiBonaventura, M. (2023). Comparing g-computation, propensity score-based weighting, and targeted maximum likelihood estimation for analyzing externally controlled trials with both measured and unmeasured confounders: A simulation study. BMC Med. Res. Methodol., 23.10.1186/s12874-023-01835-6 ↗
73. The calculus of M-estimation10.1198/000313002753631330 ↗
74. Causal inference without balance checking: Coarsened exact matching10.1093/pan/mpr013 ↗
75. Yang, S., and Ding, P. (2017). Asymptotic causal inference with observational studies trimmed by the estimated propensity scores. arXiv.10.1093/biomet/asy008 ↗
76. Doubly robust estimation of causal effects10.1093/aje/kwq439 ↗
77. Comment: Performance of double-robust estimators when “inverse probability” weights are highly variable10.1214/07-STS227D ↗
78. Machine learning for causal inference: On the use of cross-fit estimators10.1097/EDE.0000000000001332 ↗
79. Challenges in obtaining valid causal effect estimates with machine learning algorithms10.1093/aje/kwab201 ↗
80. Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies10.1093/pan/mpr025 ↗