Title: Missing Data in Alcohol Clinical Trials: A Comparison of Methods
Abstract: Alcoholism: Clinical and Experimental ResearchVolume 37, Issue 12 p. 2152-2160 Original Article Missing Data in Alcohol Clinical Trials: A Comparison of Methods Kevin A. Hallgren, Kevin A. Hallgren Department of Psychology , University of New Mexico, Albuquerque, New MexicoSearch for more papers by this authorKatie Witkiewitz, Katie Witkiewitz Department of Psychology , University of New Mexico, Albuquerque, New MexicoSearch for more papers by this author Kevin A. Hallgren, Kevin A. Hallgren Department of Psychology , University of New Mexico, Albuquerque, New MexicoSearch for more papers by this authorKatie Witkiewitz, Katie Witkiewitz Department of Psychology , University of New Mexico, Albuquerque, New MexicoSearch for more papers by this author First published: 24 July 2013 https://doi.org/10.1111/acer.12205Citations: 108 Reprint requests: Kevin Hallgren, MS, Department of Psychology, University of New Mexico, 1 University of New Mexico, MSC03 2220, Albuquerque, NM 87131-0001; Tel.: 505-277-4121; Fax: 505-925-2301; E-mail: [email protected]. Read the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinkedInRedditWechat Abstract Background The rate of participant attrition in alcohol clinical trials is often substantial and can cause significant issues with regard to the handling of missing data in statistical analyses of treatment effects. It is common for researchers to assume that missing data is indicative of participant relapse, and under that assumption, many researchers have relied on setting all missing values to the worst-case scenario for the outcome (e.g., missing = heavy drinking). This sort of single-imputation method has been criticized for producing biased results in other areas of clinical research, but has not been evaluated within the context of alcohol clinical trials, and many alcohol researchers continue to use the missing = heavy drinking assumption. Methods Data from the COMBINE study, a multisite randomized clinical trial, were used to generate simulated situations of missing data under a variety of conditions and assumptions. We manipulated the sample size (n = 200, 500, and 1,000) and dropout rate (5, 10, 25, 30%) under 3 missing data assumptions (missing completely at random, missing at random, and missing not at random). We then examined the association between receiving naltrexone and heavy drinking during the first 10 weeks following treatment using 5 methods for treating missing data (complete case analysis [CCA], last observation carried forward [LOCF], missing = heavy drinking, multiple imputation [MI], and full information maximum likelihood [FIML]). Results CCA, LOCF, and missing = heavy drinking produced the most biased naltrexone effect estimates and standard errors under conditions that are likely to exist in randomized clinical trials. MI and FIML produced the least biased naltrexone effect estimates and standard errors. Conclusions Assuming that missing = heavy drinking produces biased results of the treatment effect and should not be used to evaluate treatment effects in alcohol clinical trials. Supporting Information Filename Description acer12205-sup-0001-SupplementMaterial.pdfapplication/PDF, 107.2 KB Data S1. Combined supplementary materials.pdf: example R syntax for using multiple imputation via the mice package, Mplus syntax for using full information maximum likelihood, and example data set. acer12205-sup-0002-SupplementaryResultmissing5-10.xlsxMS Excel, 36 KB Data S2. Supplementary results table, missing 5-10 percent.xlsx: tables of numeric results presented in Fig. 1 including results for n = 200 and n = 500 simulations when dropout rates ranged from 5 to 10%. acer12205-sup-0003-SupplementaryResultmissing25-30.xlsxMS Excel, 34.7 KB Data S3. Supplementary results table, missing 25-30 percent.xlsx: tables of numeric results presented in Fig. 2 including results for n = 200 and n = 500 simulations when dropout rates ranged from 25 to 30%. Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article. References Allison PD (2001) Missing Data. Sage, Thousand Oaks, CA. American Psychiatric Association (1994) Diagnostic and Statistical Manual (DSM-IV). 4th ed. American Psychiatric Association, Washington DC. Anton RF, O'Malley SS, Ciraulo DA, Cisler RA, Couper D, Donovan DM, Gastfriend DR, Hosking JD, Johnson BA, LoCastro JS, Longabaugh R, Mason BJ, Mattson ME, Miller WR, Pettinati HM, Randall CL, Swift R, Weiss RD, Williams LD, Zweben A (2006) Combined pharmacotherapies and behavioral interventions for alcohol dependence: the COMBINE study: a randomized controlled trial. JAMA 295: 2003–2017. Arndt S (2009) Stereotyping and the treatment of missing data for drug and alcohol clinical trials. Subst Abuse Treat Prev Policy 4: 2. Asparouhov T, Muthén B (2010) Multiple imputation with Mplus version 2. Sept. 29. Available at: www.statmodel.com/download/Imputations7.pdf. Accessed March 28, 2013. Baraldi AN, Enders CK (2010) An introduction to modern missing data analyses. J Sch Psychol 48: 5–37. Barnes SA, Larsen MD, Schroeder D, Hanson A, Decker PA (2010) Missing data assumptions and methods in a smoking cessation study. Addiction 105: 431–437. Burton A, Altman DG, Royston P, Holder RL (2006) The design of simulation studies in medical statistics. Stat Med 25: 4279–4292. van Buuren S, Boshuizen HC, Knook DL (1999) Multiple imputation of missing blood pressure covariates in survival analysis. Stat Med 18: 681–694. van Buuren S, Groothuis-Oudshoort K (2011) mice: multivariate imputation by chained equations in R. J Stat Softw 45: 1–67. Collins LM, Schafer JL, Kam CM (2001) A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychol Methods 6: 330–351. COMBINE Study Research Group (2003) Testing combined pharmacotherapies and behavioral interventions in alcohol dependence: rationale and methods. Alcohol Clin Exp Res 27: 1107–1122. Enders CK (2011) Missing not at random models for latent growth curve analyses. Psychol Methods 16: 1–16. Falk D, Wang XQ, Liu L, Fertig J, Mattson M, Ryan M, Johnson B, Stout R, Litten RZ (2010) Percentage of subjects with no heavy drinking days: evaluation as an efficacy endpoint for alcohol clinical trials. Alcohol Clin Exp Res 34: 2022–2034. FDA (2006) Medical Review of Vivitrol: 21–897. US Government, Rockville, Maryland. Fertig JB, Ryan ML, Falk DE, Litten RZ, Mattson ME, Ransom J, Rickman WJ, Scott C, Ciraulo D, Green AI, Tiouririne NA, Johnson B, Pettinati H, Strain EC, Devine E, Brunette MF, Kampman K, Tompkins DA, Stout R (2012) A double-blind, placebo-controlled trial assessing the efficacy of levetiracetam extended-release in very heavy drinking alcohol-dependent patients. Alcohol Clin Exp Res 36: 1421–1430. First MB, Gibbon M, Spitzer RL, Williams JBW, Benjamin LS (1997) Structured Clinical Interview for DSM-IV Axis II Personality Disorders (SCID-II). American Psychiatric Press, Inc., Washington, DC. Graham JW (2009) Missing data analysis: making it work in the real world. Annu Rev Psychol 60: 549–576. Hallgren K (2013) Conducting simulation studies in the R programming environment. Tutor Quant Method Psychol 9: 43–60. Hedden SL, Woolson RF, Carter RE, Palesch Y, Upadhyaya HP, Malcolm RJ (2009) The impact of loss to follow-up on hypothesis tests of the treatment effect for several statistical methods in substance abuse clinical trials. J Subst Abuse Treat 37: 54–63. Hedeker D, Gibbons RD (1997) Application of random-effects pattern-mixture models for missing data in longitudinal studies. Psychol Methods 2: 64–78. Hedeker D, Mermelstein RJ, Demirtas H (2007) Analysis of binary outcomes with missing data: missing = smoking, last observation carried forward, and a little multiple imputation. Addiction 102: 1564–1573. Johnson BA, Rosenthal N, Capece JA, Wiegand F, Mao L, Beyers K, McKay A, Ait-Daoud N, Anton RF, Ciraulo DA, Kranzler HR, Mann K, O'Malley SS, Swift RM (2007) Topiramate for treating alcohol dependence: a randomized controlled trial. JAMA 298: 1641–1651. Lane P (2008) Handling drop-out in longitudinal clinical trials: a comparison of the LOCF and MMRM approaches. Pharm Stat 7: 93–106. Little RA, D'Agostino R, Cohen ML, Dickersin K, Emerson SS, Farrar JT, Frangakis C, Hogan JW, Molenberghs G, Murphy SA, Neaton JD, Rotnitzky A, Scharfstein D, Shih WJ, Siegel JP, Stern H (2012) The prevention and treatment of missing data in clinical trials. N Engl J Med 367: 1355–1360. Little RA, Rubin DB (2002) Statistical Analysis with Missing Data. 2nd ed. Wiley, New York. Liu G, Gould AL (2002) Comparison of alternative strategies for analysis of longitudinal trials with dropouts. J Biopharm Stat 12: 207–226. Mackenzie A, Funderburk FR, Allen RP, Stefan RL (1987) The characteristics of alcoholics frequently lost to follow-up. J Stud Alcohol 48: 119–123. Mallinckrodt CH, Clark SW, David SR (2001) Accounting for dropout bias using mixed-effects models. J Biopharm Stat 11: 9–21. Mallinckrodt CH, Lane PW, Schnell D, Peng Y, Mancuso JP (2008) Recommendations for the primary analysis of continuous endpoints in longitudinal clinical trials. Drug Inf J 42: 303–319. Miller WR, Del Boca FK (1994) Measurement of drinking behavior using the form 90 family of instruments. J Stud Alcohol Suppl 12: 112–118. Molenberghs G, Thijs H, Jansen I, Beunckens C (2004) Analyzing incomplete longitudinal clinical trial data. Biostatistics 5: 445–464. Muthén LK, Muthén B (2012) Mplus Users Guide. 7th ed. Muthén & Muthén, Los Angeles, CA. National Research Council (2010) The Prevention and Treatment of Missing Data in Clinical Trials. National Academies Press, Washington, DC. Available at: http://www.nap.edu/catalog.php?recordid-12955). Accessed Papp KA, Fonjallaz P, Casset-Semanaz F, Krueger JG, Mittkowski KM (2008) Approaches to reporting long-term data. Curr Med Res Opin 24: 2001–2008. Postel MG, De Haan HA, Ter Huurne ED, Van der Palen J, Becker ES, De Jong CAJ (2011) Attrition in web-based treatment for problem drinkers. J Med Internet Res 13: e117. Prisciandaro JJ, Rembold J, Brown DG, Brady KT, Tolliver BK (2011) Predictors of clinical trial dropout in individuals with co-occurring bipolar disorder and alcohol dependence. Drug Alcohol Depend 118: 493–496. R Development Core Team (2012) R: A Language and Environment for Statistical Computing [Computer Software] Version 2.15.0. R Foundation for Statistical Computing, Vienna, Austria. Rubin DB (1976) Inference and missing data. Biometrika 63: 581–592. Schafer JL, Graham JW (2002) Missing data: our view of the state of the art. Psychol Methods 7: 147–177. Siddiqui O (2011) MMRM versus MI in dealing with missing data—a comparison based on 25 NDA data sets. J Biopharm Stat 21: 423–436. Siddiqui O, Hung HMJ, O'Neill R (2009) MMRM vs LOCF: a comprehensive comparison based on simulation study and 25 NDA datasets. J Biopharm Stat 19: 227–246. Sobell LC, Sobell MB, Maisto SA (1984) Follow-up attrition in alcohol treatment studies: is "no news" bad news, good news or no news? Drug Alcohol Depend 13: 1–7. Suh JJ, Pettinati HM, Kampman KM, O'Brien CP (2008) Gender differences in predictors of treatment attrition with high dose naltrexone in cocaine and alcohol dependence. Am J Addict 17: 463–468. Witkiewitz K, Bush T, Magnusson LB, Carlini BH, Zbikowski SM (2012) Trajectories of cigarettes per day during the course of telephone tobacco cessation counseling services: a comparison of missing data models. Nicotine Tob Res 14: 1100–1104. Wu MC, Carroll RJ (1988) Estimation and comparison of changes in the presence of informative right censoring by modeling the censoring process. Biometrics 44: 175–188. Zhang W, Yiu-Fai Y (2011) A tutorial on structural equation modeling with incomplete observations: multiple imputation and FIML methods using SAS. July 21. Available at: http://support.sas.com/rnd/app/stat/papers/imps2011_FIML.pdf. Accessed March 28, 2013. Citing Literature Volume37, Issue12December 2013Pages 2152-2160 ReferencesRelatedInformation