University of Southern California Dissertations and Theses

An exploration of nonresponse with multiple imputation in the Television, School, and Family Project

(USC Thesis Other)

An exploration of nonresponse with multiple imputation in the Television, School, and Family Project

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content INFORMATION TO USERS This manuscript has been reproduced from the microfilm master. UMI films the text directly from the original or copy submitted. Thus, some thesis and dissertation copies are in typewriter face, while others may be from any type of computer printer. The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleed through, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send UMI a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion. Oversize materials (e.g., maps, drawings, charts) are reproduced by sectioning the original, beginning at the upper left-hand comer and continuing from left to right in equal sections with small overlaps. Each original is also photographed in one exposure and is included in reduced form at the back of the book. Photographs included in the original manuscript have been reproduced xerographically in this copy. Higher quality 6” x 9” black and white photographic prints are available for any photographs or illustrations appearing in this copy for an additional charge. Contact UMI directly to order. UMI A Bell & Howell Information Company 300 North Zeeb Road, Ann Arbor MI 48106-1346 USA 313/761-4700 800/521-0600 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. AN EXPLORATION OF NONRESPONSE WITH MULTIPLE IMPUTATION IN THE TELEVISION, SCHOOL, AND FAMILY PROJECT by William Black Howells A Thesis Presented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree MASTER OF SCIENCE (Applied Biometry and Epidemiology) December 1996 Copyright 1996 William Black Howells Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. UMI Num ber: 13 8 3 5 3 1 UMI Microform 1383531 Copyright 1997, by UMI Company. All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. UMI 300 North Zeeb Road Ann Arbor, MI 48103 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. UNIVERSITY O F SOUTHERN CALIFORNIA TH E GRADUATE SCHOOL UNIVERSITY PARK LOS ANGELES. CALIFORNIA 9 0 0 0 7 This thesis, written by William Black Howells under the direction of h .iS ...T hesis Committee, and approved by all its members, has been pre sented to and accepted by the Dean of The Graduate School, in partial fulfillment of the requirements fo r the degree of Applied Biometry and Epidemiology Dean T int* October 10, 1996 THESIS COMMITTEE Ckairman Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. ACKNOWLEDGEMENTS I realize that many people have contributed to my steady progress through this degree and toward this thesis. I want to take this opportunity to acknowledge and thank them. Thanks to John Graham for sparking my interest in analysis with missing data and for coaching a memorable softball team. Thanks to Joseph Schafer for the use of his programs and prompt email suggestions. Thanks to Jean Richardson, Brian Flay and the TVSFP writing group for the use of their data and for their support. Thanks to Rich Pinder for help with the DMV data. Thanks to Stan Azen for his patient guidance. Thanks to Jonathan Buckley and the Childrens Cancer Group for employment and the use of computers. Thanks to the thesis committee, chaired by Clyde Dent with Jim Gauderman and Wendy Mack. A special thanks to my new wife Analyn for her patience throughout this process. And finally, thanks to God without whom nothing is possible. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. iii TABLE OF CONTENTS Acknowledgements........................................................................................................ii List of Tables ..............................................................................................................v Abstract vi Page 1. Introduction..............................................................................................................1 1.1 Missing data theory..........................................................................................3 1.1.1 The response mechanism ................................................................... 3 1.1.2 ML estimation..................................................................................... 5 1.1.3 ML estimation with mixed continous and categorical data ..............7 1.1.4 Multiple imputation ........................................................................... 9 1.1.5 Proper imputation via data augmentation......................................... II 1.2 Missing data in practice ................................................................................ 13 1.2.1 Hypertension tria l..............................................................................13 1.2.2 Testing the MCAR assumption......................................................... 14 1.2.3 Missing data in psychosocial research ............................................. 14 1.2.4 The collection of additional data....................................................... 17 1.3 The TVSFP/ Latchkey study.......................................................................... 18 2. Methods ...........................................................................................................20 2.1 Plan of analysis..............................................................................................21 2.2 Survey d a ta ....................................................................................................22 2.3 DMV d a ta ......................................................................................................27 2.4 Multiple imputation models ..........................................................................30 2.4.1 Evaluating the response mechanism.................................................30 2.4.2 Choosing DMV variables for the imputation model........................ 31 2.4.3 Data augmentation parameters.........................................................33 2.4.4 Choosing the number of imputations ...............................................35 2.5 Analysis of interest........................................................................................35 2.5.1 Obtaining repeated-imputation inferences ...................................... 36 2.5.2 Comparison of models .....................................................................37 2.6 Computer resources ......................................................................................38 3. Results ...........................................................................................................38 3.1 Preparation of data ........................................................................................38 3.1.1 Factor analysis .................................................................................38 3.1.2 Standardizing and averaging item s.................................................. 41 3.2 Analysis with missing d ata............................................................................43 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. iv 3.2.1 Selection of DMV variables..............................................................43 3.2.2 Tests for M CAR................................................................................ 46 3.2.3 Ninth Grade item missingness ..........................................................48 3.2.4 Data augmentation param eters..........................................................49 3.2.5 Descriptive statistics of imputed datasets..........................................51 3.3 Comparison of models ....................................................................................52 3.3.1 Complete case vs ignorable model without DMV (I vs II).................54 3.3.2 Complete case vs nonignorable model without DMV (I vs IV) .... 55 3.3.3 Ignorable models without and with DMV (II vs III) ....................... 57 3.3.4 Nonignorable models without and with DMV (TV vs V ) ................. 57 3.3.5 Log transformation of tobacco use (12th)..........................................59 4. Discussion .............................................................................................................59 4.1 Response mechanism ......................................................................................59 4.2 Ignorable models............................................................................................. 63 4.3 Nonignorable models ..................................................................................... 65 4.4 DMV models...................................................................................................66 4.5 Model mispecification ................................................................................... 67 4.6 Conclusions..................................................................................................... 69 Bibliography .............................................................................................................71 Appendix A .............................................................................................................76 Appendix B .............................................................................................................80 Appendix C .............................................................................................................82 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. LIST OF TABLES Table Page 1 Validity of means and regression coefficients under three missing data patterns .................................................................................................... 6 2 Factor loadings and Cronbach's alpha for Ninth and Twelfth Grade surveys 39 3 Mean, standard deviation and skewness for Ninth and Twelfth Grade items and standardized composite variables .................................................... 42 4 Univariate DMV predictors of attrition at Twelfth Grade ..............................44 5 Multivariate DMV predictors of attrition at Twelfth Grade ............................. 45 6 Univariate tests of the MCAR assumption for continuous and discrete variables...............................................................................................46 7 Ninth Grade item missingness for stayers and droppers at Twelfth Grade 48 8 Cell probabilities (tt) for multiple imputation model of tobacco use (12th) with DMV variables.......................................................................................... 49 9 Cell means (p) of other females for multiple imputation model of tobacco use (12th) with DMV variables ...................................................... 50 10 Column of tobacco use (12th) in the variance-covariance matrix (2) for multiple imputation model of tobacco use (12th) with DMV variables ..........50 11 Descriptive statistics for imputed datasets of imputation models for tobacco use (12th) ...................................................................................... 52 12 Repeated-imputation inferences for linear regression coefficients for all models of tobacco use (12th).................................................................. 53 13 Differences in observed significance levels between models I and I I ............. 55 14 Differences in observed significance levels between models I and I V 56 15 Differences in observed significance levels between models II and III 58 16 Differences in observed significance levels between models IV and V ......... 58 17 Correlations between corresponding Ninth and Twelfth Grade composite variables ...................................................................................................61 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. vi ABSTRACT The Television, School, and Family Project (TVSFP), a study of health behavior beginning in 1986, followed a cohort of adolescents from Seventh Grade through two years post high school with loss of subjects at each of four followup survey measurements. The most substantial attrition (45%) occurred between the Ninth and Twelfth Grades and provided the focus for this paper. A linear regression of several Twelfth Grade outcomes was adopted as the analysis of interest. The outcomes were Twelfth Grade tobacco, alcohol, marijuana, and hard drug use as well as depression and drinking/ riding with drug use. Missing data resulted both from subject attrition and item missingness. The analysis presented here followed a general strategy set out by Schafer (1991) who employed a model for mixed continuous and categorical data with missing values (Little and Schluchter, 1985), data augmentation (Tanner and Wong, 1987; Li, 1988), and multiple imputation (Rubin, 1987). Other features of the analysis were the evaluation of the response mechanism, the incorporation of archival data in the form of motor vehicle records (DMV), and the exploration of the sensitivity of inferences to nonignorable nonresponse. Conclusions: Regression coefficients from models of ignorable nonresponse were similar in observed significance levels to those of complete case analyses. In contrast, inferences proved sensitive to nonignorable models. Imputations based on DMV data were not very different from ignorable models without DMV variables. In this study, if researchers believe that the data were not missing at random and thus the response nonignorable, complete case linear regression coefficients should be interpreted with caution. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 1 1. Introduction Research involving human subjects faces the murky reality of missing observations. Analysis with missing data exists in the complex nexus of theory, methods, design, and data collection. The practice is further complicated by the diverse nature of research designs, the multiplicity of research areas, and the variety of approaches to missing data analysis. In the foundational work Statistical Analysis with Missing Data. Little and Rubin (1987) provide the following taxonomy for missing data strategies: complete case, imputation, weighting, and model-based procedures. Complete case analysis is probably the easiest and most often used but may result in severe bias if a strong assumption for the missing data is not met. Imputation procedures are distinguished by the method used to fill in missing values and allow the use of complete data methods of analysis. Weighting procedures are most often used in sample surveys to adjust for different probabilities of response. Model-based procedures, the focus of Little and Rubin’s work, define a model for the data and then base inferences on the likelihood of that model. The categories of the taxonomy are not mutually exclusive. For example, the approach taken in this paper is to assume a model for the data, estimate the parameters of that model, and then impute missing values based on those parameters (Schafer, 1991). The missing data strategy will be applied to data from the Television, School, and Family Project (TVSFP), a panel study of adolescent health behavior in Southern California (Sussman et al. 1986,1989; Flay et al., 1988; Brannon et al., 1989; Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Richardson et al., 1989, 1993; Dwyer et al., 1990). In this study a group of adolescents was followed from Seventh Grade through two years post-high school with loss of subjects at each measurement subsequent to the first. Missing data resulted both from subject attrition and item missingness, the failure of subjects to answer specific questions. The most substantial attrition occurred between the Ninth and Twelfth Grade measurements, 45% of Ninth Grade sample dropped out. To narrow the scope of the analysis, the Ninth Grade sample will be used as the baseline, and the focus will be to analyze several outcomes using models for imputation and regression analysis. Through an agreement with the California Department of Motor Vehicles (DMV), motor vehicle records were obtained for 84% of the Ninth Grade sample. A further goal will be to incorporate the DMV data into the analysis. The motivation to employ a missing data analysis is nothing less than the validity of the study. The criticism that major findings are biased due to missing data is often difficult to address. An analysis with missing data attempts to quantify the impact of missing data and thus provides investigators a sense of the trustworthiness of complete case results. The analysis presented here is exploratory in that the sensitivity of inferences to different models of nonresponse is explored. The results should be viewed as an aid to interpretation rather than as providing absolutely valid estimates. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 3 1.1 Missing Data Theory The presentation of missing data theory typically begins with simple problems of univariate missingness, continuous normal variables, and data that are missing completely at random. However, real life datasets often contain very complex types of missing data with general patterns of missingness, mixed continuous and categorical data, and data that are not missing at random. Fortunately, work on these problems has proceeded both in theory and also in the development of algorithms implemented by computer programs. These sophisticated techniques will soon be available for broad use by data analysts and researchers. For example, BMDP AM release 8 is scheduled to contain the model for mixed categorical and continuous data with missing values used in this paper (Little and Schluchter, 1985; Jamshidian, 1995). To avoid misinterpretation of results, attention to the assumptions of these methods is necessary and will be outlined below. The main assumptions concern the response mechanism, the parameters to be estimated, and the method of estimation. 1.1.1 The Response Mechanism. The key assumption in analysis with missing data is the response mechanism. This is an assumption because in practice the true response mechanism will not usually be known. Response mechanisms are classified as either ignorable or nonignorable. The patterns of missing data resulting from nonresponse are also classified. Terminology was first introduced by Rubin (1976) and has since been discussed extensively, for example, Little and Rubin (1987), Schafer (1991), Little (1992,1995), and Greenland and Finkle (1995). Assume that Y is a Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 4 rectangular dataset with a general pattern of missing data with n observations and p variables. A general pattern of missing data is one in which any variable may be missing for any subject. Y may be partitioned into an observed part, Yo*, and a missing part, Ym is , so that Y = (Yo b s , Ym js ). Let the response mechanism be represented by R, an indicator variable for Y that takes the value R=1 for observed data and R=0 for missing data. Y and R could be represented as Y^ and R^, but subscripts will be suppressed for notational simplicity. Missing data may be categorized into the following three types of missingness according to the relationship between R and Ym is : missing completely at random (MCAR), missing at random (MAR), and not missing at random (non-MAR). The data are missing completely at random (MCAR) if the probability of response depends neither on the observed data (Yo b s ) nor on the missing data (Ym is ), which may be written Pr(R | Y) = Pr(R). The data are only missing at random (MAR), a weaker assumption, if the probability of response depends on the observed data but not on the missing data, Pr(R | Y) = Pr(R | Yo b s ). The data are not missing at random (non-MAR) if R depends on the missing data in some way, either Pr(R | Y) = Pr(R | Ym is ) or Pr(R | Y) = Pr(R | Y). Little and Rubin (1987) clarify these potentially confusing distinctions with an example. Consider a sample of subjects with two variables, U = age, which is fully observed for all subjects, and V = income, which is missing for some subjects V = (Vo b s , Vm js ). In terms of the above notation, Y is the entire n x 2 matrix with Yo b s = (U, Vo b s ) and Ym is = Vm js . The assumption concerning the response mechanism Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. (R) may be illustrated in this way. If the probability that a subject responded does not depend on their age or their income, then the data are MCAR. The missing data is simply a random subsample of the original sample. If, however, young subjects were less likely to respond, then the data are MAR because R depends on age, and age is fully observed. In this case, the missing data may be considered a random sample within subcategories of age. There are fewer young subjects, which results in less efficient estimates, but the distribution of income is not biased within young age categories. Theoretically, if R depended only on Vo b s , the observed incomes, to the exclusion of Vm is , the missing incomes, then the data would still be MAR. In the third case, if wealthy subjects were less likely to respond, then the data are non-MAR; the response depends on the variable with missing data. 1.1.2 ML Estimation. The goal of Rubin (1976) was to define the weakest assumption among patterns of missing data under which the response mechanism could be ignored. He showed that this pattern was MAR. The main theme of Little and Rubin (1987) was that with MAR data, ML estimation produces valid estimates. The arguments have been laid out nicely by Diggle, Liang and Zeger (1994). Let f(yo b s,ym is > r) represent the joint probability density function of (Yo b s ,Ym is ,R). This may be factored as follows: ym is, r) = flyo b l, ym J J[r I yo b s , ym is ) The joint density of the observed variables (Yo b s ,R) is needed for ML estimation and may be obtained by integrating over the missing data, Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 6 flyo b s r) = I fiyg b s , ym is ) fir \ yo b j, ym is ) dym ij If the data are MAR, then f (r | yo b s , ym is ) does not depend on ym is and thus may be taken outside the integral, foot? r> = Kr I ya J / /O w ym is ) dym is Taking the integral, Ryo b ,> r> = f a I ya J the Log-likelihood becomes L = log fir | yo b s ) + log fiyo b s ) These two terms are maximized separately. Because the first term contains no information on the distribution of yo b s , it may be ignored for inferences on Yo b s . Table 1 presents a simple schema for the validity of means and regression coefficients for the three patterns of missing data and complete case vs ML estimation with missing data. Under MCAR, both analyses yield valid estimates of both means and regression coefficients, although the complete case analysis will be less efficient Table 1 Validity o f Means and Regression Coefficients under Three Missing Data Patterns Pattern Complete Case ML Estimates of All Cases Means Coefficients Means Coefficients MCAR valid valid valid valid MAR not valid valid valid valid non-MAR not valid not valid not valida not valida a bias reduced Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. due to a smaller sample size. In the case of MAR data, ML analysis gives valid estimates of both parameters while complete case analysis yields valid estimates only for regression coefficients. To understand the difference, consider a complete case regression analysis with the dependent variable partially missing. With MAR data, the regression coefficients are unbiased because the mean of the dependent variable is conditioned on the fully observed covariates within which missingness in the dependent variable is random. On the other hand, the unconditional mean is open to bias because missingness depends on the value of the dependent variable. If subjects with a high value on the dependent variable are more likely to drop out, the mean will be biased downward. Interpreting a regression coefficient as the slope of a line, the location of the line may be shifted, but the value of the slope will not change. On the other hand, ML estimates of means with MAR data are unbiased because the estimates are found with models that include the observed covariates, within which missingness is random. For both analyses, non-MAR data will result in biased estimates though ML will reduce the bias. Little and Rubin (1989) state, "In practice, it is often hard to decide from the data whether the MAR assumption is appropriate; however ML [maximum likelihood estimation] often reduces nonresponse bias even when the MAR assumption is not strictly valid." Clearly, ML analyses are superior with respect to validity and should be preferred over complete case analysis when practical. 1.1.3 ML Estimation with Mixed Continuous and Categorical Data. Many datasets contain both continuous and categorical data so methods for analyzing such Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 8 data are desirable. Little and Schluchter (1985) developed a mixed model by extending the general location model of Olkin and Tate (1961) and adding maximum likelihood estimation for missing data through a modification to the EM algorithm (Dempster etal., 1977). Notation will follow Schafer’s (1991) summary of the model. With p categorical, q continuous variables and n subjects; let W be the n x p matrix of categorical variables and Z be the n x q matrix of continuous variables so that the entire dataset is Y = (W, Z). With W„ W2, ... Wp categorical variables, let d, denote the number of categories for the jth variable so that the total number of cells for all p P categorical variables is D = E For notational convenience arrange the cells in a linear order indexed by d = 1,2,... D. Each subject will fall into one of the D cells, and let {xd} denote the cell counts for the p-dimensional contingency table. Further let U be an n x D matrix with rows u,, s = 1,2,... n. Each row u ,. is a D-size vector with a 1 in position d where the subject falls and 0's elsewhere. The general location model may be described in terms of two components, the marginal distribution of the categorical variables, P(W|t c ), and the conditional distribution of the continuous variables, P(Z|W). The marginal distribution of W is multinomial on the cell counts xd , p { w \t z ) = — n < ' E x ; r f = 1 < * = i where the cell probabilities are denoted by 7 t = {7td} and 2 7 td = 1. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 9 The conditional distribution of the continuous variables is modeled as multivariate normal. Let Ed denote the D-size vector with a 1 in position d and 0's elsewhere. Given that u ,. = Ed ,Z is distributed as N(pd , 2) where pd is the mean for each cell and 2 is the variance-covariance matrix, assumed the same for each cell. The complete likelihood function may then be written as a product of the likelihoods for a multinomial and mixture of multivariate normals, L(Q\Y) « [ I I / s] * Pi" 7 e x p [ - i £ E ( z , " n / E ' V M d= I d= 1 s e B t where Bd = {s : p. = Ed}, S is the simplex {tc : T td £ 0, 2 7 u d = 1}, and 0 = (% , p, 2) where p = (p„ p2, ... pD )T is the D x q matrix of cell means. This model has (D -1) + Dq + q(q+l)/2 parameters. An important point is that if any of the cell counts is zero, then the corresponding mean vector pd drops out and becomes inestimable. Therefore, the model works best with n large enough to ensure that no cell is empty. 1.1.4 Multiple Imputation. Multiple imputation was first proposed by Rubin (1978) and then developed systematically in Rubin (1987). Rubin was concerned about three aspects. First, the method should allow the use of complete-data methods. Second, it should produce correct estimates of the standard errors which are underestimated by single imputation. Thirdly, it should allow the testing of the sensitivity of inferences to different models of nonresponse. The procedure consists of replacing each missing value with m imputed values. The resulting m filled-in datasets Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 10 are then analyzed with complete data methods and combined to produce valid estimates of the standard errors. Correct standard errors are obtained in a manner that accounts for "the increase in variance of estimation due to nonresponse". An estimate of the desired parameter Q is obtained by averaging over the m datasets, < T . ■ E e > /* ! The estimate of variance consists of a within imputation component, = £ U Jm n w * [ /* i and between imputation component, *. - Ece.rW'ce.relW”-') /* ! The total variance is then the sum, with correction for finite m. When there is little information about the response mechanism or when one has reason to believe that the data are non-MAR, Rubin (1987) recommended testing the sensitivity of inferences to different models of nonresponse. A simple and easily communicated method is to set (nonignorable imputed Y) = 1.2 * (ignorable imputed Y), which is to say that if dropouts had responded, they would have responded 20% higher in the missing variables than respondents. One of the chief critics of multiple imputation is Fay (1992, 1993). He leveled two main criticisms against multiple imputation. His first criticism was that multiple imputation is not satisfactory in complex sampling situations such as those encountered Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 11 at the U.S. Bureau of the Census. The second criticism was that when the imputation model and the model of analysis are very different, as may happen when the imputer and analyst are two different people perhaps in two different organizations, then the analysis may yield biased results. In answer to this criticism, Meng (1994) coined the term “congenial” to refer to the situation where the model of analysis and model used for imputation are not in conflict. He offered some original notation to describe the conditions when this is true. Unfortunately, the distinction remains a theoretical one, and there is no test of congeniality in practice. 1.1.5 Proper Imputation via Data Augmentation. In the presence of missing data, Little and Schluchter (1985) recommended the EM-modified general location model to obtain ML estimates for imputation. However, Schafer (1991) noted that for an imputation to be proper, as defined by Rubin (1987), the parameters of a model must be a true draw from their posterior distribution. Therefore, imputations based on the EM algorithm are not proper in that the algorithm produces the expected value of the parameters rather than a random draw from a distribution. As a solution, Schafer proposed the use of data augmentation (Tanner and Wong, 1987), which is closely related to Gibbs sampling. Data augmentation is a two step iterative procedure that is analogous to the EM algorithm in that the E step of the EM algorithm is replaced by an I step and the M step is replaced by a P step. In the E step, the EM algorithm calculates the expected value of the desired set of parameters while data augmentation simulates a random draw of the data, conditional on current values of the parameters. The EM Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 12 algorithm then maximizes the complete data likelihood while data augmentation creates a random draw of the parameters from their complete data posterior. In notation, if Y = (Yo b s , Ym is ) and 0 represents the parameters of the model, with a current value of 0(t) the I step of data augmentation is a random draw from In the P step, a new value of 0 is drawn from its complete data posterior, Note that the P step involves the assignment of a prior distribution t c (0) as in P(0|7) « 71(0)1(017) • When applied to the mixed model described above, Schafer assigned a Dirichlet prior for the cell probabilities k and a Jeffreys prior for (|i, E). The I step may best be described by analogy to the E step of the EM algorithm which considers the complete data likelihood as a linear function of three sufficient statistics. The E step replaces these statistics with their expectations given the observed data and the current estimate of 0. The I step of data augmentation replaces them with a random draw from their predictive distribution given the observed data and 0. A derivation of the necessary distributions may be found in Schafer (1991). The P step is a draw from the complete data posterior which factors as D P(0|7) = P (k \W)P(L\Z,W) n > 0 ‘< /|2,Z,fF) i= 1 with W the n x p matrix of categorical variables and Z the n x q matrix of continuous Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 13 variables. This distribution is simulated by sampling from a Dirichlet, a Wishart, and D independent multivariate normals in succession. 1.2 Missing Data in Practice Of practical concern is the plausibility and the prevalence of MCAR and MAR data in real life research. Greenland and Finkle (1995) considered MCAR to be rare in epidemiology where completeness of questionnaires is often related to exposure or disease. For example, a subject who dies or who is disabled from a toxic exposure will not complete a follow-up questionnaire or will be less likely to do so. They point out that the MAR assumption is not always reasonable especially when sensitive behaviors are probed. For example, failure to answer a question on sexual preference most likely depends on sexual preference. Laird (1988) stated that MAR data are common in longitudinal research, especially clinical trials, where the past experience of subjects affects their future participation. 1.2.1 Hypertension Trial. An example of a MAR dropout process occurred in a hypertension trial (Rosendorf and Murray, 1986) and was discussed by Murray and Finley (1988) and Little (1995). In this study the outcome of interest was diastolic blood pressure (DBP). Treatment lasted 12 weeks with clinic visits at 0, 2,4, 8, and 12 weeks. Patients with DBP greater than 110 mmHg at either weeks 4 or 8 were to skip the rest of treatment and enter a follow-up phase, thus creating missing data at weeks 8 or 12. As expected, analyzing fully observed data showed that patients who skipped Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 14 treatment according to protocol were systematically different from those who remained. However, the data were MAR because the missingness depended on recorded values of DBP. Inferences based on these data are valid if the recorded values are included in the model and maximum likelihood methods are used. In this context “model” refers to a specialized missing data model such as the EM algorithm (Dempster et al., 1977) which is capable of analyzing all the data. Otherwise, if “model” meant a complete case regression model and the outcome of interest were DBP at week 12, then those subjects missing week 12 data would be deleted along with their week 4 or 8 DBP, thus creating non-MAR data. 1.2.2 Testing the MCAR Assumption. The MCAR assumption is one that may be tested in practice. A simple test for a variable with missing data may be carried out by forming two groups, droppers and stayers, and comparing the distribution of a fully observed variable on the basis of the two groups. This may consist of a series of t-tests or 2 x C contingency tables where C is the number of categories for a categorical variable. Little (1988) noted the multiple testing problem when many such tests are performed and developed a global test of the MCAR assumption. 1.2.3 Missing Data in Psychosocial Research. Prevention research is often conducted with longitudinal study designs in which subject attrition is a major source of missing data. Prevention researchers are concerned both with the different prevalence of use between stayers and dropouts as well as how attrition affects the validity of conclusions regarding the experimental conditions. These concerns are Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 15 often phrased in terms of threats to both internal and external validity, roughly defined as the ability to generalize within and outside the sample (Campbell and Stanley, 1966). In the terminology of the statistical literature, many of these approaches are variations on the test for MCAR data. Jurs and Glass (1971) offered an early attempt to analyze attrition in the context of traditional ANOVA experiments in education. They recommended the measurement of any variable that might be related to attrition, comprising pretest and biographical data. These became dependent variables in a MANOVA with the following effects: treatment (T), attrition group status (dropout vs. stayer) (G), and treatment by attrition interaction (T x G). In short, their interpretation was that a significant G effect meant non-random attrition within groups and therefore external validity was threatened, and a significant (T x G) effect meant non-random attrition between treatment groups and thus internal validity was threatened. Hansen et al. (1985) attempted to standardize the analysis of attrition in prevention research with four specific questions concerning 1) the difference between stayers and droppers on pretest and demographic characteristics, 2) patterns of attrition over time, 3) rates of attrition among experimental conditions, and 4) pretest scores across experimental conditions. They applied the four tests to two tobacco prevention studies and concluded that both studies were generalizable only to nonusers, but only one of the studies was threatened with differential loss of subjects among conditions. Biglan et al. (1987) examined another school-based tobacco-cessation program with Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 16 somewhat different tests of attrition bias. To assess external validity, they conducted a series of one-way ANOVA's on various pretest dependent variables with attrition status as the factor. With this analysis they found that the analysis of treatment effect at posttest was conducted with a sample that under-represented smokers and those at risk. A second paper by Biglan et al. (1991) further refined attrition analyses. For external validity, they presented a discriminant analysis which yielded a list of variables on which droppers and stayers were significantly different. Looking back on the body of attrition literature, Snow et al. (1992) observed that attrition rates differed by type of pretest substance use and that different measurement and statistical analyses had been used. Based on earlier findings, they proposed the following three hypotheses: dropouts are more likely to be substance users than stayers; attrition is most consistent for tobacco and marijuana and less consistent for alcohol and hard drugs; and mean use comparisons would find significant differences between stayers and droppers more often than use-nonuse analyses. In their longitudinal study of 1117 adolescents in New England, followed from Sixth through Twelfth Grade, they found support for all three hypotheses. It has become almost axiomatic in prevention research that study dropouts are heavier substance users than stayers. To directly test this hypothesis, Pirie et al. (1988) tracked a cohort of dropouts from two adolescent smoking studies in the Minneapolis/ St. Paul area. The original cohort of 7124 students was formed in the Seventh Grade, and 5573 were still in their original schools in the Eleventh and Twelfth Grades. The Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 17 remaining 1551 dropouts were tracked, and 90% of them were measured in short phone interviews. Three groups of students were formed on the basis of their school attendance: absentees, transfers, and dropouts. The researchers found that smoking prevalence was much higher in all three groups, most notably the school dropouts, compared to adolescents who stayed in school. A challenge that faces the prevention researcher is to apply the statistical theory and methods of missing data analysis to the context of attrition in prevention studies. To this end, Graham and Donaldson (1993) conceptualized the response mechanism as accessible or inaccessible. A response mechanism is accessible if it has been measured and is included in the model. Alternatively, a response mechanism is inaccessible if it has not been measured and therefore is not included in the model. They focused on differential attrition which they defined as both different dropout rates among experimental conditions and the statistical interaction between experimental condition and attrition status. They showed with simulated data that even when there was differential attrition and missing data in the covariates and dependent variable; if the response mechanism was accessible, EM algorithm ML estimates of linear regression parameters were not biased. 1.2.4 The Collection o f Additional Data. If a substantial amount of data is missing, it makes sense to collect additional information on the response mechanism. Rubin (1987) discussed the use of information privy to the imputer but not the analyst; Glynn et al. (1986) provided an example of an analysis that used follow-up data on a Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 18 subsample of nonrespondents, and Graham (personal communication) collected data on a random subsample of nonrespondents to show that reasons for nonresponse were not correlated with outcome. Ashurst et al. (1988) collected additional information and used it to inform their strategy for assessing and correcting for nonresponse bias. Little (1995) advised researchers to collect data on the reasons for attrition and to include the data in the model. Little and Rubin (1989) stated, Where possible, studies should be designed to limit the impact of nonrandom missing-data mechanisms. For income nonresponse, for example, measurable covariates that correlate with income should be collected, such as owner/renter status, square footage of house, number of cars owned, income above or below $40,000. 1.3 The TVSFP/Latchkey Study The Television, School, and Family Project (TVSFP) was an experimental study in tobacco prevention and cessation among early adolescents and their families. The purpose of the study was to test the mode of delivery of a social influences curriculum, whether classroom, television or both. The study design included four possible combinations of instruction: both TV and classroom, TV only, classroom only, and two control groups, neither with TV and one with attention-control instruction. The experimental classroom conditions received the social influences curriculum (Flay et al., 1988). In early 1986 approximately 7000 Seventh Graders in 47 schools in Los Angeles and San Diego Counties were tested before and after receiving the program. Researchers found no significant difference in substance use among experimental Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. conditions (Richardson, personal communication). However, TVSFP advanced health education research by achieving equivalent expectancies of success between experimental and control conditions (Sussman et al., 1989). Expectancies of success were measured by asking students to quantify their beliefs about how likely what they learn about smoking will affect their smoking attitudes and behaviors. This was important for placebo implementation because simply the expectancy that a program might reduce smoking has been shown to predict actual smoking. Published work on the project also covered program acceptance by the target audience. They found that both the classroom and television formats were significant predictors of acceptance both separately and together (Brannon et al., 1989). In the Eighth and Ninth Grades the original cohort, supplemented by additional "drop-ins", was surveyed again. With the addition of new questions on self-care, TVSFP transitioned to the Latchkey Project, an observational cohort of adolescents. One finding was that subjects who most often took care of themselves were twice as likely to use substances (Richardson et al., 1989). The cohort of Ninth Graders was followed and measured again in the Twelfth Grade and two years thereafter. To facilitate tracking of subjects, vehicle records were obtained from the California Department of Motor Vehicles (DMV). Although the right to privacy is not mentioned explicitly in the U.S. Constitution or Bill of Rights, state law has provided the basis for lawsuits in such diverse areas as reproduction, access to medical records, and the media. The use of archival data for research purposes certainly touches on the Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 20 right to privacy and has the potential for creating friction between the research community and the public. Public inspection of DMV records is covered under the California Vehicle Code, Section 1808, which reads . . . all records of the department relating to the registration of vehicles, other information contained on an application for a driver's license, abstracts of convictions, and abstracts of accident reports . . . shall be open to public inspection during office hours (West's Annotated Vehicle Code, 1990). An agreement between the University of Southern California Department of Preventive Medicine and the California Department of Motor Vehicles served as the basis for digital access to the data used for this paper. Even though consent was not obtained from study participants, confidentiality of DMV records was maintained, and names were separated from sensitive information for analysis. 2. Methods Multiple imputation based on a model for the data was chosen as the method to evaluate the impact of missing data in this application. An attractive feature of this method is the use of complete data methods which are combined to obtain repeated- imputation inferences. The approach accommodated maximum likelihood estimation and explored the bias from potential non-randomly missing data. The results from several imputation models were compared to those of a complete case analysis. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 21 2.1 Plan of Analysis The strategy for analyzing the Ninth and Twelfth Grade TVSFP data involved the choice of two models, one for imputation and the other for analysis. The imputation model was the model for mixed continuous and categorical data described above. The analysis model was the well known linear regression model. Unless specified, the term "model" will refer to the combination of imputation and analysis models. This strategy consisted of the following steps: • Choose analysis model • Formulate ideas concerning response mechanism • Test the MCAR assumption • Evaluate the DMV data for association with attrition • Choose appropriate imputation models and impute the data • Obtain repeated-imputation inferences • Compare inferences from different models The following five models were created and are numbered for reference: I. Complete case model II. Ignorable response without DMV variables III. Ignorable response with DMV variables IV. Nonignorable response without DMV variables V. Nonignorable response with DMV variables These models were examined for each of the outcomes of interest which were Twelfth Grade tobacco, alcohol, marijuana, and hard drug use, depression, drinking/riding with substance use, and a log transformation of tobacco use. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 22 2.2 Survey Data The TVSFP cohort was formed from Seventh Grade students in Los Angeles and San Diego Counties in 1986. They were followed two years post high school for a total of five measurements. Questionnaires were administered in the classroom by project staff. Absentees were contacted by telephone and mailed surveys. All questionnaires were confidential, identified only by a numeric ID and birth date. To narrow the scope of the analysis, the Ninth and Twelfth Grade measurements were used as the focus for this paper with the Ninth Grade sample considered the baseline. The transition from Ninth to Twelfth marked the period of highest subject attrition. A total o f4442 subjects were measured in the Ninth Grade, all of whom were eligible for measurement in the Twelfth Grade at which time 2451 (55%) of the Ninth Grade sample responded. An additional 475 students who “dropped in” at Twelfth Grade were not included in this analysis. The Ninth Grade questionnaire consisted of 136 items that covered a broad range of health behaviors and psychosocial constructs. Composite variables were created from individual survey items to represent a latent construct. “Item” will be used to refer to each survey question while “variable” or “construct” will stand for the composites created from a number of related items. The following ten constructs from the Ninth Grade survey were considered: tobacco, alcohol, marijuana, risk-taking, hostility, stress, depression, family conflict, self-care, and school grades. Sex and ethnicity were also included as covariates. A student’s ethnicity was determined by Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 23 self-reports over several waves of measurement. In the case of conflicting self descriptions, a non-white ethnicity besides “Other” was given priority. The Ninth Grade survey employed a three-form design which grouped items into a core section X and sections A, B, and C. The core section was placed first on each form, and sections A, B, and C were permuted with the intention that nonresponse at the end of the questionnaire would be distributed randomly among the three sections. The Twelfth Grade survey consisted of 120 items with exact or similar wording to corresponding Ninth Grade items. All students received the same form. Twelfth Grade outcomes of interest were tobacco, alcohol, marijuana, hard drug use, driving/riding with substance use and depression. A log transformation of tobacco use (LOGTOB12) was created to better meet the assumption of normality of the imputation and linear regression models. These outcomes were chosen to represent a range of types of drug use. Depression was included as a representative psychological construct. The driving/riding items were examined for their potential higher correlation with DMV substance use violations. Appendix A contains the complete listing of Ninth and Twelfth Grade constructs with their constituent items and variable names. The questionnaires allowed the subject a range of responses which were ordered to represent a quantity such as “1 to 2 drinks” or “number of days drinking”. Most items were coded from " 1" to the number of categories for that item. The exception was items which allowed a fill-in response, for example, “How many days in the LAST MONTH (30 days) have you had alcohol to drink?” Values that were outside the range Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 24 [0, 30] were coded as missing. Although variables with ordered categories are technically ordinal, they were analyzed as continuous variables because the underlying construct, amount of use, is continuous though not necessarily normally distributed. Items were recoded in the same direction with a greater value representing a worse level of the variable. For example, more depression, worse grades, and more self-care all were coded with higher values. The Twelfth Grade questionnaire directed the subject to skip a group of questions based on the response to the first item in a series. For example, if the subject answered “None” to a lifetime use question, they were directed to skip monthly, weekly, and daily use items. Therefore, subjects who answered “None” for lifetime alcohol, tobacco, and marijuana use were coded as “None” for more frequent use questions and “Zero” for days of use in the past 30 days. The hard drug use items were prefaced by an instruction which read, “If you have never used any of these drugs check here and skip to question. . . . ” Unfortunately, the response to this question was not keypunched; and therefore, true item missingness could not be determined for the hard drug use items. Composite variables were created from related groups of standardized and averaged items to reduce the number of variables in the model and make interpretation easier. The survey was designed to measure certain underlying theoretical constructs or factors by asking a series of questions related to the construct. As a prerequisite to combining items, exploratory factor analysis was employed to quantify the relationship Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 25 of groups of items to the theoretical, latent constructs. This was done by examining the loading of each item on a factor. A factor loading is similar to a regression coefficient except that it represents the regression of a latent or unobserved variable on observed variables. Factor loadings range from 0 to 1 with values of 0.80 and 0.90 considered to be high. Items were therefore entered in a series of principal components factor analyses (Gorsuch, 1983). A promax rotation was used to separate the factors. The strategy was to enter a large number of items within a broad theoretical area such as substance use or psychosocial and examine the loadings on a given number of factors. Various numbers of factors were examined. Thus an item was allowed to load on a factor on which it was not intended to load. Based on the factor loadings and other considerations, the decision was made whether to keep items that did not load on the desired factor. Due to the large amount of missing data and the complete case approach of the factor analysis, care was taken to group items with similar amounts of missingness so that the number of complete cases would not be substantially lowered. For example, when hard drug use items were included with the other substance use items, only 761/4442=17% of the subjects had complete data as compared to 2143/4442=48% of the subjects without considering hard drug use. Hard drug use items were therefore considered separately from other Twelfth Grade substance use. The items were entered in the following groups: Ninth Grade substance use, Ninth Grade psychosocial, Twelfth Grade substance use, Twelfth Grade hard drug use, and Twelfth Grade depression. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 26 Cronbach’s alpha was also used to describe the internal consistency of each construct (Cronbach, 1951). After examining each construct and making necessary adjustments, the items were combined to form composite variables. Because items within a group had different ranges, each item was first standardized to a mean of 10 and standard deviation of 1 and then averaged within each subject. A mean of 10 was used to avoid negative numbers because a nonignorable imputation model was later created by multiplying each ignorable imputed value by 1.2. Due to item missingness, varying numbers of items within a construct were observed which presented a problem when averaging all items for a subject. As a conservative approach, the composite variable was set to missing if the subject failed to respond to even one of the items in a related group of items. This was done by setting each item to missing before standardizing so that the same set of subjects were standardized for each composite variable. Therefore, a composite variable represented a subject who responded to all items in a group. Twelfth Grade hard drug use was handled differently than the other variables due to the sensitive nature of the questions and the corresponding lower response. Individual hard drug use items were still standardized to a mean of 10 and a standard deviation of 1. However, instead of the average, the maximum response among the items was coded as the composite variable. This was done because it was important to know if even one of the drugs was used. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 27 2.3 DMV Data There were two requests for California Department of Motor Vehicle (DMV) records of TVSFP subjects, one in Spring 1991 and the second in Spring 1993. In 1990/91 TVSFP subjects were surveyed in the Twelfth Grade. The data from the 1993 submission was used for this analysis because DMV records are cumulative with more recent violations added on the end of each record. The data for less serious violations is taken off the record after three years (State of California Vehicle Code 1991,1992) so that the 1993 data encompassed roughly one year prior and two years subsequent to the Twelfth Grade survey. The entire cohort of approximately 9000 subject ID’s, names, and birth dates was submitted on computer disk, and 6134 were matched by computer at the DMV. Subjects not matched by computer were resubmitted on paper forms for a more thorough search by DMV personnel. Of the remaining 3000, only those that were being tracked in the post high school data collection (i.e. those measured at Twelfth Grade) were resubmitted on paper. This effort resulted in about 500 additional matches bringing the total percentage of matches to approximately 6500/9000 = 72%. The percentage of Ninth Grade DMV matches was higher at 3727/4442 = 84%. The computer matches were received as ASCII files and read into SAS datasets. The paper matches were keypunched into SAS datasets. The DMV ASCII files and keypunched paper printouts were merged. Each DMV record consisted of the names, addresses and violations for one individual. However, one individual could have multiple records. The violation data Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 28 were entered from oldest to most recent in groups of incidents with each incident representing one time the subject was detained or pulled over. Each incident consisted of the following data: item code, violation/accident date, conviction date, sections violated, statute violated (e.g. vehicle code, penal code), DMV points, and, if appropriate, location of accident. DMV points are accrued by drivers for each violation; specific points are designated in the vehicle code with more serious violations carrying more points. There was also room for multiple violations per incident. For example, a subject may have been stopped for speeding and also cited for a seat belt violation. Although the majority of the violations involved the California Vehicle Code, violations of other state codes such as the Penal Code and Health & Safety Code that were related to vehicles were also included, for example, auto theft and possession of an open container. The middle section of each record consisted of comments and restrictions. Examples were a requirement to wear corrective lenses or the comment that a license had been returned by another state, indicating the subject had moved. The final section consisted of actions taken by the DMV in response to a violation, for example, probation or suspension of license. There were also reasons listed for the actions such as excessive blood alcohol levels. Several types of variables were created from this data and are listed in Appendix B. Most were binary and the rest were categorical groupings of counted data. The first group of variables was created from the individual violations of the DMV Code. A description of each code was recorded from the 1991 California DMV Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 29 Vehicle Code and other references (West's Annotated California Codes, 1990). From these descriptions, the following seven categories of violation codes were created: • speeding • other moving • license-related • equipment • other non-moving • substance-related • crime A subject was coded "1" if they had at least one violation in a particular category and "0" if not. The five most frequent violations in each category are listed in Appendix C. Another group of categories was created to organize the authority codes that served as the basis for the DMV actions. Descriptions of these codes were also found in the 1991 California Vehicle Code and grouped into five categories: • substance use • failure to appear/pay fine/report • financial responsibility • negligent driving/damage caused • procedural codes A complete listing of the individual codes within each category is also found in Appendix C. Several items in the DMV record were counted data. Examples are the number of times the subject was pulled over and cited, the number of violations per subject, and the number of accidents per subject. Most of the counts consisted of zeros and ones with a long tail. Therefore, subjects with high counts were incorporated in a "x+" category. For example, subjects with one or more non-vehicle code violations (i.e., Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 30 Penal, Business and Professional, Welfare and Institution laws) were coded "1+". This was an attempt to reduce bias due to arrest or pullover rates that varied with local law enforcement practice. Other variables with multiple categories were cumulative DMV points and highest DMV points per incident. The final group of variables consisted of written phrases in the record which were standard enough to serve as the basis for a 0/1 variable. The exact wording of the comments, restrictions, and reasons for court actions is listed in Appendices B and C. 2.4 Multiple Imputation Models Data augmentation and multiple imputation were carried out with SPLUS programs and FORTRAN subroutines with a limit of 30 variables per model (Schafer, 1991). For this reason and because of the practical limitations of computing time and disk storage, imputation was done at the level of composite variables instead of individual survey items. A different set of imputations was created for each outcome to lessen the amount of missingness in the imputation model and to maintain congeniality between the imputation and analysis model. Nonignorable models were created by multiplying each ignorable imputed value by 1.2. 2.4.1 Evaluating the Response Mechanism. The MCAR assumption was tested by forming two groups based on Twelfth Grade attrition (stayers and droppers). T-tests were used to compare their responses on Ninth Grade composite variables. Categorical variables, for the most part DMV, were compared using x2 tests. Significant Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 31 differences indicated whether the response, or equivalently the nonresponse, at Twelfth Grade was MCAR. Tests of MCAR could only be done for those subjects who completed items at Ninth Grade or who were matched by the DMV. Evaluating whether the data were MAR or non-MAR was a more difficult matter due to the fact that statistical tests cannot be applied to data that are missing. In addition, the response was complicated by item missingness at both measurements as well as missing DMV data. As explained above, the statistical theory of missing data shows that the response mechanism is ignorable if it does not depend on the missing data. The application of this theory to real-life missing data problems is not straightforward. In the case of the TVSFP data, arguments may be advanced on either side and will be discussed below. 2.4.2 Choosing DMV Variables for the Imputation Model. Although no firm rules exist for choosing variables to include in an imputation model, one guideline is that they be correlated with the missing data. Another strategy is to include variables that predict nonresponse. The idea behind this approach is represented in other contexts by Heckman (1979), Dent (1988), and Leigh et al. (1993). Preliminary analyses demonstrated a low correlation (r < 0.20) of the DMV variables with both Ninth and Twelfth Grade observed data. Therefore, the criteria of predicting the response mechanism was adopted. This was evaluated through a logistic regression analysis with the binary outcome of attrition at the Twelfth Grade. This consisted of a univariate screening of the 73 DMV variables at alpha=0.20 which was followed by a Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 32 forward, stepwise logistic regression analysis to select independent DMV predictors of nonresponse. The next step was to conduct the multiple imputation with the smaller set of DMV variables, all the Ninth Grade variables of interest, and each Twelfth Grade outcome. In order to test the relative importance of the DMV data in this setting, two separate sets of imputations were created, one with the DMV variables and one without. During initial runs with all of the significant DMV variables in the model, convergence was not reached. The reason for this was thought to be sparse data from too many categorical variables with missing data, specifically the DMV variables. To remedy this, the binary DMV variables were considered continuous variables. Because the imputation model required continuous variables to be multivariate normal, the DMV variables that were extremely skewed, less than 10% in one category, were deleted. The DMV substance use violations, represented by the 0/1 DMV substance use variable, were considered to have the best potential of correlating with the missing substance use data on the questionnaires. However, only 41 (1.1%) subjects had one or more DMV substance use violations. Therefore, DMV substance use could not be reasonably modeled as a normal, continuous variable as the other DMV variables were modeled. Preliminary models with DMV substance use entered as a categorical variable along with sex and ethnicity failed to converge after 1000 iterations. Examination of the 20 cell contingency table of sex by ethnicity by DMV substance use Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 33 found that there was at least one subject in each model that could be used to estimate the means for the missing cells. For example, for the model of tobacco use (12th), there were no subjects in the female by other ethnicity by DMV substance use=l+ category but there was one subject in the female by other ethnicity by DMV substance use=missing category that theoretically could be used to estimate the cell mean. This model converged after 5040 iterations, but the cell probability (n) for the female, other, substance violations =1+ cell was <0.0000000001, in practical terms zero. This resulted in undefined cell means (p) for that cell. Each of the other models with DMV substance use as a categorical variable converged within 10,000 iterations but had the same zero cell probability. This was no doubt due to the 16% missing data in the DMV substance use variable and the fact that there were very few subjects with non missing Twelfth Grade variables who were female and of other ethnicity. Regrettably, substance use was excluded from the DMV imputation models, leaving sex and ethnicity the only categorical variables. 2.4.3 Data Augmentation Parameters. As stated above in the description of the mixed model for continuous and categorical data with missing values there were (D -1) + Dq + q(q+l)/2 parameters to be estimated where D is the number of cells for the contingency table of all categorical variables, and q is the number of continuous variables. Sex and ethnicity created a two by five table with a total of 10 cells and therefore 9 parameters (u) to estimate. All survey variables were considered continuous, relying on large sample properties to meet the normal distributional Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 34 assumptions of the model. For each of the models without DMV variables there were nine Ninth Grade variables and one Twelfth Grade outcome for a total of ten continuous variables. The matrix of cell means (p) of size Dq therefore contained 10*10 = 100 parameters. The variance-covariance matrix (2) consisted of 10*(10+l)/2 = 55 parameters. The three sets of parameters for the non-DMV models therefore totaled 9 + 100 + 55 = 164 parameters. With the addition of five DMV variables, considered as continuous variables, the number of parameters rose to 9 + 150 + 120 = 279 parameters. If all five DMV variables had been entered as categorical variables, including those that were highly skewed, the number of parameters would have been 320+ 100 + 55=475. For each model, starting values of parameters were obtained from the EM algorithm. At this point improper imputations could have been generated by randomly drawing the missing data from the prescribed distribution using the one set of parameter estimates. However, to create proper imputations, the parameters themselves were drawn from their estimated posterior distribution. To accomplish this in practice, the data augmentation algorithm was halted every 100 iterations for a random draw of parameters. This assumed that the estimates stabilized after 100 iterations (Schafer, 1991). Based on each set of parameters, missing data was imputed by a random draw from the posterior distribution given the observed data and the current estimates of the model parameters. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 35 2.4.4 Choosing the Number o f Imputations. Both theoretical and practical considerations came into play when deciding the number of imputations. Practical considerations were the size of each completed dataset, the availability of disk space, and the run time for each program. These factors made fewer imputations more attractive. From a theoretical perspective, more imputations are better because Rubin’s theory depends on an infinite number of imputations. Rubin (1987) showed, however, that with modest amounts of missing data (around 20%), the coverage of nominal interval estimates is good with as little as three imputations. The percentage of missing data was substantially more than 20% in the TVSFP data, ranging from 45% to 80%. Five imputations were created for this analysis to provide a satisfactory balance between theoretical and practical concerns. A different set of imputations was created for each of the seven outcomes, including the log transformation of tobacco use (12th). Four different imputation models were employed for a total of (5 repetitions) * (7 outcomes) * (4 imputation models) = 140 imputed datasets with each dataset roughly one megabyte in size. 2.5 Analysis o f Interest Data for this project was typically analyzed to see how variables measured at earlier waves predicted health-behavioral outcomes at later waves of measurement. This question was addressed with a multivariate linear regression analysis which examined six separate outcomes at Twelfth Grade: alcohol, tobacco, marijuana, hard drug use, Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 36 depression, and driving/riding with substance use. The natural logarithm of tobacco was also created to address the deviation from normality. The covariates included sex, ethnicity, Ninth Grade letter grades as well as Grade Nine survey measurements of alcohol, tobacco, marijuana, risktaking, stress, depression, family conflict and self-care. The DMV variables were not included in these models because their relation to outcome was not of interest. Sex and ethnicity were entered as 0/1 dummy variables with "Male" and "White" as baseline categories. 2.5.1 Obtaining Repeated-Imputation Inferences. The parameters of interest were the p’s for the Ninth Grade independent variables in the linear regression models. Five sets of P’s were obtained for each model. The null hypotheses were P; = 0 for each of 14 covariates (counting dummy variables), given all but the ith covariate in the model. This was the partial F-test based on the sums of squares for the last covariate added. A repeated-imputation inference for an individual P was obtained in the manner described above by first running a regression model on each imputed dataset, obtaining the partial F-test estimates, and then averaging the five resulting P's. Repeated imputation estimates of the standard errors were found by combining within and between imputation estimates of the variance over the imputed datasets. Special formulas were used to provide estimates of the degrees of freedom (Rubin, 1987). The formula for the degrees of freedom (v) is v = (« - 1)(1 + - 4 2 r m where rm is the relative increase in variance due to nonresponse, Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. with Bm and Um the between and within imputation variance as defined above. 2.5.2 Comparison o f Models. Inferences from multiple imputation models, or "repeated-imputation" inferences, were obtained for each model and outcome. To examine the effects of ignorable nonresponse on inferences, complete case analyses were compared to models of ignorable nonresponse (Models I vs II). To explore the effect of a 20% higher nonignorable nonresponse, complete case analyses were compared to models of nonignorable nonresponse without adjustment by DMV variables (Models I vs IV). Model IV was created by multiplying each ignorable imputed value in Model II by 1.2 and then running the regression analyses. Imputations for Model V were created in a similar manner except that ignorable values of Model III, which had been adjusted by DMV data, were multiplied by 1.2. To see the effect of including the DMV predictors of attrition in the imputation model, differences between Models II and III were examined as well as Models IV and V. The log transformation of tobacco use (12th) was compared to tobacco use (12th) for all models. The models were compared with respect to observed significance levels or p-values. Although p-values are affected by several aspects of the sample, they were considered the statistic of most practical relevance. For example, betas and standard errors might differ somewhat among models, but if the differences do not produce significant changes in p-values, then basic conclusions are not affected. The choice was made to explore the effect of missing data on different outcomes rather than a more detailed examination of Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 38 the statistical properties of the models. Differences in p-values between models occurred in two ways: they changed from significant to non-significant, or they changed from non-significant to significant. Significance was decided at the traditional 0.05 level. Both types of changes were noted, and p-values that crossed the 0.05 level by small amounts were still noted. 2.6 Computer Resources The data were managed and analyzed using desktop personal computers. Statistical analysis was done with SAS and Epilog (SAS Institute, 1988; Epicenter Software, 1993). The imputation models were run on a UNIX mainframe computer with SPLUS and FORTRAN programs written by Schafer (1991). 3. Results 3.1 Preparation o f Data 3.1.1 Factor Analysis. The results of the complete case principal components factor analysis are listed in Table 2. Only loadings on the respective factor are given with the exception of DRIVE12A and DRIVE 12B which is explained below. Of the three Ninth Grade substance use factors, tobacco use was the most cohesive with four items loading from 0.73 to 1.0 and a Cronbach’s alpha of 0.90. Two tobacco items regarding current tobacco use did not load with the other items and were excluded. The alcohol variable consisted of three items of which ALC9C, “days drank alcohol last Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Table 2 Factor Loadings and Cronbach's Alpha fo r Ninth and Twelfth Grade Surveys Ninth Grade Substance Use Twelfth Grade Substance Use Loading Item Description Cron, a Tobacco 0.90 0.73 TOB9A Ever smoked 0.80 TOB9B Cigarettes whole life 1.00 TOB9E Hours last cigarette 0.93 TOB9F Cigarettes currently Alcohol 0.80 1.00 ALC9A Drinks whole life 0.96 ALC9B Drinks at one time 0.28 ALC9C Days last month alcohol M arijuana 0.61 0.27 MAR9A Marijuana whole life 1.00 MAR9B Days last month used Ninth Grade Psychosocial Loading Item Description Cron, a Risktaking 0.75 0.65 RISK9A Don’t want to wait 0.91 RISK9B Like people think older 0.98 RISK9C Trouble for fun OK 1.00 RISK9D Like to take risks 0.95 RISK9E Enjoy doing opposite Stress 0.79 0.99 STRESS9A Upset over unexpected 0.87 STRESS9B Unable control things 0.99 STRESS9C Felt nervous last m onth 1.00 STRESS9D Angry things can't control 0.79 STRESS9E Afraid when alone Depression 0.77 0.96 DEP9A I do most things wrong 1.00 DEP9B Bad things will happen 0.96 DEP9C I hate myself 0.93 DEP9D All bad things are my fault 0.63 DEP9E I am tired all the time 0.75 DEP9F I feel alone 0.95 DEP9G Nobody loves me Family Conflict 0.84 1.00 FAM9A Family looks things to nag 0.91 FAM9B Family doesn’t understand 0.95 FAM9C Arguments with family Self-Care 0.59 1.00 SELF9A Number days self-care 0.98 SELF9B Hours self-care per day 0.52 SELF9C School grade self-care Loading Item Description Cron, a Tobacco 0.92 0.54 TOB12A Cigarettes whole life 0.89 TOB12B Cigarettes past month 0.97 TOB12C Cigarettes last week 1.00 TOB12D Cigarettes past 24 hours Alcohol 0.89 0.97 ALC12A Drinks whole life 0.99 ALC12B Drinks past month 0.99 ALC12C Drinks past week 1.00 ALC12D Drinks at one time 0.95 ALC12E Days last month alcohol Drunk/Driving * 0.74 0.63 DRIVE12A Drive after drink /month 0.62 DRIVE12B Ride drink/drive M arijuana 0.89 0.59 MAR12A Marijuana whole life 0.90 MAR12B Marijuana last month 0.63 MAR12C Joints per occasion of use 1.00 MAR12D Days past month used Twelfth Grade Hard Drug Use Loading Item Description Cron, a H ard Drugs 0.58 HARD12A 0.66 HARD12B 0.69 HARD12C 0.66 HARD12D 0.66 HARD12E 0.69 HARD12F 0.74 Heroin, morphine, opium LSD or acid “downers” “uppers” inhalants to get high PCP Twelfth Grade Depression Loading Item Description Cron, a Depression 0.34 DEP12A 0.59 DEP12B 0.74 DEP12C 0.64 DEP12D 0.45 DEP12E 0.68 DEP12F 0.62 DEP12G 0.68 I do most things wrong Bad things will happen I hate myself All bad things are my fault I am tired all the time I feel alone Nobody loves me * based on a 3-factor principal components analysis; Cronbach’s Alpha calculated for two items; factor loadings for alcohol factor Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 40 month”, had a low loading of 0.28. This item loaded higher (0.52) on the marijuana construct. However, the Cronbach’s alpha of 0.80 indicated good internal consistency; therefore, it was retained in the alcohol construct. Marijuana was also problematic in that there were only two items in the survey, and they did not appear to load together (factor loadings of 0.27 and 1.0) with a Cronbach’s alpha of 0.61. The construct was retained because it was considered important to have a separate measure of marijuana use at Ninth Grade. There were five Ninth Grade psychosocial constructs, most of which formed fairly good factors. The lowest factor loading was 0.52 for SELF9C on the self-care factor (Cronbach's Alpha=0.59). The other four constructs had internal consistency measures ranging from 0.75 (risktaking) to 0.84 (family conflict). All original items were retained with the exception of two items intended to load on a “hostility” construct. Subjects were asked how often the following two statements applied to themselves: “I am quick tempered” (HOST9A) and “When I get mad I say nasty things” (HOST9B). These two items failed to load on their own factor. Instead, HOST9A had a loading of -0.43 with RISK9A, and HOST9B had a loading of 0.62 on the stress construct. Although some argument could be made for inclusion with these other factors, the hostility items were excluded from the analysis. A factor analysis was also conducted for the Twelfth Grade constructs (Table 2). Internal consistency for the three major substances, tobacco, alcohol, and marijuana, centered close to 0.90 as indicated by Cronbach’s alpha. Some items had loadings in the 0.50 and 0.60 range, but all items were retained for these factors. Two drinking/driving Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 41 items (DRIVE 12A and DRIVE 12B) were included because of their potentially close correlation with DMV data. Together they had a Cronbach’s alpha of 0.74 and loaded on the alcohol variable with 0.63 and 0.62 factor loadings. They were placed in their own construct to isolate, at least on a conceptual level, the information regarding driving and using substances. The factor loadings for the six hard drug use items ranged from 0.58 to 0.69 with a Cronbach’s alpha of 0.74. Twelfth Grade depression also had low factor loadings in the range of 0.33 to 0.74 and an internal consistency measure of 0.68. All Twelfth Grade depression items were retained in order that the construct would have the same group of items as Ninth Grade depression. 3.1.2 Standardizing and Averaging Items. Table 3 lists the results of the standardization and averaging of the individual items to form composite variables. The N, percent observed, mean, standard deviation, and skewness of each item and composite variable is given. One may see that items within a construct experienced similar percentages of missingness so that no construct was greatly diminished by one item with exorbitant missingness. The percent missing for the composite variables was less than or equal to the percent missing of the least observed item in the group. This occurred because the composite was set to missing if one of the items per subject was missing. The exception was hard drug use which was non-missing if only one item per subject was observed. Each substance use variable had one or more items skewed over 2.0. The standardizing and averaging process resulted in composite variables with skewness between the range of skewness of the constituent items. With the exception of alcohol use (12th), the other Twelfth Grade substance use composite variables had skewness of Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Table 3 Mean, Standard Deviation and Skewness for Ninth and Twelfth Grade Items and Standardized Composite Variables 42 Ninth Grade Variable N (% ) Mean Std Skew TOB9A 4404 (99) 1.95 0.87 0.10 TOB9B 4399 (99) 2.94 2.00 0.61 TOB9E 4395 (99) 1.50 1.01 3.46 TOB9F 4379 (99) 2.17 1.86 2.17 Tobacco 4349 (98) 10.00 0.88 1.12 ALC9A 4396 (99) 3.74 2.27 0.25 ALC9B 4378 (99) 2.31 1.34 0.69 ALC9C 4239 (95) 1.48 4.04 4.56 Alcohol 4213 (95) 10.00 0.84 1.14 MAR9A 4399 (99) 1.67 1.37 2.08 MAR9B 4194 (94) 0.52 2.96 7.82 M arijuana 4186 (94) 10.00 0.85 4.27 R JSK .9A 3582 (81) 3.16 1.33 -0.06 RISK9B 3613 (81) 2.72 1.28 0.35 RJSK9C 3612 (81) 2.30 1.23 0.71 RISK9D 3605 (81) 2.71 1.28 0.29 RISK9E 3605 (81) 2.15 1.16 0.92 Risktaking 3520 (79) 10.00 0.71 0.39 STRESS9A 3341 (75) 2.60 0.94 -0.10 STRESS9B 3318 (75) 2.37 1.02 0.15 STRESS9C 3301 (74) 2.58 1.06 -0.12 STRESS9D 3305 (74) 2.45 1.04 0.05 STRESS9E 3307 (74) 1.84 0.97 0.88 Stress 3273 (74) 10.00 0.73 0.06 DEP9A 3213 (72) 1.23 0.51 2.21 DEP9B 3170 (71) 1.43 0.61 1.12 DEP9C 3168 (71) 1.26 0.53 1.99 DEP9D 3154 (71) 1.32 0.57 1.60 DEP9E 3171 (71) 1.50 0.70 1.06 DEP9F 3157 (71) 1.42 0.61 1.16 DEP9G 3162 (71) 1.24 0.53 2.14 Depression 3009 (68) 10.00 0.65 1.42 FAM9A 3656 (82) 2.47 1.30 0.48 FAM9B 3652 (82) 2.58 1.30 0.38 FAM9C 3633 (82) 2.70 1.36 0.27 Family Conf. 3623 (82) 10.00 0.87 0.37 SELF9A 3536 (80) 3.43 1.95 0.16 SELF9B 3485 (78) 3.06 1.35 -0.16 SELF9C 3478 (78) 2.55 1.03 0.78 Self-Care 3430 (77) 10.00 0.74 -0.15 Grades 4388 (99) 3.68 1.83 0.51 Twelfth G rade Variable N (% ) Mean Std Skew TOB12A 2437 (55) 2.81 2.06 0.88 TOB12B 2430 (55) 1.62 1.46 2.44 TOB12C 2428 (55) 1.37 1.09 3.14 TOB12D 2425 (55) 1.18 0.68 4.02 Tobacco 2420 (54) 10.00 0.90 2.72 Log(Tobacco) 2420 (54) 2.30 0.08 2.53 ALC12A 2420 (54) 4.70 2.30 -0.43 ALC12B 2430 (55) 2.52 1.80 0.89 ALC12C 2431 (55) 1.69 1.23 1.83 ALC12D 2430 (55) 2.84 1.55 0.57 ALC12E 2311 (52) 229 4.10 3.34 Alcohol 2283 (51) 10.00 0.84 0.83 MAR12A 2418 (54) 2.21 1.96 1.44 MAR12B 2404 (54) 1.38 1.06 3.31 MAR12C 2413 (54) 1.77 1.32 1.75 MARI 2D 2355 (53) 0.98 3.98 5.63 M arijuana 2338 (53) 10.00 0.87 2.67 HARD12A 889 (20) 1.07 0.47 6.12 HARD12B 875 (20) 1.45 1.20 2.91 HARD12C 880 (20) 1.37 1.04 3.39 HARD12D 882 (20) 1.69 1.62 2.37 HARD12E 881 (20) 1.29 0.96 3.92 HARD12F 873 (20) 1.17 0.73 5.54 H ard Drugs 906 (20) 10.81 1.71 2.32 DEP12A 2397 (54) 1.48 0.54 0.49 DEP12B 2386 (54) 1.34 0.54 1.29 DEP12C 2391 (54) 1.17 0.42 2.55 DEP12D 2390 (54) 1.28 0.50 1.57 DEP12E 2389 (54) 1.64 0.73 0.68 DEP12F 2387 (54) 1.41 0.57 1.04 DEP12G 2390 (54) 1.14 0.41 3.02 Depression 2360 (53) 10.00 0.59 1.49 DRIVE12A 2435 (55) 1.96 1.01 1.92 DRIVE12B 2417 (54) 2.38 1.26 1.29 Drink/Drive 2410 (54) 10.00 0.89 1.47 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 43 greater than 2.0. This reflected the nature of the substance use data with larger percentages of no use/ infrequent use and long tails of higher use. 3.2 Analysis with Missing Data 3.2.1 Selection o f DMV Variables. The first step in evaluating the DMV variables for their relationship to Twelfth Grade nonresponse and subsequent inclusion in the imputation model was the logistic regression of attrition status on each DMV variable. Only the 84% of subjects with non-missing DMV data could be analyzed. As shown in Table 4,43 of the 73 DMV variables were significant in univariate models at the 0.20 level. COMRET (CA license returned by state) was not significant at the univariate level but was included in the multivariate model for substantive reasons, namely the potential for identifying subjects who had moved out of state. However, subjects with "1+" COMRET comments were not at greater risk of attrition (OR=T .1, p=0.76). All but three of the significant predictors were risk factors for attrition. Patients at a lower risk of attrition were those who were required to wear corrective lenses (COMEYE), had a greater number of accidents (NUMACC), and attended traffic school (TRAFSCHL). Forty-four DMV variables were then entered in a forward, stepwise logistic regression analysis. Of the 44 variables, the following eight variables were selected with an alpha-to-include of 0.05: LICENSE (license-related violations), NONOTHER (non moving violations other than speeding), CRIME (more serious criminal violations), EQUIPMNT (equipment-related), SUBSTNCE (substance-related), COMID (subject was Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 44 Table 4 Univariate DMV Predictors o f Attrition at Twelfth Grade a b Variable Description OR SE P COMLIC license canceled 2.8 0.73 0.16 COMFIN financial proof required 1.4 0.22 0.10 COMPRF proof code x date 2.2 0.25 <0.01 COMRET CA license returned by <state> 1.1 0.31 0.76 e COMNOCA does not hold valid CA license 2.5 0.09 0.00 COMDUP ID duplicate or no fee issued 1.8 0.35 0.09 COMPEND pending application 1.4 0.15 0.02 COMEYE must wear corrective lenses 0.6 0.11 <0.01 COMID subject issued ID card 2.1 0.07 <0.01 COMPERM no permanent CA license issued 4.4 0.31 <0.01 NUMPULL number of violation entries 1J 0.03 <0.01 NUMVIOL number of violation codes 1.4 0.03 <0.01 NUMACC number of accidents 0.9 0.10 0.20 NUMFTA number of failure to appear 2.4 0.10 <0.01 NUMABST number of violations not FTA 1.3 0.04 <0.01 NUMVC number of vehicle code violations 1.4 0.03 <0.01 NUMNOVC number of non-VC violations 2.5 0.25 <0.01 NUMACT number of court actions 2.0 0.10 <0.01 NUMSECT number of court action codes 2.0 0.10 <0.01 NUMSUSP number of suspensions 1.9 0.10 <0.01 SPEEDING speeding violation 1 .1 0.09 0.11 MOVOTHER other moving violation 1.8 0.10 <0.01 LICENSE license-related violation 3.1 0.10 <0.01 EQUIPMNT equipment-related violation 2.2 0.10 <0.01 NONOTHER other non-moving violation 3.2 0.11 <0.01 SUBSTNCE substance-related violation 5.3 0.37 <0.01 CRIME criminal violation 18.8 0.74 <0.01 RFRA financial responsibility— accident 1.6 0.27 0.09 RFRC financial responsibility-citation 3.4 0.50 0.02 RALC excessive blood alcohol 3.1 0.51 0.03 RFTA failure to appear 3.7 0.19 <0.01 RNEGPRF negligent operator proof required 6.2 0.65 0.01 RNEGOP negligent operator 3.1 0.38 <0.01 RPFTA provisional failure to appear 6.8 1.12 0.09 DPROB probation 3.2 0.39 <0.01 DSUSP suspended 2.8 0.14 <0.01 DSUBST substance use (court action) 2.7 0.48 0.04 DFAIL failure to appear (court action) 3.2 0.16 <0.01 DFINC financial responsibility (ct action) 1.9 0.25 0.01 DNEG negligent driving/damage (ct action) 5.5 0.57 <0.01 DPROC procedural (court action) 1.9 0.52 0.21 PT HIGH highest DMV point 1.4 0.07 <0.01 PT CUMUL DMV points sum 1.2 0.04 <0.01 TRAFSCHL traffic school 0.7 0.30 0.18 ‘based on 3727/4442 = 84% of subjects with non-missing data b droppers = 1390 (37%), stayers = 2337 (63%) c not statistically significant in univariate but included in stepwise procedure for substantive reasons Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 45 issued a California E D card), COMNOCA (subject does not hold a valid CA license), and COMPERM (no permanent CA license). It was noteworthy that the NONOTHER variable consisted mainly of “failure-to-appear” violations. The distinction in meaning between COMPERM and COMNOCA could not be determined. The multivariate model, consisting of the eight DMV variables from the stepwise procedure, showed that each variable was still significant at p < 0.01 by Wald's test (Table 5). Subjects with criminal violations were at the highest risk of attrition (OR=13.2, p<0.01). Also at high risk of attrition were subjects with substance use violations (OR=3.3, p<0.1) and those with no permanent California license (OR=4.1,p<0.01). Unfortunately, the three variables (CRIME, SUBSTNCE, and COMPERM) that exhibited the highest risk of attrition were also the most highly skewed. They were therefore excluded from the imputation model to avoid violation of the assumption of normality. This left the remaining five DMV Table 5 Multivariate DMV Predictors o f Attrition at Twelfth Grade a b Variable Description OR SE P LICENSE license-related violation 1.7 0.13 <0.01 COMID subject issued ID card 1.7 0.09 <0.01 NONOTHER other non-moving violation 2.0 0.13 <0.01 COMNOCA does not hold valid CA license 1.9 0.12 <0.01 COMPERM no permanent CA license issued 4.1 0.32 <0.01 CRIME criminal violation 13.2 0.76 <0.01 EQUIPMNT equipment-related violation 1.5 0.11 <0.01 SUBSTNCE substance-related violation 3.3 0.39 <0.01 a model resulting from forward stepwise logistic regression; alpha to include = 0.05; variables listed in order of entry b based on 3727/4442 = 84% of subjects with non-missing DMV data Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 46 variables for inclusion in the imputation model. These five variables showed that subjects in the " 1+" category were at a significantly higher risk of attrition, though in the 1.5 to 2.0 range (p < 0.01). 3.2.2 Tests for MCAR. Univariate tests for the assumption that the data were missing completely at random (MCAR) were performed for Ninth Grade and DMV variables. This consisted of a series of t-tests for continuous variables and crosstabulations for categorical variables with comparison groups formed by attrition at Twelfth Grade (Table 6). Continuous variables were recoded so that higher values represented more or worse levels of the variable. Dropouts had significantly higher levels of Ninth Grade tobacco, alcohol, and marijuana use as well as worse grades, more risktaking behavior, depression, family conflict, and self-care after school (p<0.01). Stress (9th) was the only variable for which dropouts and stayers did not differ (p=0.58). The difference between the means of the two groups even for the significantly different variables was not great, ranging from 0.08 for risktaking, depression, and self-care to 0.54 for grades. Due to the large sample size, the standard errors were small and the degrees of freedom were large, contributing to highly significant p-values. The DMV categorical variables showed the same pattern with higher percentages of violations among droppers. This is not surprising, however, because the logistic regression model of attrition used to select the DMV variables was essentially the same as the % 2 tests for MCAR. Droppers and stayers did not differ on COMRET, the DMV comment that related to moving out of state. Dropouts were Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 46 Table 6 Univariate Tests o f the MCAR Assumption for Continuous and Discrete Variables Continuous Variable (9th)a Mean Stayer Dropper P Tobacco 9.89 10.13 <0.01 Alcohol 9.94 10.07 <0.01 Marijuana 9.92 10.11 <0.01 Grades 9.76 10.30 <0.01 Risktaking 9.97 10.05 <0.01 Stress 10.01 9.99 0.58 Depression 9.97 10.05 <0.01 Family Conflict 9.95 10.07 <0.01 Self-Care 9.97 10.05 <0.01 Discrete Percent b Variable Stayer Dropper P Sex Male 45 52 <0.01 Female 55 48 Ethnicity White 32 26 <0.01 Asian 12 6 Black 9 17 Hispanic 44 46 Other 2 5 License returned 1 1 0.76 No CA license 10 22 <0.01 ID issued 29 47 <0.01 No permanent license 1 <0.01 License-related 7 19 <0.01 Equipment 9 18 <0.01 Non-moving,other 7 19 <0.01 Substance-related 0.4 2.2 <0.01 Crime-related 0.09 1.58 <0.01 a variables recoded so that higher values represent a higher quantity or worse level of variable b percentages for DMV variables represent subjects in the "yes" category Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 48 more often male and of Black, Hispanic, or Other ethnicity. On the other hand, stayers were more likely to be female, White, and Asian. Together, these set of tests provided evidence that the missing data was not MCAR. 3.2.3 Ninth Grade Item Missingness. Table 7 lists the results of an examination of item missingness at Ninth Grade for stayers and droppers at Twelfth Grade. Item missingness for survey data at Ninth Grade was significantly related to attrition at Twelfth Grade with almost twice as much item missingness for dropouts (p<0.01). The few subjects missing sex and ethnicity were also dropouts at Twelfth Grade. Matching of DMV data was significantly lower for those who attrited at Twelfth Grade (pO.Ol). Table 7 Ninth Grade Item Missingness for Stayers and Droppers at Twelfth Grade " Variable N Stayers (%) N Droppers (%) P Tobacco 34 (1) 59 (3) <0.01 Alcohol 95 (4) 134 (7) <0.01 Marijuana 102 (4) 154 (8) <0.01 Grades 16 (1) 38 (2) <0.01 Risktaking 383 (16) 539 (27) <0.01 Stress 476 (19) 693 (35) <0.01 Depression 642 (26) 791 (40) <0.01 Family Conflict 346 (14) 473 (24) <0.01 Self-Care 434 (18) 578 (29) <0.01 Sex 0 (0) 6 (0.3) <0.01 Ethnicity 0 (0) 4 (0.2) 0.03 DMVb 114 (5) 601 (30) <0.01 a based on n=2451 (55%) Stayers and n=1991 (45%) Droppers at Twelfth Grade b all DMV variables experienced same amount of missingness because DMV records were either matched or not matched Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 49 3.2.4 Data Augmentation Parameters. The number of parameters was too large for tabular presentation (279 per DMV model, for example); however, a selection of parameters is presented to provide a sense of the data augmentation process. Tables 8 through 10 show selected starting values and model parameters from five imputed datasets of the ignorable response model of tobacco use (12th) with DMV. The complete set of cell probabilities (ft) is given in Table 8. One may note that each column adds to one and no cell probability is zero. The estimates of cell probabilities, which are higher for Whites and Hispanics, seem reasonable given the demographics of the cohort. Table 9 shows the column of cell means (p.) for females of Other ethnicity. This group was chosen for illustrative purposes only. Again the estimates seem reasonable with Ninth Grade means around 10 and DMV variables between 1 and 2 Table 8 Cell Probabilities (n) for Multiple Imputation Model o f Tobacco Use (12th) -with DMV Variables Sex Ethnicity 0 100 Iteration 200 300 400 500 Male White 0.1429 0.1430 0.1405 0.1463 0.1489 0.1488 Male Asian 0.0491 0.0445 0.0565 0.0475 0.0445 0.0493 Male Black 0.0614 0.0674 0.0626 0.0623 0.0652 0.0567 Male Hispanic 0.2144 0.2052 0.2139 0.2355 0.2326 0.2151 Male Other 0.0148 0.0147 0.0113 0.0236 0.0233 0.0202 Female White 0.1515 0.1504 0.1463 0.1489 0.1400 0.1459 Female Asian 0.0464 0.0478 0.0475 0.0445 0.0516 0.0447 Female Black 0.0636 0.0618 0.0623 0.0652 0.0627 0.0628 Female Hispanic 0.2367 0.2466 0.2355 0.2326 0.2399 0.2340 Female Other 0.0190 0.0185 0.0236 0.0233 0.0234 0.0226 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 50 Table 9 Cell Means (ft) of Other Females for Multiple Imputation Model of Tobacco Use (12th) with DMV Variables Variable 0 100 Iteration 200 300 400 500 Non-moving,other 1.04 1.00 1.04 1.01 1.04 1.02 Equipment 1.06 1.08 1.12 1.09 1.12 1.07 License-related 1.04 0.99 1.04 1.02 1.09 0.97 ID issued 1.44 1.36 1.43 1.50 1.37 1.37 No CA license 1.15 1.07 1.09 1.13 1.06 1.14 Tobacco (9th) 10.07 10.10 9.99 10.00 10.03 9.84 Alcohol (9th) 9.98 9.96 10.01 9.97 10.14 9.75 Marijuana (9th) 9.86 10.01 10.01 9.89 9.81 9.69 Risktaking 10.05 10.09 10.11 10.02 10.08 10.06 Stress 10.29 10.26 10.19 10.40 10.40 10.25 Depression 10.22 10.13 10.16 10.14 10.14 10.09 Family Conflict 10.36 10.22 10.44 10.37 10.26 10.26 Self-Care 10.01 10.04 10.12 10.07 10.07 9.93 Grades 9.77 9.79 9.65 9.68 9.65 9.80 Tobacco (12th) 10.19 10.16 10.25 10.14 10.45 10.12 Table 10 Column of Tobacco Use (12th) in the Variance-Covariance Matrix (2 J ) for Multiple Imputation Model of Tobacco U se (12th) with DMV Variables 0 100 Iteration 200 300 400 500 Non-moving,other 0.0317 0.0234 0.0343 0.0333 0.0431 0.0318 Equipment 0.0256 0.0274 0.0333 0.0267 0.0232 0.0197 License-related 0.0408 0.0500 0.0465 0.0282 0.0541 0.0407 ID issued 0.0077 0.0116 -0.0089 0.0083 0.0205 0.0042 No CA license -0.0033 -0.0026 -0.0021 0.0056 0.0019 -0.0077 Tobacco (9th) 0.3715 0.3665 0.3888 0.3943 0.3801 0.3777 Alcohol(9th) 0.2435 0.2312 0.2545 0.2680 0.2476 0.2406 Marijuana (9th) 0.1731 0.1833 0.1768 0.2008 0.1893 0.1643 Risktaking 0.1139 0.1065 0.1426 0.1180 0.1086 0.1268 Stress 0.0732 0.0715 0.0501 0.0905 0.0587 0.0731 Depression 0.0657 0.0669 0.0546 0.0857 0.0676 0.0511 Family Conflict 0.1191 0.1067 0.0892 0.1275 0.1283 0.1359 Self-Care 0.0530 0.0379 0.0786 0.0605 0.0693 0.0376 Grades 0.1392 0.1274 0.1166 0.1544 0.1555 0.1502 Tobacco (12th) 0.8041 0.7914 0.8206 0.8019 0.8220 0.7928 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 51 (DMV variables were recoded for imputation from [0,1] to [1,2]). Table 10 gives one row of the variance-covariance matrix (2) for tobacco use (12th) with all other the three sets of parameters were stable in that they did not vary dramatically among the five "random" draws at each 100th iteration and were very close to the starting values provided by the EM algorithm. 3.2.5 Descriptive Statistics o f Imputed Datasets. The descriptive statistics for Twelfth Grade outcomes of all 140 datasets could not be listed, but the 20 sets of imputations of tobacco use (12th) are described in Table 11. Similar patterns were observed for other outcomes. One characteristic of the imputed data was that for ignorable models the minimum value was consistently lower than the complete case minimum while the maximum value was the same. Imputed outcomes were less skewed than complete cases, and outcomes of nonignorable models were less skewed than those of ignorable models. The imputed data variance was larger than the complete cases, but the SE was lower due to more observations (observed plus imputed) although the SE for nonignorable models was generally larger than the complete case SE. Means for ignorable responses were near the mean of the complete cases at 10.0. Models of nonignorable response shifted the means to the 11.0 range. The exception was hard drug use with nonignorable means around 12.3. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 52 Table 11 Descriptive Statistics for Imputed Datasets o f Imputation Models for Tobacco Use (12th) Dataset N Min Max Mean Variance SE Skewness I. Complete Case 2420 9.52 14.88 10.00 0.8071 0.0183 2.7248 II. Ignorat Dataset #1 > le/ No DMV 4442 6.63 14.88 10.04 0.8528 0.0139 1.4216 Dataset #2 4442 7.32 14.88 10.05 0.8381 0.0137 1.4407 Dataset #3 4442 7.21 14.88 10.05 0.8751 0.0140 1.4012 Dataset #4 4442 7.02 14.88 10.04 0.8223 0.0136 1.5136 Dataset #5 4442 6.79 14.88 10.04 0.8688 0.0140 1.3905 III. Ignora Dataset #1 ble/ DMV 4442 7.41 14.88 10.05 0.8220 0.0136 1.4889 Dataset #2 4442 7.27 14.88 10.05 0.8402 0.0138 1.4894 Dataset #3 4442 7.36 14.88 10.07 0.8187 0.0136 1.5304 Dataset #4 4442 7.54 14.88 10.07 0.8339 0.0137 1.4981 Dataset #5 4442 6.74 14.88 10.06 0.8338 0.0137 1.4505 IV. Nonig] Dataset #1 norable/ No DMV 4442 7.96 15.97 10.96 2.1415 0.0220 0.6784 Dataset #2 4442 8.78 16.41 10.97 2.1374 0.0219 0.6523 Dataset #3 4442 8.65 15.81 10.97 2.1838 0.0222 0.7073 Dataset #4 4442 8.42 15.81 10.95 2.0801 0.0216 0.6637 Dataset #5 4442 8.14 16.38 10.96 2.1564 0.0220 0.6978 V. Nonignr Dataset #1 arable/ DMV 4442 8.89 16.67 10.97 2.1234 0.0219 0.6378 Dataset #2 4442 8.72 17.35 10.97 2.1317 0.0219 0.6803 Dataset #3 4442 8.83 16.68 10.99 2.1512 0.0220 0.6368 Dataset #4 4442 9.05 16.80 10.99 2.1839 0.0222 0.6496 Dataset #5 4442 8.08 16.26 10.98 2.1454 0.0220 0.6432 3.3 Comparison o f Models As an example of a complete set of results, all five models of tobacco use (12th) are listed in Table 12. Differences between tobacco use (12th) and log tobacco use (12th) are noted Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced w ith permission o f th e copyright owner. Further reproduction prohibited without permission. Table 12 Repeated-Imputation Inferences for Linear Regression Coefficients for all Models o f Tobacco Use (12 th) Covariates I. Complete Case Beta(SE) p II. Ignorable No DMV Beta(SE) p III. Ignorable DMV Beta(SE) p IV. Nonignorable No DMV Beta(SE) p V. Nonignorable DMV Beta(SE) p Tobacco (9 th) 0.44(0.037) <0.01 0.47(0.033)0.01 0.47(0.026)0.01 0.62(0.043)0.01 0.60(0.037)0.01 Alcohol (9th) 0.10(0.036)<0.01 0.08(0.035) 0.05 0.07(0.026) 0.01 0.05(0.050) 0.32 0.04(0.040) 0.35 Marijuana (9 th) -0.07(0.038) 0.07 -0.05(0.024) 0.06* -0.04(0.033) 0.22 -0.02(0.035) 0.52 -0.01(0.041) 0.80 Grades 0.03(0.026) 0.29 0.01(0.022) 0.75 0.02(0.016) 0.16 0.23(0.030)0.01 0.25(0.024)0.01 Risktaking -0.01(0.038) 0.70 0.01(0.043) 0.81 0.02(0.029) 0.59 -0.00(0.060) 0.95 0.00(0.042) 0.91 Stress 0.05(0.036) 0.16 0.04(0.025) 0.14 0.01(0.037) 0.72 0.01(0.035) 0.80 -0.02(0.043) 0.73 Depression -0.05(0.040) 0.18 -0.03(0.032) 0.32 -0.04(0.026) 0.15* -0.03(0.049) 0.55 -0.05(0.040) 0.26 Family Conflict 0.04(0.031) 0.20 0.02(0.023) 0.50 0.02(0.031) 0.63 0.05(0.032) 0.15 0.04(0.042) 0.38 Self-Care -0.01(0.031) 0.71 -0.04(0.030) 0.25 -0.01(0.030) 0.67 -0.02(0.042) 0.63 0.00(0.043) 0.99 Sex -0.07(0.045) 0.13 -0.06(0.032) 0.10 -0.03(0.031) 0.33 -0.18(0.048)<0.01 -0.15(0.048)<0.01 Ethnicity= Asian -0.17(0.069) 0.01 -0.23(0.059)<0.01 -0.24(0.051)<0.01 -0.32(0.087)0.01 -0.33(0.079)0.01 Black -0.34(0.086)<0.01 -0.39(0.067)0.01 -0.39(0.055)<0.01 -0.14(0.092) 0.16* -0.14(0.082) 0.10* Hispanic -0.23(0.050)0.01 -0.28(0.049)0.01 -0.26(0.037)<0.01 -0.24(0.066)0.011 -0.23(0.056)0.011 Other 0.07(0.142) 0.63 -0.15(0.086) 0.09 -0.09(0.135) 0.54 0.22(0.126) 0.09* 0.30(0.180) 0.13* * significant for LOGTOB12 t not significant for LOGTOB12 U> 54 and discussed below. Subsequent comparisons between models list only those coefficients that changed observed significance levels between models. P-values were obtained for complete case models from the partial F-test or, in this case the equivalent t-test with n-p-2 degrees of freedom where n is the complete case sample size and p is the number of covariates. For complete cases p=14 and n ranged from 1324 to 1380 for all outcomes except hard drug use (12th) for which n=498. Therefore, df>1300 for each test except for hard drug use which had df=482. P-values for repeated-imputation inferences were obtained using a t-test of the average of the imputed betas, standard errors based on the within and between imputation variance and n=4442, and a formula for the degrees of freedom (v) developed by Rubin (1987). The formulas for these statistics were described in section 2.5.1. Values of v, taking Model II of tobacco use for example, were between 6.5 and 29.1, substantially lower than those of complete case analysis. 3.3.1 Complete Case vs Ignorable Model without DMV (I vs II). Changes in the observed significance levels between Models I and II are documented in Table 13 and occurred mainly in alcohol use (12th), marijuana use (12th), and hard drug use (12th). For alcohol use both marijuana use (9th) and letter grades (9th) fell from p<0.03 to p>0.15. For marijuana use (12th), letter grades was less significant in Model II (p=0.24), but Black ethnicity was a more significant predictor (p<0.01). In the hard drug use (12th) models, both marijuana use (9th) and risktaking (9th) were less significant in Model II. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 55 Table 13 Differences in Observed Significance Levels between Models land II Grade 12 Grade 9 I. Complete Case II. Ignorable/ No DMV Outcome Covariate Beta(SE) p Beta(SE) p Alcohol Marijuana -0.10(0.033)0.01 -0.09(0.057) 0.19* Grades 0.05(0.023) 0.02 0.03(0.018) 0.16* Depression -0.09(0.036)0.01 -0.06(0.029) 0.06* Marijuana Grades 0.07(0.026) 0.01 0.02(0.015) 0.24* Depression -0.08(0.040) 0.04 -0.06(0.034) 0.11* Ethnicity=Black -0.12(0.086) 0.16 -0.24(0.062) <0.01 f Hard Drugs Marijuana 0.40(0.105)0.01 0.30(0.153) 0.11* Risktaking 0.27(0.117) 0.02 0.22(0.133) 0.16* Ethnicity=Hispanic -0.32(0.159) 0.05 -0.43(0.177) 0.06* Depression Alcohol -0.05(0.024) 0.02 -0.05(0.023) 0.07* Log(Tobacco) Marijuana -0.01(0.003) 0.06 -0.00(0.002) 0.04t * significant for Model I f non-significant for Model I 3.3.2 Complete Case vs Nonignorable Model without DMV (I vs IV). The comparison of Models I and IV found changes in p-values for all six outcomes and log tobacco use (Table 14). Differences will be reported with Model I as baseline. Among Ninth Grade substance use, marijuana use was less significant in predicting alcohol use, hard drug use, and depression. Alcohol use (9th) was less significant for tobacco use, and its log transformation. Tobacco use (9th) was more significant (p<0.01) for depression (12th). Among Ninth Grade psychological constructs, depression (9th) was less significant for alcohol use, marijuana use, and drinking/driving. Risktaking (9th) changed to a non-significant status for hard drug use and drinking/driving; and stress Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Table 14 Differences in Observed Significance Levels between Models la n d IV 56 Grade 12 Grade 9 I. Complete Case IV. Nonignor./ No DMV Outcome Covariate Beta(SE) p Beta(SE) p Tobacco Alcohol 0.10(0.036)0.01 0.05(0.050) 0.32* Ethnicity=Black -0.34(0.086)0.01 -0.14(0.092) 0.16* Grades 0.03(0.026) 0.29 0.23(0.030)<0.0lf Sex -0.07(0.045) 0.13 -0.18(0.048)0.0If Alcohol Marijuana -0.10(0.033)0.01 -0.06(0.067) 0.40* Depression -0.09(0.036)0.01 -0.05(0.037) 0.15* Ethnicity=Black -0.23(0.077)0.01 -0.11(0.071) 0.11* Marijuana Depression -0.08(0.040) 0.04 -0.07(0.047) 0.15* Hard Drugs Marijuana 0.40(0.105)0.01 0.37(0.183) 0.10* Risktaking 0.27(0.117) 0.02 0.24(0.150) 0.18* Ethnicity=Hispanic -0.32(0.159) 0.05 -0.43(0.216) 0.10* Depression Marijuana -0.05(0.026) 0.05 -0.01(0.028) 0.60* Stress 0.10(0.023)0.01 0.04(0.032) 0.28* Tobacco 0.03(0.025) 0.23 0.l0(0.033)<0.01t Grades 0.02(0.018) 0.36 0.25(0.026)<0.01f Ethnicity=Black 0.03(0.058) 0.66 0.35(0.068)<0.01f Ethnicity=Other -0.00(0.095) 0.96 0.43(0.156) 0.02t Drinking/Driving Risktaking 0.13(0.040)0.01 0.13(0.060) 0.06* Stress 0.09(0.038) 0.02 0.06(0.049) 0.22* Depression -0.08(0.042) 0.05 -0.08(0.054) 0.17* Ethnicity=Black -0.21(0.092) 0.02 -0.03(0.086) 0.75* Grades 0.03(0.028) 0.24 0.23(0.025) <0.0 If Log(Tobacco) Alcohol 0.01(0.003)0.01 -0.00(0.007) 0.97* Ethnicity=Hispanic -0.02(0.005)0.01 -0.01(0.009) 0.20* Grades 0.00(0.002) 0.33 0.05(0.004)0.0 If Sex -0.01(0.004) 0.13 -0.03(0.008)<0.01f Ethnicity=Other 0.01(0.013) 0.67 0.07(0.021)<0.01t * significant for Model I t non-significant for Model I Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 57 (9th) exhibited a similar change for depression and drinking/driving. Letter grades (9th) changed from non-significant to significant in predicting tobacco use, depression, drinking/driving, and log tobacco use. Changes were also evident in the sex and ethnicity variables. Black ethnicity became non-significant in tobacco use, alcohol use, and drinking/driving but became significant in depression. Hispanic ethnicity was non-significant in hard drug use and log tobacco use. Other ethnicity increased significance levels in depression and log tobacco use. Sex also increased significance levels in tobacco use and log tobacco use. 3.3.3 Ignorable Models without and with DMV(II vs III). Changes between ignorable models without and with DMV are listed in Table 15 with Model II considered the reference model. Hard drug use exhibited the most changes which occurred in marijuana use (9th), risktaking (9th), and Black, Hispanic, and Other ethnicity. These covariates were more significant predictors with DMV although changes were marginal for Black and Hispanic ethnicity. Marijuana use (9th) was also more significantly predictive of alcohol use but less so for depression and log tobacco use. Depression (9th) was significantly predictive of log tobacco use, and letter grades (9th) was significant for depression. 3.3.4 Nonignorable Models without and with DMV (TV vs V). Changes in significance levels were also noted between DMV and non-DMV nonignorable models (Table 16). The same five covariates of hard drug use in the Model II and III comparison changed for Models IV and V, namely marijuana use (9th), risktaking (9th) Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Table IS Differences in Observed Significance Levels between Models Hand III 58 Grade 12 Grade 9 II. Ignorable/ No DMV III. Ignorable/DMV Outcome Covariate Beta(SE) P Beta(SE) P Alcohol Marijuana -0.09(0.057) 0.19 -0.07(0.031) 0.05t Depression -0.06(0.029) 0.06 -0.06(0.029) 0.05f Hard Drugs Marijuana 0.30(0.153) 0.11 0.27(0.088) 0.02f Risktaking 0.22(0.133) 0.16 0.20(0.067) 0.0 If Ethnicity=BIack -0.47(0.204) 0.06 -0.67(0.111) <0.0 If Ethnicity=Hispanic -0.43(0.177) 0.06 -0.38(0.083) <0.0 If Ethnicity=Other -0.70(0.421) 0.16 -0.76(0.169) <0.0 it Depression Marijuana -0.04(0.018) 0.03 -0.04(0.025) 0.17* Alcohol -0.05(0.023) 0.07 -0.05(0.021) 0.05t Grades 0.03(0.017) 0.15 0.03(0.010) <0.0 it Drinking/Driving Depression -0.09(0.036) 0.04 -0.06(0.030) 0.06* Log(Tobacco) Marijuana -0.00(0.002) 0.04 -0.00(0.003) 0.19* Depression -0.00(0.003) 0.30 -0.01(0.002) 0.03t * significant for Model II f non-significant for Model II Table 16 Differences in Observed Significance Levels between Models IV and V Grade 12 Grade 9 II. Nonignori No DMV III. Nonignor./ DMV Outcome Covariate______________Beta(SE) g________ Beta(SE)______ p Alcohol Ethnicity=Other 0.31(0.162) 0.08 0.35(0.146) 0.03t Marijuana Risktaking 0.11(0.042) 0.01 0.09(0.056) 0.15* Hard Drugs Marijuana 0.37(0.183) 0.10 0.33(0.103) 0.02t Risktaking 0.24(0.150) 0.18 0.22(0.082) 0.02t Ethnicity=BIack -0.36(0.246) 0.19 -0.61(0.138) <0.0lt Ethnicity=Hispanic -0.43(0.216) 0.10 -0.37(0.103) <0.01t Ethnicity=Other -0.72(0.507) 0.21 -0.79(0.214) <0.0It Depression Alcohol -0.09(0.033) 0.01 -0.08(0.041) 0.06* Drinking/Driving Risktaking 0.13(0.060) 0.06 0.10(0.043) 0.03t Stress 0.06(0.049) 0.22 0.08(0.036) 0.03t Sex -0.11(0.057) 0.06 -0.18(0.047) <0.0It * significant for Model IV t non-significant for Model IV Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 59 and Black, Hispanic, and Other ethnicity. Risktaking (9th) was less significant for marijuana use, and stress (9th) was more significant for drinking/driving. 3.3.5 Log Transformation o f Tobacco Use (12th). Overall, few changes occurred with the log transformation of tobacco use. Changes in p-values are noted in Table 12. The complete case analysis of log tobacco use yielded identical results to the non-transformed variable. Marijuana use (9th) was marginally more significant in Model II of log tobacco use (p=0.04), and depression (9th) was significant for Model III of log tobacco use (p=0.03). The majority of differences were found in the ethnicity variable for Models IV and V. Black and Other ethnicity were significant for log tobacco use while Hispanic ethnicity was not significant in comparison to tobacco use. 4. Discussion 4.1 Response Mechanism It is important to have a working hypothesis for the pattern of missing data in any study. Greenland and Finkle (1995) provided examples that suggested the non-MAR assumption is often plausible in epidemiology. Laird (1988) suspected that MAR data are common in longitudinal studies where attrition may depend on past performance that was observed. One may reason, therefore, that if attrition depended on missing data concurrent with attrition, the data would be non-MAR. Examination of the response mechanism in TVSFP was an important part of the analysis. Fortunately, there are only three choices: MCAR, MAR, or non-MAR. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Introduced above was the example of a MAR pattern of missing data in the hypertension trial (Rosendorff and Murray, 1986) in which dropout was planned for subjects with DBP above a prespecified level. In discussing the dropout or response mechanism, Murray and Finley (1988) and Little (1995) concluded the data were MAR because the cause of missingness was observed, namely DBP earlier in the study. Although the hypertension trial was a clinical trial and TVSFP had transitioned to an observational study, the differences in missing data are enlightening. Both studies involved repeated measurements of outcome, and the final measurement was partially missing in both studies. Both studies faced the question of whether response depended on the missing outcome data at the final measurement. In the hypertension trial, investigators knew that response to the week 12 DBP did not depend on week 12 DBP but rather on DBP at weeks 4 or 8. They knew this because they took the patients off study if their DBP was above the prespecified level in the protocol. In contrast, TVSFP did not take students off study if they had high levels of Grade Nine outcomes, for example alcohol use (9th). In addition, it was impossible for investigators to know whether response depended on Grade Twelve measures of alcohol use. Therefore, the difference between the two studies was that in the hypertension study the response mechanism was known while in TVSFP it was not known. A response mechanism may only be known with any certainty if it is under the control of the investigators. This leaves two alternatives for missing data in TVSFP. Consider one of the Twelfth Grade outcomes, alcohol use (12th), although the same reasoning applies to the Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 61 other outcomes. Either nonresponse depended on alcohol use (12th) or it did not. If nonresponse did not depend on alcohol use (12th), then the data were either MCAR or MAR. The examination of Ninth Grade covariates showed that the data were not MCAR, leaving the possibility of MAR, the best case scenario. On the other hand, if nonresponse did depend on alcohol use (12th), then the data were non-MAR. The distinction is crucial because with MAR data, the response mechanism is ignorable and even complete case estimates of regression coefficients are valid (Table 1). With non-MAR data, response is nonignorable, and currently there is no general method for obtaining valid parameter estimates. Assuming the worst case scenario for a moment, bias due to nonignorable nonresponse is reduced by imputing data with variables correlated with the outcome that experienced missingness. Among complete cases, Table 17 shows a positive correlation between Ninth Grade data and corresponding Twelfth Grade outcomes, Table 17 Correlations between Corresponding Ninth and Twelfth Grade Composite Variables Twelfth Ninth Grade Grade Tobacco Alcohol Marijuana Depression Tobacco 0.46 0.33 0.22 0.12 Alcohol 0.36 0.52 0.22 0.11 Marijuana 0.36 0.37 0.34 0.12 Depression 0.06 0.04 0.01 0.36 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 62 namely tobacco, alcohol, marijuana use, and depression. Inclusion of these variables in the imputation models would tend to reduce bias due to nonignorable response. With this line of argument, the question becomes one of degree of reduction of bias. Even if one uses ML estimation and includes variables correlated with outcomes which are missing data, in the absence of a gold standard it is difficult to say how much the bias is reduced. A potential problem with the Ninth Grade covariates in their ability to reduce bias is the item missingness. A small percentage of students (1% to 6%, Table 3) did not answer drug use questions even though the questions appeared at the beginning of the survey. These students were also more likely to drop out at Twelfth Grade (Table 7). Ninth Grade psychosocial variables were missing between 18% to 29% due to the three-form design. Because the forms were randomly assigned, the missingness could be considered random with respect to experimental condition. However, completion of these items was associated with attrition at Twelfth Grade. Therefore, it seems likely that the same mechanism that caused attrition at Twelfth Grade was responsible for at least some missing items at Ninth Grade and raises the possibility that the Ninth Grade item missingness is partially non-MAR. If the variables that were correlated with missing data were also missing data in a non-MAR pattern, then their effectiveness for reducing bias through the imputation model would be reduced. Because the percentage of missing Ninth Grade data is small, at least for substance use, this presents less of a problem. However, the same argument applies to the DMV data Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 63 which had a moderate amount of missing data. The potential reduction of bias by including Ninth Grade variables that were correlated with Twelfth Grade outcomes cannot be evaluated because these variables were included in all models. 4.2 Ignorable Models Given the best case scenario of a MAR missing data pattern, and thus an ignorable response mechanism, both Model I (complete case) and Model II (ignorable/no DMV) provide valid estimates of regression coefficients. Differences between the two models (Table 13) occurred mostly in three outcomes: alcohol use, marijuana use, and hard drug use. If one accepts the methodology behind the imputation process, the interpretation is straightforward. Had all subjects responded at Ninth and Twelfth Grade, the regression coefficients listed in Table 13 would have changed significance levels in the directions indicated with the associated observed significance levels (p-values). The important assumption behind Model II is that the response was ignorable, that is, dependent only on observed data. Intuitively, this means that the data imputed for droppers may have been systematically different than observed data for stayers but that the differences were random within levels of observed covariates. All but two of the changes in p-values listed in Table 13 were such that the regression coefficients were less significant in the ignorable model. If one imagines a regression coefficient as the slope of a line, the ignorable model had the effect of washing out the slopes. The regression coefficients that were less significant in Model II were closer to Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 64 zero than their complete case counterparts. However, the standard errors of Model II were generally smaller. Not shown in Table 13 are the special degrees of freedom (v) (see section 2.5.1) for the partial F-test or equivalent t-test for the regression coefficients. The special degrees of freedom decrease as the between imputation variance Bm increases, thus reflecting uncertainty due to nonresponse. The df for the repeated-imputation inferences were almost always less than the complete case df, thus accounting for the less significant results. Rather than being interpreted as spurious, this result reflects one of the purposes of multiple imputation, namely taking into account the uncertainty due to nonresponse. Given the large percentage of missing data, and thus more uncertainty, it is surprising that there were not more changes than were found. The fact that most of the regression coefficients did not change in observed significance level in Model II, however, gives researchers some measure of confidence in the complete case results, assuming ignorable nonresponse. An example of a substantive conclusion that might be drawn from Model II is the importance of letter grades (9th) in predicting alcohol use and marijuana use. In the complete case analysis, students with poor grades at Ninth Grade were more likely to use alcohol and marijuana at Twelfth Grade. However, under the assumption of an ignorable nonresponse, if all subjects had responded at Twelfth Grade, poor grades would no longer be significantly predictive of alcohol or marijuana use at Grade Twelve. If a theory existed for such a relationship, then either Model II provides evidence against the theory or the assumptions of the model could be disputed. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 65 4.3 Nonignorable Models In contrast to the ignorable models, comparison of the complete case analyses to the models of nonignorable nonresponse showed many changes (Table 14). These models tested the sensitivity of p-values to a hypothetical 20% higher response by droppers and those leaving items blank at Twelfth Grade. It should be emphasized that these results, though intriguing, do not reflect the true response mechanism, which is unknown. There are many ways droppers could have responded to create non-MAR data. The higher drug use created in Model IV (nonignorable/ no DMV) is consistent, though, with what is known about the substance use of school and study dropouts; and therefore, the model is at least plausible. Each of the six outcomes experienced changes from complete case p-values, though only one change was seen in marijuana use. Model IV exhibited changes both to less significant and more significant p-values. The changes to more significant results occurred mostly in tobacco use and depression. Continuing the example of letter grades (9th), if investigators were looking for a relationship between poor grades and Twelfth Grade tobacco use, depression, or drinking and driving/riding; then they would not have found evidence for this relationship in complete case analyses but would have found a significant relationship had droppers responded 20% higher in the above Twelfth Grade outcomes. The relatively large number of differences between Models I and IV suggests that less confidence should be placed in inferences concerning complete case regression coefficients. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 66 4.4 DMV Models DMV variables were chosen for their prediction of response but were not highly correlated with observed Twelfth Grade outcomes. The purpose of including the DMV data in two of the imputation models was to model the response mechanism and thus adjust the imputations for differences in response. There were some differences between models with and without DMV, but not as many as were produced by the nonignorable models. This confirmed Little and Rubin's (1989) skepticism regarding the use of variables that predict attrition but that are not correlated with variables with missing data. The exception was hard drug use for which the DMV- adjusted imputations produced more significant prediction from higher marijuana use and risktaking at Ninth Grade and significantly lower hard drug use by students of Black, Hispanic, and Other ethnicity. Why it did so for hard drug use and not for other outcomes is an interesting question but one that is only open to speculation. One possibility is that because hard drug use experienced the highest percentage of missing data (80%), it had the most to benefit from additional data. Also, hard drug use did not have a corresponding Ninth Grade covariate to reduce bias in the case of non-MAR data. Another possibility is that DMV variables were highly correlated with hard drug use, but the high correlation was not evident for the small percentage of observed data. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 67 4.5 Model Mispecification Little and Rubin (1989) stated, "Some might argue that ad hocery in the formation of estimates has been replaced by ad hocery in the assumptions of the model." They went on to say that model mispecification is also an issue in complete case analysis and imply that the same sort of approach may be used in model specification with incomplete data, for example, adding polynomial terms to the model. Laird (1988) warned, With complete data, likelihood analyses are fairly robust to use of an incorrect data model. With incomplete data there may be more sensitivity to model mispecification, since implicitly the data model is used to 'fill in' for the missing values on the basis of f(Ym | Y0 , X, 0)." Along these lines, Little and Rubin (1987) examined a nonignorable model that assumed log-transformed income data to be normal for respondents alone. They found substantially different intercepts but very similar regression coefficients. There is some evidence for mispecification in the imputation models presented here. The data were imputed outside the range of the observed data. This might be expected for a nonignorable model, but it also happened for the ignorable models in which the minimum value of the imputed outcome was consistently lower than the observed values (Table 11). This leaves the impression that the imputation model was trying to fill out the lower tail of a normal distribution. More evidence is that the imputed datasets were less skewed and thus more consistent with normality. An attempt to address this issue was the log transformation of tobacco use because a log transformation is known to normalize positive, L-shaped distributions Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 68 such as these substance use variables. It is doubtful whether this transformation worked because the skewness of tobacco use was only reduced from 2.72 to 2.53 (Table 3). The effect of the transformation on inferences was found to be minimal and was relegated mainly to the nonignorable models and the ethnicity variables (Table 12). To the extent that the log transformation created a more normal variable, this provides some evidence that model mispecification did not affect inferences on regression coefficients. It should also be noted that alcohol use (12th) was the least skewed variable (skewness=0.83, Table 3), and yet the ignorable model still produced changes from complete case analysis (Table 13). Another modeling issue is congeniality which is the agreement between the imputation and analysis model. Two different models were employed for imputation and analysis, the mixed model of Little and Schluchter (1985) and the linear regression model. There were two main differences. In the mixed model discrete variables were modeled as cells of a contingency table while in the regression model they were coded as dummy variables. The second difference was the inclusion of DMV variables in some imputation models but not in the corresponding regression models. In retrospect, it may have been prudent to do so to avoid the criticism of uncongeniality. The effect of the departure from congeniality is unknown. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 69 4.6 Conclusions This analysis was intended to explore nonresponse, and results should be interpreted in that light. Speculation concerning why certain covariates were significant predictors of outcomes in some models but not others seems less important than the fact that there were or were not many differences. An examination of the response mechanism showed that the data were not MCAR, but it could not be determined whether the data were MAR or non-MAR. Thus models representing both ignorable and nonignorable nonresponse were employed. Although the true model of nonresponse was not known, differences from complete case analyses were considered an important criteria. A quotation by F. H. C. Marriot (1974) provides some perspective in view of the massive methodological machinery and the vast array of numerical results: If the results disagree with informed opinions, do not admit a simple logical interpretation, and do not show up clearly in a graphical presentation, they are probably wrong. There is no magic about numerical methods, and many ways in which they can break down. They are a valuable aid to the interpretation of data, not sausage machines automatically transforming bodies of numbers into packets of scientific fact. The differences between complete case analyses and ignorable models do have a simple logical interpretation, namely that complete case inferences did not take into account uncertainty due to nonresponse and therefore were too sharp, that is, were significant too often. This is supported by the fact that all but two of the changes in signficance levels in Table 13 were in the direction of less significance for the ignorable models. Confidence in these results lies in the theory behind multiple Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 70 imputation which was designed to provide better estimates of variance than either complete case or single imputation. The method is based on a model and might be more sensitive to departures from the model than model mispecification without missing data. In comparison to the nonignorable models, there were fewer changes in the ignorable models. This gives some measure of confidence in complete case results for those who hold to the MAR assumption. On the other hand, under the assumption of one specific non-MAR pattern, inferences would have changed substantially. This is not to say that every non-MAR pattern would have produced such changes because other patterns were not examined. However, the imputation of a 20% higher response for droppers is consistent with what is known about substance use for dropouts of prevention studies. For those who hold the non-MAR assumption, the sensitivity of inferences to the simple yet plausible nonignorable nonresponse model should prevent over-confidence in complete case results. The attempt to adjust for differences between droppers and stayers by including auxiliary DMV data predictive of attrition did not produce many differences from imputation models without DMV except for the hard drug use outcome with 80% missing data. This finding confirms other work which suggests that it is not helpful to include variables in imputation models unless they are correlated with the variable that is missing. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. BIBLIOGRAPHY 7 1 Ashurst, J. T., De La Rocha, O. and Tobis, J. (1988). Analysis of collateral data: a method for assessing and correcting the effects of attrition on internal validity. Journal o f Experimental Education, 60,215-234. Azen, S. P., Van Guilder, M. and Hill, M. A. (1989). Estimation of parameters and missing values under a regression model with non-normally distributed and non-randomly incomplete data. Statistics in Medicine, 8,217-228. Biglan, A., Severson, H., Ary, D., Faller, C., Gallison, C., Thompson, R., Glasgow, R. and Lichtenstein, E. (1987). Do smoking prevention programs really work? Attrition and the internal and external validity of an evaluation of a refusal skills training program. Journal o f Behavioral Medicine, 10,159-171. Biglan, A., Hood, D., Brozovsky, P., Ochs, L., Ary, D. and Black, C. (1991). Subject attrition in prevention research. In Drug abuse prevention intervention research: Methodological issues (NIDA Research Monograph No. 107), C. B. Leukfeld and W. Bukowski (eds.), 213-234. Washington DC: U. S. Government Printing Office. Brannon, B. R., Dent, C. W., Flay, B. R., Smith, G., Sussman, S., Pentz, M. A., Johnson, C. A. and Hansen, W. B. (1989). The television, school, and family project: V. the impact of curriculum delivery format on program acceptance. Preventive Medicine, 18,492-502. Campbell, D. T. and Stanley, J. C. (1966). Experimental and Quasi-Experimental Designs for Research on Teaching. Chicago: Rand McNally. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16,297-334. Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977). Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion). Journal o f the Royal Statistical Society, B39, 1-38. Dent, C. W. (1988). Using SAS linear models to assess and correct for attrition bias. SAS User's Group International Proceedings o f the 13 th Annual Conference. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 72 Dwyer, K. M., Richardson, J. L., Danley, K. L., Hansen, W. B., Sussman, S. Y., Brannon, B., Dent, C. W., Johnson, C. A. and Flay, B. R. (1990). Characteristics of eighth-grade students who initiate self-care in elementary and junior high school. Pediatrics, 86,448-454. Epicenter Software (1993). Epilog Plus, Version 3. Pasadena, CA: Epicenter Software. Fay, R. E. (1992). When are inferences from multiple imputation valid? ASA Proceedings o f Survey Research Methods Section. Alexandria, VA: American Statistical Association. Fay, R. E. (1993). Valid Inferences from imputed survey data. ASA Proceedings o f Survey Research Methods Section. Alexandria, VA: American Statistical Association. Flay, B. R., Brannon, B. R., Johnson, C. A., Hansen, W. B., Ulene, A. L., Whitney- Saltiel, D. A., Gleason, L. R., Sussman, S., Gavin, M. D., Glowacz, K. M., Sobol, D. F. and Spiegel, D. C. (1988). The television, school, and family project: I. theoretical basis and program development. Preventive Medicine, 17, 585-607. Glynn, R., Laird, N. and Rubin, D. B. (1986). Selection modelling versus mixture modelling with nonignorable nonresponse. In Drawing Inferences from Self- Selected Samples, H. Wainer (ed), 115-142. New York: Springer-Verlag. Graham, J. W. and Donaldson, S. I. (1993). Evaluating interventions with differential attrition: the importance of nonresponse mechanisms and use of follow-up data. Journal o f Applied Psychology, 78,119-128. Graham, J. W., Hofer, S. M. and Piccinin, A. M. (1994). Analysis with missing data in drug prevention research. In Advances in Data Analysis fo r Prevention Intervention Research (NIDA Research Monograph No. 142), L. M. Collins and L. Seitz (eds.). Washington DC: U. S. Government Printing Office. Greenland, S. and Finkle, W. D. (1995). A critical look at methods for handling missing covariates in epidemiologic regression analsyes. American Journal o f Epidemiology, 142,1255-1264. Gorsuch, R. L. (1983). Factor Analysis, 2nd Edition. Hillsdale, NJ: L. Erlbaum Associates. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 73 Hansen, W. B., Collins, L. M., Malotte, C. K., Johnson, C. A. and Fielding, J. E. (1985). Attrition in prevention research. Journal of Behavioral Medicine, 8, 261-275. Heckman, J. J. (1979). Sample selection bias as a specification error. Econometrica, 47,153-161. Jamshidian, M. (1995). Statistical analysis with missing data using BMDP Release 8. Unpublished manuscript. Jurs, S. G. and Glass, G. V. (1971). The effect of experimental mortality on the internal and external validity of the randomized comparative experiment. Journal o f Experimental Education, 40, 62-66. Laird, N. M. (1988). Missing data in longitudinal studies. Statistics in Medicine, 7, 305-315. Leigh, J. P., Ward, M. M. and Fries, J. F. (1993). Reducing attrition bias with an instrumental variable in a regression model: results from a panel of rheumatoid arthritis patients. Statistics in Medicine, 12. Li, K. H. (1988). Imputation using Markov chains. Journal o f Statistical Computing Simulation, 30, 57-79. Little, R. J. A. (1988). A test of missing completely at random for multivariate data with missing values. Journal o f the American Statistical Association, 83, 1198- 1202. Little, R. J. A. (1992). Regression with missing X's: a review. Journal o f the American Statistical Association, 87, 1227-1237. Little, R. J. A. (1995). Modeling the drop-out mechanism in repeated-measures studies. Journal o f the American Statistical Association, 90,1112-1121. Little, R. J. A. and Rubin, D. B. (1987). Statistical Analysis with Missing Data. New York: Wiley. Little, R. J. A. and Rubin, D. B. (1989). The analysis of social science data with missing values. Sociological Methods and Research, 18,292-326. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 74 Little, R. J. A. and Schluchter, M. D. (1985). Maximum likelihood estimation for mixed continous and categorical data with missing values. Biometrika, 72,497- 512. Marriot, F. H. C. (1974). The Interpretation o f Multiple Observations. London: Academic Press. Quoted in Applied Multivariate Statistical Analysis, 3rd edition, R. A. Johnson and D. W. Wichem. New Jersey: Prentice-Hall. Meng, X. L. (1994). Multiple-imputation inferences with uncongenial sources of input (with discussion). Statistical Science, 9,538-573. Murray, G. D. and Findlay, J. G. (1988). Correcting for the bias caused by drop-outs in hypertension trials. Statistics in Medicine, 11,621-631. Olkin, I. and Tate, R. F. (1961). Multivariate correlation models with mixed discrete and continuous variables. Annals o f Mathematical Statistics, 32,448-465. Pirie, P. L., Murray, D. M. and Luepker, R. V. (1988). Smoking prevalence in a cohort of adolescents, including absentees, dropouts, and transfers. American Journal o f Public Health, 78,176-178. Richardson, J. L., Dwyer, K., McGuigan, K., Hansen, W. B., Dent, C., Johnson, C. A., Sussman, S. Y., Brannon, B. and Flay, B. (1989). Substance use among eighth- grade students who take care of themselves after school. Pediatrics, 84, 556- 566. Richardson, J. L., Radziszewska, B., Dent, C. W. and Flay, B. R. (1993) Relationship between after-school care of adolescents and substance use, risk taking, depressed mood, and academic achievement. Pediatrics, 92,32-38. Rosendorff, C. and Murray G. D. (1986). Ketanserin versus metoprolol and hydrochlorothiazide in essential hypertension: only ketanserin's hypotensive effect is age related. Journal o f Hypertension, 4, S109-S 111. Rubin, D. B. (1976). Inference and missing data. Biometrika, 63, 581-592. Rubin, D. B. (1987). Multiple Imputation for Nonresponse in Surveys. New York: Wiley. Rubin, D. B. (1991). EM and beyond. Psychometrika, 56,241-254. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 75 SAS Institute Inc. (1994). The SAS System for Windows 6.10. Cary, NC: SAS Institute Inc. Schafer, J. L. (1991). Algorithms for producing multiple imputations for incomplete multivariate data. Unpublished PhD thesis, Department of Statistics, Harvard University. Snow, L. S., Tebes, J. K. and Arthur, M. W. (1992). Panel attrition and external validity in adolescent substance use research. Journal o f Consulting and Clinical Pyschology, 60, 804-807. State o f California Vehicle Code 1991 (1992). Sacramento, CA: Department of Motor Vehicles. Sussman, S., Brannon, B. R., Flay, B. R., Gleason, L., Senor, S., Sobol, D. F., Hansen, W. B. and C. A. Johnson (1986). The television, school, and family project: II. formative evaluation of television segments by teenager and parents -- implications for parental involvement in drug education. Health Education Research, I, 185-194. Sussman, S., Dent, C. W., Brannon, B. R., Glowacz, K., Gleason, L. R., Ullery, S., Hansen, W. B., Johnson, C. A. and Flay, B. R. (1989). The television, school, and family project: IV. controlling for program success expectancies across experimental and control conditions. Addictive Behaviors, 14,601-610. Tanner, M. A. and Wong, W. H. (1987). The calculation of posterior distributions by data augmentation (with discussion). Journal of the American Statistical Association, 82, 528-550. West's Annotated California Codes: Business and Professional Code (1990). St. Paul, MN: West Publishing. West's Annotated California Codes: Penal Code (1990). St. Paul, MN: West Publishing. West's Annotated California Codes: Vehicle Code (1990). St. Paul, MN: West Publishing. West's Annotated California Codes: Welfare and Institution Code (1990). St. Paul, MN: West Publishing. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 76 APPENDIX A SURVEY CONSTRUCTS AND ITEMS Ninth G rade: Construct Variable Survev Item Tobacco Use TOB9A Have you ever smoked a cigarette? TOB9B How many cigarettes have you smoked in your W HOLE LIFE? TOB9C How many cigarettes have you smoked in the LAST W EEK (7 days)? TOB9D How many cigarettes have you smoked in the last 24 HOURS? TOB9E How many hours has it been since your LAST cigarette? TOB9F How many cigarettes do you currently smoke? TOB9 composite variable (excluding TOB9C and TOB9D) Alcohol Use ALC9A How many alcohol drinks have you ever had in your W HOLE LIFE? ALC9B When you drink alcohol, how many drinks do you USUALLY have? ALC9C How many days in the LAST MONTH (30 days) have you had alcohol to drink? ALC9 composite variable M arijuana Use MAR9A How many times have you used marijuana in your W HOLE LIFE? MAR9B How many days in the LAST MONTH (30 days) have you used m arijuana? MAR9 composite variable Risktaking RISK9A I don't want to wait several years before I can live the life that suits me. RISK9B I like people to think I'm older than I really am. RISK9C It is worth getting into trouble to have fun. RISK9D I like to take risks. RISK9E I enjoy doing things people say shouldn't be done. RISK9 composite variable Hostility HOST9A I am quick tempered. HOST9B When I get mad I say nasty things. HOST9 composite variable Stress STRESS9A In the last month, how often have you been upset because of something that happened unexpectedly? STRESS9B In the last month, how often have you felt unable to control the important things in your life? STRESS9C In the last month, have you felt nervous and stressed? STRESS9D In the last month, how often have you been angered because of things that happened that were outside of your control? STRESS9E In the last month, how often have you felt afraid when you were by yourself? STRESS9 composite variable Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 77 Construct Depression Family Conflict Self-Care Sociodemographic Twelfth G rade: Tobacco Use Variable DEP9A DEP9B DEP9C DEP9D DEP9E DEP9F DEP9G DEP9 Survey Item a. I do most things okay b. I do many things wrong c. I do everything wrong a. I think about bad things happening to me once in a while b. I worry that bad things will happen to me c. I am sure that terrible things will happen to me a. I hate myself b. I do not like myself I like myself a. all bad things are my fault b. many bad things are my fault c. bad things are not usually my fault a. I am tired once in a while b. I am tired many days c. I am tired all the time a. I do not feel alone I feel alone many times 1 feel alone all the time Nobody really loves me I am not sure if anybody loves me c. 1 am sure that somebody loves me composite variable c. FAM9A My family looks for things to nag me about.. . FAM9B My family doesn't understand me. .. FAM9C I have a lot of arguments with my family ... FAM9 composite variable SELF9A How many days do you take care of yourself in the afternoon or evening after school without an adult being there? SELF9B Think of those days that you take care of yourself in the afternoon or evening after school without an adult being there. How many hours do you usually take care of yourself? SELF9C W hat grade were you in when you FIRST started taking care of yourself with no adult at home at least one afternoon or evening a week? SELF9 composite variable GRADES9 W hat grades do you usually get in school? SEX <male, female> RACE <black, white, hispanic, asian, other> TOB12A How many cigarettes have you smoked in your whole life? TOB12B How many cigarettes have you smoked in the past month (30 days)? TOB 12C How many cigarettes have you smoked in the last week (7 days)? TOB 12E How many cigarettes have you smoked in the past 24 hours? TOB 12 composite variable Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 78 Construct Variable Alcohol Use ALC1 2A ALC12B ALC12C ALC12D ALC12E ALC 12 M arijuana Use MAR12A MAR12B MAR12C MARI 2D MAR12 H ard Drugs HARD12A HARD12B HARD12C HARD 12D HARD12E HARD12F HARD12G HARD 12 Depression DEP12A DEP12B DEP12C DEP12D DEP12E Survey Item How many drinks of alcohol have you had in your whole life? How many drinks of alcohol have you had in the past month (30 days)? How many drinks of alcohol have you had in the last week (7 days)? When you drink alcohol, how many drinks do you usually have at one time? How many days in the past month (30 days) have you had alcohol to drink? composite variable On how many occasions have you used marijuana (pot) or hashish in your whole life? On how many occasions have you used marijuana (pot) or hashish in the last month? When you use m arijuana or hashish, how many joints (or equivalents) do you usually have? (If you smoked with others, count only the amount you smoked) How many days in the past month (30 days) have you used marijuana or hashish? composite variable How many times have you used each of the following drugs in your whole life? drugs like heroin, morphine, or opium (other than prescribed by a doctor) LSD or acid "downers" like sleeping pills, barbiturates, tranquilizers, valium or Quaaludes (other than prescribed by a doctor) "uppers" like speed, amphetamines, or ice (other than prescribed by a doctor) sniffed (not smelled) glue, paint, gasoline, or other inhalants to get high PCP, angel dust or sherms composite variable a. I do most things okay b. I do many things okay c. I do everything wrong a. I think about bad things happening to me once in a while b. I worry that bad things will happen to me c. I am sure that terrible things will happen to me a. I hate myself b. I do not like myself c. I like myself a. all bad things are my fault b. many bad things are my fault c. bad things are usually not my fault a. I am tired once in a while b. I am tired many days c. I am tired all the time Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 79 Construct Depression (continued) Drunk Drive/Ride Variable Survey item DEP12F a. I do not feel alone b. I feel alone many times c. I feel alone all the time DEP12G a. Nobody really loves me b. I am not sure if anybody loves me c. I am sure that somebody loves me DEP12 composite variable DRIVE12A In the last m onth (30 days), how many times did you drive after drinking alcohol or using drugs? DRIVE 1 2B In the last month (30 days), how many times were you with someone sho was driving after drinking alcohol or using drugs? DRIVE 12 composite variable Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. APPENDIX B LIST OF DMV VARIABLES I. Violations Variable DescriDtion SPEEDING Speeding MOVOTHER Other moving violations LICENSE License-related EQUIMPNT Equipment NONOTHER Other non-moving violations SUBSTNCE Substance use CRIME Crime UNKNOWN Unknown NUMPULL Incidents NUMVIOL Section violations NUMACC Accidents NUMFTA Failure to appear NUMABST Abstracts of incidents NUMVC Vehicle code violations NUMNONVC Non-vehicle code violations PT CUMUL DMV points, cumulative PT HIGH DMV points, highest per incident TRAFSCHL Traffic school II. Comments COMLIC Cancelled mm/dd/yy. Voluntary surrender COMFIN Fr prf req mmddyy term mmddyy COM3 Medical expires mm-dd-yy COMPRF Proof code x eff date mm/dd/yy terminates mm/dd/yy (proof codes present data were A, B, C, K ; meaning unknown) COM5 Ambulance driver cert issued mm/dd/yy exp mm/dd/yy COMRET Calif lie returned by state COM7 Caution - mult records - same name/bd COMNOCA - does not hold valid calif lie COM9 Fr proof on file COM 10 IDcard canc - lost COMDUP ID dup or no fee iss mm/dd/yy COM 12 Pending renewal-by-mail COMP END Pending application COMM Proof filed SR22 operators coverage COM 15 Proof filed SR22 owners coverage COMM Proof filed SR22 broad coverage COM 17 Renewal-by-mail pending SSN COMEYE Must wear corrective lenses when driving COM 19 Class C/3 driving limited to vehicle with automatic transmission COM20 Limited to vehicles without air brakes COM21 Restricted to driving to/from/during course of employment per CVC 16072 COM22 Court restriction - drive to and from employment COM27 Class A/1 - B/2 - limited to vehicle with automatic transmission Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 81 COM28 Class M/4 driving limited to vehicle with automatic transmission COM29 M ay not drive in interstate commerce COM33 M ay drive a tour bus with appropriate class/endorsement COMID Subject issued id card 00/00/00 <expires mm/dd/yy> COM3S Subject reported deceased COMPERM X number index only - no permanent calif lie issued COM37 C ert code a, sch bus cert issued mmddyy, exp yy COM38 License valid pending review III. Court Actions Variable DescriDtion DCANC Cancelled DPROB Probation DRSTR Restriction DREVK Revoked DSUSP Suspended DSUBST Substance use DFAIL Failure to appear/pay fine/report DFINC Financial Responsibility DNEG Negligent driving/damage caused DPROC Procedural Rl 17711 and 17712 cancellations RFRA Accident - fr RFRC Citation - fr RALC Excessive blood alcohol level RFTA Failure to appear notice R6 Failure to surrender license R 9 Insurance cert cancelled R10 M anslaughter RNEGPRF Neg op proof req RNEGOP Negligent operator RPFTA Provisional fra NUMACT DMV Actions NUMSECT DMV Action section codes Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 82 APPENDIX C DESCRIPTION OF DMV VIOLATIONS AND COURT ACTIONS I. Violation Codes (five most frequent violations for each category) Category Description Speeding 1. Unsafe speed for prevailing conditions (use for all prima facie limits) 2. Exceeding S3 mph maximum speed limit 3. Speeding (out of state) 4. Exceeding 60 or 65 mph speed limit, as posted 5. Exhibition of speed, engaged aid or abet Other-Moving 1. Red or stop, vehicles stop at limit line or x-walk 2. Traffic control sign, failure to obey regulatory provisions 3. Driving without lights during darkness 4. Required or prohibited turn, failure to obey official sign 5. Turning without signaling last 100 feet License I. Driver, unlicensed 2. Vehicle unregistered or in violation of air pollution regulations 3. Evidence of financial responsibility 4. Driving when privilege suspended or revoked for other reasons 5. Driver's license, not in possession Equipment 1. Safety belts, drivers, or passengers 4 years or older, in private motor vehicle 2. Passenger over 16 years restrained by safety belt 3. Unlawful for parent to transport child without child passenger restraint system 4. Defective windshield or rear window, impairing driver’s view 5. Unsafe condition of vehicle, load or equipment Other Non-moving 1. Failure to appear, after signing citation or court continuance 2. Notice to department (failure to appear, failure to pay) 3. Payment of fines 4. Failure to pay installment fine for violation of div 1 1 5. Violation of promise to appear or pay fine (general) Substance Use I. Driving with a BAC of 0.08% or more 2. Under influence of alcohol, drug, or combination, drive a vehicle 3. Driver under 21, knowingly operating vehicle carrying alcohol 4. Controlled substances or alcohol-related offense 5. Passenger under 21, personal possession of alcohol in motor vehicle Crime 1. Auto theft 2. Theft by fraud 3. Injuring or tampering with vehicle or contents 4. Petty theft 5. Carrying concealed weapon in vehicle Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. II. Court Actions (all actions listed) Category Substance use Failure to appear/ pay fine/report Financial Negligent driving/ damage caused Procedural Description 1. Minor: revocation of driving privilege for DUI 2. Influence of alcohol or drugs, excess blood alcohol, addiction, or speed contest 3. Suspension (6 mo) for driving under the influence of alcohol 4. Suspension (18 mo) for driving under the influence of alcohol 5. Suspension (3 yr) for driving under the influence of alcohol 6. Refusal of chemical test (officer had reasonable cause to believe DUI) 7. Immediate suspension of license for blood alcohol of .08 or more 8. Suspension and restriction: commercial drivers license for DUI 9. Seizure of drivers license; suspension for blood alcohol > .08 1. Suspension for failure to appear 2. No formal hearing due to failure to respond to notice 3. Mandatory suspension of license for failure to report an accident 4. Failure to appear or pay fine 1. Suspension for failure to m aintain proof of ability to respond in damages after DUI 2. Proof of financial responsibility 3. Suspension for failure to prove financial responsibility at time of accident 4. Period of suspension: restriction alternative for failure to show financial responsibility 5. Failure to prove financial responsibility 1. Negligent operator; violation points 2. Required revocation for felony with motor vehicle involved 3. Required revocation for reckless driving causing bodily injury 4. Required revocation for manslaughter resulting from operation of motor vehicle 5. Release from liability for damages caused by minor by signing license application 1. Provisional license for minors: demonstration program 2. Surrender of license erroneously issued 3. Re-examination of licensee by department 4. Alternative action of suspending or revoking license after re-examination 5. Person involved to be notified of decision upon conclusion of hearing Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

A comparative study of environmental factors associated with multiple sclerosis in disease-discordant twin pairs

PDF

Familiality and environmental risk factors of peptic ulcer: A twin study

PDF

Immune recovery vitritis in AIDS: Incidence, clinical predictors, sequellae, and treatment outcomes

PDF

Descriptive epidemiology of thyroid cancer in Los Angeles County, 1972-1995

PDF

beta3-adrenergic receptor gene Trp64Arg polymorphism and obesity-related characteristics among African American women with breast cancer: An analysis of USC HEAL Study

PDF

Imputation methods for missing items in the Vitality scale of the MOS SF-36 Quality of Life (QOL) Questionnaire

PDF

Determinants of mammographic density in African-American, non-Hispanic white and Hispanic white women before and after the diagnosis with breast cancer

PDF

Cognitive dysfunction and occupations with likely significant magnetic field exposure: A cross-sectional study of elderly Mexican Americans

PDF

Associations between lung function growth and air pollution in two cohorts in Southern California children

PDF

Do reproductive factors have the same effect on breast cancer risk in African-American and White women?

PDF

Does young adult Hodgkin's disease cluster by school, residence and age?

PDF

Associations of weight, weight change and body mass with breast cancer risk in Hispanic and non-Hispanic white women

PDF

Development and evaluation of standardized stroke outcome measures in a population of stroke patients in rural China

PDF

Analysis of gene-environment interaction in lung cancer

PDF

Imputation methods for missing data in growth curve models

PDF

A joint model for Poisson and normal data for analyzing tumor response in cancer studies

PDF

Counter -matching in nested case -control studies: Design and analytic issues

PDF

The Effect Of Non-Sampling Measurement Errors On Relative Risk Estimates Of Coronary Heart Disease In Framingham Women

PDF

A study of pediatric oncology nurses' attitudes to and knowledge of genetic testing

PDF

Effects of myeloperoxidase (MPO) polymorphism and tobacco smoke on asthma and wheezing in Southern California children

Asset Metadata

Creator Howells, William Black (author)

Core Title An exploration of nonresponse with multiple imputation in the Television, School, and Family Project

School Graduate School

Degree Master of Science

Degree Program Applied Biometry

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag biology, biostatistics,health sciences, public health,OAI-PMH Harvest,sociology, theory and methods,Statistics

Language English

Contributor Digitized by ProQuest (provenance)

Advisor Dent, Clyde (committee chair), Gauderman, William James (committee member), Mack, Wendy (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c16-8261

Unique identifier UC11341504

Identifier 1383531.pdf (filename),usctheses-c16-8261 (legacy record id)

Legacy Identifier 1383531.pdf

Dmrecord 8261

Document Type Thesis

Rights Howells, William Black

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA