University of Southern California Dissertations and Theses

Understanding the methodological limitations In the ecological momentary assessment of physical activity

(USC Thesis Other)

Understanding the methodological limitations In the ecological momentary assessment of physical activity

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Running head: UNDERSTANDING LIMITATIONS IN EMA OF PHYSICAL ACTIVITY Copyright 2017 Eldin Dzubur UNDERSTANDING THE METHODOLOGICAL LIMITATIONS IN THE ECOLOGICAL MOMENTARY ASSESSMENT OF PHYSICAL ACTIVITY by Eldin Dzubur A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy (PREVENTIVE MEDICINE (HEALTH BEHAVIOR)) Institute for Health Promotion and Disease Prevention Research Department of Preventive Medicine University of Southern California August 2017 i DEDICATION To my family, for their sacrifice and support ii ACKNOWLEDGEMENTS First and foremost, I would like to acknowledge my mentor and committee chair Genevieve Dunton for her guidance, inspiration, and unwavering, selfless support. I am truly grateful for your mentorship and could not have wished for a more ideal advisor. I would also like to thank Stephen Intille from Northeastern University for developing all of the applications that make this dissertation possible, serving as a member of my dissertation proposal committee, and working with me throughout my doctoral program. Your work moved me to learn programming in-depth and subsequently broaden my career aspirations. Next, I want to acknowledge Jimi Huh and Adam Leventhal for providing valuable feedback and serving on my dissertation committee, and Donna Sprujit-Metz for providing advice and encouragement throughout my time at USC. I want to acknowledge current and past members of REACH Lab, who made the studies in this dissertation possible. On Mobile TEENS, Cesar Aranguri & Keito Kawabata and on MATCH, Frank Cedeno, Jing Ke, Gigi Lopez, Jaclyn Maher, Sydney O’Connor, Karen Ra, Lissette Ramirez, & Brian Redline. I want to thank all other interns, students, staff, and faculty of REACH Lab and the MATCH Working Group for their support. I am especially grateful for Marny Barovich, who consistently dedicates herself to the doctoral students in Preventive Medicine with seemingly around-the-clock support. I am also thankful for having supportive cohort members Angelica Delgado-Rendon, Cheng Freddy Wen, Robert Garcia, & Jennifer Tsai and the rest of Health Behavior Research who have attended talks, provided constructive feedback, and made the past five years a joyous experience. I am thankful for Alma Jusufagic and my friends in Los Angeles, especially for their patience during my dissertation process. iii Finally, I would like to express my gratitude and appreciation to my parents (Becir and Safeta Dzubur), who were forced out of Bosnia and Herzegovina in my toddler years and sacrificed much of their adulthood to make sure I would have a home, a healthy life, and an education. I am also grateful for my grandparents (Sabit and Ulfeta Dzubur and Velija and Celebija Custovic), my sister (Elma Dzubur), and the rest of my family in Michigan (Emir, Suada, Dzenis, & Medina Dzubur ) and abroad who supported me throughout my academic years. This dissertation was made possible by a two-year cancer control and epidemiology research training grant from the National Institutes of Health (2T32CA009492-31, PI: Pentz). iv Table of Contents DEDICATION I ACKNOWLEDGEMENTS II LIST OF TABLES VII LIST OF FIGURES VIII ABSTRACT IX CHAPTER 1: INTRODUCTION 1 BACKGROUND AND SIGNIFICANCE 1 Physical Activity Measurement 1 Limitations of Measurement with Accelerometers 3 Ecological Momentary Assessment 4 Addressing Limitations in Accelerometry with Ecological Momentary Assessment 7 Challenges of Ecological Momentary Assessment 7 OVERVIEW OF DISSERTATION STUDIES 10 CHAPTER 2: THE ASSOCIATIONS BETWEEN ENERGY INTAKE AND EXPENDITURE BEHAVIORS AND COMPLIANCE TO ECOLOGICAL MOMENTARY ASSESSMENT PROTOCOLS DESIGNED TO MEASURE ENERGY- BALANCE BEHAVIORS 12 ABSTRACT 12 INTRODUCTION 14 Energy Intake Behaviors 14 Energy Expenditure Behaviors 16 Gaps in Compliance Literature 19 Study Overview 20 Specific Aims and Hypotheses 20 METHODS 21 Participants 21 Procedure 22 Measures 23 Data Analysis 26 Power Analysis 30 RESULTS 32 Descriptive Statistics 32 Physical Activity and Compliance 35 Self-Reported Behavior and Compliance 40 DISCUSSION 44 Physical Activity and Compliance 44 Self-Reported Behavior and Compliance 46 Limitations 48 Implications 50 Future Directions 51 v CHAPTER 3: MODELING STUDY BURDEN AND PARTICIPANT FATIGUE IN ECOLOGICAL MOMENTARY PROTOCOLS WITH SUBJECTIVE SELF-REPORT AND OBJECTIVE SURVEY METADATA 53 ABSTRACT 53 INTRODUCTION 55 Factors Affecting Study Burden 55 Gap in Current Knowledge 58 Conceptualizing Participant Fatigue 59 Study Overview 61 Specific Aims and Hypotheses 61 METHODS 62 Participants 63 Procedure 63 Measures 64 Data Analysis 67 Power Analysis 71 RESULTS 73 Descriptive Statistics 73 User Satisfaction and Compliance 77 Investigating Survey Completion Time and Item Response Variance 78 Fatigue and Compliance 80 DISCUSSION 81 User Satisfaction and Compliance 82 Investigating Survey Completion Time and Item Response Variance 83 Fatigue and Compliance 84 Limitations 84 Implications 86 Future Directions 87 CHAPTER 4: REACTIVITY TO A LONGITUDINAL SMARTPHONE-BASED TIME- INTENSIVE PHYSICAL ACTIVITY ASSESSMENT 89 ABSTRACT 89 INTRODUCTION 91 Reactivity to Accelerometers 91 Reactivity to Ecological Momentary Assessment 93 Literature Gaps 94 Study Overview 94 Specific Aims and Hypotheses 95 METHODS 96 Participants 96 Procedure 96 Measures 98 Data Analysis 99 Power Analysis 101 RESULTS 102 Descriptive Statistics 102 Reactivity to Multimethod Protocols 103 vi Order Effects and Time 106 Order Effects and Compliance 109 DISCUSSION 109 Reactivity to Multimethod Protocols 110 Order Effects and Time 111 Order Effects and Compliance 112 Limitations 113 Implications 114 Future Directions 115 CHAPTER 5: DISCUSSION AND CONCLUSIONS 116 IMPLICATIONS 119 Methodological Implications 119 Theoretical Implications 121 Intervention Implications 123 FUTURE RESEARCH DIRECTIONS 125 CONCLUDING REMARKS 126 REFERENCES 127 vii LIST OF TABLES Table 1. Ecological momentary assessment schedule for mothers and children .......................... 23 Table 2. Demographic Characteristics at Baseline and EMA Descriptive Statistics .................... 34 Table 3. EMA compliance as a function of sedentary time 15, 30, and 60 minutes before a prompt in mothers and children ............................................................................................ 37 Table 4. EMA compliance as a function of light activity time 15, 30, and 60 minutes before a prompt in mothers and children ............................................................................................ 38 Table 5. EMA compliance as a function of MVPA time 15, 30, and 60 minutes before a prompt in mothers and children ......................................................................................................... 39 Table 6. EMA compliance as a function of self-reported activity at the prior prompt in mothers and children ........................................................................................................................... 42 Table 7. EMA compliance as a function of self-reported eating at the prior prompt in mothers and children ........................................................................................................................... 43 Table 8. Survey Item Response Variance Items for Mobile TEENS ........................................... 65 Table 9. End-of-Study EMA Satisfaction Items ........................................................................... 67 Table 10. Demographic and Ecological Momentary Assessment Descriptive Statistics ............. 75 Table 11. End-of-Study Ecological Momentary Assessment Satisfaction Descriptive Statistics 76 Table 12. Compliance as a function of EMA and end-of-study protocol satisfaction .................. 78 Table 13. Linear growth curve models of EMA survey completion time and survey item response variance ................................................................................................................................. 80 Table 14. Compliance as a function of EMA survey completion time and survey item response variance ................................................................................................................................. 81 Table 15. Accelerometer-measured activity as a function of real-time data capture protocol ... 104 Table 16. Interaction of day in study and protocol order on accelerometer-measured activity .. 107 Table 17. Day-level EMA compliance as a function of protocol order ...................................... 109 viii LIST OF FIGURES Figure 1. Sample screenshot of eating and activity item for the MATCH study .......................... 24 Figure 2. Power Curve for Research Questions 1-3 ..................................................................... 32 Figure 3. Basic Conceptual Model of Burden, Fatigue, and Compliance .................................... 58 Figure 4. Complete Conceptual Model of Burden, Fatigue, and Compliance .............................. 60 Figure 5. Power Curve for Research Question 4 & 6 ................................................................... 72 Figure 6. Power Curve for Research Question 5 .......................................................................... 73 Figure 7. Sample Screenshot of Mobile TEENS End-of-Day Recall Activity ............................ 98 Figure 8. Power Curve for Research Questions 7 & 8 ................................................................ 102 Figure 9. Power Curve for Research Questions 9 ....................................................................... 102 Figure 10. Measurement protocol type by order interaction predicting light activity time ....... 105 Figure 11. Measurement protocol type by order interaction predicting sedentary time ............ 105 Figure 12. Measurement protocol type by order interaction predicting MVPA time ................ 106 Figure 13. Time by measurement protocol order interaction predicting MVPA time ............... 108 Figure 14. Time by measurement protocol order interaction predicting sedentary time ........... 108 ix ABSTRACT This dissertation was primarily methodological in nature and explored limitations found in studies using ecological momentary assessment (EMA) as a sampling strategy to measure physical activity. The dissertation examines 1) the association between energy balance behaviors (i.e., diet and physical activity) and EMA prompt compliance, 2) subjective and objective indicators of participant fatigue and their association with EMA prompt compliance, and 3) changes in participant physical activity (i.e., reactivity) as an effect of EMA protocols. Subjective measures of physical activity and compliance data collected from EMA surveys, supplemented with objective measures of physical activity from accelerometers, were used to test time-varying (e.g., within-person and between-person) systematic non-compliance using multilevel models. Results revealed that participants were less likely to comply to EMA prompts when engaged in physical activity and more likely to comply to EMA prompts while sedentary. Next, the time it took participants to complete surveys decreased, while the variance in survey item response decreased throughout the measurement period. Also, participants who spent more time completing surveys, on average and for each survey, were less likely to comply to EMA prompts. Finally, participants who received an EMA plus accelerometer protocol at the beginning of the study had increasing levels of activity over time, similar to an intervention effect. However, participants with an accelerometer only training period prior to EMA had decreasing levels of activity throughout the study, suggestive of reactivity. These findings indicate that EMA studies measuring physical activity should employ strategies to reduce differential compliance, measure and minimize participant fatigue in real-time, and diminish the effects of reactivity with statistical modeling and optimized study design. 1 CHAPTER 1: INTRODUCTION Background and Significance Physical activity is widely recognized as having a positive effect on quality of life and overall health across the lifespan. Improvements in cognitive function are known to exist in individuals who engage in more frequent physical activity, and studies have revealed dose- response relationships between physical activity and decreased depressive symptoms (Dunn, Trivedi, & O'Neal, 2001; Hillman, Erickson, & Kramer, 2008). Even though physical inactivity is associated with higher body-mass-index (BMI, kg/m 2 ), and subsequently obesity (BMI ≥ 30, in adults), physical activity can be protective against all-cause mortality even in obese individuals when compared to inactive individuals with normal BMI (Fogelholm, 2010). Given the importance of physical activity as a predictor of health, researchers have validated and standardized measurement of physical activity across studies with an array of subjective and objective methods (Friedenreich et al., 2006; Hendelman, Miller, Baggett, Debold, & Freedson, 2000; Sallis, Buono, Roby, Micale, & Nelson, 1993). Physical Activity Measurement For purposes of standardizing findings across studies, physical activity and inactivity is generally divided into four intensity levels based on metabolic rate expenditure: sedentary, such as a seated resting position; light activity, such as standing or playing an instrument; moderate activity, such as brisk walking or casual bicycling; and vigorous activity, such as basketball or running (Ainsworth et al., 2000; Trost, Loprinzi, Moore, & Pfeiffer, 2011). The World Health Organization (WHO) provides multi-tiered evidence-based guidelines for adults (ages 18-64), suggesting 150 minutes of moderate-intensity aerobic activity (or 75 minutes of vigorous- 2 intensity, or some combination of both) per week with bouts lasting longer than 10 minutes, and recommending 300 minutes of moderate-intensity aerobic activity with muscle-strengthening at least two days a week (WHO, 2010). Hence, the classification of physical activity, along with WHO recommendations, provide guidelines for subsequent research and serve as a basis for implications from these findings. In order to differentiate sedentary behavior, light activity, and moderate-vigorous physical activity, and measure the frequency and duration of discrete intensity levels in free- living situations, researchers use a variety of methods such as participant self-report and waist- worn accelerometers (Freedson, Melanson, & Sirard, 1998; Troiano et al., 2008). Seven-day and three-day physical activity recall (PAR) instruments have been validated as cost-effective and low burden methods of capturing physical activity using systematic questionnaires (Atkin et al., 2012; Sallis & Saelens, 2000). However, recent studies have implicated everyday tasks (i.e., light physical activity) and interrupted sedentary behavior as potential correlates of health-related outcomes (Belcher et al., 2015; McManus, 2007). Moreover, relationships between predictors of poor health outcomes (e.g., waist circumference) and sedentary time remain significant, even after controlling for moderate-vigorous physical activity. This suggests that the use of passive objective measures to exhaustively capture free-living situations is especially advantageous to studying physical activity (Healy et al., 2008). For instance, motion data collected from accelerometers can be parsed with validated algorithms to determine whether participants are wearing their accelerometers, while advanced equations also allow researchers to obtain the estimated energy expenditure when the accelerometer is worn (Choi, Liu, Matthews, & Buchowski, 2011; Lyden, Kozey, Staudenmeyer, & Freedson, 2011). 3 Furthermore, as technology has improved, accelerometers have advanced participant wear-time, device interoperability, and passive activity recognition with improvements in battery life, wireless connectivity, and new algorithms that allow participants to wear accelerometers on their wrists (Troiano, McClain, Brychta, & Chen, 2014). Smartphones with accelerometers and dedicated computing power for accelerometry (i.e., motion co-processors) are becoming ubiquitous among consumers. Moreover, a recent study revealed that smartphones with motion co-processors (e.g., Apple iPhone 5s) yielded results similar to those of waist and wrist-worn accelerometers (Case, Burwick, Volpp, & Patel, 2015). Despite advantages found in passive real- time data collection and developments in sophisticated algorithms, accelerometers are still limited in their ability to categorize and capture contextual factors of physical activity and inactivity. Limitations of Measurement with Accelerometers Accelerometers are limited to capturing motion data, and they are sensitive to algorithm choices when computing time spent sedentary compared to accelerometer non-wear (Cain, Sallis, Conway, Van Dyck, & Calhoon, 2013). Studies have cross-validated accelerometers and utilized various non-wear cut points to determine the amount of time necessary to classify sedentary behavior as accelerometer non-wear (Winkler et al., 2012). As expected, changes in non-wear algorithm cut-points yielded misclassification of non-wear as sedentary behavior up to a third of the time and sedentary behavior as non-wear nearly half of the time. Furthermore, strength training and vigorous activities such as biking, where the waist or arms are limited in motion, cannot be captured by accelerometers alone. While some authors dismiss such activities as infrequent in the scope of a waking (i.e., 16-hour) day, other studies have had success with ankle-placement of accelerometers and multiple accelerometers worn by participants (Tudor- 4 Locke & Myers, 2001). However, ankle-placement does not capture subtleties in arm and waist movement, while multiple accelerometers can be burdensome to participants and costly to researchers (He et al., 2014; Mannini, Intille, Rosenberger, Sabatini, & Haskell, 2013). Lastly, contextual settings of activity, such as whether the activity was performed alone, and psychological information (e.g., whether the activity was enjoyable) about activities cannot be recorded by accelerometers. Social context and the physical environment are strong correlates of physical activity, obesity, and distal health outcomes (Dunton, Kawabata, Intille, Wolch, & Pentz, 2012; Durand, Andalib, Dunton, Wolch, & Pentz, 2011). One study examined the effect of peer and friend companionship in a biking task among normal and overweight children. Despite both groups showing increased motivation and activity duration while biking with a friend, only overweight children were positively influenced by peer companionship (Salvy et al., 2009). Contextual correlates are found among adults as well; for instance, a systematic review found that generally positive perceptions of neighborhood aesthetics, amenities, and safety were associated with physical activity and decreased sedentary behavior (Dunton, Atienza, Castro, & King, 2009; Dunton, Berrigan, Ballard-Barbash, Graubard, & Atienza, 2009; Humpel, Owen, & Leslie, 2002). Results from the aforementioned studies suggest that missing information from an accelerometer-only approach leaves behind both a biased sample of available activity data and missing contextual information. The inability for accelerometers to capture contextually-relevant information and load-bearing activities may have negative clinical implications given weekly strength-training recommendations by public health agencies. Ecological Momentary Assessment In order to augment or replace accelerometry as a measure of physical activity, a method must overcome limitations of accelerometry such as the inadequate ability to capture activities 5 with minimal waist or wrist movement as well as obtain contextual or psychological information about activities. Given advancements in technology and ubiquity of smartphones, a subjective self-report sampling strategy known as ecological momentary assessment (EMA) can be implemented through the development and implementation of smartphone applications (apps). Ecological momentary assessment is a real-time sampling strategy that is designed to overcome limitations in traditional cross-sectional and longitudinal study design (Shiffman, Stone, & Hufford, 2008). Cross-sectional and longitudinal studies sample participants one time or at infrequent intervals (e.g., six-month), subsequently masking variations within individuals throughout the course of the study. Moreover, data is gathered outside of a free-living context, such that lab settings no longer represent settings encountered in real life (i.e., representative design), and measured or unmeasured conditions may be incorrectly associated with outcomes (i.e., ecological validity) (Araujo, Davids, & Passos, 2007; Brunswik, 1955). Lastly, participants are often expected to recall extended periods of time with retrospective surveys (i.e., recall bias), especially in studies with health-related outcomes. EMA prompts participants with surveys in real-time, often using smartphones or other mobile devices (Shiffman et al., 2008). Participants are not confined to laboratory settings, and cues are generally associated with outcomes, thereby inherently possessing representative design and maximizing ecological validity. The time-intensive nature of prompting multiple times a day allows researchers to develop short surveys that do not ask participants to recall extended periods of time, subsequently reducing recall bias. Therefore, EMA serves as a foundation for assessing variables that are likely to vary on a day-to-day basis, as well as variables that are especially sensitive to context (Shiffman et al., 2008). 6 Historically, designs like ecological momentary assessment predate modern smartphones. Daily diaries have been used to record time-use and physical activity for decades; however, these studies could not capture within-day variation, suffered from recall biases similar to those found in cross-sectional studies, failed to capture context, and could not prevent participants from completing missed days retrospectively (Stone, Shiffman, Atienza, & Nebeling, 2007). The first modern ecological momentary assessment studies emerged in psychology (known as experience sampling methodology) using larger portable computing devices (e.g. Personal Digital Assistants). EMA methodology had also been adapted to text-message triggered surveys and phone-call surveys and continues to be an alternative when technical resources are constrained (O'Reilly & Spruijt-Metz, 2013). Early EMA studies used fixed time points to prompt participants with surveys, while modern studies are capable of prompting participants randomly within a time-window to improve representative design by limiting anticipation. For example, in a study capturing leisure time-use in adolescents, EMA prompts were scheduled to occur every fifteen minutes on four random days querying participants about their activity with a single item (Biddle, Gorely, Marshall, & Cameron, 2009). With the advent of smartphones, EMA prompts have advanced from less sophisticated signal-contingent (i.e., random) prompting schedules to advanced event-contingent (i.e., context- sensitive) prompting that can trigger surveys based on factors other than time (Intille, 2007). Studies have used device interoperability across wireless protocols to trigger context-sensitive EMA surveys, such as when participants use asthma inhalers equipped with specialized sensors in order to predict antecedents to asthma attacks by assessing stress, mood states, and activities after the use of an inhaler (Dzubur et al., 2015). 7 Addressing Limitations in Accelerometry with Ecological Momentary Assessment By utilizing advancements in computing and EMA methodology, limitations in accelerometry have been addressed through the supplementation of standard physical activity study design with EMA sampling. A prompting schedule can be arranged to capture the extent of a day and eliminate any vagueness that is classified as non-wear by an accelerometry algorithm. Moreover, at least one recent application has the capability to use built-in accelerometers to prompt participants after extended periods of no activity, activity, or missing data in order to minimize the amount of missing activity data (Dunton et al., 2014). Using self-report surveys, EMA is able to capture detailed information about activities (e.g., whether the activity involved ascending or descending stairs or a hill), as well as perceptions of activities (e.g., difficulty of an activity), that are impossible to distinguish using an accelerometer alone. Moreover, researchers are able to query participants on mood states, social context, physical context, and other psychosocial factors that are relevant to the study. In a longitudinal study designed to examine long-term effects of Smart Growth communities on health outcomes, participants were monitored for four days with external GPS sensors, accelerometers, and mobile devices. Participants were queried on activity type, social context, physical context, and other measures, and results revealed that activity in children often occurred outdoors and in the presence of others (Dunton, Kawabata, et al., 2012). Hence, supplementing standard physical activity study design with EMA addresses many of the limitations found in accelerometry and provides detailed insight on correlates of physical activity. Challenges of Ecological Momentary Assessment However, EMA study design itself has limitations that can lead to biased samples, poor participant satisfaction, and reduced statistical power. While sampling using EMA reduces biases 8 found in traditional recall study design (e.g., recall bias, ecological validity) and supplements physical activity measurement (e.g., measures of context), the challenges associated with this relatively novel approach are not thoroughly understood (Shiffman et al., 2008). First, compliance (i.e., response to a scheduled EMA prompt in a timely manner as defined by researchers) to EMA may be associated with an array of observed and unobserved variables that may bias sampling, especially in physical activity studies. Next participants are expected to complete an EMA protocol, and it is unclear how much fatigue participants experience on a daily basis because of study design parameters (e.g., measurement period duration). Lastly, while the knowledge of being measured by an EMA protocol may not affect behavior on its own (i.e., reactivity), there are no known studies estimating the extent to which reactivity occurs when EMA and accelerometers are used to measure physical activity. Compliance to EMA prompts is critical to the internal and external validity of a study, as poor compliance may result in a sampling bias toward a specific demographic of participants or toward responses completed at times of convenience. More importantly, systematic non- compliance may occur as a function of a measure of interest. Therefore, the variables assessed by EMA surveys may be associated with whether or not participants respond to surveys. This is especially true for variables of interest such as diet or physical activity that may be affected by social desirability or participant attentiveness to mobile devices (Tudor-Locke & Myers, 2001). The majority of EMA studies report compliance rates exclusively without analyzing associations between covariates or variables of interest and compliance (Dunton, Whalen, Jamner, & Floro, 2007). Still, findings across a small subset of EMA studies have revealed that temporal (e.g., time of day) and psychological variables (e.g., positive affect) are associated with compliance to subsequent EMA (Courvoisier, Eid, & Lischetzke, 2012; Dunton, Liao, Kawabata, & Intille, 9 2012). However, among studies that have analyzed physical activity as a correlate of EMA compliance, results have been mixed. There were no significant associations between physical activity and compliance in children, yet adults showed decreased compliance following physical activity (Dunton, Liao, Intille, Spruijt-Metz, & Pentz, 2011; Dunton, Liao, et al., 2012). While physical activity, psychological variables, and temporal variables have been acknowledged in studies, it is not known whether EMA compliance is associated with eating and other energy- intake behaviors. Hence, the resulting missing data patterns may not be random, and by ignoring associations between variables of interest and EMA compliance, subsequent models may produce biased estimates. Participant fatigue to a study protocol may occur as a function of the burden caused by the study. Burden caused by the EMA protocol varies depending study design choices made by researchers. To reduce burden and hypothetically improve compliance, Hufford and Schiffman have suggested strategies in EMA design such as decreasing the number of surveys participants receive per day, decreasing the time it takes to complete a given survey, shortening the measurement period of EMA, simplifying survey items and improving face validity, and enhancing the user experience with the EMA application (2003). Still, despite identifying factors thought to affect study burden, the association between these design choices and compliance is inconsistent across studies (Bond et al., 2013; Carels, Douglass, Cacciapaglia, & O'Brien, 2004; Kikuchi, Yoshiuchi, Ohashi, Yamamoto, & Akabayashi, 2007; Mitchell et al., 2014). The discrepancies across studies suggest that participant fatigue may occur prior to any changes in compliance; the latent effect of fatigue on compliance is further substantiated by decreases found in compliance over time in lengthy EMA studies. Yet, participant fatigue is only infrequently measured, usually at the conclusion of studies, and is rarely measured on a real-time basis (Cain, 10 Depp, & Jeste, 2009; King et al., 2008). Consequently, if participant fatigue is high, participants may be unlikely to repeat studies, refer others to the study, and yield survey responses that decline in validity over time. Moreover, highly fatigued participants may compromise the integrity of answers by trying to quickly complete surveys. Reactivity, defined as a change in behavior due to knowledge that one is being measured, is a threat to internal validity common across studies using accelerometers (Scott, Morgan, Plotnikoff, Trost, & Lubans, 2014). Specifically, children, but not adults, are known to improve their physical activity in the first few days of a study. Nonetheless, adults also show similar patterns of reactivity when asked to record step counts on a daily basis (Clemes & Parker, 2009). On the other hand, EMA protocols show little to no reactivity, even among measures such as body image or pain that are sensitive to reactivity in standard cross-sectional study design (Heron & Smyth, 2013; Stone et al., 2003). However, no such research exists for studies examining physical activity using EMA or multimethod (i.e., accelerometer and EMA) approaches. Assumptions made about the lack of reactivity in EMA studies can produce Type-II errors, where temporal trends caused by reactivity are thought to occur as a result of a predictor other than time. Overview of Dissertation Studies While non-compliance, participant fatigue to the protocol, and reactivity may pose threats to the internal validity of studies assessing physical activity using EMA, this sampling method may still be a valuable tool in the assessment of physical activity and other health behaviors, especially as a supplement to passive sensors such as accelerometers. By using secondary data analysis from existing EMA studies, this proposed dissertation will examine patterns of non- compliance, conceptualization of fatigue, and trends in physical activity to understand how each 11 of these limitations may affect study results. The three studies independently address each of the three aforementioned challenges faced in the design and implementation of EMA measurement of physical activity. The first study systematically examines the associations between diet and physical activity, and compliance to EMA prompts among adults and children. The second study describes and tests a novel conceptualization of participant fatigue in EMA studies. The third study examines if and how participants react to a time-intensive multimethod EMA and end-of- day activity recall protocol. Together, the studies comprehensively examine to what extent compliance, fatigue, and reactivity limit researchers when implementing EMA protocols in physical activity studies. Importantly, the potential findings from each study provide opportunities for guidance in designing future research proposals. Predictors of compliance may reveal biases in sampling, conceptualizing fatigue may be used to reduce burden studies place on participants, and understanding reactivity may explain changes in activity across a measurement period. 12 CHAPTER 2: THE ASSOCIATIONS BETWEEN ENERGY INTAKE AND EXPENDITURE BEHAVIORS AND COMPLIANCE TO ECOLOGICAL MOMENTARY ASSESSMENT PROTOCOLS DESIGNED TO MEASURE ENERGY- BALANCE BEHAVIORS Abstract Purpose: Ecological momentary assessment (EMA) is a real-time sampling strategy often used to measure energy intake (i.e., diet) and energy expenditure (i.e., activity) behaviors. While EMA studies are prone to missing data as a result of intensive longitudinal prompting, little is known about the how measured behavioral outcomes systematically influence compliance to EMA survey prompts. The purpose of this study was to test the association between EMA compliance and objective and subjective measures of physical activity and dietary behaviors. Methods: Mothers and their 8-12 year old children (N=200 dyads at baseline) from the first two measurement periods of an ongoing longitudinal study were instructed to complete EMA surveys up to eight times a day for a total of seven days, while simultaneously wearing accelerometers. Multilevel logistic regressions were used to test the likelihood of compliance to an EMA prompt as a function of time-lagged subjectively reported physical activity and eating behaviors and objective accelerometer-measured moderate-vigorous physical activity (MVPA), light activity, and sedentary time. Results: Mothers and children were more likely to comply to an EMA prompt if they were engaged in more than their own average sedentary time, less than their own average MVPA, and less than their own average light activity prior to an EMA prompt. Mothers were more likely to comply to an EMA prompt when reporting unhealthy eating or sedentary screen behavior at the prior EMA prompt compared to other unreported behaviors, while children were more like to comply to an EMA prompt when reporting healthy eating, sedentary 13 screen behavior, or physical activity behaviors at the prior prompt compared to unreported behavior. Children with less MVPA and greater sedentary time than other children, on average, were more likely to respond to any given EMA prompt, while mothers with greater than average light activity time were more likely to respond to any given prompt. Conclusions: Studies examining physical activity using EMA may inadvertently oversample sedentary behavior and undersample exercise and non-exercise activities at the prompt level because participants are less likely to respond to EMA prompts while they are active. Further, self-reporting of behavior may influence subsequent compliance, suggesting a psychological mechanism (e.g., social desirability bias) that explains differential compliance to EMA prompts. Hence, additional steps may need to be exercised when implementing EMA protocols in studies of energy balance behaviors. 14 Introduction Although obesity rates in the United States have stabilized over the past decade, almost 35% of adults and nearly 18% of children are classified as obese. Moreover, an additional 35% of adults and 12% of children are overweight and at risk for obesity (Flegal, Carroll, Ogden, & Johnson, 2002; Skinner & Skelton, 2014). The scope of this issue underlines the importance in studying the energy-balance behaviors that can be used to shape obesity prevention policy and development of obesity interventions. To overcome limitations in laboratory research, cross- sectional research, and standard longitudinal designs, researchers often use ecological momentary assessment (EMA) to measure energy intake and expenditure behaviors. EMA is a real-time sampling strategy that reduces recall biases and improves ecological validity by capturing responses in free-living situations (Shiffman et al., 2008). Notwithstanding advantages in study design, time-intensive data collection is sensitive to missing data caused by survey non- compliance that may lead to sampling biases when measured variables are associated with patterns of compliance to survey prompts. Furthermore, the extent to which energy intake and expenditure behaviors influence EMA survey compliance is a critical gap in the literature. Energy Intake Behaviors Energy intake behaviors, such as the consumption of food and beverages, are critical targets of weight-loss interventions given their role in an energy balance equation. Without a calorie restricted diet, physical activity interventions yield diminished effect sizes, and in some cases, may produce weight gain (Thomas et al., 2012). While combined lifestyle changes across diet and activity are ideal, the ease of dietary restrictions, in comparison to the integration of physical activity into daily habit, shows great success in short-term weight loss, especially in older adults (Messier et al., 2013). Still, the prevalence and low cost of fast food, junk food, and 15 sugar-sweetened beverages present significant environmental hurdles to balancing caloric intake with a nutritious diet (Powell & Nguyen, 2013). Moreover, the associations between consumption of calorie-dense food and negative emotions (i.e., eating to reduce a stress response) suggests that further research is needed to determine psychological antecedents of energy intake behaviors (Tsenkova, Boylan, & Ryff, 2013). Real-time Data Capture of Dietary Intake Ecological momentary assessment provides a real-time approach to assessing the relationships between psychological and environmental variables and energy intake behaviors, and has been implemented in several studies. For instance, to examine how the environmental presence of food affects consumption differentially in normal and overweight/obese individuals, Thomas and colleagues deployed a mobile EMA study that prompted adult participants up to six times a day over the course of a week (2011). The surveys queried participants on food they ate since last interacting with their mobile device, and the presence of other palatable foods during that same time frame. The study found that individuals with higher BMI were more likely to overeat in the presence of other foods as compared to their leaner counterparts. Participants completed approximately 70% of the mobile prompts, and the study found no temporal predictors of compliance (Thomas et al., 2011). However, given the importance of temporality between prompts, and the potential for significant recall bias if multiple prompts are missed, the EMA surveys that were completed may have oversampled eating frequency. As expected, there was a high frequency of reported overeating as noted by the authors, and this could be an artifact of the design, as opposed to participant perception (Thomas et al., 2011). The findings from this study highlight how missing data in EMA studies may influence estimates, and how it is possible to examine behavioral predictors, such as eating frequency, to determine if the estimates are biased. 16 EMA studies utilizing energy intake behaviors as outcomes or predictors generally fail to report correlates of EMA compliance, and the previous study by Thomas and colleagues represents a common gap, not an exception. For example, a study by Grenard and colleagues used EMA to examine consumption and predictors of sugar-sweetened beverages and snacks in adolescents over a seven day period using up to five signal-contingent prompts (2013). Given the mixed study design, compliance was only available for signal-contingent prompts, and the study reported approximately a 70% compliance rate. There was no analysis completed to determine correlates of compliance, despite the possibility of differential over or under-reporting of eating events with event-contingent prompts. Signal-contingent prompts contributed an additional 25% of data to event-contingent prompts, and identifying predictors of compliance could determine whether or not the sampling was biased (Grenard et al., 2013). While no knowledge exists on whether or not energy intake behaviors affect EMA prompt compliance, energy expenditure has been investigated as a correlate of compliance in several studies. Energy Expenditure Behaviors Energy expenditure behaviors, defined as any calorie expenditure split into sedentary, light, and moderate-vigorous physical activity categories, are associated with all-cause mortality and an array of diseases including diabetes and hypertension (Kodama et al., 2013; van der Ploeg, Chey, Korda, Banks, & Bauman, 2012). Despite mixed results as a target in interventions for weight loss, increase in energy expenditure is associated with improvements in cardiovascular health, and has been shown to attenuate the negative health effects associated with high BMI (Barry et al., 2014; Thomas et al., 2012). Furthermore, physical activity is associated with global improvements in sleep, alertness, memory, affect, stress, and other psychological outcomes (Bherer, Erickson, & Liu-Ambrose, 2013; Martikainen et al., 2013; 17 McClain, Lewin, Laposky, Kahle, & Berrigan, 2014). Moreover, new findings show that non- exercise activity thermogenesis (NEAT), commonly defined as light, moderate, or vigorous activity occurring as a result of everyday tasks, may be a more significant predictor of overall health than previously thought (Villablanca et al., 2015). As a result of exercise, studies show that individuals consume more calories to compensate for perceived expenditure; however, compensatory energy intake is limited when individuals do not recognize activity as exercise, such as with NEAT (Werle, Wansink, & Payne, 2011). Therefore, it is becoming increasingly crucial for researchers to classify activity using validated measures of physical activity, and to merge this data with repeated measures that allow for participant self-report. Real-time Data Capture of Physical Activity EMA is able to integrate objective data from accelerometry along with subjective reports of physical activity and psychological variables. This allows researchers to differentiate perceptions of physical activity, as well as to examine psychological antecedents and outcomes. Across studies examining physical activity with EMA, compliance to prompts is widely reported and similar to EMA studies outside physical activity measurement (Dunton et al., 2007). Generally, 70-80% compliance to EMA prompts in studies examining physical activity is considered acceptable and widely reported (Biddle et al., 2009; Liao, Skelton, Dunton, & Bruening, 2016). However, compliance may decrease sharply across a measuring period, with at least one EMA study assessing physical activity revealing almost a 50% decline in compliance over seven days (Spook, Paulussen, Kok, & Van Empelen, 2013). While studies that examine physical activity do often report rates of answered surveys, analyzing patterns of unanswered surveys and determining how activity is correlated to these patterns is limited. For instance, in a small sample of older adults, antecedents of physical 18 activity were measured using an electronic diary on a PDA over a period of two weeks (Dunton, Atienza, et al., 2009). Participants received four prompts per day and answered 87% of surveys. Social interaction, self-efficacy, positive affect, and energy were associated with compliance to prompts, yet physical activity was not assessed as a potential correlate despite its role as an outcome (Dunton, Atienza, et al., 2009). Still, there are inconsistent results even among studies that have assessed physical activity as a predictor of EMA compliance. For instance, a study by Dunton and colleagues assessed the acceptability of an EMA protocol and examined validity of physical activity responses with accelerometer wear (2011). The study provided a comprehensive analysis of temporal, affective, and demographic predictors of compliance, including physical activity. Children were provided with mobile phones, and they received a total of 20 prompts across four days outside of school hours. Results showed acceptability (80% compliance), feasibility, and validity of activity report gathered from EMA. Contrary to hypothesized predictors in methods literature, physical activity before an EMA prompt was not associated with answered surveys, and EMA surveys did not interrupt physical activity (Shiffman et al., 2008; Tudor-Locke & Myers, 2001). Ethnicity was the only significant predictor of compliance, with White children answering surveys more often than those of other ethnic backgrounds (Dunton et al., 2011). However, the findings in the study did not use a continuous measure of MVPA, nor did they analyze sedentary or light activity as reported by the accelerometer. The results may not generalize to all populations, however, as a study with adults enrolled in an identical protocol examining activity using EMA and accelerometry found contrasting results (Dunton, Liao, et al., 2012). MVPA prior to an EMA prompt was associated with a decreased likelihood of answering a survey, and overweight individuals were more likely to become more sedentary after answering 19 an EMA prompt, suggesting that EMA surveys may disrupt activity and increase sedentary behavior in adults. Given mixed findings among these two studies, more research is needed to expose whether or not physical activity is associated with EMA prompt compliance, and to determine whether the aforementioned differences between adults and children are significant. Gaps in Compliance Literature The use of EMA protocols in diet (i.e., energy intake) and physical activity (i.e., energy expenditure) research provides numerous advantages, as well as the ability to assess both outcomes simultaneously for energy balance research. Although few studies have examined the association between physical activity and EMA prompt compliance, results were not consistent (e.g., MVPA was associated with decreased compliance in adults, but not children) and analyses were not comprehensive (i.e., MVPA, light activity, and sedentary behavior were not independently assessed). Furthermore, it is unclear how patterns of unanswered EMA prompts are associated with diet, as no known studies have examined this association. Moreover, hypotheses about the causal directionality in the relationship between diet and EMA prompt compliance are vague, given the lack of research. While physical activity research using EMA posits that individuals may not answer EMA prompts due to their inattentiveness to prompts while engaging in higher levels of physical activity, the logic may not extend to eating behaviors. While there is a likelihood that participants may not answer surveys because they are busy eating, the act of eating does not preclude the use of a smartphone or similar device. Furthermore, research on binge eating and related eating disorders shows little evidence of EMA compliance causing changes in eating behavior, but no findings have tested the effects of diet on compliance (Smyth et al., 2007). For example, participants may intentionally ignore EMA prompts after eating unhealthy foods as a means for social desirability, leading to an over- 20 reporting of healthy foods. Lastly, it is unclear if predictors of compliance in diet and physical activity studies differ between adults and children. Study Overview The study addressed conceptual gaps in physical activity and diet-related EMA literature by testing physical activity and eating behavior as predictors of compliance to EMA prompts in adults and children enrolled in an ongoing EMA study examining health-related behaviors. Furthermore, potential differences between adults and children were examined, as physical activity and eating behaviors are known to differ throughout the lifespan and may influence compliance differentially (Malina, 1996). Specific Aims and Hypotheses 1. To test the associations among continuous measures of activity (i.e., accelerometry) and EMA recall measures (i.e., activity questions) of activity with EMA prompt compliance. Accelerometry-measured MVPA time and EMA-reported sports or exercise were hypothesized to be inversely associated with EMA compliance at the subsequent prompt. Overall MVPA or proportions of EMA-reported sports or exercise were hypothesized to show no significant associations with overall EMA compliance. On the other hand, accelerometry- measured sedentary time and EMA-reported screen behaviors were hypothesized to be positively associated with EMA compliance at the subsequent prompt. Similarly, overall sedentary time or proportions of screen-behaviors were hypothesized to be positively associated with overall EMA compliance. 2. To test the associations between EMA measures of energy intake (i.e., food consumption) and EMA prompt compliance. 21 Diet-related predictors of compliance to EMA were not known. Given the lack of literature, this is an exploratory aim with no a priori hypotheses. 3. To test whether adults and children differ in their association between energy intake and expenditure behaviors and EMA prompt compliance. MVPA just before an EMA prompt was hypothesized to be inversely associated with compliance in adults, but not children. Overall sedentary behavior was hypothesized to be positively associated with overall compliance in adults, but not children. The relationship between EMA prompt compliance and light activity was an exploratory aim. Methods This study used two measurement periods (i.e., waves) of data from an ongoing dyadic study examining the effects of mothers’ stress and parenting behaviors on their children’s eating and physical activity. The study, known as Mothers’ and Their Children’s Health (MATCH), is a longitudinal mixed-methods case-crossover dyadic study utilizing dietary recall, EMA, salivary cortisol, accelerometry, and surveys over a period of six waves of data collection separated by six months each. The proposed study utilized EMA and accelerometry at both measurement waves, along with baseline survey and anthropometric data obtained at the first wave. Ancillary measures and procedures in the MATCH study are detailed elsewhere (Dunton et al., 2015). Participants Participants in the MATCH study were recruited in the Greater Los Angeles area at school and community sites. An ethnically diverse sample of working mothers and their eligible 8-12 year-old children were recruited as dyads, and those who expressed interest were contacted to participate in the study. Inclusion criteria for the study included: (1) having a child that was in 3rd-6th grade; (2) having at least 50% custody of the child; and (3) both members of the dyad 22 being able to read English or Spanish. Participants were excluded from the study if they met the following criteria: (1) reported taking medications for thyroid function or psychological conditions; (2) reported health issues that limited physical activity. Children were excluded if they were: (3) enrolled in a special education program at school; (4) used oral/inhalant corticosteroids for asthma; (5) were underweight based on age-sex adjusted BMI percentile. Mothers were excluded if (6) they worked more than eight hours on any weekend day or any time on more than two weekday evenings. 201 dyads were enrolled in the study, and 200 dyads (99.5%) completed the EMA protocol with data available for at least one participant at baseline. Procedure Participants were assessed with surveys and measured for anthropometric data at the beginning of each period of data collection. Mothers and children were each provided with an Actigraph accelerometer and a Moto G (Motorola Mobility) smartphone with the custom MATCH application installed. Each participant in a mother-child pair was instructed to wear their accelerometer; research staff also provided a tutorial on the use of the EMA application. Each period of data collection in the MATCH study consisted of six days of EMA plus two half- days for a total of seven complete days of EMA. Participants were prompted up to eight times on weekends and four times on weekdays outside of school or work hours (Table 1). Mothers were prompted at random during the first half of an hour-long window and children were prompted at random during the second half of the window. Mothers and children had the ability to specify bedtime and wake time to prevent the application from prompting surveys when participants were asleep. 23 Table 1. Ecological momentary assessment schedule for mothers and children Weekdays Weekend Days 7:00am - 8:00am No Assessment EMA 9:00am - 10:00am No Assessment EMA 11:00am - 12:00pm No Assessment EMA 1:00pm - 2:00pm No Assessment EMA 3:00pm - 4:00pm EMA EMA 5:00pm - 6:00pm EMA EMA 7:00pm - 8:00pm EMA EMA 9:00pm - 9:30pm (Mothers Only) EMA EMA Note: Mothers received prompts between 0:00-0:29. Children received prompts between 00:30- 00:59. Mothers and children did not receive prompts during school time on weekdays. Measures Self-Reported Eating and Activity Participants were prompted with an item querying food consumption and activity in the past two hours at each EMA prompt. Figure 1 shows a sample screenshot of the EMA item and the options presented to the participant. The occurrence of eating was coded as a categorical variable with the following four selections: unhealthy foods (e.g., chips or fries), healthy foods (i.e., fruits and vegetables), both unhealthy and healthy foods, and no food selected. In addition, a separate eating variable was coded as the count of any eating items selected at the prompt. Self-reported (i.e., subjective) activity was obtained from the same EMA item querying mothers and children. Responses were coded as a categorical variable with four selections: screen behaviors (i.e., TV, videos, or video games), active (i.e., exercise or sport) behaviors, both screen and active behaviors, and no selected behavior type. The reference group for both categorical variables was the no selection condition. 24 Figure 1. Sample screenshot of eating and activity item for the MATCH study Physical Activity Objectively-recorded physical activity was obtained as activity counts at 30-second epochs from Actigraph accelerometers (GT3x, Firmware v1.5.0) and was aggregated at 15, 30, and 60 minute windows prior to each scheduled EMA prompt. To create time windows, minutes of sedentary, light activity, and moderate-vigorous physical activity time were calculated using standard age-adjusted cut points (Freedson et al., 1998). Non-wear was calculated as sixty minutes of consecutive zero counts (Troiano et al., 2008). A custom script looped through each time-stamped EMA prompt to aggregate and collapse data (i.e., non-wear, sedentary time, light activity, and MVPA) per minute based on each of the three (i.e., 15, 30, 60) windows. Aggregated windows containing 100% non-wear time were coded as missing to limit falsely reporting non-wear as sedentary time. Anthropometric and Demographic Measures Participant height and weight was measured in duplicate using a Tanita scale and BMI was calculated using age and sex-adjusted BMI percentiles for children and standard formulas for adults (kg/m 2 ) (Vidmar, Carlin, Hesketh, & 25 Cole, 2004). BMI categories (normal, overweight, obese) were used for data analysis and coded with normal as the reference group. Mothers reported household income, own ethnicity, child's ethnicity, own age, and own gender. Children reported their own age and gender. Ethnicity was coded as a dichotomous variable (1=Hispanic, 0=Not Hispanic), socioeconomic status was coded as a categorical variable based on household income quartiles (reference=25th percentile), and gender was coded dichotomously (1=Female, 0=Male). Temporal Measures Each EMA prompt was time-stamped and recoded for analysis as the day in the study (1-8), day of the week, and time of day. Measurement period (i.e., wave) was screened as a potential covariate. Day of the week was coded dichotomously as weekend and weekday (reference =weekday) and time of day was coded as morning (7AM-12PM), afternoon (12PM-5PM), and evening (5PM-10PM) (reference=evening). Compliance Measures Compliance was defined as a participant having answered an EMA prompt and completing at least one item in the survey. Compliance rates were defined as the proportion of prompts that were answered with at least one item completed divided by the number of actual prompts received. Compliance recovery rates were defined as the proportion of prompts that were answered with at least one item completed divided by the number of prompts an individual should have received. Compliance recovery rates were used to assess the extent to which data was missing due to technical issues. EMA completion was defined as having answered and fully completed an EMA prompt and EMA completion rates were defined as the proportion of prompts fully completed to the number answered. Compliance was coded dichotomously with non-compliance as the reference group. 26 Data Analysis Multilevel logistic regressions was used to test the associations between energy intake and expenditure and EMA prompt compliance proposed in this study using melogit in Stata 14.2. Each research question was analyzed as a two-level model, and the first prompt of each day was excluded from models with time-lagged EMA reported predictors, as the prior prompt was no longer a proximal predictor of compliance. All models were screened for normality and covariance structures. Variables were transformed to satisfy assumptions and an exchangeable covariance structure was assumed unless information criteria indicated otherwise (e.g., first- order autoregressive). Given the dyadic nature of the study, each hypothesis was tested separately in adults and children to minimize inflation of level-two sample size (i.e., Type I error) and reduce overgeneralization of estimates (i.e., Type II error). Predictor variables were interpreted as level-one (within-subject) effects and level-two (between-subject) effects by disaggregating variance of each predictor into two variables: a grand-mean (or proportion) centered person-level variable (BS) and person-centered prompt level variable (WS) (Curran & Bauer, 2011). Research Question 1. To test the associations among continuous measures of activity (i.e., accelerometry) and EMA measures (i.e., activity surveys) of activity with EMA prompt compliance, 10 independent multilevel logistic regressions were run separately in children and mothers. Each model tested EMA prompt compliance as the outcome and adhered to the following model conditional on the subject-specific random intercept (U j ): !(# $% = 1 ) = *+,(- .. + 0 % ) 1 + *+,(- .. + 0 % ) where Y ij is the compliance at time i in participant j with a binomial (Bernoulli) distribution, U j represents a participant's random intercept (i.e., deviation from the cumulative log-odds), and γ 00 27 represents the cumulative participant-specific natural log of the odds of complying to a prompt. The likelihood of any given participant with U j = 0 of complying to a prompt was given by the following formula: *+,- .. The lowest level residual variance in all logistic regressions is 1 2 3 , resulting in variance and coefficient rescaling and the inability to compare regression coefficients or variance components across models (Hox, Moerbeek, & van de Schoot, 2010). Therefore, models were individually screened for variance-covariance structure using Akaike’s/Bayesian Information Criterion (AIC/BIC) without fitting an unconditional model and a priori person-level and temporal covariates were added as confounders (Schwarz, 1978). Covariance structures were screened, including exchangeable/compound symmetry (i.e., all random effects have a common pairwise covariance in addition to a single variance parameter), identity/variance components (i.e., all random effects have a covariance of 0 in addition to a single variance parameter), and first-order autoregressive (i.e., all variance parameters are identical, but correlations are influenced exponentially by the closest measurement). Person-level covariates included demographics that may be associated with compliance or were known to be associated with physical activity, including age, gender, BMI category, socioeconomic status (SES), and race/ethnicity (Sallis, Prochaska, & Taylor, 2000). The likelihood of complying to an EMA prompt is thought to change over the course of a study, differ between weekends and weekdays, and prior literature shows improved evening compliance to EMA prompts (Courvoisier et al., 2012). Therefore, temporal covariates included measurement period (WAVE), day in the study (DIS), day of the week (DOW), and time of day (TOD) of the prompt. 28 Accelerometer-derived activity variables, specifically sedentary time and light activity time, are highly collinear and could not be examined simultaneously. Hence, the following between-subject (BS) and within-subject (WS) predictors were tested separately in each model: 1) sedentary time, 15 minutes before a prompt 2) light activity, 15 minutes before a prompt 3) MVPA, 15 minutes before a prompt 4) sedentary time, 30 minutes before a prompt 5) light activity, 30 minutes before a prompt 6) MVPA, 30 minutes before a prompt 7) sedentary time, 60 minutes before a prompt 8) light activity, 60 minutes before a prompt 9) MVPA, 60 minutes before a prompt 10) EMA reported activities at the prior prompt. Self-reported active behavior was represented by three dummy-coded variables (physical activity, sedentary behavior, and both selections) with a reference group of no selection. Dummy-coded variables were not represented with their own coefficients to allow the model to generalize to continuous predictors. Multilevel logistic regressions may be written in agreement with multilevel linear regression theory to improve readability (Hedeker, 2008): 456 , $% 1−, $% = - .. +- 8. !9 $% +- .8 !9 % +- .: 9;< % + - .3 ;<=><? % +- .@ ABC % +- .D E<E % +- .F ?9G< % +- :. >CE $% +- 3. >HI $% +- @. JH> $% + - D. I9K< $% + 0 % where PA represents each WS physical activity predictor tested, !9 represents each BS physical activity predictor tested, and each - represents the regression coefficient of its respective covariate. The likelihood of complying to an EMA prompt at i+1 by an individual j with a given physical activity at prompt i after adjusting for all covariates is given by the following equation: WS: *+,- 8. !9 $% BS: *+,- .8 !9 % 29 Research Question 2. To test the associations between EMA measures of energy intake (e.g., reported healthy eating) and EMA prompt compliance, two independent time-lagged multilevel logistic regressions were run separately in mothers and children. Similar to the first research question, compliance at prompt i was used to as the outcome with a binomial (Bernoulli) distribution and a model conditional on the subject-specific random intercept (U j ). Models were screened for variance-covariance structures using AIC/BIC criteria and adjust for covariates a priori. Each model was adjusted for age, gender, BMI category, socioeconomic status (SES), race/ethnicity, day in the study (DIS), day of the week (DOW), time of day (TOD), and measurement period (WAVE) of the prompt. The model was time-lagged so the following EMA-reported predictors at prompt i-1 will be used to predict compliance at prompt i: 1) healthy or unhealthy eating behavior 2) number of foods consumed. Eating behavior was represented by three dummy-coded variables (healthy, unhealthy, and both selections) with a reference category of “None.” 456 , $% 1−, $% = - .. +- 8. <A $L8% +- .8 <A % + - .: 9;< % +- .3 ;<=><? % +- .@ ABC % +- .D E<E % +- .F ?9G< % +- :. >CE $% +- 3. >HI $% +- @. JH> $% +- D. I9K< $% + 0 % where <A $L8% represents the eating behavior of each participant, j, at prompt i-1 and <A % represents the mean or proportion of each eating behavior for participant j. The likelihood of complying to an EMA prompt i by an individual j with eating behavior at prompt i-1 after adjusting for all covariates was given by the following equation: WS: *+,(- 8. <A $L8% ) BS: *+,(- .8 <A % ) 30 Research Question 3. To test whether adults and children differ in their association between energy intake and expenditure behaviors and EMA prompt compliance, the analyses from the first and second research questions were repeated with additional terms added to test multilevel interactions of dyadic status (i.e. mother or child). Instead of testing each subgroup separately, data from mothers and children was combined and a new term (MOTHER j ) was added and coded dichotomously (reference=child). To test group differences of the BS effect and the WS effect, each energy balance behavior predictor (MA % and HB ij ) was multiplied by dyadic status and entered into the model. 456 N OP 8LN OP = - .. +- 8. MA $% +- .8 MA % +- .: 9;< % +- .3 ;<=><? % +- .@ ABC % + - .D E<E % +- .F ?9G< % +- .Q BHJM<? % +- .R BHJM<? % ∗MA % +- :. >CE $% +- 3. >HI $% + - @. JH> $% +- D. BHJM<? % ∗MA $% +- F. I9K< $% +0 % where - .R represents the regression coefficient for the differences in the BS effect of energy balance behaviors (HB) on EMA prompt compliance between mothers and children and - D. represents the regression coefficient for the differences between mother and child in the WS effect of each predictor on EMA prompt compliance. Therefore, as compared to her child, the likelihood of a mother j with a given predictor complying to an EMA prompt i after adjusting for all covariates was given by the following equation: WS: *+,- D. BHJM<? % ∗MA $% BS: *+,- .R BHJM<? % ∗MA % Power Analysis A power analysis using G*Power 3.1 was conducted to determine if the proposed analysis was feasible given the sample size. The Type-I and 1- β error rates were set at 0.05 and 0.80 in all models, respectively. The ICC (ρ) for all variables was originally estimated at 0.20 to 31 represent the variance within individuals for compliance as an outcome; the true intra-class correlation for compliance was 0.27 and 0.24 in mothers and children, respectively . The probability of compliance was assumed to be 0.80 when all covariates and the predictor are one standard deviation greater than the mean. The probability of compliance was assumed to be 0.75 when all covariates and the predictor are at their mean. This resulted in an effective detectable effect size of 0.05 and an odds ratio of 1.33. Multilevel models are more efficient at detecting level-1 effects, given a design effect of D=1- ρ (Snijders, 2005). Therefore, it is only necessary to detect adjusted sample size for level-2 effects. Covariates are expected to have a moderate to high correlation with the within-subjects predictor (R 2 = 0.3). Given between and within-subject disaggregation of effects, a continuous between-subject predictor was assumed with a population SD and mean at 1 and 0, respectively. The unadjusted minimum level-1 (i.e., prompt-level) sample size given these parameters was 740. Figure 2 shows the power curve demonstrating power as a function of sample size given a decrease and an increase in the detectable effect size. The design effect for the within-subject is given by the equation D=1+(m-1)*ρ, where m is the estimated number of available within-group observations calculated by multiplying expected compliance by the number of EMA prompts (n=26 for mothers, n=21 for children, n=47 combined) (Snijders, 2005). The power necessary to detect an effect on compliance is given by multiplying the design effect by the unadjusted estimate (6 for mothers, 5 for children, and 10.2 for combined), resulting in a minimum of 4,440, 3,700, and 8,140 observations across participants necessary to detect an effect. Assuming that on average 200 dyads comply to 75% of prompts, a total of 5,200, 4,200, and 9,400 observations will be available for mothers, children, and the combined sample, respectively at each measurement period (i.e., a grand total of 10,400, 32 8,400, and 18,800; respectively). Therefore, there was enough power to detect the desired effects within this sample. Figure 2. Power Curve for Research Questions 1-3 Results Descriptive Statistics Children were between 8-12 years old with approximately half of children identifying as Female and reporting Hispanic ethnicity. Mothers were 26-57 years old with half reporting annual household income less than $75,000 and Hispanic ethnicity. More than half of children and two-thirds of mothers were overweight or obese. Table 2 provides detailed person-level demographic characteristics and prompt-level EMA descriptive statistics. Approximately half of all EMA prompts occurred on weekends and in the evening. Participants answered 79% of EMA prompts at the baseline measurement period and 77% of prompts at the second measurement period (p<0.001, z=-4.72). There were no significant differences in EMA prompt compliance between mothers and children (p>0.05, z=1.87), and there was no significant difference in EMA prompt compliance between mothers and children across measurement periods (p>0.05, z=-0.56). Participants fully completed 94.96% of all 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 0 500 1000 1500 2000 Total sample size = 1.43333 = 1.33333 = 1.23333 Odds ratio z tests - Logistic regression Tail(s) = Two. Pr(Y=1|X=1) H0 = 0.75. R2 other X = 0.3. X distribution = Normal. X parm μ = 0. X parm ( = 1. ) err prob = 0.05 Power (1-β err prob) 33 prompts answered; participants fully completed more prompts (98%) at the second measurement period than at the baseline measurement period (93%)(p<0.001,z=11.85). EMA completion rates for mothers and children did not differ significantly at the baseline measurement period, but children (99%) fully completed more surveys at the second measurement period than their mothers (97%)(p<0.001,z=-6.25). In aggregate, mothers and children reported predominantly TV, videos, or video games, eating fruits or vegetables, or doing no listed health behaviors. However, children were significantly more likely to report TV, videos, or video games (p<0.001,z=-11.59), exercise or sports (p<0.001,z=-13.47), eating chips or fries (p<0.001,z=-5.23), eating pastries or sweets (p<0.01,z=-3.11), and drinking soda or energy drinks (p<0.05,z=-2.02) than mothers. Mothers were more likely to report doing no listed health behavior than children (p<0.001,z=9.36). There were no differences in EMA-reported health behaviors across measurement period. Participants were predominantly sedentary at all aggregate accelerometer windows before each EMA prompt. Mothers were significantly more sedentary (p’s<0.001; z’s=13.87, 15.17, and 15.85 for 15, 30, and 60 minute windows, respectively) and engaged in significantly less light activity (p’s<0.001; z’s=-9.35, -9.47, and -9.24 for 15, 30, and 60 minute windows, respectively) than children. Mothers recorded less MVPA time (p’s<0.001; z’s=-14.69, -15.86, and -16.08 for 15, 30, and 60 minute windows, respectively) than children. Overall, participants were significantly less active (i.e., MVPA) at the second measurement period compared to the baseline measurement period (p’s<0.001; z’s=-3.45, -3.49, and -3.80 for 15, 30, and 60 minute windows, respectively). 34 Table 2. Demographic Characteristics at Baseline and EMA Descriptive Statistics N (Mean) % (SD) N (Mean) % (SD) N (Mean) % (SD) 40.93 6.16 9.61 0.92 25.43 16.30 0 0.0 95 48.7 95 24.1 200 100.0 100 51.3 300 76.0 103 51.5 91 46.7 194 49.1 97 48.5 104 53.3 201 50.9 55 27.6 52 26.8 107 27.2 58 29.2 59 30.4 117 29.8 49 24.6 47 24.2 96 24.4 37 18.6 36 18.6 73 18.6 66 33.9 117 62.6 183 47.9 64 32.8 39 20.9 103 27.0 65 33.3 31 16.6 96 25.1 N % N % N % 2185 20.5 2004 23.4 4189 21.8 8495 79.5 6572 76.6 15067 78.3 5569 52.1 4199 49.0 9768 50.7 5111 47.9 4377 51.0 9488 49.3 5824 54.5 3937 45.9 9761 50.7 3058 28.6 2898 33.8 5956 30.9 1798 16.8 1741 20.3 3539 18.4 1704 20.7 2883 45.4 4587 31.4 830 10.1 2119 33.4 2949 20.2 458 5.6 690 10.9 1148 7.9 751 9.1 872 13.7 1623 11.1 420 5.1 422 6.6 842 5.8 2288 27.8 1720 27.1 4008 27.5 498 6.0 533 8.4 1031 7.1 3466 42.1 1679 26.4 5145 35.3 Mean SD Mean SD Mean SD 9.96 3.93 8.23 4.20 9.21 4.14 19.69 7.17 16.08 7.52 18.12 7.54 38.30 13.52 30.94 13.91 35.10 14.17 4.39 3.44 5.39 3.36 4.82 3.44 8.64 6.02 10.56 5.86 9.47 6.03 16.83 10.55 20.53 10.68 18.44 10.77 0.41 1.32 1.08 2.09 0.70 1.73 0.81 2.24 2.19 3.75 1.41 3.07 1.61 3.76 4.32 6.56 2.79 5.34 Hispanic Sedentary Time (60 min.) Light Time (15 min.) Light Time (30 min.) Light Time (60 min.) Eaten Fruits or Vegetables Drank Soda or Energy Drinks None of These Things Sedentary Time (15 min.) Accelerometer Measures* Health-Related Behaviors Gender Male Female Ethnicity Non-Hispanic Compliance Did Not Comply Household Income <$34,999 >$35,000-$74,999 >$75,000-$114,999 >$115,000 BMI Category Afternoon Morning Eaten Fast Food Eaten Pastries or Sweets Eaten Chips or Fries Exercise or Sports TV, Videos, or Video Games MVPA Time (15 min.) MVPA Time (30 min.) *Accelerometer measures are aggregated to minutes before an EMA prompt. MVPA Time (60 min.) Total (N=395) Sedentary Time (30 min.) Demographic Characteristics Mothers (N=200) Children (N=195) Age Complied Day of Week Week Day Weekend Day Time of Day Evening Normal Overweight Obese Ecological Momentary Assessment Prompts 35 Physical Activity and Compliance Tables 3, 4, and 5 shows detailed results of the multiple multilevel logistic regressions examining EMA prompt compliance as a function sedentary time, light activity time, and MVPA time at 15, 30, and 60 minutes before each EMA prompt for mothers and children (i.e., accelerometer-measured component of Research Question 1). After adjusting for covariates, mothers were significantly more likely to comply to an EMA prompt when they were more sedentary than usual (i.e., WS effect) 15, 30, or 60 minutes before the prompt (p’s<0.01). On the other hand, mothers were significantly less likely to comply to an EMA prompt when they engaged in either light activity or MVPA greater than usual (i.e., WS effect) 15, 30, or 60 minutes before an EMA prompt (p’s<0.05). However, mothers with greater-than-average aggregate (i.e., BS effect) recorded light activity 60 minutes before an EMA prompt were more likely to comply to any given EMA prompt compared to mothers with average aggregate light activity (p<0.01). Children were significantly less likely to comply to an EMA prompt when they were engaged in MVPA or light activity greater than their own mean (i.e., WS effect) 15, 30, or 60 minutes before an EMA prompt after adjusting for all covariates (p’s<0.01). Moreover, compared to children with average aggregate MVPA time, children who engaged in greater-than- average overall (i.e., BS effect) MVPA 15, 30, or 60 minutes before an EMA prompt were less likely to comply to any given EMA prompt (p’s<0.05). Children were significantly more likely to comply to an EMA prompt when they were more sedentary than usual (i.e., WS effect) 15 or 30 minutes before an EMA prompt (p’s<0.01). Similarly, children with greater-than-average mean (i.e., BS effect) sedentary time 30 and 60 minutes before an EMA prompt were more likely 36 to comply to any given EMA prompt than children with average aggregate sedentary time (p’s<0.05). A subsequent set of multilevel logistic regressions revealed no cross-level interactions between mothers and children for MVPA, light activity, and sedentary time on compliance at 15, 30, and 60 minute windows before an EMA prompt (p’s>0.05)(i.e., Research Question 3). 37 Table 3. EMA compliance as a function of sedentary time 15, 30, and 60 minutes before a prompt in mothers and children Mothers Children (1) (2) (3) (4) (5) (6) 15min. 30min. 60min. 15min. 30min. 60min. O.R. 95%C.I. O.R. 95%C.I. O.R. 95%C.I. O.R. 95%C.I. O.R. 95%C.I. O.R. 95%C.I. Within-SubjectsEffect 1.03 ⇤⇤⇤ [1.01,1.04] 1.01 ⇤⇤ [1.01,1.02] 1.01 ⇤⇤ [1.00,1.01] 1.04 ⇤⇤⇤ [1.02,1.05] 1.01 ⇤⇤ [1.00,1.02] 1.00 [1.00,1.01] Between-SubjectsEffect 0.98 [0.85,1.13] 1.00 [0.93,1.07] 1.02 [0.98,1.05] 1.09 [0.97,1.23] 1.07 ⇤ [1.00,1.14] 1.04 ⇤ [1.01,1.08] Age 0.99 [0.96,1.02] 0.99 [0.96,1.02] 0.99 [0.96,1.02] 0.93 [0.78,1.10] 0.91 [0.76,1.08] 0.90 [0.76,1.07] BodyMassIndex(ref =Normal) Overweight 0.77 [0.52,1.15] 0.77 [0.51,1.14] 0.78 [0.52,1.17] 0.84 [0.56,1.24] 0.85 [0.57,1.25] 0.85 [0.58,1.26] Obese 1.16 [0.76,1.76] 1.15 [0.75,1.75] 1.19 [0.78,1.82] 0.89 [0.57,1.39] 0.90 [0.58,1.40] 0.88 [0.57,1.36] Ethnicity(ref =Non Hispanic) Hispanic 0.79 [0.54,1.14] 0.78 [0.54,1.14] 0.79 [0.54,1.15] 0.84 [0.60,1.17] 0.87 [0.63,1.21] 0.89 [0.64,1.23] AnnualHouseholdIncome(ref = <$34,999) $35,000-74,999 1.53 [0.97,2.39] 1.51 [0.97,2.37] 1.43 [0.91,2.25] 1.62 ⇤ [1.06,2.46] 1.62 ⇤ [1.07,2.46] 1.58 ⇤ [1.04,2.38] $75,000-114,999 1.52 [0.95,2.42] 1.49 [0.94,2.38] 1.43 [0.89,2.28] 1.22 [0.79,1.90] 1.21 [0.79,1.88] 1.24 [0.80,1.91] >$115,000 1.65 [0.98,2.78] 1.62 [0.96,2.73] 1.60 [0.95,2.71] 1.18 [0.71,1.95] 1.20 [0.72,1.97] 1.20 [0.73,1.97] DayofWeek(ref =Weekday) WeekendDay 1.27 ⇤⇤⇤ [1.11,1.46] 1.30 ⇤⇤⇤ [1.13,1.49] 1.26 ⇤⇤⇤ [1.10,1.44] 1.30 ⇤⇤⇤ [1.12,1.51] 1.29 ⇤⇤⇤ [1.11,1.50] 1.28 ⇤⇤⇤ [1.11,1.49] DayintheStudy 1.08 ⇤⇤⇤ [1.04,1.12] 1.08 ⇤⇤⇤ [1.05,1.12] 1.08 ⇤⇤⇤ [1.04,1.12] 0.99 [0.95,1.03] 0.98 [0.95,1.02] 0.98 [0.95,1.02] TimeofDay(ref =Evening) Afternoon 0.71 ⇤⇤⇤ [0.62,0.81] 0.71 ⇤⇤⇤ [0.62,0.81] 0.70 ⇤⇤⇤ [0.61,0.80] 0.84 ⇤ [0.72,0.97] 0.83 ⇤ [0.72,0.96] 0.82 ⇤⇤ [0.71,0.94] Morning 0.79 ⇤ [0.64,0.99] 0.79 ⇤ [0.63,0.98] 0.80 ⇤ [0.65,0.99] 1.12 [0.89,1.41] 1.14 [0.91,1.43] 1.12 [0.90,1.40] Waveinstudy 0.83 ⇤⇤ [0.73,0.94] 0.82 ⇤⇤ [0.72,0.93] 0.80 ⇤⇤⇤ [0.71,0.91] 0.85 ⇤ [0.73,0.97] 0.85 ⇤ [0.74,0.98] 0.85 ⇤ [0.74,0.98] Gender(ref =Male) Female 1.17 [0.85,1.60] 1.16 [0.85,1.58] 1.18 [0.87,1.61] Constant 4.37 ⇤⇤⇤ [2.59,7.37] 4.47 ⇤⇤⇤ [2.65,7.55] 4.70 ⇤⇤⇤ [2.77,7.96] 4.61 ⇤⇤⇤ [2.72,7.81] 4.58 ⇤⇤⇤ [2.71,7.72] 4.55 ⇤⇤⇤ [2.72,7.63] Between-SubjectsVariance 1.06 ⇤⇤⇤ [0.77,1.35] 1.07 ⇤⇤⇤ [0.77,1.36] 1.08 ⇤⇤⇤ [0.79,1.38] 0.88 ⇤⇤⇤ [0.62,1.13] 0.86 ⇤⇤⇤ [0.61,1.11] 0.85 ⇤⇤⇤ [0.61,1.10] N 7931 8031 8219 6069 6168 6343 Exponentiatedcoefficients;95%confidenceintervalsinbrackets Referencegroup: DidnotcomplytoEMAprompt ⇤ p <0.05, ⇤⇤ p <0.01, ⇤⇤⇤ p <0.001 1 38 Table 4. EMA compliance as a function of light activity time 15, 30, and 60 minutes before a prompt in mothers and children Mothers Children (1) (2) (3) (4) (5) (6) 15min. 30min. 60min. 15min. 30min. 60min. O.R. 95%C.I. O.R. 95%C.I. O.R. 95%C.I. O.R. 95%C.I. O.R. 95%C.I. O.R. 95%C.I. Within-SubjectsEffect 0.98 ⇤ [0.96,1.00] 0.98 ⇤⇤ [0.97,0.99] 0.99 ⇤ [0.99,1.00] 0.96 ⇤⇤⇤ [0.94,0.98] 0.98 ⇤⇤⇤ [0.97,0.99] 0.99 ⇤⇤ [0.98,1.00] Between-SubjectsEffect 1.16 [0.99,1.36] 1.10 ⇤ [1.01,1.19] 1.07 ⇤⇤ [1.02,1.11] 0.99 [0.85,1.15] 0.99 [0.91,1.08] 1.00 [0.96,1.04] Age 0.99 [0.97,1.02] 0.99 [0.97,1.02] 1.00 [0.97,1.02] 0.95 [0.80,1.13] 0.94 [0.79,1.12] 0.95 [0.79,1.13] BodyMassIndex(ref =Normal) Overweight 0.79 [0.53,1.17] 0.79 [0.53,1.17] 0.81 [0.55,1.20] 0.84 [0.56,1.25] 0.84 [0.56,1.26] 0.84 [0.56,1.25] Obese 1.17 [0.77,1.78] 1.16 [0.77,1.76] 1.20 [0.79,1.82] 0.92 [0.59,1.44] 0.93 [0.60,1.46] 0.91 [0.59,1.42] Ethnicity(ref =Non Hispanic) Hispanic 0.78 [0.54,1.13] 0.77 [0.53,1.11] 0.79 [0.55,1.14] 0.84 [0.60,1.17] 0.85 [0.61,1.18] 0.84 [0.60,1.18] AnnualHouseholdIncome(ref = <$34,999) $35,000-74,999 1.62 ⇤ [1.04,2.52] 1.64 ⇤ [1.06,2.55] 1.66 ⇤ [1.07,2.57] 1.62 ⇤ [1.06,2.47] 1.62 ⇤ [1.06,2.47] 1.56 ⇤ [1.02,2.38] $75,000-114,999 1.54 [0.97,2.44] 1.53 [0.96,2.42] 1.50 [0.95,2.37] 1.21 [0.78,1.88] 1.20 [0.77,1.87] 1.19 [0.77,1.86] >$115,000 1.65 [0.98,2.77] 1.62 [0.97,2.72] 1.64 [0.98,2.75] 1.20 [0.72,1.99] 1.20 [0.72,1.99] 1.20 [0.72,1.99] DayofWeek(ref =Weekday) WeekendDay 1.27 ⇤⇤⇤ [1.10,1.46] 1.30 ⇤⇤⇤ [1.13,1.49] 1.26 ⇤⇤ [1.10,1.44] 1.31 ⇤⇤⇤ [1.12,1.52] 1.29 ⇤⇤⇤ [1.11,1.50] 1.28 ⇤⇤ [1.10,1.48] DayintheStudy 1.08 ⇤⇤⇤ [1.04,1.12] 1.08 ⇤⇤⇤ [1.04,1.12] 1.08 ⇤⇤⇤ [1.04,1.12] 0.99 [0.95,1.03] 0.98 [0.94,1.02] 0.98 [0.95,1.02] TimeofDay(ref =Evening) Afternoon 0.71 ⇤⇤⇤ [0.62,0.81] 0.71 ⇤⇤⇤ [0.62,0.81] 0.70 ⇤⇤⇤ [0.61,0.80] 0.85 ⇤ [0.73,0.98] 0.84 ⇤ [0.73,0.97] 0.83 ⇤ [0.72,0.96] Morning 0.78 ⇤ [0.62,0.97] 0.77 ⇤ [0.62,0.95] 0.76 ⇤ [0.62,0.94] 1.10 [0.88,1.39] 1.11 [0.88,1.39] 1.08 [0.86,1.34] Waveinstudy 0.82 ⇤⇤ [0.72,0.94] 0.81 ⇤⇤ [0.72,0.93] 0.80 ⇤⇤⇤ [0.70,0.91] 0.85 ⇤ [0.74,0.98] 0.85 ⇤ [0.74,0.98] 0.85 ⇤ [0.74,0.97] Gender(ref =Male) Female 1.19 [0.86,1.63] 1.18 [0.86,1.62] 1.21 [0.88,1.65] Constant 4.29 ⇤⇤⇤ [2.56,7.20] 4.40 ⇤⇤⇤ [2.63,7.37] 4.49 ⇤⇤⇤ [2.69,7.50] 4.55 ⇤⇤⇤ [2.67,7.74] 4.65 ⇤⇤⇤ [2.73,7.92] 4.73 ⇤⇤⇤ [2.79,8.04] Between-SubjectsVariance 1.04 ⇤⇤⇤ [0.75,1.33] 1.03 ⇤⇤⇤ [0.75,1.32] 1.03 ⇤⇤⇤ [0.75,1.32] 0.90 ⇤⇤⇤ [0.64,1.15] 0.89 ⇤⇤⇤ [0.64,1.15] 0.89 ⇤⇤⇤ [0.64,1.15] N 7931 8031 8219 6069 6168 6343 Exponentiatedcoefficients;95%confidenceintervalsinbrackets Referencegroup: DidnotcomplytoEMAprompt ⇤ p <0.05, ⇤⇤ p <0.01, ⇤⇤⇤ p <0.001 2 39 Table 5. EMA compliance as a function of MVPA time 15, 30, and 60 minutes before a prompt in mothers and children Mothers Children (1) (2) (3) (4) (5) (6) 15min. 30min. 60min. 15min. 30min. 60min. O.R. 95%C.I. O.R. 95%C.I. O.R. 95%C.I. O.R. 95%C.I. O.R. 95%C.I. O.R. 95%C.I. Within-SubjectsEffect 0.94 ⇤⇤ [0.90,0.98] 0.96 ⇤⇤ [0.94,0.99] 0.98 ⇤ [0.96,1.00] 0.93 ⇤⇤⇤ [0.91,0.96] 0.96 ⇤⇤⇤ [0.95,0.98] 0.98 ⇤⇤⇤ [0.97,0.99] Between-SubjectsEffect 1.37 [0.79,2.38] 1.22 [0.90,1.65] 1.11 [0.95,1.30] 0.74 ⇤ [0.56,0.98] 0.82 ⇤ [0.71,0.96] 0.89 ⇤⇤ [0.82,0.97] Age 0.99 [0.97,1.02] 1.00 [0.97,1.02] 1.00 [0.97,1.03] 0.92 [0.77,1.09] 0.89 [0.75,1.06] 0.89 [0.75,1.05] BodyMassIndex(ref =Normal) Overweight 0.76 [0.51,1.13] 0.76 [0.51,1.14] 0.80 [0.54,1.20] 0.77 [0.51,1.15] 0.76 [0.51,1.13] 0.77 [0.52,1.14] Obese 1.17 [0.77,1.78] 1.17 [0.77,1.78] 1.23 [0.81,1.88] 0.84 [0.53,1.31] 0.83 [0.53,1.30] 0.79 [0.51,1.24] Ethnicity(ref =Non Hispanic) Hispanic 0.79 [0.54,1.14] 0.79 [0.54,1.15] 0.81 [0.55,1.17] 0.82 [0.59,1.14] 0.83 [0.59,1.15] 0.84 [0.61,1.16] AnnualHouseholdIncome(ref = <$34,999) $35,000-74,999 1.56 ⇤ [1.01,2.43] 1.58 ⇤ [1.02,2.46] 1.58 ⇤ [1.01,2.46] 1.63 ⇤ [1.07,2.47] 1.67 ⇤ [1.10,2.53] 1.62 ⇤ [1.07,2.45] $75,000-114,999 1.53 [0.97,2.44] 1.53 [0.96,2.43] 1.49 [0.94,2.38] 1.23 [0.79,1.91] 1.22 [0.79,1.88] 1.23 [0.80,1.89] >$115,000 1.65 [0.98,2.77] 1.62 [0.96,2.72] 1.62 [0.96,2.73] 1.24 [0.75,2.05] 1.25 [0.76,2.06] 1.28 [0.78,2.10] DayofWeek(ref =Weekday) WeekendDay 1.26 ⇤⇤ [1.09,1.44] 1.28 ⇤⇤⇤ [1.11,1.47] 1.24 ⇤⇤ [1.08,1.42] 1.29 ⇤⇤⇤ [1.11,1.50] 1.28 ⇤⇤ [1.10,1.49] 1.27 ⇤⇤ [1.09,1.47] DayintheStudy 1.08 ⇤⇤⇤ [1.04,1.12] 1.08 ⇤⇤⇤ [1.05,1.12] 1.08 ⇤⇤⇤ [1.04,1.12] 0.98 [0.95,1.02] 0.98 [0.94,1.02] 0.98 [0.94,1.02] TimeofDay(ref =Evening) Afternoon 0.71 ⇤⇤⇤ [0.62,0.81] 0.71 ⇤⇤⇤ [0.62,0.81] 0.70 ⇤⇤⇤ [0.61,0.80] 0.83 ⇤ [0.72,0.96] 0.83 ⇤⇤ [0.72,0.95] 0.82 ⇤⇤ [0.71,0.94] Morning 0.78 ⇤ [0.63,0.97] 0.77 ⇤ [0.62,0.96] 0.78 ⇤ [0.63,0.96] 1.10 [0.88,1.39] 1.10 [0.88,1.39] 1.07 [0.86,1.34] Waveinstudy 0.82 ⇤⇤ [0.72,0.94] 0.81 ⇤⇤ [0.72,0.93] 0.80 ⇤⇤⇤ [0.70,0.90] 0.84 ⇤ [0.73,0.97] 0.84 ⇤ [0.73,0.97] 0.84 ⇤ [0.73,0.96] Gender(ref =Male) Female 1.07 [0.77,1.49] 1.06 [0.77,1.46] 1.04 [0.75,1.44] Constant 4.38 ⇤⇤⇤ [2.61,7.37] 4.41 ⇤⇤⇤ [2.62,7.42] 4.48 ⇤⇤⇤ [2.66,7.56] 5.12 ⇤⇤⇤ [3.01,8.73] 5.23 ⇤⇤⇤ [3.08,8.86] 5.32 ⇤⇤⇤ [3.15,8.97] Between-SubjectsVariance 1.05 ⇤⇤⇤ [0.76,1.34] 1.06 ⇤⇤⇤ [0.77,1.35] 1.08 ⇤⇤⇤ [0.78,1.37] 0.87 ⇤⇤⇤ [0.62,1.12] 0.86 ⇤⇤⇤ [0.61,1.11] 0.85 ⇤⇤⇤ [0.61,1.10] N 7931 8031 8219 6069 6168 6343 Exponentiatedcoefficients;95%confidenceintervalsinbrackets Referencegroup: DidnotcomplytoEMAprompt ⇤ p <0.05, ⇤⇤ p <0.01, ⇤⇤⇤ p <0.001 3 40 Self-Reported Behavior and Compliance Table 6 presents the results of multilevel logistic regressions predicting compliance as a function of self-reported activity behaviors in mothers and children (i.e., EMA-measured component of Research Question 1). Compared to reporting no activity, both mothers and children were more likely to comply to the concurrent prompt if they reported a sedentary screen behavior at the prior EMA prompt (i.e., WS effect) after adjusting for all covariates (p’s<0.05). Additionally, children who reported a greater-than-average proportion (i.e., BS effect) of sedentary screen behavior were more likely to comply to any given EMA prompt than children who reported an average proportion of sedentary screen behavior (p<0.05). Children were more likely to comply to an EMA prompt if they reported sports and exercise at the previous prompt (i.e., WS effect) compared to reporting no activity (p<0.01). A subsequent multilevel logistic regression revealed no cross-level interactions between mothers and children for activity on compliance (i.e., Research Question 3)(p’s>0.05). Table 7 shows complete output of four multilevel logistic regressions predicting compliance as a function of self-reported eating behaviors in mothers and children (i.e., Research Question 2). Mothers were more likely to comply to the concurrent EMA prompt when reporting eating unhealthy food at the previous prompt (i.e., WS effect) compared to reporting no eating behavior (p<0.01). On the other hand, children were more likely to comply to an EMA prompt if they reported eating healthy food at the prior prompt (i.e., WS effect) compared to no eating behavior (p<0.01). Mothers reporting an overall greater-than-average number (i.e., BS effect) of food items were more likely to comply to EMA prompts than mothers reporting an overall average number of food items (p<0.05). Subsequent multilevel logistic regressions revealed a 41 cross-level interaction between mothers and children for the average number of reported food items (i.e., BS effect) on compliance (i.e., Research Question 3) (p<0.05). 42 Table 6. EMA compliance as a function of self-reported activity at the prior prompt in mothers and children (1) (2) Mothers Children O.R. 95%C.I. O.R. 95%C.I. EMA-ReportedActivityType(ref =None) SedentaryScreenBehavior(BS) 1.25 [0.49,3.21] 2.95 ⇤ [1.27,6.81] SedentaryScreenBehavior(WS) 1.30 ⇤ [1.01,1.68] 1.42 ⇤⇤ [1.09,1.85] SportsandExercise(BS) 3.36 [0.92,12.30] 3.10 ⇤ [1.10,8.75] SportsandExercise(WS) 1.11 [0.81,1.51] 1.50 ⇤⇤ [1.10,2.03] ScreenBehavior&Sports/Exercise(BS) 0.17 [0.01,3.09] 1.08 [0.44,2.68] ScreenBehavior&Sports/Exercise(WS) 1.36 [0.73,2.52] 1.35 [0.97,1.87] Age 0.99 [0.97,1.02] 0.94 [0.80,1.10] BodyMassIndex(ref =Normal) Overweight 0.78 [0.53,1.13] 1.11 [0.76,1.63] Obese 1.14 [0.77,1.69] 1.04 [0.68,1.59] Ethnicity(ref =Non Hispanic) Hispanic 0.73 [0.51,1.03] 0.92 [0.67,1.28] AnnualHouseholdIncome(ref = <$34,999) $35,000-74,999 1.24 [0.83,1.87] 1.14 [0.76,1.71] $75,000-114,999 1.18 [0.76,1.82] 1.03 [0.67,1.59] >$115,000 1.26 [0.77,2.06] 0.86 [0.53,1.41] DayofWeek(ref =Weekday) WeekendDay 0.96 [0.76,1.23] 1.44 ⇤⇤ [1.12,1.86] DayintheStudy 1.03 [0.97,1.10] 1.06 [0.99,1.13] TimeofDay(ref =Evening) Afternoon 0.73 ⇤ [0.55,0.96] 0.77 [0.58,1.03] Morning 0.87 [0.65,1.16] 0.92 [0.68,1.25] Waveinstudy 0.79 ⇤ [0.66,0.96] 0.78 ⇤ [0.65,0.94] Gender(ref =Male) Female 1.07 [0.79,1.44] Constant 9.58 ⇤⇤⇤ [5.25,17.47] 5.59 ⇤⇤⇤ [3.07,10.19] Between-SubjectsVariance 0.65 ⇤⇤⇤ [0.40,0.89] 0.59 ⇤⇤⇤ [0.36,0.82] N 4215 3834 Exponentiatedcoefficients;95%confidenceintervalsinbrackets Referencegroup: DidnotcomplytoEMAprompt ⇤ p <0.05, ⇤⇤ p <0.01, ⇤⇤⇤ p <0.001 4 43 Table 7. EMA compliance as a function of self-reported eating at the prior prompt in mothers and children Mothers Children (1) (2) (3) (4) Discrete Count Discrete Count O.R. 95%C.I. O.R. 95%C.I. O.R. 95%C.I. O.R. 95%C.I. EMA-ReportedFoodConsumption(ref =None) UnhealthyFood(BS) 1.10 [0.31,3.89] 1.58 [0.74,3.36] UnhealthyFood(WS) 1.54 ⇤⇤ [1.19,1.99] 1.37 [1.00,1.90] HealthyFood(BS) 2.39 [0.87,6.57] 1.31 [0.56,3.07] HealthyFood(WS) 1.00 [0.80,1.26] 1.57 ⇤⇤ [1.14,2.17] UnhealthyandHealthyFood(BS) 13.98 ⇤ [1.41,138.82] 1.86 [0.69,5.03] UnhealthyandHealthyFood(WS) 1.25 [0.86,1.84] 1.38 [0.95,2.00] #ofFoodCategoriesReported(BS) 2.42 ⇤ [1.21,4.86] 1.22 [0.87,1.71] #ofFoodCategoriesReported(WS) 1.09 [0.96,1.24] 1.00 [0.90,1.11] Age 0.99 [0.96,1.02] 0.99 [0.97,1.02] 0.96 [0.81,1.13] 0.95 [0.81,1.10] BodyMassIndex(ref =Normal) Overweight 0.65 ⇤ [0.45,0.94] 0.68 ⇤ [0.48,0.98] 1.04 [0.71,1.52] 1.02 [0.71,1.45] Obese 1.11 [0.75,1.65] 1.09 [0.75,1.58] 1.12 [0.72,1.74] 1.02 [0.68,1.52] Ethnicity(ref =Non Hispanic) Hispanic 0.70 ⇤ [0.49,0.98] 0.72 [0.52,1.01] 0.88 [0.64,1.22] 0.91 [0.67,1.22] AnnualHouseholdIncome(ref = <$34,999) $35,000-74,999 1.22 [0.82,1.82] 1.28 [0.87,1.89] 1.16 [0.76,1.75] 1.20 [0.82,1.76] $75,000-114,999 1.08 [0.71,1.65] 1.31 [0.87,1.98] 0.88 [0.57,1.35] 0.99 [0.67,1.48] >$115,000 1.15 [0.70,1.88] 1.30 [0.81,2.08] 0.81 [0.49,1.34] 0.89 [0.56,1.42] DayofWeek(ref =Weekday) WeekendDay 0.92 [0.75,1.14] 0.93 [0.77,1.14] 1.40 ⇤ [1.05,1.86] 1.30 ⇤ [1.03,1.63] DayintheStudy 1.03 [0.97,1.09] 1.03 [0.98,1.08] 1.04 [0.97,1.11] 1.04 [0.98,1.10] TimeofDay(ref =Evening) Afternoon 0.97 [0.74,1.26] 0.85 [0.66,1.08] 0.73 ⇤ [0.53,0.99] 0.78 [0.60,1.02] Morning 0.94 [0.71,1.25] 0.87 [0.67,1.14] 1.00 [0.70,1.43] 0.93 [0.70,1.24] Waveinstudy 0.83 ⇤ [0.70,0.99] 0.87 [0.74,1.02] 0.76 ⇤ [0.62,0.94] 0.80 ⇤ [0.67,0.95] Gender(ref =Male) Female 1.00 [0.73,1.35] 1.04 [0.78,1.38] Constant 10.47 ⇤⇤⇤ [5.96,18.37] 9.04 ⇤⇤⇤ [5.27,15.50] 6.40 ⇤⇤⇤ [3.37,12.17] 6.12 ⇤⇤⇤ [3.51,10.67] Between-SubjectsVariance 0.68 ⇤⇤⇤ [0.43,0.92] 0.69 ⇤⇤⇤ [0.46,0.92] 0.50 ⇤⇤⇤ [0.26,0.75] 0.55 ⇤⇤⇤ [0.35,0.76] N 5206 6029 2953 4348 Exponentiatedcoefficients;95%confidenceintervalsinbrackets Referencegroup: DidnotcomplytoEMAprompt UnhealthyFoods: ChipsorFries,PastriesorSweets,FastFood,SodaorEnergyDrinks HealthyFoods: FruitsorVegetables ⇤ p <0.05, ⇤⇤ p <0.01, ⇤⇤⇤ p <0.001 5 44 Discussion The study used a multi-method real-time data capture protocol to investigate how objectively measured and self-reported health-related behaviors affect compliance to ecological momentary assessment in mothers and children. Systematic missingness due to non-compliance is prevalent in intensive longitudinal research, and may result in biased data that threaten the validity of results. By using accelerometers as a passive measure, the study was able to examine effects of physical activity on EMA compliance at various levels of prompt proximity without response biases inherent in self-report. However, self-reported physical activity can provide context to behavior, and EMA is frequently used to measure dietary intake. Therefore, the study also examined the association of these self-reported health behaviors with EMA compliance using time-lagged self-reports. The intensive longitudinal nature of the study further allowed for disaggregation of variance to determine if an effect was due to between-person differences or within-person differences. The study is one of the first to examine self-reported and passively measured behavioral predictors of compliance to EMA surveys simultaneously in adults and children. Physical Activity and Compliance As hypothesized, engagement in activity classified as non-sedentary (i.e., MVPA or light) just before an EMA prompt was highly predictive of prompt non-compliance and conversely, sedentary time before an EMA prompt was predictive of prompt compliance in both mothers and children. The findings mirror those of a similar study in adults that found higher levels of MVPA in the 15 minutes preceding and following an unanswered EMA prompt compared to an answered EMA prompt (Dunton, Liao, et al., 2012). On the other hand, the aforementioned study found no association between EMA compliance and sedentary time, nor was light activity 45 examined as a predictor of compliance. With respect to MVPA, participants may have lacked either the ability to suspend participation in an activity or concentrate on exercise while listening for EMA auditory signals thus making it difficult for them to respond. Alternatively, participants may have deliberately set aside devices when they anticipated participation in physical activity. Children (but not mothers) who were more active, on average, may have been preoccupied with after-school physical activities and were subsequently less likely to comply to EMA prompts. Additionally, organized team sports may prohibit the use of smartphones, thus children may not have had the opportunity to answer EMA prompts even if an auditory signal was acknowledged. An inverse association between compliance and light activity in children suggests non- compliance to EMA protocols may generalize beyond exercise-induced inattentiveness. Instead, common activities such as household chores, childcare, and occupational tasks classified as light activity in the Compendium of Physical Activities also result in decreased likelihood of EMA prompt compliance (Ainsworth et al., 2000). Participants may have been unwilling or unable to interrupt their workflow, or may have set aside devices to focus on the task at hand. Interestingly, mothers who engaged in more frequent light activity, on average, were more likely to comply to EMA prompts. Although the exact nature of this relationship is not certain, a recent study indicates that busy adults may, in fact, be more motivated to complete tasks and likely to complete them in a timely manner (Wilcox, Laran, Stephen, & Zubcsek, 2016). Prompt compliance was higher after sedentary behaviors because participants who were sedentary in the moments before a prompt were likely able to attend to their devices or hear auditory alerts. Moreover, individuals who report more smartphone use are more likely to substitute physical activity in favor of sedentary behaviors that are conducive to smartphone use (Lepp, Barkley, Sanders, Rebold, & Gates, 2013). This suggests that sedentary participants were 46 more likely to be operating their mobile devices while a survey was triggered than active participants. Active use of a smartphone requires reduced effort to initiate and complete surveys, and attentiveness was likely enhanced due to additional stimuli (i.e., pop-up notification). Cross- sectional studies further corroborate findings showing that children with greater-than-average levels of sedentary time were more likely to comply to an EMA prompt (Vandelanotte, Sugiyama, Gardiner, & Owen, 2009). Given the strong positive correlation between sedentary behavior and cell phone use, these children likely also had supplemental familiarity with mobile devices. Lastly, it is unclear why sedentary time 60 minutes prior to an EMA prompt was not associated with EMA prompt compliance in children. The association between sedentary time and compliance may be weaker in children in general and a distal aggregate measure of activity before a prompt may have attenuated this effect. Self-Reported Behavior and Compliance The study examined EMA-reported eating behaviors as an exploratory aim and thus hypotheses were limited. Whereas mothers were more likely to comply to an EMA prompt after reporting eating unhealthy food, children were more likely to comply to an EMA prompt after reporting eating healthy food. Results among children are corroborated by dietary recall literature, but explanatory mechanisms for mothers are unclear. Social desirability (i.e., the tendency to report outcomes that are perceived as acceptable or positive) has been shown to influence dietary reports of children, especially in the context of parental supervision and may therefore partially explain to the relationship between reports of healthy food and EMA compliance (Börnhorst et al., 2013). On the other hand, contrary to the association between EMA compliance and reports of eating unhealthy food, studies among women show that participants are more likely to underreport caloric intake in general, with a dietary reporting bias 47 approximately twice as large as men (Hebert, Clemow, Pbert, Ockene, & Ockene, 1995; Hebert et al., 1997). Based on this literature, it is also unclear why mothers who overreported eating items were more likely to comply to EMA prompts. The literature, however, shows that adults are inclined to engage in a good-subject role (e.g., selecting items they think a researcher is interested in) when they are aware of outcomes of interest in an experiment, which could serve as a potential explanation for the greater reporting of eating items in highly compliant mothers (Nichols & Maner, 2008). Although underreporting of food recall among children is noted across the literature, there was no association between food categories reported by children and EMA compliance (Livingstone, Robson, & Wallace, 2004). Consequently, an explanatory mechanism of the interaction between mothers and children for the association between count of food items recorded and EMA compliance is unknown. Contrary to hypothesized effects, self-reporting of either sedentary screen behavior or sports and exercise was associated with increased compliance in children at the subsequent prompt. Likewise, children reporting a greater proportion of sedentary screen behaviors or sports and exercise compared to none were more likely to comply with EMA prompts. The findings suggest that children with tendencies to report time-use behaviors will be more likely to comply to EMA prompts, although whether this reporting is a function of a valid response (i.e., children who engaged in more screen time or exercise also completed more EMA prompts), overreporting (i.e., children who selected more items also completed more EMA prompts), or some other response bias (e.g., social desirability) is unclear. Psychometric literature partially supports findings from this study, as self-reported physical activity is positively associated with social desirability (Jago, Baranowski, Baranowski, Cullen, & Thompson, 2007). Yet, the same study found an inverse association between social desirability and reports of sedentary behavior that 48 contradict the relationship between EMA compliance and reports of sedentary screen behavior found here. On the other hand, mothers were only more likely to comply to an EMA prompt if they had previously reported sedentary behavior. Reasons for this relationship are unclear, although non-significant trends in the between-subject effect seem to suggest that this finding is a result of sedentary mothers being more likely to have opportunities to engage with their smartphones. However, this assumes that mothers were sedentary two hours later and the response itself was valid. This trend in mothers, while non-significant, is corroborated by cross-sectional research in adults that reveals overreporting of activity and overestimation of caloric expenditure as a result of social desirability (Adams et al., 2005). Limitations The study was primarily limited by missing data from EMA prompts and accelerometer non-wear. Each analysis used a time-lagged EMA predictor that relied on the presence of data at the prior prompt, such that missing data due to EMA non-compliance was shifted by one time- point. Subsequent attempts to analyze this new missing data results in recursion of the same limitation. This challenge is especially problematic given the importance of multiple missed EMA prompts and the inability to detect the temporal length (i.e., strength) of an effect of a predictor on compliance. However, given the large prompt-level sample size, it is unlikely that the significance of an association was masked due to a missing temporal length parameter. Available data was screened and found that this higher order effect occurs in approximately 13% (n=2,498) of the sample. This type of missing data may be handled with multilevel multiple imputation, but this is beyond the scope of the study. 49 Similarly, windows with accelerometer data were missing if they consisted entirely of non-wear. The literature indicates that conservative non-wear cut-points significantly decrease sedentary time, and therefore, the data may under-sample sedentary behavior. However, by sampling multiple time windows prior to EMA prompts, the study minimizes the likelihood of an entire window containing missing data. On the other hand, windows with partial non-wear are more likely to contain sedentary time, because all non-wear is classified as sedentary behavior at zero counts. The final dataset was screened for the percentage of incomplete accelerometer windows to determine the impact of this limitation. Mothers had approximately 5% of accelerometer windows with partial non-wear, while children had approximately 7% of accelerometer windows with partial non-wear. Therefore, it is unlikely that this limitation produced biased results, given the relative homogeneity of wear time before an EMA prompt. The study was also limited in design by the sampling schedule of EMA surveys. The within-subject effect was limited in interpretation as a proximal predictor of EMA compliance, because time-lagged predictors from EMA surveys were reported approximately two hours prior to a prompt. Activity and eating behaviors are unlikely to be sustained for long periods of time, therefore the likelihood of Type-II error increases for the within-subject effect of EMA-reported physical activity and eating. However, this study eliminated the first observation per day to reduce any error introduced by the previous day's observation. Additionally, alternative approaches such as three-level models or multiple imputation do not entirely address these limitations and introduce additional problems (e.g., limited level-two sample size, complex computing). EMA self-reported eating and activity behavior was limited to a single item and the list of potential activities was limited to only two options. Furthermore, the option "None of these" was 50 ambiguous and may refer to either activities or foods. Therefore, the reference group in all dichotomous EMA variables may mean the participant is engaging in other physical activity, sedentary behavior, or eating an unlisted food. However, ambiguous EMA data was validated post-hoc against accelerometry to determine the extent to which this limitation affects results. MVPA and sedentary time just prior to EMA-reported physical activity and sedentary prompts were significantly higher than other prompts, respectively (p’s<0.05). Lastly, the lack of male parents, exclusion of non-working mothers, and missed prompting during work and school hours limited the external validity (i.e., generalizability) of results in the study. Implications The study contributes methodological knowledge to the field of obesity-related studies that use real-time assessment protocols such as EMA. The study was one of the first studies to systematically examine (subjective and objective) physical activity and eating behaviors as predictors of compliance to EMA prompts in EMA studies measuring these behaviors. Specifically, significant prompt-level findings across all three classifications of activity may have broad implications for researchers interested in understanding physical activity in adults and children. Given typically low mean levels of MVPA across participants in EMA studies, the results suggest that researchers may choose to adopt context-sensitive protocols with aggressive auditory prompting schedules in order to record post-MVPA self-report surveys. Otherwise, studies risk obtaining EMA data with limited variance in activity levels (i.e., predominantly sedentary time). Particularly, the study uniquely implicates light activity as a predictor of EMA non-compliance. Researchers should exercise caution when interpreting results from EMA data time-linked to physical activity measures, as the protocol may substantially undersample the 51 majority of free-living situations (e.g., walking, work, chores), not just MVPA as previously thought. The study further contributes to the sparse EMA psychometric literature with findings indicating participants may choose to comply to a prompt differentially based on their prior response. Researchers may leverage these findings by implementing real-time data analysis to predict the likelihood that a participant will fail to comply in the future, especially if they fail to report food or activity behavior. Moreover, the findings imply that social desirability (among other psychometric issues) may directly impact EMA surveys. Hence, prompts following physical activity in children, for instance, may be oversampled, due to a child’s desire to report behaviors perceived as positive. Still, the findings solicit additional questions, such as the exact psychological mechanisms, and at a minimum, the findings warrant caution and extra research in EMA measures of activity and diet. Future Directions The findings from this study revealed that sedentary time is systematically oversampled in EMA studies, while MVPA and light activity time is undersampled. Further, the study found evidence for the existence of underlying response biases that systematically affect compliance. Future studies may seek to generalize findings to male adults, while also examining adolescents and the elderly. Additional research is needed to determine behavioral or psychological mechanisms responsible for poor compliance following reports of certain eating or active behaviors in adults and children. These findings may be used to guide researchers in developing alternate EMA protocols to improve compliance across groups. For instance, external accelerometers that communicate with phones could be used to delay or intentionally prompt participants at opportune moments. Hence, future research may choose to test the success of 52 context-sensitive protocol changes in reducing systematic non-compliance. Lastly, as EMA studies begin to integrate multiple imputation, results from this study may be used as predictors of missingness in data imputation. Multiple imputation depends on the assumption that data is unrelated to observed or unobserved variables (missing completely at random, MCAR), but as noted in other EMA studies, missingness (i.e., non-compliance) is often associated with observed variables (missing at random, MAR). Moreover, missing data is likely associated with some unobserved variable (missing not at random, MNAR). This study elucidates passively measured and self-reported behavioral outcomes that were previously MNAR and can guide researchers in controlling for them as covariates in MAR models, thereby reducing biases in estimates with imputed data (Little & Rubin, 1989). 53 CHAPTER 3: MODELING STUDY BURDEN AND PARTICIPANT FATIGUE IN ECOLOGICAL MOMENTARY PROTOCOLS WITH SUBJECTIVE SELF-REPORT AND OBJECTIVE SURVEY METADATA Abstract Purpose: Although ecological momentary assessment (EMA) can address limitations of typical cross-sectional research and longitudinal research with infrequent observations, the intensive nature of repeated measurements found in EMA protocols may create significant burden for participants. There is limited literature examining the reasons for and extent to which burden from EMA protocols leads to an increase in participant fatigue (e.g., repetitive responses, hastened completion of surveys) and subsequent failure to comply to EMA survey prompts. The purpose of this study was to explore the relationships of cross-sectional and EMA measured indicators of participant fatigue with EMA prompt compliance. Methods: Adolescents between the ages of 14 and 18 (N=44) were enrolled in a methodological study testing the feasibility of a mobile physical activity assessment tool using a cross-over design (i.e., with and without a mobile physical activity assessment tool). During the mobile physical activity assessment session, participants completed EMA prompts at two-hour intervals and after moments of inactivity or activity, while wearing accelerometers and completing an assisted end-of-day recall activity. Data from this component was used to obtain measures of participant fatigue by assessing end-of-study satisfaction measures, collecting real-time reports of phone usage, and computing survey metadata (i.e., survey completion time and survey item response variance). A series of multilevel regressions were used to examine temporal trends in measures of participant fatigue and the likelihood of compliance to an EMA prompt as a function of objective and subjective measures of participant fatigue. Results: End-of-study satisfaction measures, EMA 54 reported phone usage, and survey item response variance did not predict EMA compliance. Both EMA survey completion time and survey item response variance decreased across the measurement period. Survey completion time at the prior prompt was inversely associated with subsequent EMA compliance. Similarly, individuals who spent more time on average completing surveys were less likely to comply with any EMA prompts. Conclusions: Researchers seeking to improve EMA compliance by reducing burden on participants should utilize survey metadata, especially survey completion time, or implement real-time compliance improvement strategies among participants who are the most likely to provide valid responses. Given reductions in both variance and completion time throughout the measurement period, studies employing prolonged and sophisticated protocols should consider the possibility that compliance rates may not necessarily be valid indicators of data quality. Additional research is needed to develop real-time and end-of-study self-reported measures of study burden that can be used to predict EMA compliance. 55 Introduction Ecological momentary assessment (EMA) is a real-time sampling strategy used to capture time-varying processes in behavioral studies. EMA protocols are time-intensive and can prompt participants multiple times a day at regular schedules (i.e., signal-contingent prompts) or trigger surveys contingent on an event (e.g., following an activity bout). Hence, EMA minimizes the biases found in other types of study design and can supplement objective measurement devices such as accelerometers in the context of physical activity studies. Moreover, EMA is deployed in free-living situations, thereby limiting biases found in laboratory settings or in the presence of an experimenter (Shiffman et al., 2008). Factors Affecting Study Burden Despite advantages in time-intensive sampling, studies using EMA can cause participant fatigue to protocol, thereby introducing sampling bias and reduction in data validity. Participant fatigue may result in reduced compliance to EMA and is affected by burden on the participant resulting from the EMA protocol (i.e., study burden). There are at least five factors posited by Hufford to likely affect study burden and subsequently reduce compliance: 1) sampling density, 2) survey length, 3) measurement period length, 4) item complexity, and 5) user satisfaction (2007). However, the relationships between these variables and compliance is mixed at best. Sampling Density First, participants may be burdened by a protocol that prompts surveys frequently (Hufford, 2007). Sampling density (i.e., the number of survey prompts per day) varies across studies from as little as two prompts per day up to near-continuous prompting every 15 minutes. For instance, in a study by Gorely and colleagues designed to investigate time-use and physical activity among adolescents, participants were asked to complete 44 short paper-and- pencil surveys on three randomly assigned weekdays; participants also completed 68 paper and 56 pencil surveys on one randomly assigned weekend day during waking hours (2007). Each survey queried 15-minute activity recall and context. Approximately half of all surveys were completed, despite a lack of prompt reminders. However, nearly 40% of participants reported at least an hour-long delay in filling out each survey, indicating delayed response behaviors (i.e., "hoard and backfill") consistent with other paper-and-pencil diaries (Gorely et al., 2007; Shiffman et al., 2008). Still, sampling frequency is not consistently associated with compliance. In a study of obese women assessing dieting relapse and physical activity in the final week of an intervention, overall compliance was approximately 68% despite only participants receiving four prompts per day (Carels et al., 2004). In contrast, a study examining the relationship between tension headaches and physical activity in a primarily female sample reported overall compliance exceeding 95% during a week with four prompts per day (Kikuchi et al., 2007). Survey Length Next, the length of each individual survey is thought to affect the amount of effort required by the participant at each EMA prompt (Hufford, 2007). Survey design strategies such as branching and subsetting can be used to limit the number of questions a participant receives per survey while allowing for an increased scope of question topics. Item branching (i.e., skip logic) can reduce survey length by limiting supplementary questions (e.g., context) to activities of interest (Morren, van Dulmen, Ouwerkerk, & Bensing, 2009). Still, participants may learn that a particular selection of questions results in shortened surveys, thereby limiting the sample size of surveys containing an outcome of interest. Item subsetting, on the other hand, relies on the increase in within-subject sample size native to EMA, and presents items at set frequencies (e.g., 60%) for each survey (Christensen, Barrett, Bliss-Moreau, Lebo, & Kaschub, 2003). However, in larger models, this becomes problematic as data reduction is multiplicative, such that three variables prompted at 60% frequency result in only 22% available 57 data at best. The length of time it takes a participant to complete a survey is infrequently reported, although studies often exclude responses if participants do not respond in timely manner (e.g., 15 minutes) (Stone & Shiffman, 2002). Studies generally report completion times ranging between two and five minutes. However, it is unclear how survey length affects compliance as survey length across studies is relatively homogenous and compliance rates, if at all reported, do not vary consistently with changes in survey length (Bond et al., 2013; Dunton, Kawabata, et al., 2012; King et al., 2008; Mitchell et al., 2014). Measurement Period Duration Similar to the length of individual surveys, the duration of a measurement period within a study is directly related to attrition thought to be caused by increased burden (Hufford, 2007). The literature consistently uses week-long measurement periods as a standard for measurement of physical activity (Mata et al., 2012). However, other studies have reported success with protocols as long as ten weeks (Gauvin, Rejeski, & Norris, 1996; Kanning & Schlicht, 2010). For example, to assess a PDA-based physical activity intervention designed for sedentary, but healthy adults, participants were prompted twice daily over a period of eight weeks (King et al., 2008). The 36-item survey assessed physical activity, social and physical context, and other factors and was completed in 2-3 minutes. Participants complied with 68% of prompts on average; however, the study showed considerable attrition week-over-week. While overall compliance was moderate, decrease in compliance across the study period is especially problematic when results may attenuate the effect of an intervention. Item Complexity and User Satisfaction Finally, the complexity of individual items presented within each prompt and user satisfaction with the EMA protocol are thought to be indicators of participant burden (Hufford, 2007). Researchers can address these issues by improving item comprehension, and by enhancing the user experience of hardware and software. 58 However, these constructs have not been measured objectively in EMA studies, as they require access to metadata such as user engagement with the application and the time it takes to answer each individual item. Given these constraints, EMA studies have used protocol training to address item complexity (Hamar, Biddle, Soos, Takacs, & Huszar, 2010). Furthermore, studies receive generally positive feedback from post-protocol surveys assessing participant satisfaction with EMA (Cain et al., 2009; King et al., 2008). A study by Tsai et. al. examined the feasibility of a EMA-like caloric balance application in a small group of obese participants compared to a paper-based diary (2007). Compared to approximately 60% compliance in the paper diary, the application yielded nearly 100% compliance across 30 days. The study is unique in its use of a computing ubiquity questionnaire (e.g., security, interaction, etc.) in addition to a protocol satisfaction questionnaire (e.g., helpfulness, frequency of use, etc.) following the completion of the measurement period. However, there were no statistically significant differences between electronic and paper protocols, likely due to the small sample size (Tsai et al., 2007). Moreover, the brevity of EMA surveys, in general, has limited researchers from adding real-time feedback questionnaires. Gap in Current Knowledge While prior literature has conceptualized compliance as a proximal outcome to determine burden, it is evident that an intermediary variable may be affected by burden and in turn, may affect compliance (Figure 3). Figure 3. Basic Conceptual Model of Burden, Fatigue, and Compliance 59 As substantiated by mixed associations between compliance and the five aforementioned study design choices thought to improve study burden, there is a gap in the literature in terms of conceptualizing participant fatigue caused by EMA study protocols. Without a proper measure of fatigue, researchers may be designing protocols that adversely impact participants even if compliance is not reduced. For instance, in studies examining correlates of physical activity with psychosocial variables such as stress, the consequences of participant fatigue may result in observations that show biased temporal trends. Furthermore, there is a critical gap in knowledge on participant fatigue as a time-varying process. Given the loss of data in longer studies, participant fatigue likely increases over time and sampling may be differentially biased depending the length of a measurement period (King et al., 2008; Spook et al., 2013). Conceptualizing Participant Fatigue Three targets exist as possible time-varying methods to operationalize participant fatigue: 1) EMA items specifically querying study burden; 2) completion time, defined earlier as the length of time it takes to finish a survey; and 3) survey item response variance, defined as differences in the range of response options participants select within a survey. As noted in other studies, survey completion time may not have a linear relationship as an indicator of fatigue and may not predict compliance on its own. For instance, participants may finish surveys slower initially during a learning process, followed by gradual improvements in speed as they master the use of EMA application. Alternately, they may choose to comply with surveys, despite finding them bothersome, changing over time contingent on a participant’s threshold for tolerance of a protocol. However, results from these surveys may not be valid if participants try to complete the surveys as fast as possible with the least amount of effort. The best compromise of effort and time, therefore, would be to click the same option repeatedly until the survey is finished. Within- 60 survey item response variance would complement these findings, because it provides a general estimate of a participant's choice variety throughout the prompt. Therefore, individuals who finish surveys faster as a consequence of EMA mastery should not show changes in variance over time. In other words, even if true scores of an item change over time, there should not be a systematic decline in item response variance, given that item response variance is a between-item measure aggregated at the survey level. However, participants who experience fatigue or are bothered by prompts should show decreased variance due to repetition of choice (i.e., conscious selection of the same answer to minimize effort). As seen in Figure 4, participant fatigue is a dynamic process and therefore conceptualization of fatigue captures change over time, as opposed to a conceptualization of study burden that consists of one-time study design decisions. However, unlike sampling density, measurement protocol duration, survey length, and item complexity, a user’s satisfaction is not controlled by researchers at study design and is generally measured at the conclusion of a study. Figure 4. Complete Conceptual Model of Burden, Fatigue, and Compliance 61 Study Overview The proposed study addressed gaps in measuring momentary participant fatigue in studies that utilize EMA. EMA studies suggest that psychological, behavioral, and temporal factors influence EMA compliance, but less is known about study design choices that may affect overall compliance and satisfaction with the study (Courvoisier et al., 2012; Dunton, Liao, et al., 2012). Hufford suggests that sampling density, survey length, measurement period length, item complexity, and user satisfaction likely affect the burden a study places on participants, but studies show mixed effects on compliance (2007). However, EMA studies show decreased compliance over time, suggesting that study burden may cause participant fatigue that is associated with compliance (King et al., 2008). Moreover, there are no known studies measuring participant fatigue (directly or otherwise) from the burden of a study and its effect on compliance. Specific Aims and Hypotheses 4. To examine associations between end-of-study satisfaction measures (e.g., likelihood to repeat the study), proportion of time where the phone was intentionally not in possession (e.g., too burdensome to carry), and EMA compliance. The likelihood of complying to an EMA prompt session was inversely associated with the proportion of prompts where the phone was not in possession due to burden. Compliance was hypothesized to be positively associated with end-of-study satisfaction. 5. To observe how EMA completion times and response patterns (i.e., survey item response variance) change throughout the course of the assessment period. 62 If participants were not systematically fatigued over time, survey completion times were hypothesized to decrease over the course of the study due to learning, while response patterns were not hypothesized to show significant changes over the study period. 6. To test the main effects and interactions of EMA completion time and response patterns (i.e., survey item response variance) on EMA compliance. Survey completion time was expected to interact significantly with response patterns at the person-level when predicting prompt compliance. As a result of familiarity with EMA and not fatigue, prompt compliance was hypothesized to be greater than average if survey completion time was shorter than average, but variance was greater than average. Inversely as a result of fatigue or avoidance, prompt compliance was hypothesized to be lower than average if survey completion time was longer than average, but variance was lower than average Methods The study used EMA data from a 28-day protocol testing the feasibility of a supplementary end-of-day recall method to examine time-use in adolescents. Mobile TEENS was a quasi-experimental crossover study utilizing external accelerometry and an Android application. The application was capable of using data from onboard accelerometers to prompt participants contextually and at specified time intervals, as well as generating an accelerometry- guided end-of-day recall activity to assist participants in remembering activities throughout the day. Participants were also queried for application feedback following the study. The proposed study used the EMA and feedback data, in addition to anthropometric and demographic measures. A detailed description of the end-of-day recall activity is available elsewhere (Dunton et al., 2014). 63 Participants Participants in the Mobile TEENS study were recruited from a local high school (grades 9-12) in the Greater Los Angeles area. Individuals interested in the study were contacted and screened for eligibility and included an ethnically diverse sample of 14-18 year-old adolescents. Inclusion criteria for the study were: (1) being enrolled as a 9th-12th grade student at the recruitment site (2) having the ability to comprehend written English, (3) not being diagnosed with any medical issues that may limit physical activity (e.g., asthma), and (4) being in possession of an Android smartphone or any GSM-compatible smartphone with a mobile phone plan. 44 participants were enrolled in the study, of which 40 (91%) completed EMA data suitable for this study. Procedure Participants were assessed at the start of the study for demographic information and anthropometric measures. Adolescents with Android smartphones running a compatible operating system version were allowed to use their own device. Participants with a smartphone equipped with a GSM-capable radio were provided with Nexus 4 (LG) device, and these participants had their SIM card swapped to maintain cellular plans. Signal-contingent (i.e., random) and event-contingent (i.e., context-sensitive) EMA surveys were scheduled for a period of two weeks. Random prompts occurred at two-hour intervals from 3 PM to 9 PM on weekdays (three prompts per day) and 7 AM to 9 PM on weekends (seven prompts per day). Context- sensitive EMA (CS-EMA) prompts were prompted if the smartphone's accelerometer detected one of three scenarios: no data, a period of at least 10 minutes of missing activity followed by 1 minute of activity; no activity, a period of at least 60 minutes of low activity followed by 2 minutes of moderate activity; and activity, a period of at least 15 minutes of intense activity 64 followed by 10 minutes of low activity. Participants received up to three total audible pings per survey and prompts were separated by at least 30 minutes. Equipment was collected at the end of the study and participants provided feedback on their experience with the EMA portion of the study. Measures Compliance Compliance was coded as a binary variable (reference=non-compliant) at the prompt level and defined as having completed at least the main activity item within a prompt. Anthropometric and Demographic Measures Height and weight were measured in duplicate using a Tanita scale and BMI were calculated using age and sex-adjusted BMI z-scores for adolescents (Vidmar et al., 2004). Participants reported their own age and gender (dichotomous, reference=Male). Ethnicity was coded as a dichotomous variable (1=Hispanic, 0=Other). Temporal Measures Each EMA prompt was time-stamped and recoded for analysis as the day in the study (1-14), day of the week (weekend versus weekday; reference=weekday), and time of day (morning and afternoon versus evening; reference=evening). Survey Completion Time Survey completion time was calculated as the difference in time from the moment a prompt was presented to the time a participant finished the survey. Timestamps to differentiate prompt time and the time a participant started a survey were unavailable. To account for this, only prompts that were answered at the first reminder were included. Prompts were excluded from analysis if survey completion time exceeded 10 minutes (i.e., untimely response). Survey completion time was not adjusted for the number of items in a survey. 65 Survey Item Response Variance Survey item response variance was calculated by first coding all EMA items in numerical order based on presented position (See Table 8). Text entry and numerical entry questions were excluded from the analysis. The main activity item and up to four social context items were multiple choice and required special transformation. If a participant selected multiple options, the coding was summed and divided by the number of selected items. Conceptually, this provided the average position of a participant's selection. The survey item response variance was calculated as the sum-of-squares deviation across all single- choice and transformed multiple-choice items divided by one less than the number of items. Table 8. Survey Item Response Variance Items for Mobile TEENS Number of Items Question 7* What have you been DOING for the past hour? Choose all that apply. 6 While using technology (TV, phone), were you: Excluded Approximately how many minutes did you spend [ANSWER 1]? 5^ While [ANSWER 1], were you: 6 While going somewhere, were you: 6 How did you have the PHONE while [ANSWER 1]? 7 Please indicate your reason for not carrying your phone while [ANSWER 1]? 6^ What was the MAIN PURPOSE of [ANSWER 1]? 2 Was this eating or drinking a meal or a snack? 5^ How ENJOYABLE was [ANSWER 1]? 2^ Were you [ANSWER 1] because YOU want to do it? 2^ Were you [ANSWER 1] because YOUR PARENTS want you to do it? 2^ Were you [ANSWER 1] because YOUR FRIENDS want you to do it? 2^ Were you [ANSWER 1] because YOUR TEACHERS want you to do it? 2^ Were you [ANSWER 1] ALONE? 5^* While [ANSWER 1], were you with: Choose all that apply. 8 What type of sports or exercise activity? Excluded Approximately how many MINUTES did you spend participating in this sport or exercise activity? 5* Did the sport or exercise activity involve: Choose all that apply. 5 How much extra weight were you carrying during the sport or exercise activity? 4 Did the sport or exercise activity involve: 5 How much PAIN/SORENESS did you feel during the sport or exercise activity? 6 What was the MAIN PURPOSE of participating in the sport or exercise activity? 4 Where did you participate in the sport or exercise activity? 5 WHERE was this OTHER place? 2 Did you participate in the sport or exercise activity OUTDOORS? 5 How did you have the PHONE while participating in the sport or exercise activity? 66 7 Please indicate your reason for not carrying your phone while participating in the sport or exercise activity: 12 What was this other activity? Excluded Approximately how many minutes did you spend [ANSWER 4a]? 5 When [ANSWER 4a], were you: 6 How did you have the PHONE while [ANSWER 4a]? 7 Please indicate your reason for not carrying your phone while [ANSWER 4a]: Excluded Can you tell us what you were doing for the past 30 minutes that you answered as something else? Excluded Approximately how many minutes did you spend doing this activity: 5 For this activity, were you: 5 How did you have the PHONE while doing this activity? 7 Please indicate your reason for not carrying your phone while doing this activity: * Multiple choices ^Asked up to 4 times per survey EMA-Reported Phone Usage EMA prompts queried how the phone was carried (How did you have the PHONE while:) with the following available response options: On my belt; In my pocket; In my handbag/purse/backpack; Holding in my hand; Within reach, but not on me; and Not with me. If the participant selected Not with me, a follow-up question queried reasons for failing to carry the device (Please indicate your reason for not carrying your phone while:). The EMA phone usage items were grouped into a single variable with four items representing non- burdened non-wear (Forgot it; Battery died; Did not want to damage it; and Not allowed to carry it), other instances where the device was worn coded as wear, and the remaining selections representing burdened non-wear (Too bulky, Too uncomfortable, and Embarrassed to carry it). Instances where multiple activity selections received a phone usage follow-up question were excluded. The item was dummy-coded into two dichotomous variables, with wear as the reference group. Survey-Reported Satisfaction with EMA Protocol As part of an exit survey, participants completed 11 items assessing satisfaction with the mobile phone surveys (See Table 9). Each item was coded on a Likert-type four-point scale with higher numbers representing greater satisfaction. 67 Table 9. End-of-Study EMA Satisfaction Items Item Overall, how satisfied are you with the CS-EMA system? Overall, how easy/difficult was it to use the CS-EMA system? Overall, the CS- EMA system interrupted my daily activities.* Overall, using the CS-EMA system was an enjoyable experience. Overall, using the CS-EMA system required too much of my time.* Overall, the CS-EMA system was distracting during class.* Overall, the CS-EMA system distracted me from my homework.* Overall, the CS-EMA system distracted me from my household chores.* Overall, the CS-EMA system was uncomfortable to wear.* Overall, the CS-EMA system was embarrassing to wear.* I would be willing to participate in a study testing the CS-EMA system again. *Item was reverse-coded. Data Analysis Multilevel logistic regressions were used to test the associations between compliance and participant usage satisfaction surveys, survey completion time, and survey item response variance. Multilevel linear regressions were used to test temporal trends in survey completion time and survey item response variance across the study. All models were adjusted for age, gender, and ethnicity. Normality and covariance structure specification were screened separately for each model. Stata 14.1 was used to factor analyze data and test multilevel models. Research Question 4. To examine associations between end-of-study satisfaction measures (e.g., likelihood to repeat the study), proportion of time where the phone was intentionally not in possession (e.g., too burdensome to carry), and prompt compliance, a single multilevel logistic regression was used. First, the end-of-study satisfaction reports were reduced using an average score to a single item for use in the model. To accomplish this, the 11 items 68 were analyzed for common factor loadings using a principal axis factor (PAF) method (Ford, Maccallum, & Tait, 1986). Principal-component analysis (PCA) assume that factors are uncorrelated, while maximum likelihood (ML) factor analysis assumes a multivariate normal distribution among the items, even though satisfaction measures are likely skewed due to social desirability bias (i.e., participants are more likely to report a measure as favorable). Unlike alternate approaches, PAF does not make assumptions on distribution or factor correlation. Factor loadings were rotated obliquely (i.e., rotation with a correlated factor assumption), as orthogonal rotation assumes that factors are uncorrelated. The mean of all items that loaded onto a single factor with an Eigenvalue of at least 1 at a fair factor loading greater than 0.45 were used in the analysis as a person-level measure of satisfaction (SATISFY j )(Tabachnick & Fidell, 2007). Two dummy variables representing observations of device non-wear (burdened and non- burdened) were time-lagged to allow analysis of compliance at the subsequent prompt. Each variable was centered to person-level means at each observation (burdened non-wear, B_NONWEAR i-1j ; and non-burdened non-wear, NONWEAR i-1j ) to create within-subject (WS) terms. Between-subject (BS) terms were created by taking the person-level proportion of each variable and centering it to the proportion across all participants (!_#$#%&'( ) and #$#%&'( ) ). The model was screened for variance-covariance structures using AIC/BIC criteria and adjusted for age, gender, BMI z-score, race/ethnicity, day in the study (DIS), day of the week (DOW), and time of day (TOD) of the prompt. +,- . /) 1−. /) = 3 44 +3 64 !_#$#%&'( /76) +3 46 !_#$#%&'( ) + 3 84 #$#%&'( /76) +3 48 #$#%&'( ) +3 49 :';<:=> ) +3 4? '@& ) +3 4A @&#B&( ) +3 4C !D<E ) +3 4F ('G& ) +3 94 B<: /) +3 ?4 B$% /) +3 A4 ;$B /) +H ) 69 As compared to an individual with wear at the prior prompt, the likelihood of complying to an EMA prompt at i by an individual j reporting burdened non-wear or non-burdened non-wear at prompt i-1 after adjusting for all covariates was given by the following equation: WS: IJ. 3 64 !_#$#%&'( /76) and IJ. 3 84 #$#%&'( /76) BS: IJ. 3 46 !_#$#%&'( ) and IJ. 3 48 #$#%&'( ) After adjusting for all covariates, the association between end-of-study satisfaction of an individual j with compliance to an EMA prompt was: IJ. 3 49 :';<:=> ) Research Question 5. To observe how survey completion times and response patterns (i.e., survey item response variance) change throughout the course of the assessment prior to examining their association with EMA compliance, two sets of multilevel linear regressions were used to test the effects of temporal variables on each outcome (unadjusted and adjusted for the alternate outcome). An unconditional means model was first be fitted to assess intra-class correlation for each outcome (Y ij ): > /) = 3 44 +K ) +I /) where γ 00 represents the grand mean of the outcome variable, u j represents each participant's deviation from the grand mean, and e ij represents the error in the outcome for each person j at prompt i. Temporal trends across the study period (DIS), across the day (TOD), and between days (DOW) were tested as predictors, adjusting for age, gender, and BMI z-score. Auto-regressive residuals were specified post-hoc and a log-likelihood test was used to determine whether or not they improved model fit. > /) = 3 44 +3 64 B<: /) +3 84 B$% /) +3 94 ;$B /) +3 46 '@& ) +3 48 @&#B&( ) +3 49 !D<E ) +3 4? ('G& ) +K ) +I /) 70 where γ 01 represents the change in the outcome for each subsequent day in the study, γ 20 represents the change in the outcome on weekends as compared to weekdays, and γ 30 represents the change in the outcome throughout the day after adjusting for all covariates. Research Question 6. To test the main effects and interactions of survey completion time and response patterns (i.e., survey item response variance) on prompt compliance, a pair of multilevel logistic regressions were run using time-lagged predictors. Survey completion time (TIME i-1j ) and item response variance (VARY i-1j ) were time-lagged to the prior prompt and disaggregated in order to allow for analysis of BS and WS effects on compliance at the subsequent prompt. Each model was individually screened for autoregressive residuals and variance-covariance structures using AIC/BIC criteria. A priori covariates included age, gender, BMI z-score, race/ethnicity, day in the study (DIS), day of the week (DOW), and time of day (TOD) of the prompt. +,- . /) 1−. /) = 3 44 +3 64 L'(> /76) +3 46 L'(> ) + 3 84 ;<D& /76) +3 48 ;<D& ) +3 49 '@& ) +3 4? @&#B&( ) +3 4A !D<E ) +3 4C ('G& ) +3 94 B<: /) +3 ?4 B$% /) +3 A4 ;$B /) +H ) The main effects model was used to predict the likelihood of a participant complying to a prompt given their item response variance or completion time after adjusting for covariates: WS: IJ. 3 64 L'(> /76) and IJ. 3 84 ;<D& /76) BS: IJ. 3 46 L'(> ) and IJ. 3 48 ;<D& ) To examine completion time by item response variance interactions, two within-level terms and two-cross level terms were added to the model: 3 C4 L'(> /76) ∗;<D& /76) ,3 F4 L'(> ) ∗ 71 ;<D& /76) ,3 O4 L'(> /76) ∗;<D& ) , and 3 4F L'(> ) ∗;<D& ) . Significant interactions were examined and visualized by repeating the main effects model without the survey item response variance term for observations two standard deviations above and below mean survey item response variance. Power Analysis A power analysis using G*Power 3.1 was conducted to determine if the proposed analyses were feasible with Type-I and 1- β error set at 0.05 and 0.80. Separate analyses were conducted for multilevel linear and logistic regressions. The ICC (ρ) for all variables was estimated at 0.20 to account for clustering within individuals; the true ICC’s for compliance, survey completion time, and survey item response variance were 0.21, 0.07, and 0.09. For multilevel logistic regressions, the probability of compliance was assumed to be 0.85 when all covariates and the predictor are one standard deviation greater than the mean and 0.75 when all covariates and the predictor are at their mean (OR=1.89). Covariates were expected to have a moderate to high correlation with the within-subjects predictor (R 2 = 0.3) and the between- subject predictor parameters were set at 1 and 0 for population SD and mean, respectively. Figure 5 shows the power curve demonstrating power as a function of sample size given a decrease and an increase in the detectable effect size. 72 Figure 5. Power Curve for Research Question 4 & 6 The unadjusted minimum sample size given these parameters was 166 and the design effect was calculated based a cluster size of 44 (75% compliance, D=9.6). Therefore, 1,594 observations are necessary to detect level-2 effects in the model. Given an estimated 1,760 level-1 observations, there is enough power to detect level-2 effects. For multilevel linear regressions, an effect size of 0.10 was used with 7 total predictors in the model. Figure 6 shows the power curve demonstrating power as a function of sample size given a decrease and an increase in the detectable effect size. The unadjusted minimum sample size given these parameters was 151 and the minimum sample size after adjusting for clustering was 1,450. Therefore, there was enough power to detect level-2 predictors in multilevel linear regressions. 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 0 50 100 150 200 250 300 Total sample size = 1.98889 = 1.88889 = 1.78889 Odds ratio z tests - Logistic regression Tail(s) = Two. Pr(Y=1|X=1) H0 = 0.75. R2 other X = 0.3. X distribution = Normal. X parm μ = 0. X parm ( = 1. ) err prob = 0.05 Power (1-β err prob) 73 Figure 6. Power Curve for Research Question 5 Results Descriptive Statistics Detailed demographic and EMA descriptive statistics are available in Table 10. Participants with data available for analysis were predominantly Hispanic between 14 and 18 years old, with just over half of the sample classified as either overweight or obese. Out of approximately 80% of EMA prompts participants answered in the study (n=3,006), 82% (n=2,465) were completed at the first auditory signal (prompt) and completed within 7.2 seconds to 6.8 minutes. Participants reported having the phone on hand or in their pocket most often when carrying the device. Participants noted feeling uncomfortable carrying the phone or not wanting to damage the device as the most common reason for not carrying their smartphone, while no participants indicated they were embarrassed to carry the device. Table 11 details a complete report of the end-of-study surveys of satisfaction with EMA. Overall, the majority of participants reported being satisfied with the EMA component of the study, and found it enjoyable and easy to use; only one participant reported being unwilling to 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 0 100 200 300 400 Total sample size = 0.15 = 0.1 = 0.05 Effect size f² F tests - Linear multiple regression: Fixed model. R2 deviation from zero Number of predictors = 7. # err prob = 0.05 Power (1-β err prob) 74 repeat the study. Similarly, very few participants indicated that the EMA component of the study distracted them from coursework, chores, or homework. Most participants indicated that the phone was not uncomfortable or embarrassing to wear and agreed that the protocol did not take up too much time. Factor analysis of items listed in Table 11 for the construct representing end- of-study satisfaction with the EMA protocol yielded an eight item solution with a single factor at an Eigenvalue of 3.56. EMA ease of use, comfort, and embarrassment were excluded and the final solution yielded an average factor loading of 0.66 (range: 0.55-0.85). 75 Table 10. Demographic and Ecological Momentary Assessment Descriptive Statistics N (Mean) % (SD) 15.87 1.21 17 44.7 21 55.3 13 35.1 24 64.9 18 47.4 14 36.8 6 15.8 38 100.0 0 0.0 12 31.6 26 68.4 N (Mean) % (SD) 703 19.0 3006 81.0 2683 72.3 1026 27.7 2268 61.1 1441 38.9 536 21.5 95 3.8 810 32.5 164 6.6 11 0.4 880 35.3 32 9.4 94 27.5 0 0.0 28 8.2 47 13.7 29 8.5 112 32.8 0.87 0.79 4.99 3.35 iOS *Survey item response variance is the standardized variance in the position of selected responses for each individual survey. Obese Does Not Own Smartphone Participant Smartphone Ownership Type of Smartphone Owns Smartphone Phone Location Morning & Afteroon Evening Time of Day Too uncomfortable Too bulky Not allowed to carry it Forgot it Embarrassed to carry it Did not want to damage it Gender Age Demographic Characteristics Hispanic Non-Hispanic Ethnicity Female Male Ecological Momentary Assessment Prompts Android Overweight Normal BMI Category Weekend Day Week Day Day of Week Complied Did Not Comply Compliance Within reach, but not on me Battery died Reasons for Not Carrying Phone Survey Item Response Variance* Survey Completion Time (minutes) Holding in my hand In my handbag/purse/backpack In my pocket Not with me On my belt 76 Table 11. End-of-Study Ecological Momentary Assessment Satisfaction Descriptive Statistics N % 12 32.4 22 59.5 3 8.1 0 0.0 23 62.2 12 32.4 2 5.4 0 0.0 3 8.1 7 18.9 20 54.1 7 18.9 7 18.9 26 70.3 4 10.8 0 0.0 2 5.4 6 16.2 22 59.5 7 18.9 0 0.0 3 8.1 21 56.8 13 35.1 1 2.7 5 13.5 20 54.1 11 29.7 1 2.7 3 8.1 23 62.2 10 27.0 0 0.0 8 21.6 21 56.8 8 21.6 0 0.0 4 10.8 24 64.9 9 24.3 19 51.4 17 45.9 1 2.7 0 0.0 Strongly Disagree Disagree Agree Strongly Agree EMA Interrupted Daily Activities Very Difficult Somewhat Difficult Somewhat Easy Very Satisfied Satisfaction with EMA Very Easy Ease of Use of EMA Very Dissatisfied Dissatisfied Satisfied EMA Was Enjoyable Strongly Agree Agree Disagree Strongly Disagree EMA Required Too Much Time Strongly Disagree Disagree Neither Agree nor Disagree Agree Strongly Agree Agree Disagree Strongly Disagree Agree Disagree Strongly Disagree EMA Distracted From Homework Strongly Agree Agree Disagree Strongly Disagree Willing to Participate Again Strongly Agree Agree Disagree Strongly Disagree EMA Distracted From Chores EMA Distracted In Class Phone Was Embarrassing To Wear Strongly Disagree Disagree Agree Strongly Agree Strongly Disagree Disagree Agree Strongly Agree Phone Was Uncomfortable To Wear Strongly Agree 77 User Satisfaction and Compliance Table 12 provides the results from a multilevel logistic regression examining compliance as a function of EMA and end-of-study protocol satisfaction (i.e., Research Question 4). After adjusting all covariates and EMA reported location of device, end-of-study satisfaction with EMA did not predict compliance at any given EMA prompt. Additionally, burdened and not burdened EMA reported non-wear compared to EMA-reported wear at the prior prompt (i.e., WS effect) did not predict EMA compliance at the subsequent prompt. The average proportion of burdened and not burdened EMA reported non-wear (i.e., BS effect) to EMA-reported wear time did not predict EMA compliance at any given EMA prompt. 78 Table 12. Compliance as a function of EMA and end-of-study protocol satisfaction Investigating Survey Completion Time and Item Response Variance After adjusting for age, gender, ethnicity, BMI z-score, day in the study, day of week, and EMA reported wear, end-of-study satisfaction was positively associated (β=0.16, SE=0.08) with (z=2.18, p<0.05). Additionally, the proportion of prompts reported as non-burdened non- wear to wear (i.e., BS effect) was positively associated (β=1.10, SE=0.51) with EMA completion time (z=2.16, p<0.05) after adjusting for all covariates and end-of-study satisfaction. The proportion of prompts reported as burdened non-wear to wear (i.e., BS effect) and non-wear O.R. 95%C.I. End-of-StudySatisfactionwithEMA 0.81 [0.44,1.49] EMAReportedLocationofDevice(ref =Wear) Non-Wear,NotBurdened(WS) 1.44 [0.74,2.81] Non-Wear,NotBurdened(BS) 0.07 [0.00,2.40] Non-Wear,Burdened(WS) 1.02 [0.59,1.77] Non-Wear,Burdened(BS) 1.21 [0.04,39.34] Age 1.16 [0.92,1.46] Gender(ref =Male) Female 0.85 [0.50,1.43] BMIz-score 1.03 [0.76,1.40] TimeofDay(ref =Evening) Morning&Afternoon 0.78 [0.57,1.07] DayintheStudy 0.99 [0.96,1.03] Ethnicity(ref =Non Hispanic) Hispanic 2.98 ⇤⇤⇤ [1.64,5.43] DayofWeek(ref =Weekday) WeekendDay 1.25 [0.89,1.75] Constant 4.65 ⇤⇤⇤ [2.64,8.22] Variance(Randomeffects) 1.44 ⇤ [1.08,1.91] N 1925 Exponentiatedcoefficients;95%confidenceintervalsinbrackets Referencegroup: DidnotcomplytoEMAprompt ⇤ p<0.05, ⇤⇤ p<0.01, ⇤⇤⇤ p<0.001 1 79 (burdened and not burdened) reported at the prior EMA prompt (i.e., WS effect) were not associated with EMA completion time (p’s>0.05). The proportion of prompts reported as burdened non-wear to wear (i.e., BS effect) was significantly associated (β=4.74, SE=1.91) with EMA survey item response variance (z=2.48, p<0.05) after adjusting for all covariates and end-of-study satisfaction. EMA-reported non- burdened non-wear (β=0.96, SE=0.41; respectively) at the prior EMA prompt (i.e., WS effect) were significantly and marginally associated with subsequent EMA survey item response variance (z=2.34, p<0.10; respectively). The proportion of prompts reported as non-burdened non-wear to wear (i.e., BS effect), EMA reported burdened non-wear at the prior prompt (i.e., WS effect), and end-of-study satisfaction with EMA were not associated with EMA survey item response variance (p’s>0.05). Temporal trends for survey completion time and survey item response variance were explored using linear growth curve models (i.e., Research Question 5). Multilevel linear growth curve models indicated an approximate 2 second decrease in survey completion time for each subsequent day in the study, after adjusting for all covariates, including survey item response variance (see Table 13). There was a significant positive association between survey item response variance and survey completion time at the prompt level (i.e., WS effect). The model adjusting for survey item response variance found a 3.7% reduction in the day-to-day decrease in survey completion time compared to the unadjusted model. A multilevel linear growth curve model indicated reduced EMA survey item response variance for each subsequent day in the study, after adjusting for covariates (see Table 13). Moreover, after additionally adjusting for survey completion time, survey item response variance decreased 24% less per day in the study compared to the unadjusted model. 80 Table 13. Linear growth curve models of EMA survey completion time and survey item response variance Fatigue and Compliance Table 14 details a pair of multilevel logistic regressions examining compliance as a function of EMA survey completion time and survey item response variance (i.e., Research Question 6). After adjusting for all covariates and survey item response variance, participants with greater than usual EMA completion time at the prior prompt (i.e., WS effect) were less likely to complete a subsequent EMA prompt. Similarly, participants with greater-than-average mean EMA completion time (i.e., BS effect) were less likely to complete any given EMA prompt. Survey item response variance at the prior prompt (i.e., WS effect) and average survey item response variance (i.e., BS effect) did not predict EMA compliance. A second model revealed no cross-level interaction of survey item response variance and completion time on EMA compliance. Unadjusted Adjusted (1) (2) (3) (4) SurveyCompletionTime SurveyItemResponseVariance SurveyCompletionTime SurveyItemResponseVariance b S.E. b S.E. b S.E. b S.E. DayintheStudy 0.027 ⇤⇤⇤ (0.004) 0.050 ⇤⇤ (0.017) 0.026 ⇤⇤⇤ (0.004) 0.038 ⇤ (0.017) Age 0.017 (0.033) 0.023 (0.140) 0.019 (0.033) 0.037 (0.137) Gender(ref =Male) Female 0.008 (0.078) 0.894 ⇤⇤ (0.325) 0.043 (0.083) 0.886 ⇤⇤ (0.318) BMIz-score 0.011 (0.042) 0.255 (0.178) 0.004 (0.042) 0.243 (0.174) TimeofDay(ref =Evening) Morning&Afternoon 0.072 ⇤ (0.034) 0.495 ⇤⇤⇤ (0.146) 0.061 (0.034) 0.463 ⇤⇤ (0.146) Ethnicity(ref =Non Hispanic) Hispanic 0.025 (0.086) 0.136 (0.361) 0.032 (0.084) 0.161 (0.353) DayofWeek(ref =Weekday) WeekendDay 0.010 (0.037) 0.356 ⇤ (0.159) 0.002 (0.037) 0.351 ⇤ (0.158) SurveyItemResponseVariance(BS) 0.060 (0.039) SurveyItemResponseVariance(WS) 0.023 ⇤⇤⇤ (0.005) SurveyCompletionTime(BS) 0.947 (0.648) SurveyCompletionTime(WS) 0.432 ⇤⇤⇤ (0.090) Constant 1.078 ⇤⇤⇤ (0.085) 4.489 ⇤⇤⇤ (0.357) 1.107 ⇤⇤⇤ (0.085) 4.386 ⇤⇤⇤ (0.351) Variance(Randomeffects) 0.043 ⇤⇤⇤ (0.014) 0.754 (0.215) 0.041 ⇤⇤⇤ (0.013) 0.715 (0.203) Residual(Randomeffects) 0.572 ⇤⇤⇤ (0.017) 10.626 ⇤⇤⇤ (0.316) 0.566 ⇤⇤⇤ (0.017) 10.517 ⇤⇤⇤ (0.313) N 2293 2293 2293 2293 Standarderrorsinparentheses ⇤ p<0.05, ⇤⇤ p<0.01, ⇤⇤⇤ p<0.001 2 81 Table 14. Compliance as a function of EMA survey completion time and survey item response variance Discussion The study used a two-week EMA and accelerometer protocol as part of a larger study on physical activity measurement to explore how study burden (i.e., self-reported satisfaction with the study), self-reported participant fatigue (i.e., EMA-reported device wear and reasons for device non-wear), and passively measured participant fatigue (i.e., completion time and item response variance) could be used to predict compliance to EMA prompts. Although studies with EMA prompting have typically reported favorable satisfaction from participants, even in extended measurement periods, the validity of end-of-study satisfaction questionnaires as a predictor of compliance to and acceptability with a real-time data capture protocol is unknown. Moreover, end-of-study reports may be influenced by recall and response biases resulting in (1) (2) MainEffectsModel InteractionModel O.R. 95%C.I. O.R. 95%C.I. SurveyItemResponseVariance(BS) 1.09 [0.86,1.37] 1.11 [0.88,1.41] SurveyItemResponseVariance(WS) 1.01 [0.97,1.06] 1.00 [0.95,1.05] SurveyCompletionTime(BS) 0.30 ⇤⇤ [0.13,0.68] 0.34 ⇤ [0.15,0.79] SurveyCompletionTime(WS) 0.79 ⇤⇤ [0.67,0.94] 0.76 ⇤⇤ [0.64,0.90] Age 1.26 ⇤ [1.03,1.53] 1.20 [0.98,1.48] Gender(ref =Male) Female 0.68 [0.40,1.13] 0.62 [0.37,1.04] BMIz-score 0.99 [0.76,1.29] 0.98 [0.76,1.27] TimeofDay(ref =Evening) Morning&Afternoon 0.93 [0.66,1.31] 0.93 [0.66,1.31] DayintheStudy 1.01 [0.97,1.04] 1.01 [0.97,1.04] Ethnicity(ref =Non Hispanic) Hispanic 2.42 ⇤⇤⇤ [1.48,3.95] 2.10 ⇤⇤ [1.25,3.54] DayofWeek(ref =Weekday) WeekendDay 0.95 [0.68,1.35] 0.95 [0.67,1.34] SurveyCompletionTime(WS)*SurveyItemResponseVariance(BS)Interaction 1.12 [0.97,1.28] SurveyItemResponseVariance(BS)*SurveyCompletionTime(BS)Interaction 0.57 [0.25,1.33] SurveyItemResponseVariance(WS)*SurveyCompletionTime(BS)Interaction 1.04 [0.87,1.23] SurveyItemResponseVariance(WS)*SurveyCompletionTime(WS)Interaction 1.00 [0.97,1.04] Constant 6.31 ⇤⇤⇤ [3.66,10.87] 7.58 ⇤⇤⇤ [4.18,13.75] Variance(Randomeffects) 1.25 ⇤ [1.01,1.55] 1.22 [0.99,1.50] N 1932 1932 Exponentiatedcoefficients;95%confidenceintervalsinbrackets Referencegroup: DidnotcomplytoEMAprompt ⇤ p<0.05, ⇤⇤ p<0.01, ⇤⇤⇤ p<0.001 3 82 arbitrary ceiling effects on satisfaction reporting. By supplementing end-of-study satisfaction reports with real-time subjective measures, the study was able minimize response biases in order to test the association between end-of-study satisfaction and real-time wear-time questionnaires with EMA compliance. Further, modern EMA protocols on smartphones allow for real-time monitoring of response patterns, including the length of time it takes for a participant to finish a survey and the degree to which item selection varies within a survey. The study is one of the first to explore how these measures change throughout a protocol and to determine whether or not they are associated with EMA prompt compliance. User Satisfaction and Compliance Contrary to hypotheses, neither end-of-study satisfaction with EMA nor self-reported phone usage at the prior prompt were associated with compliance to EMA prompts. Even though end-of-study items loaded moderately on a single factor after reduction, there was a considerable ceiling effect across all EMA reported satisfaction items. The demand characteristics of the exit survey (e.g., researchers present, anticipated compensation, etc.) may have resulted in a desire to be a good participant (i.e., response bias)(Weber & Cook, 1972). Notably, participants also complied to approximately 80% of prompts overall, indicating little variance in both the predictor and the outcome. Individuals with low compliance may inconsistently report device non-wear, while those with high compliance may consistently report device non-wear (when applicable), resulting in a measure with poor reliability. Though within-subject effects indicate that trends, while non-significant, may agree with this explanation, the lack of any between- subject effects limit definitive conclusions. 83 Investigating Survey Completion Time and Item Response Variance Survey completion time decreased for each subsequent day in the study, indicating a possibility that individuals may have adapted to survey protocols throughout the measurement period. However, each individual survey was branched, suggesting that avoidance may explain this relationship beyond participants becoming accustomed to the EMA protocol over time. In other words, participants may have intentionally selected choices with the explicit knowledge that certain responses result in a reduced item load. Indeed, despite a theoretical maximum load of 38 items, item branching reduced EMA surveys to a mean of 5.5 items per survey, resulting in an average completion time less than one minute long. Survey item response variance decreased throughout the measurement period as well, indicating that participants were more likely to select repetitive responses. In tandem with a decreasing completion rate, repetition of item input was hypothesized to occur as a result of increasing participant fatigue caused by the length of a measurement period (Hufford, 2007). Contrary to this hypothesis, results indicated that adjusting for item response variance did not have an effect on the relationship between day in the study and survey completion time. On the other hand, adjusting for survey completion time decreased the association between day in the study and survey item response variance. Given that both completion time and item response variance are aggregate survey-level measures derived from item-level data, temporal precedence between completion time and item response variance is ambiguous and therefore prohibits mediational analyses (Chmura Kraemer, Kiernan, Essex, & Kupfer, 2008). Still, the positive association between the time it takes to complete a survey and the variance of selected items, may support the hypothesis that participants who dedicate more time to surveys may be responding with varied (and potentially, valid) answers, as opposed to 84 generating seemingly random choices to complete a survey in the least possible time (i.e., if the association between completion time and item response variance was negative). Fatigue and Compliance Contrary to hypotheses, there was no significant interaction between survey completion time and survey item response variance (i.e., objective measures of compliance) predicting EMA compliance. Instead, participants who spent more time than their own average on any given survey were less likely to comply subsequent EMA prompts. Likewise, participants with a tendency to spend more time on surveys than other participants were less likely to comply to surveys in general. The time-lagged prompt-level association suggests that participants may be mindful of the effort it took to complete the previous EMA prompt and may be less inclined to comply with the current EMA prompt. Although the person-level association corroborates this explanatory mechanism, it is unclear to what extent the association is due to momentary deliberation and not some other trait-level factors (e.g., poor reading comprehension, unfamiliarity with technology) or external variables (i.e., social or physical context) (Gammon et al., 2005; Heron & Smyth, 2010). As noted earlier, the association between variance of response patterns and day in the study was reduced significantly after adjusting for survey completion time and could elucidate the null relationship between survey item response variance and EMA compliance after adjusting for the length of time it took to complete surveys. Limitations The study is primarily limited by issues related to temporal ambiguity of effects, specifically when using the end-of-study satisfaction questionnaire in models with EMA compliance. Protocol satisfaction was measured at the end of the study and therefore could not have predicted momentary or overall compliance. Therefore, results may only be interpreted as 85 correlations, without temporality. Still, there were no statistically significant results and post-hoc analyses using a linear regression with aggregated compliance as the predictor and end-of-survey satisfaction as the outcome similarly found no association. The study is also limited by end-of-study measures of satisfaction that exhibited a ceiling effect, as individuals were queried for feedback at the exit appointment in the presence of a researcher. This may have resulted in an undetectable effect due to a lack of variance. Furthermore, reporting of phone wear and non-wear resulted in reduced proportions of non-wear time (i.e., floor effect), as individuals were less likely to have their phone away from them if they were capable of answering a survey. Still, the study had power appropriate to detect a very small effect between compliance and subjective satisfaction, thereby limiting the impact of ceiling and floor effects. Next, the study utilized a crossover design that may have initially fatigued participants to the protocol prior to deploying EMA. A group of participants wore accelerometers for 14 days prior to starting the EMA protocol, and potential differences in the outcome or predictors could have existed between these groups. Similarly, context-sensitive prompts were intentionally designed to prompt participants after bouts of activity and inactivity. Subsequently, the results from CS prompts may differentially affect participant fatigue as compared to random prompts. To address these concerns, measurement order and prompt type were tested as separate covariates in each model and did not impact results. The EMA protocol used random and context-sensitive prompts and therefore participants who were more likely to engage in behaviors that trigger bouts were likely to receive more prompts. However, the total number of received prompts was screened as a covariate and did not 86 modify the results of any study aims. Hence, it was not necessary to control for the effect of excessive prompting on participant compliance. Implications The study contributed methodological findings that may be used to assist researchers in developing health-behavior measurement protocols that are more reliable and less burdensome on participants. Trends from findings on subjectively reported satisfaction with EMA suggesting satisfied participants are less likely to comply to EMA prompts implicated significant response biases in how individuals report satisfaction with EMA protocols at the end of studies. Despite the lack of any significant relationship between end-of-study satisfaction with EMA and EMA compliance, these trends warrant caution, especially with wide use of self-report measures to determine acceptability and burden of a protocol in participants (Dzubur et al., 2015; Granholm, Loh, & Swendsen, 2008; Husky et al., 2014). Given the limited literature examining EMA metadata as a psychometric tool, the study offers an early exploratory overview of how big data collected from small samples could be used to examine participant fatigue. While the adverse impact of overall longer survey completion time on EMA compliance was not unexpected, an immediate negative association with subsequent EMA compliance may prove useful for researchers seeking to adjust protocols within participants in order to optimize data retention (Hufford, 2007). Item-branching is used as a strategy used to avoid lengthy surveys, but may actually reduce compliance (Stone & Shiffman, 2002). Instead, results suggest that alternative survey construction strategies such as timers with automatically generated questions (i.e., if the participant finishes too fast) and survey parsing (i.e., if the participant is taking too long) to equalize the amount of time spent taking the survey and not necessarily the number of items may reduce the likelihood of systematic non-compliance. Although survey item response variance 87 was not implicated as a significant predictor of EMA compliance, the findings suggested that individuals who dedicate more time to surveys provide a greater variety of answers. Yet, findings indicated that these individuals were more likely to miss surveys, potentially resulting in undersampling of valid data. However, this assumes that variance in item selection is an indicator of validity; a future study using psychometrically robust measures (e.g., reverse coding, outlier detection) may better elucidate validity of responses (Curran, 2016). Furthermore, the decline in both survey item response variance and completion time over the measurement period may be used to inform decision making in training periods for EMA protocols. Although increasing avoidance of survey items (i.e., through intentional selection of shortened survey branches) across a measurement period may be an indicator of fatigue in and of itself , the potential reduction in variance of item response across the measurement period may have to be forfeited for advantages from training periods, dependent on the outcome and protocol, such as reduction of reactivity (i.e., Hawthorne effect) with accelerometer measurement (Esliger, Copeland, Barnes, & Tremblay, 2005). Future Directions Future studies may seek to develop self-reported measures of protocol satisfaction that are robust to response biases and limit ceiling effects. For instance, EMA-delivered questionnaires querying satisfaction with EMA protocols may provide results more congruent to participant perception. Participants are not monitored by researchers in free-living situations and may perceive anonymity in EMA responses. Additionally, EMA prompted satisfaction items would be ecologically valid and elucidate changes in perceived study satisfaction across time. A confirmatory analysis of the relationship between survey item response variance, survey completion time, and EMA compliance in a more simplified EMA protocol with equivalent 88 survey length (i.e., no survey branching) and valence across all items (i.e., Likert-type only) may help elucidate the causal mechanism and generalize the results to a wide variety of protocols, populations, and outcomes. Studies utilizing EMA continue to grow in sophistication in terms of protocol complexity, external device use (e.g., UV sensors, asthma inhaler sensors), and measurement duration. However, while EMA studies in the past loaned devices to participants, a modern bring-your-own-device ("BYOD") approach can promote compliance and enable long- term use of software in a familiar environment. Still, the burden of a study can impact participant fatigue to the protocol and reduce validity of findings due to sampling biases, even if participants use their own devices. As studies become more sophisticated, the measurement of participant fatigue will grow in importance given an increased number of design decisions that will be able to impact study burden. 89 CHAPTER 4: REACTIVITY TO A LONGITUDINAL SMARTPHONE-BASED TIME- INTENSIVE PHYSICAL ACTIVITY ASSESSMENT Abstract Purpose: Reactivity, the change in behavior as a result of knowing one is being measured, is a phenomenon that is both acknowledged and well-understood in the measurement of physical activity. On the other hand, studies using ecological momentary assessment (EMA), a real-time sampling strategy, generally find little reactivity in participants, despite intensive sampling in free-living situations. However, such studies have been limited to the measurement of psychological constructs (e.g., body image perception) and behaviors unrelated to physical activity (e.g., drinking) as outcomes. There is limited literature on how the use of EMA, as well as both EMA and accelerometers, to measure physical activity as an outcome may induce reactivity (i.e., change in activity levels) in participants. The purpose of this study was to test for reactivity to the application of a longitudinal smartphone-based time-intensive physical activity measurement protocol. Methods: Adolescents between the ages of 14 and 18 (N=44) completed random EMA prompts at two-hour intervals and context-sensitive EMA prompts after moments of inactivity or activity, while wearing accelerometers and completing an assisted end-of-day recall activity for 14 days. Participants wore accelerometers and carried smartphones without EMA for another 14 days. Participants were randomized into groups receiving the full (i.e., EMA plus accelerometer) protocol in the first two weeks or the last two weeks. Multiple multilevel regressions were used to examine temporal trends and differences in physical activity and sedentary time as measured by accelerometers due to exposure to the EMA protocol and order of exposure. Results: Participants who received the EMA plus accelerometer protocol in the last two weeks had more light physical activity during the EMA plus accelerometer component than 90 accelerometer only, but had increasing sedentary time throughout the measurement period. Conversely, those who received the EMA plus accelerometer protocol in the first two weeks were more sedentary and had less MVPA during the EMA plus accelerometer protocol, as well as decreasing sedentary and increasing MVPA time throughout the study. EMA compliance did not differ between protocols, but decreased over time. Conclusions: There was evidence for differential reactivity contingent on the deployment of EMA at the beginning of a study or after a period of accelerometer measurement. To determine whether the effects of reactivity in EMA protocols may be minimized by removing the presence of external sensors, further research is needed to examine how EMA questionnaires alone affect physical activity and sedentary behavior in participants by leveraging onboard smartphone accelerometers. 91 Introduction Physical inactivity is associated with negative impacts on health-related quality of life and mental health (Biddle & Asare, 2011). The association between physical activity and psychosocial factors, however, is complex and often bidirectional (Sallis et al., 2000). In order to investigate these complex relationships in free-living situations, studies often utilize a multimethod approach with accelerometers and ecological momentary assessment (EMA). EMA is a time-intensive real-time data sampling strategy that reduces recall bias and preserves ecological validity. Participants are able to report feeling states (e.g., positive affect), stress, and contextual variables that are otherwise not measured by accelerometers. Studies have used integrated accelerometer and EMA data to determine antecedents and correlates of physical activity in adults and children (Dunton, Liao, et al., 2012). However, the internal validity of studies investigating associations among physical activity and other psychosocial factors may be limited by participant reactivity. Participant reactivity ("Hawthorne effect") occurs when participants are aware that they are being observed on the outcome of interest and subsequently change their behavior (Bassett Jr & John, 2010). While literature on reactivity caused by measurement of physical activity using accelerometers is published extensively, the literature on the effects of EMA sampling on physical activity is sparse. Reactivity to Accelerometers Reactivity is generally associated with improvements in physical activity when participants are provided with accelerometers, especially if the participant is aware of its purpose or a step count is visible. For instance, a sample of Australian adolescents were randomly assigned to three groups of pedometer wear for a period of seven days: participants that could view their steps at any time (i.e., unsealed pedometers); participants that viewed pedometer 92 readings once a day; and participants that were not able to view their steps (i.e., sealed pedometers) (Scott et al., 2014). In contrast to the sealed pedometer group, the unsealed pedometer and the daily unsealed pedometer groups showed evidence of reactivity, device tampering (e.g., shaking device to achieve step count), and a desire to impress researchers with their step counts. However, approximately half of all participants reported shaking the pedometer to increase step counts, independent of their group assignment (Scott et al., 2014). Similar crossover studies suggest that the use of sealed pedometers or accelerometers alone is not enough to eliminate reactivity (Foley, Beets, & Cardinal, 2011). Notably, reactivity occurring as a result of accelerometer-wear impacts physical activity estimates differentially over time, and it is also less likely to threaten internal validity of studies in adults. The effect is thought to disproportionately affect children because of their competitiveness and willingness to adopt novel technology (McClain & Tudor-Locke, 2009). In a secondary data analysis of eight studies assessing physical activity with accelerometers in children and adolescents, participants were up to 7% more active on the first day of a measurement period than on any other subsequent day (Dossegger et al., 2014). Among younger children, physical activity measures differed up to 10% between the first day and other days. Therefore, not only does physical activity vary over time due to reactivity, the evidence affirms the importance of assessing reactivity in children and adolescents. In contrast, a study examined a young adult sample for temporal patterns of physical activity; there was no evidence of reactivity to accelerometer wear throughout the two-week protocol (Behrens & Dinger, 2007). Similarly, adults who completed two protocols within a study showed no differences between physical activity measurement with sealed or unsealed pedometers (Matevey, Rogers, Dawson, & Tudor-Locke, 2006). 93 Reactivity to Ecological Momentary Assessment Reactivity is a well-documented threat to the internal validity of studies measuring physical activity, yet there are limited findings addressing the reactivity to time-intensive measures (e.g., EMA) of physical activity. A study by Biddle and colleagues suggests that no evidence of reactivity was found in an intensive EMA protocol on adolescents assessing time-use and activity, yet there is no systematic analysis of temporal trends nor is the four-day protocol long enough to reveal valid evidence of such an effect (2009). Studies addressing reactivity to EMA protocols have generally focused on examining self-reported outcomes measured by EMA such health-related behaviors or perceptions of symptoms (Clemes & Parker, 2009). The findings within this body of literature show mixed results, suggesting that repeated assessments impact outcomes minimally and may differ based on study design factors such as protocol length. A study in a sample of young adults contrasted results from a 30-day alcohol use recall survey with results from a 14-day alcohol use recall survey following a two-week EMA measurement protocol. The results revealed a non-significant decline in reported alcohol use, and no significant changes were found in motivations to quit drinking (Hufford, Shields, Shiffman, Paty, & Balabanis, 2002). Similarly, in two-week measurement protocols, there was no evidence for reactivity over time or based on assessment frequency when measuring pain in chronic pain sufferers or body-image perception in at-risk women (Heron & Smyth, 2013; Stone et al., 2003). However, an eight-week self-reported electronic diary protocol on a sample of adults revealed a small, but consistent, decline in drinking frequency over the course of the study (Collins et al., 1998). Despite the minimal reactivity in other EMA studies and the lack of systematic research on reactivity in EMA studies measuring physical activity, a study by Clemes & Parker suggests 94 that daily logging of physical activity may amplify reactivity (2009). Adult participants wore a pedometer under four conditions, including deception (i.e., the pedometer was identified as a body posture device), unsealed, sealed, and unsealed with a daily log. As expected, participants were more active during the unsealed diary condition as compared to any other condition, and they showed significant temporal trends consistent with reactivity (Clemes & Parker, 2009). The results are significant given that this was an adult sample that generally shows limited reactivity to accelerometer wear. However, the findings are limited in generalizability to EMA because participants were not repeatedly assessed, nor were participants self-reporting physical activity. Literature Gaps While reactivity to physical activity measurement protocols is well-documented, studies may show inconsistent results even among children. A sample of fourth-grade children was tested using a standard sealed versus unsealed pedometer protocol; the study found no evidence for reactivity (Ozdoba, Corbin, & Lemasurier, 2004). However, the protocol lasted four days, and a novelty effect may take longer to subside. Furthermore, although there are some studies that suggest little to no reactivity to EMA protocols, less is known on the effects of reactivity to an EMA protocol measuring physical activity. Most importantly, there is no known literature investigating combined diary and accelerometer approaches to activity measurement with respect to reactivity, despite evidence that an association may exist even in groups thought to be at low- risk of reactivity (i.e., adults). Study Overview To address the aforementioned gaps in studies using EMA to assess physical activity, the study tested reactivity to an EMA protocol using a crossover design with an accelerometer-only control group. The study determined if adolescents altered their physical activity in response to 95 measurement by EMA and an accelerometry-assisted end-of-day recall measure. The primary objective was to assess whether or not reactivity is a threat to internal validity in the context of physical activity measurement using EMA among adolescents. A secondary aim was to determine whether physical activity during the EMA/end-of-day recall phase shows temporal trends similar to studies using unsealed pedometers and whether a two-week accelerometer pre- wear period affected these trends (Dossegger et al., 2014). Finally, this study described day-to- day trends in EMA compliance and whether they differed if accelerometers were worn prior to EMA. Specific Aims and Hypotheses 7. To test whether implementing a multimethod activity assessment protocol consisting of momentary self-report and end-of-day self-report recall methods is associated with differences in moderate-vigorous physical activity, light activity, and sedentary behavior compared to accelerometer-only measurement of physical activity. Participants were hypothesized to be more active and less sedentary during the multimethod measurement period as compared to the accelerometer-only measurement period. 8. To examine differences in physical activity trends between individuals immediately starting an ecological momentary assessment protocol and those wearing an accelerometer for two weeks prior to beginning ecological momentary assessment. Temporal trends similar to those observed in other reactivity studies (i.e., greater physical activity at the onset of the measurement period) were hypothesized to occur at the onset of multimethod measurement in both groups, but only at the onset of accelerometer-only measurement for the accelerometer-first group. 96 9. To examine differences in ecological momentary assessment compliance trends between individuals immediately starting an ecological momentary assessment protocol and those wearing an accelerometer for two weeks prior to beginning ecological momentary assessment. Temporal trends in EMA prompt compliance were hypothesized to occur at the onset of the multimethod-first group, but not the multimethod-last group (i.e., accelerometer-first). Methods The study used accelerometry data and EMA data from Mobile TEENS, a quasi- experimental crossover study that tested the feasibility of an accelerometry-guided end-of-day activity recall method. Adolescent participants wore accelerometers for the duration of the study, but were randomly assigned to 14-day EMA monitoring groups for the first or second half of the measurement period. The study utilized the EMA plus accelerometer data from Mobile TEENS, in addition to anthropometric and demographic measures. A more thorough overview of the study design including EMA and end-of-day recall activity is available elsewhere (Dunton et al., 2014). Participants Participants recruited to the Mobile TEENS study were ethnically-diverse high school students living in the Greater Los Angeles area. 44 participants were enrolled in the study, of which 39 (89%) completed both the accelerometer and EMA portions of the study. Procedure Participants were assessed at the start of the study for demographic information and anthropometric measures and assigned to two measurement groups at random: accelerometer- first or multimethod-first. Each participant was provided with accelerometers and the Mobile 97 TEENS Android application on a loaned or study phone. Participants were instructed to wear accelerometers continuously for 28 days, excluding incompatible activities such as sleep or swimming. The study used a within-subjects crossover design with a multimethod condition and an accelerometer condition. In the multimethod condition, participants were passively monitored with a waist-worn accelerometer for 14 days while also completing an EMA protocol and an accelerometer-assisted end-of-day recall activity. In the accelerometer only condition, participants were passively monitored with a waist-worn accelerometer for 14 days while carrying their phones. Half of the participants were randomly assigned to receive the multimethod condition first, while the other half received the accelerometer only condition first. The end-of-day recall activity provided participants with a real-time graphical representation of their activity throughout the day. An algorithm detected changes in physical activity and grouped physical activity bouts ("chunks") that participants were instructed to label. Activity bouts could be merged together or split apart by interacting with the graph if more than one activity occurred (Figure 7). A participant had up to two full days to complete each daily recall and received immediate compensation via a $1 Amazon.com gift code. Signal-contingent EMA prompts occurred up to seven times per day during non-school hours, and event-contingent prompts occurred following changes detected by the smartphone accelerometer. 98 Figure 7. Sample Screenshot of Mobile TEENS End-of-Day Recall Activity Measures Demographic and Anthropometric Measures BMI z-scores were obtained from age and sex-adjusted height and weight charts (Vidmar et al., 2004). Participants self-reported age, gender (dichotomous, reference=Male), and ethnicity (1=Hispanic, 0=Other). Physical Activity Physical activity data was obtained as activity counts at 30-second epochs from Actigraph accelerometers (GT2m, Firmware v7.5.0) and processed to classify each epoch as MVPA, light activity, or sedentary time using age-adjusted cut points (Freedson et al., 1998). Epochs were also classified as non-wear based on sixty minutes of consecutive zero counts (Troiano et al., 2008). Minutes of sedentary, light activity, MVPA, and non-wear were aggregated at the daily level and days with less than 10 hours of data were coded as missing. Valid wear time was entered into the model to further control for oversampling of activity counts, specifically those classified as sedentary, in highly compliant individuals. 99 EMA Compliance Compliance to prompts was defined as having completed at least one item in an EMA survey. Compliance were aggregated as the day-level proportion of prompts complied divided by the number prompted. Temporal Measures Accelerometry and EMA epochs were time-stamped and categorized as the following variables: day in the measurement period, between days 1 and 28; and day of the week, coded dichotomously by weekend and weekday (reference = weekday). Data Analysis A series of multilevel linear regressions were used to test the main effect of exposure to EMA and end-of-day measurement, as well as temporal trends in physical activity and compliance with and without an accelerometer pre-wear period. Unconditional mean models were fit to determine intra-class correlation and models were screened for normality and covariance structure. Measurement order was coded as a binary variable, with accelerometer-first as the reference group. Age, gender, ethnicity, and BMI were entered as covariates and analyses were conducted in Stata 14. Research Question 7. To test whether implementing a multimethod EMA and end-of-day activity recall method was associated with differences in MVPA, light activity, and sedentary behavior compared to accelerometer-only measurement, three multilevel linear regressions were used with each type of physical activity as the outcome. An additional model was run for each outcome to determine if an interaction exists between the measurement exposure order (ORDER j ) and measurement exposure (EXP ij ). To assess the differences in exposure and test for order, measurement exposure (1 = multimethod measurement, 0 = accelerometer only) and order (1 = multimethod in the last two weeks, 0 = multimethod in the first two weeks) were coded dichotomously. Outcomes (Y ij ) that were tested included day-level MVPA, light activity, and 100 sedentary time. Each model was adjusted for measurement exposure order, day in the study (DIS ij ), day of the week (DOW ij ), and valid wear time (VALID ij ) in addition to standard covariates. > /) = 3 44 +3 64 &TU /) +3 46 $(B&( ) +3 84 B<: /) +3 94 B$% /) +3 ?4 L'V<B /) +3 48 '@& ) +3 49 @&#B&( ) +3 4? !D<E ) +3 4A ('G& ) +3 A4 &TU /) ∗$(B&( ) +K ) +I /) where g 10 represents the difference in the outcome between multimethod vs. accelerometer-only exposure for a participant and g 50 represents the interaction between measurement and order present only in models testing this interaction. Research Question 8. To examine differences in physical activity trends between individuals immediately starting multimethod protocol and those wearing an accelerometer for two weeks prior to beginning EMA, six multilevel linear regressions tested the interaction between multimethod exposure order (ORDER j ) and the day in the measurement period (DIS ij ) on physical MVPA, light activity, and sedentary behavior (Y ij ). Each model was further adjusted for day of the week (DOW ij ). > /) = 3 44 +3 46 $(B&( ) +3 64 &TU /) +3 84 B<: /) +3 94 B$% /) +3 ?4 L'V<B /) +3 48 '@& ) +3 49 @&#B&( ) +3 4? !D<E ) +3 4A ('G& ) +3 A4 $(B&( ) ∗B<: /) +K ) +I /) where g 01 represents the change in the outcome in the multimethod-first group, g 20 represents the trend in the outcome across days, and g 50 represents the interaction between measurement order and trend. Research Question 9. To examine differences in EMA compliance trends between individuals immediately starting an ecological momentary assessment protocol and those wearing an accelerometer for two weeks prior to beginning ecological momentary assessment, two multilevel linear regressions tested the main effect and interaction of the day in the study 101 (DIS ij ) and measurement order (ORDER j ) on day-level prompt compliance (Y ij ). The day-level model, where each i represents a different day, was additionally adjusted for the day of the week (DOW ij ). > /) = 3 44 +3 46 $(B&( ) +3 64 B<: /) +3 84 B$% /) +3 48 '@& 4) +3 49 @&#B&( ) +3 4? !D<E ) +3 4A ('G& ) +3 94 $(B&( ) ∗B<: /) +K ) +I /) where g 01 represents the person-level difference in compliance in the multimethod-first group compared to the multimethod-last group, g 10 represents the day-level trend in compliance, and g 30 represents the cross-level interaction of measurement order by trend in the study. Power Analysis A power analysis using G*Power 3.1 was conducted to determine if the proposed multilevel linear regressions were feasible with Type-I and 1- β error set at 0.05 and 0.80. ICC (ρ) for all variables was estimated at 0.10; to account for clustering within individuals, effect size was estimated as 0.10. True ICC’s for sedentary time, light physical activity, MVPA, and EMA compliance were 0.34, 0.35, 0.39, and 0.35. For models 7 and 8, there was an estimated total of 28 days of accelerometer data per person (m=28, D=3.7). For model 9, there was an estimated total of 14 days per person (m=14, D=2.3). There were 10, 10, and 8 predictors for models 7, 8, and 9, respectively. Figures 8 and 9 shows the power curve demonstrating power as a function of sample size given a decrease and an increase in the detectable effect size for each model. The unadjusted minimum sample sizes for models 7, 8, and 9 were 172, 172, and 159, respectively. After adjusting for the design effect, the adjusted sample sizes were 637, 637, and 366. Given 39 participants with 21 days (75% completed) of valid accelerometer data, there are an estimated 819 available observations. This suggests that the sample is sufficient to detect effects in Models 102 7 and 8. Given 39 participants with 10 days of available compliance (N=390), there is a sufficient sample size to detect effects in Model 9. Figure 8. Power Curve for Research Questions 7 & 8 Figure 9. Power Curve for Research Questions 9 Results Descriptive Statistics Out of 44 enrolled participants, 40 completed the EMA plus accelerometer portion of the study and 35 (87.5%) had valid data available for analysis after excluding days with insufficient 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 0 100 200 300 400 Total sample size = 0.15 = 0.1 = 0.05 Effect size f² F tests - Linear multiple regression: Fixed model. R2 deviation from zero Number of predictors = 9. # err prob = 0.05 Power (1-β err prob) 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 0 100 200 300 400 Total sample size = 0.15 = 0.1 = 0.05 Effect size f² F tests - Linear multiple regression: Fixed model. R2 deviation from zero Number of predictors = 8. # err prob = 0.05 Power (1-β err prob) 103 accelerometer wear time. Participants wore accelerometers an average of 835.65 (SD = 76.41) minutes a day across a mean 20.14 (SD = 8.37) valid wear days. The majority of average daily time was classified as sedentary (M = 594.75 minutes, SD = 85.65), while an average of 187.98 (SD = 52.29) minutes were classified as light activity. Participants were active (i.e., MVPA) for 52.91 (SD = 80.82) minutes per day and completed 81.7% (SD = 14.9%) of daily EMA prompts, on average. Subsequent t-tests revealed no significant differences between individuals who received the EMA plus accelerometer component in the first two weeks compared to those who received the EMA plus accelerometer component in the last two weeks (p’s>0.05). Reactivity to Multimethod Protocols Table 15 presents the results of multilevel linear regressions predicting sedentary, light activity, and MVPA time as a function of real-time data capture protocol type (i.e., Research Question 7). After adjusting for all covariates and measurement order, the amount of time categorized as sedentary, light activity, and MVPA was not significantly different in the days during the accelerometer and EMA protocol as compared to the accelerometer only protocol (i.e., no statistically significant main effect for exposure). Similarly, there was no difference in the amount of time categorized as sedentary, light activity, and MVPA for overall for participants who received the EMA protocol in the first two weeks compared to those who received the EMA protocol in the last two weeks after adjusting for all covariates and protocol type (i.e., no statistically significant main effect for order). Subsequent multilevel linear regressions revealed a protocol by order interaction for models predicting sedentary time, light activity time, and MVPA time (z=-2.51, p<0.05; z=2.37, p<0.05; and z=2.52, p<0.05; respectively). There were 32.5 more minutes of time categorized as light activity (Figure 10) during the accelerometer plus EMA protocol than during the accelerometer only protocol for individuals who received the 104 accelerometer plus EMA protocol in the last two weeks, but not for individuals who received the protocol in the first two weeks (Bonferroni adjusted z=2.96, p<0.05). There were 37.7 more minutes of time categorized as sedentary (Figure 11) during the accelerometer plus EMA protocol than during the accelerometer only protocol for individuals who received the accelerometer and EMA protocol in the first two weeks, but not for individuals who received the protocol in the last two weeks (Bonferroni adjusted z=3.09, p<0.05). There was 48.4% less time categorized as MVPA (Figure 12) during the accelerometer plus EMA protocol than during the accelerometer only protocol for individuals who received the accelerometer and EMA protocol in the first two weeks, but not for individuals who received the protocol in the last two weeks (Bonferroni adjusted z=-2.90, p<0.05). Table 15. Accelerometer-measured activity as a function of real-time data capture protocol (1) (2) (3) SedentaryTime LightActivityTime MVPATime b S.E. b S.E. b S.E. ProtocolType(ref =Accelerometer Only) AccelerometerandEMA 10.908 (5.939) 9.307 (5.025) 0.117 (0.081) MixedMethodsProtocolOrder(ref =First TwoWeeks) LastTwoWeeks 32.157 (30.069) 4.708 (19.088) 0.581 (0.342) Gender(ref =Male) Female 43.374 (27.361) 9.915 (17.384) 0.246 (0.311) Age 10.130 (12.415) 4.783 (7.833) 0.008 (0.141) Ethnicity(ref =Non Hispanic) Hispanic 49.147 (29.827) 13.463 (18.967) 0.788 ⇤ (0.339) BMIz-score 4.836 (14.669) 15.264 (9.278) 0.071 (0.166) DayofWeek(ref =Weekday) WeekendDay 9.060 (7.050) 20.442 ⇤⇤⇤ (5.966) 0.483 ⇤⇤⇤ (0.096) DayintheStudy 0.018 (0.373) 1.367 ⇤⇤⇤ (0.315) 0.007 (0.005) Validweartime(min.) 0.679 ⇤⇤⇤ (0.025) 0.239 ⇤⇤⇤ (0.021) 0.002 ⇤⇤⇤ (0.000) Constant 524.758 ⇤⇤⇤ (30.773) 216.189 ⇤⇤⇤ (19.867) 3.030 ⇤⇤⇤ (0.353) Variance(Randomeffects) 5727.724 ⇤⇤⇤ (1481.223) 2188.734 ⇤⇤⇤ (594.000) 0.717 (0.190) Residual(Randomeffects) 4816.763 ⇤⇤⇤ (284.658) 3456.423 ⇤⇤⇤ (204.225) 0.897 (0.053) N 606 606 606 Standarderrorsinparentheses Note: MVPAislog-transformed ⇤ p<0.05, ⇤⇤ p<0.01, ⇤⇤⇤ p<0.001 1 105 Figure 10. Measurement protocol type by order interaction predicting light activity time Figure 11. Measurement protocol type by order interaction predicting sedentary time 106 Figure 12. Measurement protocol type by order interaction predicting MVPA time Note: MVPA time is log-transformed. Order Effects and Time As noted in Table 15, each subsequent day in the study was associated with a 1.4 minute decrease in time classified as light activity time compared to the day prior, after adjusting for all covariates. However, each subsequent day in the study was not associated with a change in time classified as MVPA or sedentary compared to the day prior. Three additional multilevel linear regressions revealed a time by protocol order interaction for models predicting sedentary and MVPA time, but not light activity time, as detailed in Table 16 (i.e., Research Question 8). Interactions revealed 74.9% more time categorized as MVPA (Figure 13) on the 28 th day of the study than on the first day of the study for individuals who received the accelerometer and EMA protocol in the first two weeks, but no difference for individuals who received the protocol in the last two weeks (Bonferroni adjusted z=2.68, p<0.05). There were 71.8 more minutes of time 107 categorized as sedentary (Figure 14) on the 28 th day of the study than on the first day of the study for individuals who received the accelerometer and EMA protocol in the last two weeks, but 62.3 less minutes categorized as sedentary on the 28 th day compared to the first day for individuals who received the protocol in the first two weeks (Bonferroni adjusted z=3.20,p<0.01 and Bonferroni adjusted z=-3.07,p<0.05; respectively). Table 16. Interaction of day in study and protocol order on accelerometer-measured activity (1) (2) (3) SedentaryTime LightActivityTime MVPATime b S.E. b S.E. b S.E. DayintheStudy 2.309 ⇤⇤ (0.753) 0.374 (0.642) 0.028 ⇤⇤ (0.010) ProtocolType(ref =Accelerometer Only) AccelerometerandEMA 22.282 ⇤ (11.054) 23.484 ⇤ (9.437) 0.173 (0.152) MixedMethodsProtocolOrder(ref =First TwoWeeks) LastTwoWeeks 40.035 (36.180) 26.178 (25.772) 0.050 (0.442) Dayinthestudy*ProtocolOrderInteraction 4.967 ⇤⇤⇤ (1.401) 2.120 (1.196) 0.043 ⇤ (0.019) Gender(ref =Male) Female 44.472 (27.217) 10.407 (17.305) 0.255 (0.311) Age 9.756 (12.351) 4.938 (7.796) 0.005 (0.141) Ethnicity(ref =Non Hispanic) Hispanic 46.624 (29.676) 12.395 (18.888) 0.811 ⇤ (0.339) BMIz-score 5.266 (14.592) 15.450 (9.235) 0.075 (0.166) DayofWeek(ref =Weekday) WeekendDay 7.800 (6.985) 19.905 ⇤⇤⇤ (5.959) 0.494 ⇤⇤⇤ (0.096) Validweartime(min.) 0.677 ⇤⇤⇤ (0.025) 0.240 ⇤⇤⇤ (0.021) 0.002 ⇤⇤⇤ (0.000) Constant 576.002 ⇤⇤⇤ (33.846) 194.321 ⇤⇤⇤ (23.307) 2.582 ⇤⇤⇤ (0.405) Variance(Randomeffects) 5670.904 ⇤⇤⇤ (1463.910) 2167.175 ⇤⇤⇤ (588.692) 0.718 (0.191) Residual(Randomeffects) 4716.196 ⇤⇤⇤ (278.698) 3439.429 ⇤⇤⇤ (203.226) 0.889 ⇤ (0.053) N 606 606 606 Standarderrorsinparentheses Note: MVPAislog-transformed ⇤ p<0.05, ⇤⇤ p<0.01, ⇤⇤⇤ p<0.001 2 108 Figure 13. Time by measurement protocol order interaction predicting MVPA time Note: MVPA time is log-transformed. Figure 14. Time by measurement protocol order interaction predicting sedentary time 109 Order Effects and Compliance After adjusting for covariates and day in the study, participants who received the accelerometer and EMA protocol in the third and fourth week did not differ in daily EMA compliance rate compared to those who received the accelerometer and EMA protocol in the first and second week (i.e., Research Question 9). However, each additional day in the study was associated with a 2.1% decrease in daily EMA compliance after adjusting for covariates and measurement protocol order. As detailed in Table 17, there was no significant time by protocol order interaction for the model predicting day-level EMA compliance. Table 17. Day-level EMA compliance as a function of protocol order Discussion The study examined reactivity to the deployment of both an accelerometer only protocol and an EMA plus accelerometer protocol used to measure physical activity throughout the day in (1) (2) MainEffectsModel InteractionModel b S.E. b S.E. DayintheStudy 0.021 ⇤⇤ (0.008) 0.017 (0.010) MixedMethodsProtocolOrder(ref =First TwoWeeks) LastTwoWeeks 0.090 (0.137) 0.161 (0.175) Gender(ref =Male) Female 0.100 (0.124) 0.098 (0.124) Age 0.034 (0.057) 0.032 (0.057) Ethnicity(ref =Non Hispanic) Hispanic 0.325 ⇤ (0.136) 0.325 ⇤ (0.135) BMIz-score 0.000 (0.070) 0.001 (0.069) DayofWeek(ref =Weekday) WeekendDay 0.066 (0.070) 0.065 (0.070) Dayinthestudy*ProtocolOrderInteraction 0.010 (0.015) Constant 4.583 ⇤⇤⇤ (0.152) 4.551 ⇤⇤⇤ (0.159) Variance(Randomeffects) 0.094 ⇤⇤⇤ (0.030) 0.093 ⇤⇤⇤ (0.030) Residual(Randomeffects) 0.253 ⇤⇤⇤ (0.022) 0.252 ⇤⇤⇤ (0.022) N 289 289 Standarderrorsinparentheses Note: Dependentvariableislog-transformed. ⇤ p<0.05, ⇤⇤ p<0.01, ⇤⇤⇤ p<0.001 3 110 adolescents. Reactivity is a notable limitation to studies measuring physical activity using accelerometers, yet this does not appear to be the case for EMA studies in previous literature. However, the literature examining reactivity to EMA protocols has been limited to operationalizing reactivity as changes in psychological constructs and behavioral outcomes unrelated to physical activity, and there is no known study to date that has tested the effects of monitoring participants on reactivity using a combination of passive sensors and EMA. Researchers are increasingly implementing EMA in order to capture the environmental and psychosocial contexts of active behavior, and there is a need to understand how participants alter their own behavior in response to exposure to these measurement protocols. By leveraging crossover design, the study was able to test the extent to which reactivity in a multi-method protocol (i.e., EMA plus accelerometer) is different from an accelerometer-only protocol and examine how the order in which these protocols are deployed affects reactivity. Furthermore, studies using EMA protocols must contend with changes in compliance across lengthy measurement protocols. This study further elucidates how compliance to EMA prompts changes throughout a monitoring period and determines whether pre-deployed measures of physical activity impact compliance to a subsequent protocol change. Reactivity to Multimethod Protocols While there were no main effects that indicated differences in sedentary, light activity, or MVPA time during the accelerometer plus EMA protocol compared to the accelerometer only protocol, interaction models confirmed that the null finding occurred as a result of crossed effects due to order of protocol deployment. When participants received the EMA plus accelerometer protocol in the first two weeks (vs. last two weeks), they had less time categorized as MVPA and more time categorized as sedentary during EMA plus accelerometer than 111 accelerometer only measurement. Participants may have perceived an accelerometer as a physical activity measurement device, whereas multimethod measurement may have confounded their perception of what was being measured, thereby reducing reactivity. The decrease in MVPA may also be attributed to the inactivity required to answer EMA survey, especially context-sensitive EMA surveys that may have interrupted physical activity or encouraged sedentary behavior. However, a prior study found that EMA prompts do not interrupt physical activity, although the same study found that obese or overweight individuals were more inclined to engage in sedentary behavior following an EMA prompt (Dunton et al., 2011). When participants completed the EMA plus accelerometer portion of the study in the last two weeks recorded more light activity during the EMA plus accelerometer protocol than the accelerometer only protocol. Although general reactivity was observed among individuals who received the application of the EMA plus accelerometer protocol in the last two weeks, the relationship persisted only for activity that is more indicative of non-exercise tasks, as MVPA did not differ across protocols among these participants. Literature suggests that perceived exertion is not entirely correlated with activity cut-off points and often leads to overestimation of physical activity by participants (Burke et al., 2006). Hence, participants may not have been able to reach activity counts classified as MVPA, even if intentions to engage in more physical activity were present. Moreover, participants were constrained to school schedules and chores, suggesting that their ability to engage in activity above and beyond their usual threshold may be limited to increasing levels of non-exercise activities (Ainsworth et al., 2011). Order Effects and Time Participants who received the accelerometer only component of the protocol at the beginning of the study recorded a significantly increasing amount of sedentary time throughout 112 the measurement period. Similar to studies demonstrating temporal trends in reactivity, participants likely understood that they were being measured for physical activity and limited sedentary behavior at the onset of measurement (Dossegger et al., 2014). On the other hand, when individuals received the EMA component at the beginning of the study, they were more susceptible to increasing levels of MVPA and decreases in sedentary time throughout the measurement period. However, the pattern exhibited is not indicative of typical reactivity, where activity levels are heightened initially and subsequently decrease over time. Repetition of surveys asking about activity and EMA survey items focused on attributes of physical activity (e.g., type of activity) may have primed physical activity in participants over time. Alternate explanatory mechanisms attributed to the Hawthorne effect may apply (e.g., social desirability), however the day-over-day increase in physical activity is more suggestive of a gradual response to repetitive prompting as seen in physical activity interventions using smartphones (Thomas & Bond, 2015). Order Effects and Compliance There was no association between order of EMA deployment and EMA compliance, indicating that the EMA protocol itself was not susceptible to a pre-deployed accelerometer. However, EMA compliance decreased approximately 2% each day or approximately 32% across a full 14-day protocol. These results indicate that participants may have been susceptible to a novelty effect and fatigue from EMA that resulted in decreased compliance throughout the measurement period. Yet, the lack of an effect of protocol deployment order suggests that fatigue from study burden as a result of the EMA protocol and the accelerometer measurement protocol may be independent of one another. 113 Limitations The study was primarily limited by the inability to disentangle the effects of EMA and the end-of-day recall activity on reactivity. The end-of-day recall activity presented participants with a graphical summary of their activity levels, similar to how studies have unsealed pedometer logs on a daily level. Therefore, the end-of-day recall was likely to significantly impact reactivity, regardless of EMA. However, the time-of-day covariate may be useful in determining if EMA exposure has within-day effects on physical activity that the end-of-day recall activity is less likely to influence. Nevertheless, this method was also limited because the end-of-day recall activity was accessible at any time of the day. Therefore, even if there were within-day temporal trends, the true effect was ambiguous; participants could have reacted to EMA or they could have completed end-of-day recall throughout the day. Missing data, specifically for accelerometry caused by non-wear, may have resulted in a sampling bias if the activities individuals are engaged in during non-wear are systematically more active or sedentary. For example, if participants always removed their accelerometer during sports, sedentary behaviors would be oversampled. This may have been especially problematic if non-wear was systematically different for each measurement type. Daily valid wear time was added in all models using accelerometer data to account for this possibility, as it was significantly associated with the outcome in each model. Lastly, the sample in this study consisted of adolescents accustomed to daily use of smartphones. Subsequently, the findings may not generalize to younger children, who are more susceptible to reactivity in physical activity studies or to adults, who are less likely to seek social desirability despite a presence of a measurement device. However, since reactivity declines as 114 individuals age, the sample may be ideal because it assesses individuals just prior to adulthood, but after childhood (Behrens & Dinger, 2007). Implications The study contributed methodological findings to the measurement of physical activity using subjective real-time surveys and passive external accelerometers and is the first study of its kind to assess reactivity to physical activity measurement in a multi-method protocol. The findings implicated reactivity to EMA protocols only when these protocols were applied after an individual had been measured using an accelerometer. Consequently, EMA protocols may not be robust to reactivity as previously thought, although prior studies have focused on psychological constructs and non-activity related behaviors and therefore did not generalize to physical activity as an outcome. Interestingly, participants who received EMA plus accelerometers in the first two weeks may have experienced a dose-response relationship driven by repeated exposure, thereby causing decrease in sedentary behavior and an increase in MVPA over time. Evidence of an increase in physical activity, even without the presence of a theory-driven intervention, suggests that future mobile physical activity interventions can beneficially implement EMA to supplement treatment. Methodologically, these findings contribute additional evidence to the importance of controlling for temporal change within models of physical activity. Studies using EMA plus accelerometers that observe day-over-day differences in physical activity should exercise caution in how results are interpreted when they may be associated with protocol exposure. Moreover, these trends over time suggest that training periods for EMA protocols thought to improve compliance do not necessarily capture more valid activity levels, and in fact, may miss valid days at the beginning of the measurement period. Lastly, the lack of an association between protocol order and EMA compliance implies that studies may supplement EMA protocols with 115 sensors without impacting the missingness of EMA data. However, given the significant decline of EMA compliance over two weeks, the length of a measurement period may be of particular consequence in multi-method protocols. Future Directions Future research should seek to replicate the results of this study using a similar cross-over design, but with an EMA-only component that relies on built-in smartphone accelerometers as opposed to an accelerometer-only component. This methodology would be able to determine whether or not external accelerometers are associated with changes in EMA compliance, apart from just supplemental order effects. Moreover, by utilizing internal sensors within smartphones, these studies could establish a more robust measure of reactivity through the use of a measurement tool that not only hides physical activity counts, but one that is completely imperceptible to the average subject. Given the significant reactivity found in the deployment (and even removal of) an EMA protocol that causes changes in physical activity and sedentary behavior in participants, both overall and over time, future studies implementing EMA protocols should consider validating onboard smartphone accelerometers to yield data comparable to waist-worn accelerometers. However, such validation may require cooperation with manufacturers on development of standards, as accelerometer validity and reliability is known to vary between and within manufacturers of Android smartphones (Hekler et al., 2015). Still, results from the study by Hekler et. al. that found comparable results between accelerometers and smartphones are a promising future direction in the effort to produce data robust to effects of reactivity (2015). 116 CHAPTER 5: DISCUSSION AND CONCLUSIONS This dissertation described three separate, but related studies exploring challenges and limitations in the design and implementation of EMA used to measure physical activity-related outcomes. The goals of this dissertation were (1) to develop models that examine patterns of non-compliance related to energy intake and expenditure behaviors, (2) to explore measures of participant fatigue and their association with EMA compliance, and (3) to determine whether physical activity differs during an EMA and accelerometer component compared to an accelerometer only protocol. The study outlined in Chapter 2 systematically examined the associations between compliance to EMA prompts and energy balance behaviors among adults and children. The study was unique in its ability to disaggregate the person-level and prompt- level associations between energy balance behaviors and EMA prompt compliance. Chapter 3 described and tested a novel conceptualization of participant fatigue in EMA studies. The study was exploratory in nature and was one of the first to use EMA metadata to examine changes in participant fatigue over time. Lastly, the study described in Chapter 4 examined if and how participants reacted to a time-intensive multimethod EMA and end-of-day activity recall protocol. Building on prior reactivity literature that independently examined reactivity in EMA plus accelerometer protocols, the study was one of the first to examine whether simultaneous deployment of EMA and accelerometry inadvertently changed physical activity and sedentary behavior in participants. The dissertation revealed that physical inactivity (i.e., sedentary time) before a prompt was positively associated with EMA compliance. Participants who were more sedentary on average and took less time to complete surveys were more likely to comply to EMA prompts in general. Conversely, physical activity (i.e., light or MVPA) just before a prompt and the amount 117 of time it took to complete the prior prompt were positively associated with EMA non- compliance. Notably, heightened mean levels of light physical activity were associated with EMA compliance in mothers, but EMA non-compliance in children. Together, these results suggest that participants might comply only when an EMA prompt is perceived as convenient. The dissertation found early evidence for differential compliance based on response to a previous EMA question on eating or activity, although explanatory mechanisms are not clear. For instance, children who reported more activities (regardless of their type) were more likely to be compliant to EMA protocols than those reporting none, while mothers who tend to select a greater number of eating items were more likely to complete subsequent EMA prompts. Additionally, end-of-study measures of satisfaction and EMA self-report of device wear and non-wear were not associated with compliance. Therefore, subjective measures were not adequate predictors of EMA compliance, thus the use of sensors and EMA metadata was found to be a promising predictor of compliance throughout the measurement period. The dissertation found that EMA metadata and physical activity derived from sensors showed temporal trends and could be used to examine response patterns and reactivity. For instance, survey item response variance and survey completion time decreased throughout a measurement period and were positively correlated, suggesting that participants may be avoiding items in branched EMA protocols and selecting repetitive responses over time. Similarly, reactivity to multimethod physical activity measurement protocols (i.e., accelerometer plus EMA) was contingent on whether or not individuals were assigned to passively monitor their behavior for a period of time using accelerometers only before or after the multimethod protocol. Participants who were assigned to the multimethod measurement in the first two weeks were less active (i.e., MVPA) and more sedentary during the multimethod protocol. However, MVPA time 118 increased and sedentary time decreased for each subsequent day in the study, possibly as a result of EMA acting as an intervention. On the other hand, participants who received the multimethod protocol in the last two weeks had increasing time classified as sedentary from the beginning of the study, indicative of a reactivity effect at the onset of the multimethod protocol. Moreover, these participants also had greater light activity during the multimethod protocol, suggesting reactivity upon starting EMA that may have been limited to increased levels of everyday tasks (e.g., chores, walking). EMA allows researchers to minimize weaknesses found in cross-sectional and infrequently-measured longitudinal research, while supplementing objectively-measured physical activity from accelerometers with information about diet, context, and psychological factors. However, as this dissertation revealed, data collected using EMA may be limited by participant behavior and other mechanisms. Still, by recognizing weaknesses in EMA methodology, the dissertation was able to identify potential solutions to address these shortcomings (e.g., context-sensitive prompting, multiple imputation, smartphone accelerometers) and guide future research. Together, the three studies revealed that measurement of physical activity using EMA sampling is highly sensitive to differential compliance rates based on physical activity and inactivity, is associated with increasing participant fatigue throughout a measurement protocol, and can produce distinct patterns of reactivity contingent on accelerometer training periods. Moreover, findings on differential compliance, participant fatigue, and reactivity have broad implications for future implementation of EMA-based methodology, theory, and interventions. 119 Implications Methodological Implications Each of the three studies described in the dissertation were methodological in nature, and therefore have direct methodological implications for the development of EMA protocols used to measure physical activity. The dissertation showed that a majority of EMA surveys with complete data are likely to consist of responses from individuals who are sedentary and take less time to complete EMA surveys. These issues may be addressed during the study design phase by developing prompting strategies that take into account the likelihood of compliance contingent on real-time behaviors and previous response patterns. Using machine learning algorithms (e.g., unsupervised learning) that leverage the computing power of modern smartphones, researchers can implement a new type of context-sensitive EMA prompt that optimizes compliance (CO- EMA; compliance-optimized EMA) (Intille, 2007). For instance, a CO-EMA algorithm may be taught to simultaneously understand that an individual is sedentary during the scheduled prompt time and that his or her prior survey took a greater than average time to complete. To increase the likelihood of compliance, the CO-EMA prompt may then choose to remove a subset of questions to decrease the time it takes to complete a survey. An EMA prompt can then be activated as soon as possible in order to not miss a window of opportunity in sedentary time. In the event that these issues cannot be feasibly addressed at the study design phase, the dissertation suggested that multiple imputation should be used to create feasible values for predictors and outcomes. The dissertation, along with some prior EMA literature, revealed that EMA non- compliance may result in data that is missing not at random (MNAR), because missing data occurs as a function of the measured outcome. Weighting methods that assign probabilities to prompts may also be considered if post-hoc analyses of a sample reveal that individuals are non- 120 compliant due to only a single parameter (e.g., MVPA), although the dissertation indicated that multiple factors likely predict EMA compliance (Horton & Kleinman, 2007). The dissertation found some evidence showing that individuals not willing to select several choices when prompted with multiple selection items were less likely to comply, while also revealing that individuals may actively seek to avoid questions in EMA surveys throughout a measurement period, both by shortening EMA survey completion time and selecting repetitive items. Although multiple selection items can be simplified to limit choices (e.g., “the main activity in the past two hours”), the response patterns imply systematic biases that present threats to the psychometric properties of EMA differentially over time and between individuals. Although strategies exist to examine multilevel reliability using confirmatory factor analytic frameworks with structural equation modeling and multilevel validity using generalized structural equation modeling, EMA studies have often avoided these advanced techniques and presented results without recognizing the violations of assumptions due to clustering (Geldhof, Preacher, & Zyphur, 2014). Moreover, a multilevel implementation of the psychometric gold standard for reliability and validity – the multitrait multimethod matrix (MTMM) – is seldom used in the EMA-measured health behavior literature (Maas, Lensvelt-Mulders, & Hox, 2009). Thus, in circumstances where the research question necessitates branching and complex items, the findings strongly support the need to examine psychometric properties in a multilevel framework. Results implicating reactivity are of special methodological consideration for studies examining physical activity using EMA because (1) studies measuring physical activity tend to be several days long in order to capture an average group of weekdays and weekend days and (2) training periods (i.e., run-in periods) have been proposed as methods to improve compliance and 121 data validity (Stone & Shiffman, 2002). Without a training period, activity patterns indicative of reactivity (i.e., heightened activity in the treatment phase and a pattern of decreasing activity over time) were not found, yet the observed changes in activity levels over time imply multimethod protocols measuring physical activity should take into account the number of days in the study as a covariate to adjust for this association. Studies considering accelerometer-only training periods should be aware that EMA data gathered after the training period will likely be collected with atypical physical activity levels. Lastly, given the changes in physical activity over time coupled with findings from study two that imply EMA data may be most valid at the onset of a study when participants are less fatigued (i.e., assuming variance in item response is an indicator of data validity), the degree to which data can be trusted to be representative of an individual’s daily life may vary over time. Theoretical Implications The behavioral and psychological mechanisms responsible for differences in compliance, fatigue, and reactivity may be elucidated by further research and integration with established health behavior theories. For example, findings on compliance and participant fatigue that indicate a decline in compliance following physical activity and longer-than-average completion time at the prior prompt may be interpreted in the context of Bandura’s Social Cognitive Theory (SCT) (Bandura, 1998). Cognitive factors (e.g., the expectation that the prompt will take too long to complete), behavioral factors or prior experiences (e.g., the last prompt interrupted their activity), and environmental factors (e.g., participants may be embarrassed to answer a prompt in the presence of friends) are components of SCT posited to interact to produce a behavioral outcome such as EMA non-compliance. Similarly, results indicating differential reactivity to multimethod measurement protocols may be refined by adopting Ajzen’s theory of planned 122 behavior (TBP). TBP suggests that the intention to engage in a behavior, in this case physical activity, is a function of social norms (e.g., perception of physical activity as a positive social norm) and intrapersonal attitudes (e.g., participant’s desire to satisfy researchers). Furthermore, the relationship is thought to be moderated by perceived behavioral control (e.g., awareness of a measurement device that can be manipulated) (Madden, Ellen, & Ajzen, 1992). In fact, findings from prior literature show that participants tamper with accelerometers to produce physical activity (Scott et al., 2014). Furthermore, the study on reactivity in Chapter 4 suggested that the presence of both a smartphone and accelerometer may have reduced perceived control of the outcome in participants contingent on the order in which the EMA protocol was deployed. In general, the dissertation highlighted the importance of time-varying processes in physical activity research as participant compliance, fatigue, and reactivity were all found to vary throughout a measurement period. However, health behavior theories were developed outside the context of dynamic systems models that deal with time-varying constructs, before the advent of readily accessible real-time data capture strategies, and prior to the widespread adoption of mobile computing devices (Riley et al., 2011). Therefore, as Riley et. al. describe in a narrative review on the current state of health behavior models, the future of health behavior research relies on the adoption of concepts found in engineering literature such as dynamic system models. These models can integrate findings from literature to develop a series of components that predict behavior. More importantly, these components can be a combination of binary factors (e.g., is the individual alone), continuous variables (e.g., how much sleep did they get last night?), and conceptual outcomes (e.g., how motivated is the individual to exercise?). Agent- based modeling, a novel computational method used to advance communicable disease prevention, has been recently applied to obesity-related health behaviors (Hennessy et al., 2016). 123 These behavioral models offer the first modern attempt at developing a multilevel, multifaceted, dynamic model capable of predicting outcomes in the general population based on factors such as energy-balance, weight, height, age, resting metabolic rate, activity levels, and context. Future EMA-driven behavioral system models can follow this precedent and scale the results to the within-person level, predicting individual behaviors instead of population-level outcomes. The findings from this dissertation may then be added directly to these systems and agent-based models, for instance, to weigh the effect of an EMA-measured physical inactivity controller (i.e., model component) based on the likelihood that the measurement is valid or a result of oversampled sedentary behavior. Intervention Implications Digital health interventions have been widely implemented for various chronic diseases and associated health-related behaviors, including physical activity. Long-term lifestyle management programs, such as those used to prevent diabetes, have found success in the administration of two-stage digital health interventions consisting of an initial intensive behavioral change period that may include contact with real individuals, followed by a continuous maintenance stage using a mobile application (Sepah, Jiang, & Peters, 2015). As similar approaches are adapted to other obesity-related interventions, the findings from this dissertation can be used to (1) identify opportunities to measure outcomes and intervene on individual behavior and (2) deliver fully-automatic interventions and assessment of intervention outcomes for extended periods of time. In terms of intervention delivery, the findings from the first study supported a feasible use of ecological momentary intervention (EMI) to deliver EMA-based intervention components when individuals are sedentary because the study found that these participants were more likely 124 to comply to EMA in such a context. Among individuals who are generally more active, real- time interventions may be skipped entirely to simultaneously minimize burden and reduce risk of interrupting activity. Results from the third study found changes in physical activity similar to an intervention effect (i.e., improved physical activity over time) when EMA plus accelerometer was delivered in the first two weeks. Therefore, interventions with smartphones and external sensors may be able to effectively reduce sedentary behavior and improve physical activity over a period of time, especially if the EMI components are both theory-driven and optimized for mobile devices (e.g., battery life, user interface, etc.). In fact, increases in physical activity were observed even after cessation of the EMA protocol, suggesting that participants may continue to show improvements even after removal of an EMI prompting component. To further extend the intervention without causing undue burden on participants, Chapter 3 suggests that a reduction in the amount of content presented to participants at each intervention point throughout treatment may increase compliance. Similarly, the findings from this dissertation may enhance real-time intervention assessment, consequently enabling the use of microrandomized study design to examine time- varying processes and within-subject effects (Klasnja et al., 2015). Studies assessing interventions may delay assessment prompts until an individual has finished a bout of physical activity to improve the likelihood of prompt compliance, as suggested by Chapter 2. Similarly, such studies may reduce the number of items presented on surveys to minimize the time spent completing a survey if compliance decreases over time. Although supplementary data such as context, intention, or enjoyment may be sacrificed with this approach, the advantage of reduced fatigue may result in improvements in the measurement of the primary outcome (e.g., physical 125 activity). Randomization of ancillary item subsets, such that each survey is limited in size but varied in outcomes may offset this limitation. Future Research Directions Research on systematic compliance, participant fatigue, reactivity, and other special challenges found in the measurement of physical activity using EMA remains in its infancy relative to the vast literature that has applied EMA sampling strategies to assess health behavior outcomes. Yet, as this dissertation has demonstrated, there is a need for research to further elucidate explanatory mechanisms and generalize findings to a wider variety of participant samples and study protocols. While there exists a recognized need for methodological and psychometric research based on evidence for differential compliance, dynamic participant fatigue, and EMA-driven reactivity, researchers interested only in health behaviors and interventions may still contribute to the larger EMA literature by providing additional information in studies. First, investigators can adhere to reporting guidelines, such as those set forth by previous authors, that can be used to establish the strengths and weaknesses of an EMA protocol (Stone & Shiffman, 2002). Second, researchers should examine data for systematic response patterns as a step in reporting descriptive statistics for EMA data to reveal potential limitations in findings. Together, these suggested contributions would provide data for reference in future methodological studies (e.g., compliance rates, prompting sequence, etc.) and expand the EMA methods literature. Additionally, the issues addressed in the dissertation are further complicated by the growing interest in supplementing subjective real-time self-reported data (i.e., EMA) with objective data measured by external sensors, including accelerometers, heart rate monitors, GPS sensors, and more. Therefore, as advancements in technology allow EMA study design to answer 126 an increasingly sophisticated pool of research questions, investigators should remain cautious about potential limitations in novel protocols. For instance, additional external sensors may negatively impact participant fatigue and induce behavioral change that is not representative of a typical day. At the same time, however, the addition of external sensors and the integration of onboard sensors to smartphones may enable researchers to better model systematic non- compliance and predictors of fatigue, and to more closely examine reactivity. Concluding Remarks In review, EMA is a robust sampling strategy used in the measurement of physical activity and can be complemented by additional data obtained from accelerometers and other sensors. Still, limitations involving systematic non-compliance, fatigue, and reactivity are prevalent because of the longitudinal intensive nature of the protocol. However, as described in this dissertation, the limitations can be identified and future studies can leverage these findings to limit weaknesses in real-time data capture protocols. 127 REFERENCES Adams, S. A., Matthews, C. E., Ebbeling, C. B., Moore, C. G., Cunningham, J. E., Fulton, J., & Hebert, J. R. (2005). The effect of social desirability and social approval on self-reports of physical activity. American journal of epidemiology, 161(4), 389-398. Ainsworth, B. E., Haskell, W. L., Herrmann, S. D., Meckes, N., Bassett Jr, D. R., Tudor-Locke, C., . . . Leon, A. S. (2011). 2011 Compendium of Physical Activities: a second update of codes and MET values. Medicine and science in sports and exercise, 43(8), 1575-1581. Ainsworth, B. E., Haskell, W. L., Whitt, M. C., Irwin, M. L., Swartz, A. M., Strath, S. J., . . . Leon, A. S. (2000). Compendium of physical activities: an update of activity codes and MET intensities. Med Sci Sports Exerc, 32(9 Suppl), S498-504. Araujo, D., Davids, K., & Passos, P. (2007). Ecological validity, representative design, and correspondence between experimental task constraints and behavioral setting: Comment on Rogers, Kadar, and Costall (2005). Ecological Psychology, 19(1), 69-78. Atkin, A. J., Gorely, T., Clemes, S. A., Yates, T., Edwardson, C., Brage, S., . . . Biddle, S. J. (2012). Methods of Measurement in epidemiology: sedentary Behaviour. Int J Epidemiol, 41(5), 1460-1471. doi:10.1093/ije/dys118 Bandura, A. (1998). Health promotion from the perspective of social cognitive theory. Psychology and health, 13(4), 623-649. Barry, V. W., Baruth, M., Beets, M. W., Durstine, J. L., Liu, J., & Blair, S. N. (2014). Fitness vs. fatness on all-cause mortality: a meta-analysis. Prog Cardiovasc Dis, 56(4), 382-390. doi:10.1016/j.pcad.2013.09.002 Bassett Jr, D. R., & John, D. (2010). Use of pedometers and accelerometers in clinical populations: validity and reliability issues. Physical therapy reviews, 15(3), 135-142. 128 Behrens, T. K., & Dinger, M. K. (2007). Motion sensor reactivity in physically active young adults. Res Q Exerc Sport, 78(2), 1-8. doi:10.1080/02701367.2007.10762229 Belcher, B. R., Berrigan, D., Papachristopoulou, A., Brady, S. M., Bernstein, S. B., Brychta, R. J., . . . Yanovski, J. A. (2015). Effects of Interrupting Children's Sedentary Behaviors With Activity on Metabolic Function: A Randomized Trial. J Clin Endocrinol Metab, 100(10), 3735-3743. doi:10.1210/jc.2015-2803 Bherer, L., Erickson, K. I., & Liu-Ambrose, T. (2013). A review of the effects of physical activity and exercise on cognitive and brain functions in older adults. J Aging Res, 2013, 657508. doi:10.1155/2013/657508 Biddle, S. J., & Asare, M. (2011). Physical activity and mental health in children and adolescents: a review of reviews. Br J Sports Med, 45(11), 886-895. doi:10.1136/bjsports-2011-090185 Biddle, S. J. H., Gorely, T., Marshall, S. J., & Cameron, N. (2009). The prevalence of sedentary behavior and physical activity in leisure time: A study of Scottish adolescents using ecological momentary assessment. Preventive medicine, 48(2), 151-155. doi:10.1016/j.ypmed.2008.10.025 Bond, D. S., Thomas, J. G., Ryder, B. A., Vithiananthan, S., Pohl, D., & Wing, R. R. (2013). Ecological Momentary Assessment of the Relationship between Intention and Physical Activity Behavior in Bariatric Surgery Patients. International journal of behavioral medicine, 20(1), 82-87. doi:10.1007/s12529-011-9214-1 Börnhorst, C., Huybrechts, I., Ahrens, W., Eiben, G., Michels, N., Pala, V., . . . Bel-Serrat, S. (2013). Prevalence and determinants of misreporting among European children in proxy- reported 24 h dietary recalls. British Journal of Nutrition, 109(07), 1257-1265. 129 Brunswik, E. (1955). Representative design and probabilistic theory in a functional psychology. Psychol Rev, 62(3), 193-217. Burke, V., Beilin, L. J., Durkin, K., Stritzke, W. G., Houghton, S., & Cameron, C. A. (2006). Television, computer use, physical activity, diet and fatness in Australian adolescents. International Journal of Pediatric Obesity, 1(4), 248-255. Cain, A. E., Depp, C. A., & Jeste, D. V. (2009). Ecological momentary assessment in aging research: a critical review. J Psychiatr Res, 43(11), 987-996. doi:10.1016/j.jpsychires.2009.01.014 Cain, K. L., Sallis, J. F., Conway, T. L., Van Dyck, D., & Calhoon, L. (2013). Using Accelerometers in Youth Physical Activity Studies: A Review of Methods. Journal of Physical Activity & Health, 10(3), 437-450. Carels, R. A., Douglass, O. M., Cacciapaglia, H. M., & O'Brien, W. H. (2004). An ecological momentary assessment of relapse crises in dieting. J Consult Clin Psychol, 72(2), 341- 348. doi:10.1037/0022-006X.72.2.341 Case, M. A., Burwick, H. A., Volpp, K. G., & Patel, M. S. (2015). Accuracy of smartphone applications and wearable devices for tracking physical activity data. Jama, 313(6), 625- 626. doi:10.1001/jama.2014.17841 Chmura Kraemer, H., Kiernan, M., Essex, M., & Kupfer, D. J. (2008). How and why criteria defining moderators and mediators differ between the Baron & Kenny and MacArthur approaches. Health Psychology, 27(2S), S101. Choi, L., Liu, Z., Matthews, C. E., & Buchowski, M. S. (2011). Validation of accelerometer wear and nonwear time classification algorithm. Med Sci Sports Exerc, 43(2), 357-364. doi:10.1249/MSS.0b013e3181ed61a3 130 Christensen, T. C., Barrett, L. F., Bliss-Moreau, E., Lebo, K., & Kaschub, C. (2003). A practical guide to experience-sampling procedures. Journal of Happiness Studies, 4(1), 53-78. Clemes, S. A., & Parker, R. A. A. (2009). Increasing Our Understanding of Reactivity to Pedometers in Adults. Medicine and science in sports and exercise, 41(3), 674-680. doi:10.1249/MSS.0b013e31818cae32 Collins, R. L., Morsheimer, E. T., Shiffman, S., Paty, J. A., Gnys, M., & Papandonatos, G. D. (1998). Ecological momentary assessment in a behavioral drinking moderation training program. Exp Clin Psychopharmacol, 6(3), 306-315. Courvoisier, D. S., Eid, M., & Lischetzke, T. (2012). Compliance to a cell phone-based ecological momentary assessment study: the effect of time and personality characteristics. Psychol Assess, 24(3), 713-720. doi:10.1037/a0026733 Curran, P. G. (2016). Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology, 66, 4-19. Curran, P. J., & Bauer, D. J. (2011). The disaggregation of within-person and between-person effects in longitudinal models of change. Annu Rev Psychol, 62, 583-619. doi:10.1146/annurev.psych.093008.100356 Dossegger, A., Ruch, N., Jimmy, G., Braun-Fahrlander, C., Mader, U., Hanggi, J., . . . Bringolf- Isler, B. (2014). Reactivity to accelerometer measurement of children and adolescents. Med Sci Sports Exerc, 46(6), 1140-1146. doi:10.1249/MSS.0000000000000215 Dunn, A. L., Trivedi, M. H., & O'Neal, H. A. (2001). Physical activity dose-response effects on outcomes of depression and anxiety. Med Sci Sports Exerc, 33(6 Suppl), S587-597; discussion 609-510. 131 Dunton, G. F., Atienza, A. A., Castro, C. M., & King, A. C. (2009). Using Ecological Momentary Assessment to Examine Antecedents and Correlates of Physical Activity Bouts in Adults Age 50+Years: A Pilot Study. Annals of Behavioral Medicine, 38(3), 249-255. doi:10.1007/s12160-009-9141-4 Dunton, G. F., Berrigan, D., Ballard-Barbash, R., Graubard, B. I., & Atienza, A. A. (2009). Environmental Influences on Exercise Intensity and Duration in a US Time Use Study. Medicine and science in sports and exercise, 41(9), 1698-1705. doi:10.1249/MSS.0b013e3181a06c9b Dunton, G. F., Dzubur, E., Kawabata, K., Yanez, B., Bo, B., & Intille, S. (2014). Development of a smartphone application to measure physical activity using sensor-assisted self-report. Emerging Technologies to Promote and Evaluate Physical Activity, 88-100. Dunton, G. F., Kawabata, K., Intille, S., Wolch, J., & Pentz, M. A. (2012). Assessing the social and physical contexts of children's leisure-time physical activity: an ecological momentary assessment study. Am J Health Promot, 26(3), 135-142. doi:10.4278/ajhp.100211-QUAN-43 Dunton, G. F., Liao, Y., Dzubur, E., Leventhal, A. M., Huh, J., Gruenewald, T., . . . Intille, S. (2015). Investigating within-day and longitudinal effects of maternal stress on children's physical activity, dietary intake, and body composition: Protocol for the MATCH study. Contemp Clin Trials, 43, 142-154. doi:10.1016/j.cct.2015.05.007 Dunton, G. F., Liao, Y., Intille, S. S., Spruijt-Metz, D., & Pentz, M. (2011). Investigating Children's Physical Activity and Sedentary Behavior Using Ecological Momentary Assessment With Mobile Phones. Obesity, 19(6), 1205-1212. doi:10.1038/oby.2010.302 132 Dunton, G. F., Liao, Y., Kawabata, K., & Intille, S. (2012). Momentary assessment of adults' physical activity and sedentary behavior: feasibility and validity. Front Psychol, 3(260), 260. doi:10.3389/fpsyg.2012.00260 Dunton, G. F., Whalen, C. K., Jamner, L. D., & Floro, J. N. (2007). Mapping the social and physical contexts of physical activity across adolescence using ecological momentary assessment. Annals of Behavioral Medicine, 34(2), 144-153. doi:Doi 10.1007/Bf02872669 Durand, C. P., Andalib, M., Dunton, G. F., Wolch, J., & Pentz, M. A. (2011). A systematic review of built environment factors related to physical activity and obesity risk: implications for smart growth urban planning. Obes Rev, 12(5), e173-182. doi:10.1111/j.1467-789X.2010.00826.x Dzubur, E., Li, M., Kawabata, K., Sun, Y. F., McConnell, R., Intille, S., & Dunton, G. F. (2015). Design of a smartphone application to monitor stress, asthma symptoms, and asthma inhaler use. Annals of Allergy Asthma & Immunology, 114(4), 341-342. doi:10.1016/j.anai.2014.12.017 Esliger, D. W., Copeland, J. L., Barnes, J. D., & Tremblay, M. S. (2005). Standardizing and optimizing the use of accelerometer data for free-living physical activity monitoring. Journal of Physical Activity and Health, 2(3), 366-383. Flegal, K. M., Carroll, M. D., Ogden, C. L., & Johnson, C. L. (2002). Prevalence and trends in obesity among US adults, 1999-2000. Jama-Journal of the American Medical Association, 288(14), 1723-1727. doi:DOI 10.1001/jama.288.14.1723 133 Fogelholm, M. (2010). Physical activity, fitness and fatness: relations to mortality, morbidity and disease risk factors. A systematic review. Obes Rev, 11(3), 202-221. doi:10.1111/j.1467- 789X.2009.00653.x Foley, J. T., Beets, M. W., & Cardinal, B. J. (2011). Monitoring Children's Physical Activity with Pedometers: Reactivity Revisited. Journal of Exercise Science & Fitness, 9(2), 82- 86. Ford, J. K., Maccallum, R. C., & Tait, M. (1986). The Application of Exploratory Factor- Analysis in Applied-Psychology - a Critical-Review and Analysis. Personnel psychology, 39(2), 291-314. doi:DOI 10.1111/j.1744-6570.1986.tb00583.x Freedson, P. S., Melanson, E., & Sirard, J. (1998). Calibration of the Computer Science and Applications, Inc. accelerometer. Med Sci Sports Exerc, 30(5), 777-781. Friedenreich, C. M., Courneya, K. S., Neilson, H. K., Matthews, C. E., Willis, G., Irwin, M., . . . Ballard-Barbash, R. (2006). Reliability and validity of the Past Year Total Physical Activity Questionnaire. Am J Epidemiol, 163(10), 959-970. doi:10.1093/aje/kwj112 Gammon, D., Årsand, E., Walseth, O. A., Andersson, N., Jenssen, M., & Taylor, T. (2005). Parent-child interaction using a mobile and wireless system for blood glucose monitoring. Journal of medical Internet research, 7(5), e57. Gauvin, L., Rejeski, W. J., & Norris, J. L. (1996). A naturalistic study of the impact of acute physical activity on feeling states and affect in women. Health Psychol, 15(5), 391-397. Geldhof, G. J., Preacher, K. J., & Zyphur, M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19(1), 72. 134 Gorely, T., Marshall, S. J., Biddle, S. J. H., & Cameron, N. (2007). Patterns of sedentary behaviour and physical activity among adolescents in the United Kingdom: Project STIL. Journal of behavioral medicine, 30(6), 521-531. doi:10.1007/s10865-007-9126-3 Granholm, E., Loh, C., & Swendsen, J. (2008). Feasibility and validity of computerized ecological momentary assessment in schizophrenia. Schizophrenia bulletin, 34(3), 507- 514. Grenard, J. L., Stacy, A. W., Shiffman, S., Baraldi, A. N., MacKinnon, D. P., Lockhart, G., . . . Reynolds, K. D. (2013). Sweetened drink and snacking cues in adolescents. A study using ecological momentary assessment. Appetite, 67, 61-73. doi:10.1016/j.appet.2013.03.016 Hamar, P., Biddle, S., Soos, I., Takacs, B., & Huszar, A. (2010). The prevalence of sedentary behaviours and physical activity in Hungarian youth. Eur J Public Health, 20(1), 85-90. doi:10.1093/eurpub/ckp100 He, B., Bai, J. W., Zipunnikov, V. V., Koster, A., Caserotti, P., Lange-Maia, B., . . . Crainiceanu, C. M. (2014). Predicting Human Movement with Multiple Accelerometers Using Movelets. Medicine and science in sports and exercise, 46(9), 1859-1866. doi:10.1249/Mss.0000000000000285 Healy, G. N., Wijndaele, K., Dunstan, D. W., Shaw, J. E., Salmon, J., Zimmet, P. Z., & Owen, N. (2008). Objectively measured sedentary time, physical activity, and metabolic risk: the Australian Diabetes, Obesity and Lifestyle Study (AusDiab). Diabetes care, 31(2), 369- 371. doi:10.2337/dc07-1795 135 Hebert, J. R., Clemow, L., Pbert, L., Ockene, I. S., & Ockene, J. K. (1995). Social desirability bias in dietary self-report may compromise the validity of dietary intake measures. International journal of epidemiology, 24(2), 389-398. Hebert, J. R., Ma, Y., Clemow, L., Ockene, I. S., Saperia, G., Stanek, E. J., . . . Ockene, J. K. (1997). Gender differences in social desirability and social approval bias in dietary self- report. American journal of epidemiology, 146(12), 1046-1055. Hedeker, D. (2008). Multilevel models for ordinal and nominal variables Handbook of multilevel analysis (pp. 237-274): Springer. Hekler, E. B., Buman, M. P., Grieco, L., Rosenberger, M., Winter, S. J., Haskell, W., & King, A. C. (2015). Validation of physical activity tracking via android smartphones compared to ActiGraph accelerometer: laboratory-based and free-living validation studies. JMIR mHealth and uHealth, 3(2), e36. Hendelman, D., Miller, K., Baggett, C., Debold, E., & Freedson, P. (2000). Validity of accelerometry for the assessment of moderate intensity physical activity in the field. Med Sci Sports Exerc, 32(9 Suppl), S442-449. Hennessy, E., Ornstein, J. T., Economos, C. D., Herzog, J. B., Lynskey, V., Coffield, E., & Hammond, R. A. (2016). Peer Reviewed: Designing an Agent-Based Model for Childhood Obesity Interventions: A Case Study of ChildObesity180. Preventing chronic disease, 13. Heron, K. E., & Smyth, J. M. (2010). Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. British journal of health psychology, 15(1), 1-39. 136 Heron, K. E., & Smyth, J. M. (2013). Is intensive measurement of body image reactive? A two- study evaluation using Ecological Momentary Assessment suggests not. Body image, 10(1), 35-44. Hillman, C. H., Erickson, K. I., & Kramer, A. F. (2008). Be smart, exercise your heart: exercise effects on brain and cognition. Nature reviews neuroscience, 9(1), 58-65. doi:10.1038/nrn2298 Horton, N. J., & Kleinman, K. P. (2007). Much ado about nothing: A comparison of missing data methods and software to fit incomplete data regression models. The American Statistician, 61(1), 79-90. Hox, J. J., Moerbeek, M., & van de Schoot, R. (2010). Multilevel analysis: Techniques and applications: Routledge. Hufford, M., R,. (2007). Special methodological challenges and opportunities in ecological momentary assessment. The science of real-time data capture: Self-reports in health research, 54-75. Hufford, M. R., Shields, A. L., Shiffman, S., Paty, J., & Balabanis, M. (2002). Reactivity to ecological momentary assessment: An example using undergraduate problem drinkers. Psychology of addictive behaviors, 16(3), 205-211. doi:10.1037//0893-164x.16.3.205 Hufford, M. R., & Shiffman, S. (2003). Assessment methods for patient-reported outcomes. Disease Management & Health Outcomes, 11(2), 77-86. doi:Doi 10.2165/00115677- 200311020-00002 Humpel, N., Owen, N., & Leslie, E. (2002). Environmental factors associated with adults' participation in physical activity: a review. Am J Prev Med, 22(3), 188-199. 137 Husky, M., Olié, E., Guillaume, S., Genty, C., Swendsen, J., & Courtet, P. (2014). Feasibility and validity of ecological momentary assessment in the investigation of suicide risk. Psychiatry research, 220(1), 564-570. Intille, S. S. (2007). Technological innovations enabling automatic, context-sensitive ecological momentary assessment. The science of real-time data capture: Self-reports in health research, 308-337. Jago, R., Baranowski, T., Baranowski, J. C., Cullen, K. W., & Thompson, D. I. (2007). Social desirability is associated with some physical activity, psychosocial variables and sedentary behavior but not self-reported physical activity among adolescent males. Health Education Research, 22(3), 438-449. Kanning, M., & Schlicht, W. (2010). Be Active and Become Happy: An Ecological Momentary Assessment of Physical Activity and Mood. Journal of sport & exercise psychology, 32(2), 253-261. Kikuchi, H., Yoshiuchi, K., Ohashi, K., Yamamoto, Y., & Akabayashi, A. (2007). Tension-type headache and physical activity: an actigraphic study. Cephalalgia, 27(11), 1236-1243. doi:10.1111/j.1468-2982.2007.01436.x King, A. C., Ahn, D. K., Oliveira, B. M., Atienza, A. A., Castro, C. M., & Gardner, C. D. (2008). Promoting physical activity through hand-held computer technology. American journal of preventive medicine, 34(2), 138-142. doi:10.1016/j.amepre.2007.09.025 Klasnja, P., Hekler, E. B., Shiffman, S., Boruvka, A., Almirall, D., Tewari, A., & Murphy, S. A. (2015). Microrandomized trials: An experimental design for developing just-in-time adaptive interventions. Health Psychology, 34(S), 1220. 138 Kodama, S., Tanaka, S., Heianza, Y., Fujihara, K., Horikawa, C., Shimano, H., . . . Sone, H. (2013). Association Between Physical Activity and Risk of All-Cause Mortality and Cardiovascular Disease in Patients With Diabetes A meta-analysis. Diabetes care, 36(2), 471-479. doi:10.2337/dc12-0783 Lepp, A., Barkley, J. E., Sanders, G. J., Rebold, M., & Gates, P. (2013). The relationship between cell phone use, physical and sedentary activity, and cardiorespiratory fitness in a sample of US college students. International Journal of Behavioral Nutrition and Physical Activity, 10(1), 79. Liao, Y., Skelton, K., Dunton, G., & Bruening, M. (2016). A systematic review of methods and procedures used in ecological momentary assessments of diet and physical activity research in youth: an adapted STROBE Checklist for Reporting EMA Studies (CREMAS). Journal of medical Internet research, 18(6). Little, R. J. A., & Rubin, D. B. (1989). The Analysis of Social-Science Data with Missing Values. Sociological Methods & Research, 18(2-3), 292-326. doi:Doi 10.1177/0049124189018002004 Livingstone, M., Robson, P., & Wallace, J. (2004). Issues in dietary intake assessment of children and adolescents. British Journal of Nutrition, 92(S2), S213-S222. Lyden, K., Kozey, S. L., Staudenmeyer, J. W., & Freedson, P. S. (2011). A comprehensive evaluation of commonly used accelerometer energy expenditure and MET prediction equations. European journal of applied physiology, 111(2), 187-201. doi:10.1007/s00421-010-1639-8 Maas, C. J., Lensvelt-Mulders, G. J., & Hox, J. J. (2009). A multilevel multitrait-multimethod analysis. Methodology, 5(3), 72-77. 139 Madden, T. J., Ellen, P. S., & Ajzen, I. (1992). A comparison of the theory of planned behavior and the theory of reasoned action. Personality and social psychology Bulletin, 18(1), 3-9. Malina, R. M. (1996). Tracking of physical activity and physical fitness across the lifespan. Res Q Exerc Sport, 67(3 Suppl), S48-57. Mannini, A., Intille, S. S., Rosenberger, M., Sabatini, A. M., & Haskell, W. (2013). Activity Recognition Using a Single Accelerometer Placed at the Wrist or Ankle. Medicine and science in sports and exercise, 45(11), 2193-2203. doi:10.1249/MSS.0b013e31829736d6 Martikainen, S., Pesonen, A. K., Lahti, J., Heinonen, K., Feldt, K., Pyhala, R., . . . Raikkonen, K. (2013). Higher levels of physical activity are associated with lower hypothalamic- pituitary-adrenocortical axis reactivity to psychosocial stress in children. J Clin Endocrinol Metab, 98(4), E619-627. doi:10.1210/jc.2012-3745 Mata, J., Thompson, R. J., Jaeggi, S. M., Buschkuehl, M., Jonides, J., & Gotlib, I. H. (2012). Walk on the bright side: physical activity and affect in major depressive disorder. J Abnorm Psychol, 121(2), 297-308. doi:10.1037/a0023533 Matevey, C., Rogers, L. Q., Dawson, E., & Tudor-Locke, C. (2006). Lack of reactivity during pedometer self-monitoring in adults. Measurement in physical education and exercise science, 10(1), 1-11. McClain, J. J., Lewin, D. S., Laposky, A. D., Kahle, L., & Berrigan, D. (2014). Associations between physical activity, sedentary time, sleep duration and daytime sleepiness in US adults. Preventive medicine, 66, 68-73. doi:10.1016/j.ypmed.2014.06.003 McClain, J. J., & Tudor-Locke, C. (2009). Objective monitoring of physical activity in children: considerations for instrument selection. J Sci Med Sport, 12(5), 526-533. doi:10.1016/j.jsams.2008.09.012 140 McManus, A. M. (2007). Physical activity - a neat solution to an impending crisis. J Sports Sci Med, 6(3), 368-373. Messier, S. P., Mihalko, S. L., Legault, C., Miller, G. D., Nicklas, B. J., DeVita, P., . . . Loeser, R. F. (2013). Effects of Intensive Diet and Exercise on Knee Joint Loads, Inflammation, and Clinical Outcomes Among Overweight and Obese Adults With Knee Osteoarthritis The IDEA Randomized Clinical Trial. Jama-Journal of the American Medical Association, 310(12), 1263-1273. doi:10.1001/jama.2013.277669 Mitchell, J. T., Schick, R. S., Hallyburton, M., Dennis, M. F., Kollins, S. H., Beckham, J. C., & McClernon, F. J. (2014). Combined ecological momentary assessment and global positioning system tracking to assess smoking behavior: a proof of concept study. J Dual Diagn, 10(1), 19-29. doi:10.1080/15504263.2013.866841 Morren, M., van Dulmen, S., Ouwerkerk, J., & Bensing, J. (2009). Compliance with momentary pain measurement using electronic diaries: A systematic review. European Journal of Pain, 13(4), 354-365. doi:10.1016/j.ejpain.2008.05.010 Nichols, A. L., & Maner, J. K. (2008). The good-subject effect: Investigating participant demand characteristics. The Journal of general psychology, 135(2), 151-166. O'Reilly, G. A., & Spruijt-Metz, D. (2013). Current mHealth technologies for physical activity assessment and promotion. Am J Prev Med, 45(4), 501-507. doi:10.1016/j.amepre.2013.05.012 Ozdoba, R. S., Corbin, C. B., & Lemasurier, G. C. (2004). Does reactivity exist in children when measuring activity levels with unsealed pedometers? Pediatric Exercise Science, 158- 166. 141 Powell, L. M., & Nguyen, B. T. (2013). Fast-Food and Full-Service Restaurant Consumption Among Children and Adolescents Effect on Energy, Beverage, and Nutrient Intake. JAMA pediatrics, 167(1), 14-20. doi:10.1001/jamapediatrics.2013.417 Riley, W. T., Rivera, D. E., Atienza, A. A., Nilsen, W., Allison, S. M., & Mermelstein, R. (2011). Health behavior models in the age of mobile interventions: are our theories up to the task? Translational behavioral medicine, 1(1), 53-71. Sallis, J. F., Buono, M. J., Roby, J. J., Micale, F. G., & Nelson, J. A. (1993). Seven-day recall and other physical activity self-reports in children and adolescents. Med Sci Sports Exerc, 25(1), 99-108. Sallis, J. F., Prochaska, J. J., & Taylor, W. C. (2000). A review of correlates of physical activity of children and adolescents. Med Sci Sports Exerc, 32(5), 963-975. Sallis, J. F., & Saelens, B. E. (2000). Assessment of physical activity by self-report: status, limitations, and future directions. Research quarterly for exercise and sport, 71(sup2), 1- 14. Salvy, S. J., Roemmich, J. N., Bowker, J. C., Romero, N. D., Stadler, P. J., & Epstein, L. H. (2009). Effect of Peers and Friends on Youth Physical Activity and Motivation to be Physically Active. Journal of pediatric psychology, 34(2), 217-225. doi:10.1093/jpepsy/jsn071 Schwarz, G. (1978). Estimating the dimension of a model. The annals of statistics, 6(2), 461- 464. Scott, J. J., Morgan, P. J., Plotnikoff, R. C., Trost, S. G., & Lubans, D. R. (2014). Adolescent pedometer protocols: examining reactivity, tampering and participants' perceptions. J Sports Sci, 32(2), 183-190. doi:10.1080/02640414.2013.815361 142 Sepah, S. C., Jiang, L., & Peters, A. L. (2015). Long-term outcomes of a Web-based diabetes prevention program: 2-year results of a single-arm longitudinal study. Journal of medical Internet research, 17(4), e92. Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological momentary assessment. Annu Rev Clin Psychol, 4, 1-32. Skinner, A. C., & Skelton, J. A. (2014). Prevalence and Trends in Obesity and Severe Obesity Among Children in the United States, 1999-2012. JAMA pediatrics, 168(6), 561-566. doi:10.1001/jamapediatrics.2014.21 Smyth, J. M., Wonderlich, S. A., Heron, K. E., Sliwinski, M. J., Crosby, R. D., Mitchell, J. E., & Engel, S. G. (2007). Daily and momentary mood and stress are associated with binge eating and vomiting in bulimia nervosa patients in the natural environment. J Consult Clin Psychol, 75(4), 629-638. doi:10.1037/0022-006X.75.4.629 Snijders, T. A. (2005). Power and sample size in multilevel linear models. Encyclopedia of statistics in behavioral science. Spook, J. E., Paulussen, T., Kok, G., & Van Empelen, P. (2013). Monitoring dietary intake and physical activity electronically: feasibility, usability, and ecological validity of a mobile- based Ecological Momentary Assessment tool. J Med Internet Res, 15(9), e214. doi:10.2196/jmir.2617 Stone, A. A., Broderick, J. E., Schwartz, J. E., Shiffman, S., Litcher-Kelly, L., & Calvanese, P. (2003). Intensive momentary reporting of pain with an electronic diary: reactivity, compliance, and patient satisfaction. Pain, 104(1-2), 343-351. Stone, A. A., & Shiffman, S. (2002). Capturing momentary, self-report data: a proposal for reporting guidelines. Ann Behav Med, 24(3), 236-243. 143 Stone, A. A., Shiffman, S., Atienza, A. A., & Nebeling, L. (2007). Historical roots and rationale of ecological momentary assessment (EMA). The Science of Real-Time Data Capture: Self-Reports in Health Research: Self-Reports in Health Research, 1. Tabachnick, B., & Fidell, L. (2007). Using Multivariate Statistics. Pearson Education. Boston, MA. Thomas, D. M., Bouchard, C., Church, T., Slentz, C., Kraus, W. E., Redman, L. M., . . . Heymsfield, S. B. (2012). Why do individuals not lose more weight from an exercise intervention at a defined dose? An energy balance analysis. Obesity reviews, 13(10), 835- 847. doi:10.1111/j.1467-789X.2012.01012.x Thomas, J. G., & Bond, D. S. (2015). Behavioral response to a just-in-time adaptive intervention (JITAI) to reduce sedentary behavior in obese adults: Implications for JITAI optimization. Health Psychology, 34(S), 1261. Thomas, J. G., Doshi, S., Crosby, R. D., & Lowe, M. R. (2011). Ecological momentary assessment of obesogenic eating behavior: combining person-specific and environmental predictors. Obesity (Silver Spring), 19(8), 1574-1579. doi:10.1038/oby.2010.335 Troiano, R. P., Berrigan, D., Dodd, K. W., Masse, L. C., Tilert, T., & McDowell, M. (2008). Physical activity in the United States measured by accelerometer. Med Sci Sports Exerc, 40(1), 181-188. doi:10.1249/mss.0b013e31815a51b3 Troiano, R. P., McClain, J. J., Brychta, R. J., & Chen, K. Y. (2014). Evolution of accelerometer methods for physical activity research. Br J Sports Med, 48(13), 1019-1023. doi:10.1136/bjsports-2014-093546 144 Trost, S. G., Loprinzi, P. D., Moore, R., & Pfeiffer, K. A. (2011). Comparison of accelerometer cut points for predicting activity intensity in youth. Med Sci Sports Exerc, 43(7), 1360- 1368. doi:10.1249/MSS.0b013e318206476e Tsai, C. C., Lee, G., Raab, F., Norman, G. J., Sohn, T., Griswold, W. G., & Patrick, K. (2007). Usability and feasibility of PmEB: A mobile phone application for monitoring real time caloric balance. Mobile Networks & Applications, 12(2-3), 173-184. doi:10.1007/s11036- 007-0014-4 Tsenkova, V., Boylan, J. M., & Ryff, C. (2013). Stress eating and health. Findings from MIDUS, a national study of US adults. Appetite, 69, 151-155. doi:10.1016/j.appet.2013.05.020 Tudor-Locke, C. E., & Myers, A. M. (2001). Challenges and opportunities for measuring physical activity in sedentary adults. Sports Med, 31(2), 91-100. van der Ploeg, H. P., Chey, T., Korda, R. J., Banks, E., & Bauman, A. (2012). Sitting time and all-cause mortality risk in 222 497 Australian adults. Arch Intern Med, 172(6), 494-500. doi:10.1001/archinternmed.2011.2174 Vandelanotte, C., Sugiyama, T., Gardiner, P., & Owen, N. (2009). Associations of leisure-time internet and computer use with overweight and obesity, physical activity and sedentary behaviors: cross-sectional study. Journal of medical Internet research, 11(3), e28. Vidmar, S., Carlin, J., Hesketh, K., & Cole, T. (2004). Standardizing anthropometric measures in children and adolescents with new functions for egen. Stata J, 4(1), 50-55. Villablanca, P. A., Alegria, J. R., Mookadam, F., Holmes, D. R., Wright, R. S., & Levine, J. A. (2015). Nonexercise activity thermogenesis in obesity management. Paper presented at the Mayo Clinic Proceedings. 145 Weber, S. J., & Cook, T. D. (1972). Subject effects in laboratory research: An examination of subject roles, demand characteristics, and valid inference. Psychological Bulletin, 77(4), 273. Werle, C. O. C., Wansink, B., & Payne, C. R. (2011). Just thinking about exercise makes me serve more food. Physical activity and calorie compensation. Appetite, 56(2), 332-335. doi:10.1016/j.appet.2010.12.016 WHO. (2010). Global recommendations on physical activity for health. Wilcox, K., Laran, J., Stephen, A. T., & Zubcsek, P. P. (2016). How being busy can increase motivation and reduce task completion time. Journal of personality and social psychology, 110(3), 371. Winkler, E. A. H., Gardiner, P. A., Clark, B. K., Matthews, C. E., Owen, N., & Healy, G. N. (2012). Identifying sedentary time using automated estimates of accelerometer wear time. British journal of sports medicine, 46(6), 436-442. doi:10.1136/bjsm.2010.079699

Abstract (if available)

Abstract This dissertation was primarily methodological in nature and explored limitations found in studies using ecological momentary assessment (EMA) as a sampling strategy to measure physical activity. The dissertation examines 1) the association between energy balance behaviors (i.e., diet and physical activity) and EMA prompt compliance, 2) subjective and objective indicators of participant fatigue and their association with EMA prompt compliance, and 3) changes in participant physical activity (i.e., reactivity) as an effect of EMA protocols. Subjective measures of physical activity and compliance data collected from EMA surveys, supplemented with objective measures of physical activity from accelerometers, were used to test time-varying (e.g., within-person and between-person) systematic non-compliance using multilevel models. Results revealed that participants were less likely to comply to EMA prompts when engaged in physical activity and more likely to comply to EMA prompts while sedentary. Next, the time it took participants to complete surveys decreased, while the variance in survey item response decreased throughout the measurement period. Also, participants who spent more time completing surveys, on average and for each survey, were less likely to comply to EMA prompts. Finally, participants who received an EMA plus accelerometer protocol at the beginning of the study had increasing levels of activity over time, similar to an intervention effect. However, participants with an accelerometer only training period prior to EMA had decreasing levels of activity throughout the study, suggestive of reactivity. These findings indicate that EMA studies measuring physical activity should employ strategies to reduce differential compliance, measure and minimize participant fatigue in real-time, and diminish the effects of reactivity with statistical modeling and optimized study design.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Understanding the dynamic relationships between physical activity and affective states using real-time data capture techniques

PDF

Developing and testing novel strategies to detect inattentive responding in ecological momentary assessment studies

PDF

Sleep health and variability in youth: a real-time data capture study to examine influences on daily dietary intake patterns and longitudinal weight trajectories

PDF

The acute and longitudinal associations between sedentary behaviors, affective states, and emotional disorder symptoms among youth

PDF

Effects of the perceived and objectively assessed environment on physical activity in adults and children

PDF

Distress tolerance and mindfulness disposition: associations with substance use during adolescence and emerging adulthood

PDF

System dynamics of in-home family eating behavior: insights from intensive longitudinal data using Ecological Momentary Assessment and wearable sensors

PDF

Using ecological momentary assessment to study the impact of social-cognitive factors on paretic hand use after stroke

PDF

The Internet activities, gratifications, and health consequences related to compulsive Internet use

PDF

The acute relationship between affective states and physiological stress response, and the moderating role of moderate-to-vigorous physical activity

PDF

Mixed methods investigation of user engagement with a smoking cessation app

PDF

Effects of parent stress on weight-related parenting practices and child obesity risk

PDF

Diffusion of a peer‐led suicide preventive intervention in secondary schools: strategies to increase effectiveness of peer‐led interventions

PDF

Effects of sugar and fiber consumption in minority adolescents and self-tracking as a potential dietary intervention tool

PDF

Effects of near-roadway air pollution exposure on obesity, obesity-related behavior, and neurobehavioral deficits during peripuberty

PDF

Free-living and in-lab effects of sedentary time on cardiac autonomic nervous system function in youth with overweight and obesity

PDF

Cultural risk and protective factors for tobacco use behaviors and depressive symptoms among American Indian adolescents in California

PDF

The effect of an educational fotonovela on the prevention of secondhand smoke

PDF

The role of acculturation in the sun-safe behaviors of US Latino adults in the United States

PDF

Examining tobacco regulation opinions and policy acceptance among key opinion leaders and tobacco retailers in low socioeconomic status African American, Hispanic, and non-Hispanic White communities

Asset Metadata

Creator Dzubur, Eldin (author)

Core Title Understanding the methodological limitations In the ecological momentary assessment of physical activity

School Keck School of Medicine

Degree Doctor of Philosophy

Degree Program Preventive Medicine (Health Behavior Research)

Publication Date 06/13/2017

Defense Date 04/27/2017

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag Compliance,digital health,ecological momentary assessment,experience sampling,fatigue,health behavior,missing data,mobile health,OAI-PMH Harvest,physical activity,reactivity

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Dunton, Genevieve (committee chair), Huh, Jimi (committee member), Leventhal, Adam (committee member)

Creator Email dzubur@usc.edu,eldind@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c40-381394

Unique identifier UC11258083

Identifier etd-DzuburEldi-5392.pdf (filename),usctheses-c40-381394 (legacy record id)

Legacy Identifier etd-DzuburEldi-5392.pdf

Dmrecord 381394

Document Type Dissertation

Rights Dzubur, Eldin

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA