University of Southern California Dissertations and Theses

A mobile app for anxiety: an examination of efficacy and user perceptions

(USC Thesis Other)

A mobile app for anxiety: an examination of efficacy and user perceptions

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Running head: MOBILE APP FOR ANXIETY 1

A Mobile App for Anxiety: An Examination of Efficacy and User Perceptions

Jean Kim, M.A.
University of Southern California
Doctoral Dissertation
Doctor of Philosophy (Clinical Psychology)
December 2017

Faculty of the USC Graduate School:
Steven Lopez, Ph.D. (Chair)
Stanley Huey, Ph.D.
Richard John, Ph.D.
Maryalice Jordan-Marsh, Ph.D.

MOBILE APP FOR ANXIETY 2
Table of Contents
Abstract .......................................................................................................................................... 3
Introduction..................................................................................................................................... 4
Technology and Mental Health........................................................................................... 5
Establishing an Evidence Base for Mobile Apps for Mental Health................................ 10
Examination of Users’ Perceptions................................................................................... 14
Significance and Innovation of the Proposed Research.................................................... 19
Overview........................................................................................................................... 21
Study 1: Pilot Study...................................................................................................................... 21
Methods………………..................................................................................................... 21
Participants and Design......................................................................................... 21
Measures............................................................................................................... 22
Treatment Description.......................................................................................... 28
Data Analyses................................................................................................................... 29
Results............................................................................................................................... 30
Discussion......................................................................................................................... 33
Study 2: Randomized Controlled Trial......................................................................................... 34
Methods………………..................................................................................................... 34
Participants and Design......................................................................................... 34
Measures............................................................................................................... 36
Data Analyses and Hypotheses......................................................................................... 37
Results............................................................................................................................... 40
Discussion..................................................................................................................................... 46
References..................................................................................................................................... 54
Tables............................................................................................................................................ 74
Figures........................................................................................................................................... 86
Appendices.................................................................................................................................. 105

MOBILE APP FOR ANXIETY 3
Abstract

Despite the existence of hundreds of anxiety-related mobile phone applications (apps), there are
few studies to date testing the efficacy of an app for anxiety and comparing it to other platforms
(e.g., mobile phone versus computer-based intervention). Thus, this study sought to develop and
evaluate an app and computer-based website that teach evidence-based cognitive behavioral
therapy techniques to participants with anxiety.
The mobile app and website contain two components: assessment and intervention.
Participants were asked to rate themselves on their level of anxiety and were guided through a
cognitive reappraisal exercise. The website was identical to the mobile app and contained the
same components of assessment and intervention, in order to compare the technology modality
itself and not the content of the intervention.
The app and website were initially examined in a pilot study. Results from 25 college
students with measure-confirmed mild to moderate anxiety indicated that 72% of the
participants’ anxiety ratings decreased or remained the same after using the app. The app and
website also had high levels of perceived convenience and usability. A randomized controlled
trial was then conducted by including a waitlist control group. Results from 135 participants with
measure-confirmed mild to moderate anxiety indicated no interaction of group by time on
anxiety symptoms, although there was a main effect for time on satisfaction with life and
motivation scores. In addition, this study found higher perceived convenience in the app
compared to the website, but no difference in perceived usability. Potential reasons for these
results are discussed.
Keywords: mHealth, mobile application, anxiety, Cognitive Behavioral Therapy

MOBILE APP FOR ANXIETY 4
A Mobile App For Anxiety: An Examination of Efficacy and User Perceptions

Introduction

Mental illness is the leading cause of disability worldwide (World Health Organization,
2017), with one in four American adults suffering from a diagnosable mental disorder in any
given year (Kessler, Chiu, Demler, & Walters, 2005). Of all the types of mental illness, anxiety
disorders are the most common (Kessler et al., 2009). The estimated lifetime prevalence of any
anxiety disorder is over 15%, and the 12-month prevalence rate is more than 10% (Kessler et al.,
2009). Every year, anxiety disorders cost the U.S. more than 42 billion dollars (Greenberg et al.,
1999). People with anxiety disorders are three to five times more likely to go to the doctor and
six times more likely to be hospitalized than those without anxiety disorders (e.g., Kessler &
Greenberg, 2002). In addition, people with diagnosed anxiety have a higher mortality rate than
non-anxious people and a higher incidence of cardiovascular diseases (Agarwal & Agarwal,
2012). Subclinical anxiety, with an estimated 23-month incidence rate of 4.6% (Andrea,
Bultmann, van Amelsvoort, & Kant, 2009), has similar negative consequences, including
neurobiological complications, a higher incidence of cardiovascular events (Agarwal & Agarwal,
2012), reduced cytokine levels and T-cell functioning (Zorilla, Redei, & DeRubeis, 1994),
impaired inhibitory cognitive control (e.g., Ansari & Derakshan, 2011), and impairment in daily
functioning (Karsten, Penninx, Verboom, Nolen, & Hartman, 2013).
Although there are a number of efficacious treatments for anxiety, less than half of those
with anxiety receive treatment (Mackenzie, Reynolds, Cairney, Streiner, & Sareen, 2011).
Structural barriers (e.g., transportation problems, time constraints, cost) are cited as some of the
most prevalent obstacles to psychological care in a broad sample of primary care patients (Mohr
et al., 2006). Thus, there is a need for treatments that are more accessible and convenient.
MOBILE APP FOR ANXIETY 5
Technology and Mental Health

Technology-based interventions provide one possible way to increase accessibility and
convenience. The most common technology-based interventions for mental health are computer-
based interventions, and they have received support in the literature for anxiety disorders. A
recent meta-analysis (Davies, Morriss, & Glazebrook, 2014) has shown that computer-delivered
interventions, in comparison to inactive controls, reduce anxiety in university students (pooled
standardized mean difference = -0.56; 95% CI [-0.77 to -0.35], p < .001). In addition,
Andersson’s (2009) review of the literature on Internet-delivered Cognitive Behavioral Therapy
(CBT) has demonstrated large between-group effect sizes compared to no treatment controls (d’s
= .73 – 1.00) for Panic Disorder, Social Anxiety Disorder, and Posttraumatic Stress Disorder.
Because of the ubiquity of mobile phone use throughout the globe, decreasing costs,
portability, variety of features, and inconspicuousness, mobile phone-based technologies are
theorized to have added benefits compared to computer-based interventions. One look around a
classroom, subway car, or long queue certainly demonstrates the sheer pervasiveness of mobile
phones. Indeed, by the end of 2015, there were about as many mobile phone subscriptions (more
than 7 billion) as individuals in the world (International Telecommunication Union, 2015). In
2015, the number of mobile broadband Internet subscriptions surpassed the number of
households with Internet (International Telecommunication Union, 2015), thus supporting the
prediction that smartphones would overtake personal computers in access to Internet (Gartner,
2010). Mobile phones have already been used in a variety of other healthcare domains, including
chronic illness management (e.g., asthma, diabetes, HIV, hypertension, migraine); lifestyle
change (e.g., diet, exercise, obesity, sexual health); and medication compliance (see Boschen,
2009 & Harrison et al., 2011 for a review). This area is known as mHealth (abbreviated for the
MOBILE APP FOR ANXIETY 6
term “mobile health”)—medical and public health practice supported by mobile devices.
Although initially not as prevalent, the literature indicates that mobile phones are increasingly
being used to deliver treatment in behavioral health care (Luxton, McCann, Bush, Mishkind, &
Reger, 2011).
The research also suggests that mobile phones are more convenient than existing
treatment delivery options. First, there is evidence to suggest that mobile phones are particularly
well-suited for self-monitoring of symptoms (e.g., Riley et al., 2011). Self-monitoring has been
shown to serve as an intervention itself, improving mood and behavior. There are also a number
of studies showing that self-monitoring accounts for a significant portion of the variance in
behavior change outcomes (e.g., Michie, Abraham, Whittington, McAteer, & Gupta, 2009).
Traditionally, monitoring has been done on paper, which requires people to remember to carry
these diaries with them throughout their day. This often leads to poor adherence, memory errors,
and retrospective completion (Stone, Schiffman, Schwartz, Borderick, & Hufford, 2002). More
recently, self-monitoring electronically has been studied and found to improve adherence and
compliance rates significantly. For example, computer-based monitoring has been shown to have
compliance rates as high as 94% (Stone et al., 2002). However, even this type of electronic
journaling is perceived as inconvenient by some, requiring users to be at their computers at
particular times (Proudfoot et al., 2010). Mobile phones offer the potential of greater frequency
of use, in the context of the behavior in question, as cell phones are typically location
independent and nearly always turned on (Riley et al., 2011). Past work has even shown that
mobile methods are preferred for monitoring, especially in younger patients (e.g., Matthews,
Doherty, Coyle, & Sharry, 2008). Furthermore, self-monitoring via mobile phones can be
bolstered by short message service (SMS) to serve as reminders.
MOBILE APP FOR ANXIETY 7
There have been a number of reviews on EMA or experience sampling method (e.g.,
Heron & Smyth, 2010) that have led to recommendations for assessing symptoms. First, it is
recommended that retrospective reporting be avoided and that self-reported information occur as
close as possible to the time in which it occurred (Heron & Smyth, 2010). Retrospective recall is
susceptible to numerous, systematic biases (Ebner-Priemer & Trull, 2009), including the
affective valence and mood congruent memory effect (Kilhstrom, Eich, Sandbrand, & Tobias,
2000), duration neglect, and peak-end rule (Kahneman, Fredrickson, Schreiber, & Redelmeier,
1993). For example, people tend to remember salient events and recall events according to what
they now know about the event (Heron & Smyth, 2010). Thus, medicine, pharmacology, and the
pharmaceutical industry, as per the guidelines of the U.S. Food and Drug Administration, have
turned to real time self-reports of symptomatology, with the method of choice being electronic
diaries, or computer-assisted methodology to assess self-reported symptoms, behaviors, or
physiological processes, of real-time subjective states (Ebner-Priemer & Trull, 2009). The biases
associated with retrospective recall are significantly reduced with EMA, as people report their
current or recent states (Heron & Smyth, 2010). Second, it is recommended that assessment
occur in natural settings to maximize generalizability and ecological validity. Finally, multiple
assessments over time are recommended, as this lends to the exploration of temporal
relationships between variables (Heron & Smyth, 2010).
There is also a significant literature base supporting mobile phones’ suitability for
psychotherapeutic intervention. For anxiety, cognitive behavioral therapy (CBT) is often
considered the psychotherapy treatment of choice and has received the most empirical support
for being highly effective in treating anxiety (Craske, 2010). Meta-analyses have demonstrated
the superiority of CBT over waitlist, expectancy, and attention control conditions, as well as
MOBILE APP FOR ANXIETY 8
psychodynamic therapy (Butler, Chapman, Forman, & Beck, 2006; Hofmann & Smits, 2008).
The central treatment process in CBT is the reduction of anxiety-related thoughts and beliefs,
which leads to fear and symptom reduction (Craske, 2010). A number of CBT studies have
found that changes in cognition through reappraisal (e.g., changes in negative social cost ratings,
probability of a feared social outcome, panic-related cognitions, & negative beliefs about
anxiety-related sensations) predict or mediate symptom improvement (Arch, Wolitzsky-Taylor,
Eifer, & Craske, 2012). However, a common complaint by CBT therapists is client forgetfulness
in practicing cognitive therapy techniques (Beck, 2011). Mobile phones have the advantage of
sending SMS reminders at scheduled times to complete thought record and cognitive reappraisal
activities on that same device, wherever the person is. Mobile phones have been described as a
viable option for the delivery of simple self-management strategies and in particular have
received support for the delivery of CBT activities (Bang, Timpka, Eriksson, Holm, & Nordin,
2007). Specifically, mobile apps have been shown to be preferred for displaying a large amount
of information, compared to other mobile modalities such as text messaging. In addition, apps
are often cited as being more convenient, faster, and easier to browse than a website, including
mobile websites (Compuware, 2013).
Cognitive restructuring is a technique that is recommended for most patients (Dobson &
Dobson, 2016). The first step for cognitive restructuring is the identification of unhelpful or
dysfunctional thoughts. This involves metacognition (Wells, 2002), a technique that requires
training and practice. Cognitive restructuring has one identify the situation, feelings, and
behaviors associated with anxiety. “Many cognitive-behavioral therapists use the daily
Dysfunctional Thoughts Record (DTR; A. T. Beck et al., 1979; J. S. Beck, 1995). Indeed, the
DTR has almost become the defining characteristic of cognitive assessment, and numerous
MOBILE APP FOR ANXIETY 9
versions of the DTR have been created over time. This method is no doubt an effective strategy
for collecting thoughts, and we have used it extensively with clients . . . Once you have identified
a cognitive target for intervention, you can use three general questions to try to modify a
negative thought: 1. What is the evidence for and against this thought? 2. What are the
alternative ways to think in this situation? 3. What are the implications of thinking this way?”
(Dobson & Dobson, 2016, pp. 166-172). Cognitive restructuring is most successful when a
cognitive distortion, a misperception or distortion, exists and is identified. The main methods for
cognitive restructuring are: examining evidence related to negative thoughts; identifying
unrealistic expectations; examining attributional biases; reattributing causes using pie charts;
change labeling; changing dichotomous thinking into graduated thinking; generating and
evaluating alternative thoughts; and cultivating positive thoughts.
There is an extensive literature base on the efficacy of cognitive restructuring for anxiety.
It has been shown to reduce symptoms of obsessive-compulsive disorder (e.g., Whittal,
Robichaud, Thordarson, & McLean, 2008), panic disorder (e.g., Bouchard et al., 1996), and
social anxiety (Clark et al., 2006). Cognitive restructuring, compared to exposure therapy, is
particularly effective for immediate and long-term outcomes for those with social anxiety
disorder. For treating Generalized Anxiety Disorder (GAD), cognitive restructuring alone has not
been examined. However, studies have shown that CBT for GAD is superior to relaxation and
pharmacotherapy (Fisher, 2006; Mitte, 2005). With posttraumatic stress disorder (PTSD), it is
debated whether or not cognitive restructuring offers additional reduction of symptomatology,
compared to exposure therapy. While some researchers (e.g., Longmore & Worrell, 2007) assert
that cognitive restructuring may be unnecessary for treating PTSD, reviews by Ponniah and
MOBILE APP FOR ANXIETY 10
Hollon (2009) and Hassija and Gray (2010) have demonstrated that cognitive restructuring is
effective for PTSD, and comparable to exposure therapy.
Establishing an Evidence Base for Mobile Apps for Mental Health

To date, there are a fair amount of feasibility studies that have demonstrated the
acceptability and potential for the use of mobile phones in mental health care (e.g., Christensen,
Griffiths, & Korten, 2002; Graham, Franses, Kenwright, & Marks, 2000; Proudfoot et al., 2010).
Recent reviews (e.g., Boschen, 2009; Mohr, Burns, Schueller, & Clarke, 2013) have also
supported the feasibility of mobile phone use for the assessment and intervention of mental
health problems. In addition to the substantial amount of work demonstrating the rationale,
acceptability, and feasibility for mHealth, seven studies have examined the use of non-app
mobile phone features (i.e., serious games, photos, phone calls, interactive voice response,
virtual reality, & mobile-phone based sessions) for the management of anxiety. These studies
were found by conducting a search on PsycINFO and PubMed in November 2014 using the
following search terms: cell, cellular, or mobile; phone or telephone; psychol*, psychiat*, and
psychother*. No restrictions were placed on publication date. The search produced 1,630 articles
on PsycINFO and 957 articles on PubMed, and the search was supplemented by reviewing the
reference sections of identified studies. Each abstract was examined for relevance. Only
empirical articles in English pertaining to the use of mobile phones as a psychotherapeutic
intervention, with adult participants with anxiety were included. Exclusion criteria included
articles describing the development, feasibility, and acceptability of mHealth. This was because
of this study’s focus on the efficacy of the use of mobile phones as a psychotherapeutic
intervention for anxiety. Additional exclusion criteria included review articles, study protocol
descriptions, and commentary on articles, as these did not provide new data. The results for these
MOBILE APP FOR ANXIETY 11
studies were mixed, with two single case-design studies showing positive results in anxiety
reduction (Botella et al., 2011; Eonta et al., 2011); two controlled studies reporting mostly
positive results in anxiety reduction in the active treatment groups, with a significant reduction in
symptoms in the waiting list control group on one measure (Gorini et al., 2010; Pallavicini,
Algeri, Repetto, Gorini, & Riva, 2009); and three uncontrolled studies reporting mixed results
(Flynn, Taylor, & Pollard, 1992; Possemato et al., 2012; Vogel et al., 2012). It should be noted
that these studies suffered from a number of methodological limitations. First, only two of these
studies included a control group, and thus symptom changes cannot be definitively attributed to
the intervention. Second, sample sizes were small in these studies, the largest one being 46
participants. This suggests insufficient power, especially if the intervention only had a small to
medium effect size. To date, there are very few published studies that have tested the efficacy of
a mobile app for anxiety. This is alarming given the potential added benefits of mobile apps and
that there are already more than 800 anxiety-related apps in the iTunes app store alone.
Another literature review of PsycINFO, PubMed, the Journal of Medical Internet
Research (JMIR), and Google Scholar was conducted in February and August 2017 using the
following search terms: cell, cellular, or mobile; phone or telephone; app or application; and
anxi*. In addition, recent reviews (e.g., Sucala, 2017; Ameringen, Turna, Khalesi, Pullia, &
Patterson, 2017) were examined for additional studies. Only empirical articles in English
pertaining to the use of mobile phones as a psychotherapeutic intervention, with adult
participants with anxiety were included. Of the mobile apps designed to help with anxiety
disorders, five were found to have been studied for their efficacy. First, Enock, Hoffman, and
McNally (2014) found that participants using the AntiAnxiety app had lower scores on social
anxiety measures than waitlist participants, although there were no differences in scores
MOBILE APP FOR ANXIETY 12
compared to the control task participants. Second, Dagoo et al. (2014) evaluated mobile-CBT
(mCBT) compared to self-guided interpersonal psychotherapy (mIPT) for Social Anxiety
Disorder. They found that both groups showed improvement in social anxiety symptoms, with a
large within groups effect size in the mCBT group (d = 0.99) versus a small effect size in the
mIPT group (d = 0.43). Unfortunately in this study, there were technical issues, and 50.05% of
the participants used the intervention on a computer. Thus, it remained to be studied the effects
of a mobile intervention by itself. Third, Dennis and O’Toole (2014) found that one session of
attention bias modification delivered via a “game”-like format (i.e., gamified) on the Personal
Zen app, relative to the placebo training, decreased levels of anxiety. Fourth, Pham, Khatib,
Stanfield, Fox, and Green (2016) found that participants using their “Flowy” mobile phone game
app experienced better quality of life than waitlist participants, but there were no significant
differences in anxiety symptoms. Finally, the Worry Knot app (Mohr et al., 2017) teaches
emotion regulation and exposure techniques via lessons, distractions, a tool to address specific
problems that the user cannot stop thinking about, and statistics on progress. A pilot study of the
app, which included coaching on the use of the app via phone calls and texts, found that app
users experienced a significant decrease in GAD-7 scores (p < .001) after eight weeks of use.
However, this study did not include a waitlist or comparison group.
Published studies on mobile apps for other mental disorders also provide preliminary
evidence to support apps as a possible mode of delivery for mental health treatment. For
example, Burns et al.’s (2011) uncontrolled study examined a mobile app for Major Depressive
Disorder and found a decrease in depressive symptoms (Patient Health Questionnaire-9; t = 7.02,
p < .001), anxiety symptoms (Generalized Anxiety Disorder-7; t = 4.59, p < .001), and Major
Depressive Disorder diagnoses (z = 2.15, p = .03). Their app included mood ratings and context
MOBILE APP FOR ANXIETY 13
sensing, to suggest behavioral activation strategies as needed. Also, Watts et al. (2013) found
that cognitive behavioral therapy for Major Depression delivered via a mobile app was
associated with decreased depressive symptoms, as effective as the computer-based intervention.
Their app, called the Get Happy Program, told the story of a woman with depression who learned
to manage her symptoms, and participants practiced applying the same techniques to their lives
through homework activities and review of the lessons. Finally, Rizvi, Dimeff, Skutch, Carroll,
and Linehan’s (2011) Dialectical Behavior Therapy (DBT) Coach app for borderline personality
disorder and substance use disorder was associated with reductions in emotional intensity and
urge to use substances, although this study included no control group. This app included ratings
of daily emotional intensity and urges to use substances on a 0-10 scale and coaching in the use
of opposite action, a DBT technique. Thus, there is preliminary evidence to support the use of
mobile apps as a viable medium for introducing therapeutic techniques to decrease psychological
symptoms, but limited research on the efficacy of apps for anxiety in particular. A limitation of
these app studies is that very few included a control group (although one did include a computer-
based comparison group). Thus, in these studies, it is unknown if it was the intervention that
produced change in symptoms, or if it was a function of regression to the mean or other factors
that led to change.
Another limitation of past mHealth studies is a lack of comparison with existing methods
for health care delivery (Atienza & Patrick, 2011). This is a significant issue, as we know that
computer-based CBT is effective, compared to inactive control groups. Thus, it would be
premature to invest in mHealth if there is first no solid evidence to support that they are more
efficacious than no treatment control groups and at least as effective as existing interventions. To
date, only one study has examined both an mHealth and computer-delivered intervention. Watts
MOBILE APP FOR ANXIETY 14
et al. (2013) found no differences between the mobile and computer groups in depressive
symptom scores. However, their total sample size was 35 participants, which suggests low
power, unless the effect size was large (Cohen’s d of at least .8 for .8 power). The general lack of
attention in the electronic health literature to comparative efficacy highlights a need for
randomized controlled trials of these apps, with sufficient control and comparison groups (Price
et al., 2014).
Examination of Users’ Perceptions
Studying user perceptions has also been stated as essential for the adoption of these
technologies (e.g., Atienza & Patrick, 2011). In addition, studying perceptions is beneficial for
improving future versions of the interventions. In the technology and usability literature,
perceived convenience is increasingly being examined. It is also one of the most often cited
advantages of mHealth (e.g., Watts et al., 2013). An examination of perceived convenience has
its roots in the Theory of Reasoned Action (TRA; Ajzen & Fishbein, 1980, Fishbein & Azjen,
1975). This theory asserts that behavioral intention (e.g., to use the technology-based
intervention) is a function of attitudes and subjective norms. Put simply, one’s voluntary
behavior can be predicted by his or her attitude towards the behavior and how he or she thinks
others would view them if they engaged in that behavior. In 1989, Davis, Bagozzi, and Warshaw
adapted TRA to apply this model to technology use and thus created the Technology Acceptance
Model (TAM). This model posits that behavioral intention is a function of the specific attitudes
of perceived usefulness and ease of use. Perceived usefulness is defined as the degree to which
users perceive that learning effectiveness can be increased by the technology, and perceived ease
of use is the degree to which users perceive that the technology is easy to use. A number of
studies on technology adoption behavior have used TAM to examine, for example, participants’
MOBILE APP FOR ANXIETY 15
IT use (e.g., Ahn, Ryu, & Han, 2007; Lai & Li, 2005), e-learning (e.g., Lee, 2010; Lin, 2011),
and mobile learning (e.g., Park, Nam, & Cha 2011; Wang, Wu, & Wang, 2009), with these
studies finding that perceived usefulness and ease of use are positively associated with
behavioral intention to use the technology.
In addition to perceived usefulness and ease of use, perceived convenience has been
increasingly examined. Although usefulness and ease of use from TAM are positively associated
with behavioral intention, perceived convenience significantly predicts behavioral intent and has
been cited as being influential in the acceptance and use of new technology (Chang, Tseng,
Liang, & Yan, 2013). Perceived convenience is defined as the user’s perception of how
convenient the technology is in terms of time, place, and process during which a task is
accomplished (Yoon & Kim, 2007). Perceived convenience has been found to have a positive
relationship with consumers’ behavioral intention to online shop (Gupta & Kim, 2007), to use
Radio Frequency Identification technology (Hossain & Prybutok, 2008), and to use mobile
English learning (Chang et al., 2013). However, perceived convenience has not yet been studied
systematically as it relates to mHealth interventions.
When convenience is discussed in the mHealth literature, it is largely based in theoretical
discussions and qualitative data. For example, it is often described that one of the greatest
strengths of mobile technology is that its “strong portability and mobility facilitates learning
processes by not only breaking down time and geographical barriers but also bringing substantial
convenience, immediacy and suitability” (Chang et al., 2013, p. 373). In comparison to
traditional methods of healthcare delivery, mHealth is stated to address barriers of time and
accessibility, delivering care in a more convenient way on a device most Americans own and
carry with them. A mobile app for anxiety is thus an example of a disruptive innovation, a new
MOBILE APP FOR ANXIETY 16
technology that uses what has been learned from evidenced-based intervention science in novel
ways to increase impact. Bower and Christensen (1995) first described the concept of disruptive
innovation as a product or service that transforms an existing market by introducing simplicity,
convenience, accessibility, and affordability where complication and high costs are the status
quo. The intent of disruptive innovations is to provide more affordable and conveniently
accessible options to increase access to care.
There is preliminary qualitative evidence that suggests that mobile technology is
perceived as convenient. Qualitative studies exploring community attitudes (e.g., Watts et al.,
2013) have demonstrated that mobile apps are a convenient and acceptable way of capturing data
in context. In a qualitative study of a text messaging intervention for smoking cessation, the
convenience of not having to “carry anything extra” and being able to receive support “wherever
or whenever” was viewed as a significant advantage of mHealth (Naughton, Jamison, & Sutton,
2013). Not having “to be somewhere at a certain time and sit and listen to someone talking” was
also seen as an advantage. Some even felt that the convenience of being able to read text
messages on their own time and to be able to read through them quickly meant that they could
“take it in more”. Mobile phones also have an advantage over fixed desktop or laptop computers,
which can be perceived as inconvenient and often do not offer the ability to use the intervention
in context (Watts et al., 2013), lending more support for the convenience of modalities like
mobile apps.
On the other hand, there is evidence to suggest that not all users perceive a mobile app as
being convenient. Certainly, apps are notorious for their wide variability in usage. Some app
users are highly active, but some stop using them early on. For example, out of over 1,100
diabetes apps available, there are few sustained users of the majority of these apps (Malvey &
MOBILE APP FOR ANXIETY 17
Slovensky, 2014). It is unknown what accounts for the wide variability in actual use of mobile
apps, but measuring perceived convenience may aid in the understanding of this phenomenon.
Although many authors state that mobile-based interventions are preferred in young participants,
it is unknown if people of all ages will prefer a mobile app and perceive it as convenient. In
addition, small screens and keyboards have been highlighted as a disadvantage of mobile phones
by researchers such as Thornton and Houser (2002). Since some prefer to type on a full-sized
keyboard, it is possible that for them, a computer-based intervention will be more convenient
than an app. Furthermore, given that computer-based monitoring already has shown compliance
rates as high as 94% (Stone et al., 2002), this suggests that computer-based interventions may be
at least minimally convenient. It is unknown if mobile apps will actually have even greater
perceived convenience. One study to date has compared an app and website modality. Watts et
al. (2013) created and tested a mobile-based version of their computer intervention because they
believed mobile phones are viewed as more convenient than computers. They found there were
no differences between the two modalities of the intervention in adherence, homework
completion, and effort. However, they did not measure the more proximal variable they believed
varied between the two modalities (i.e., convenience) and therefore were not able to compare the
two groups on perceived convenience. Thus, it remains unknown how convenient mobile apps
will be perceived in relation to existing interventions like computer-based interventions,
especially when measured quantitatively.
Although convenience has been examined in the usability literature as being key to
technology adoption, one of the most widely identified variables related to behavioral change in
other literatures is intrinsic motivation. Self-Determination Theory (SDT; Deci & Ryan, 1985,
2010; Gagné & Deci, 2005; Ryan & Deci, 2000) posits that intrinsic motivation, derived from
MOBILE APP FOR ANXIETY 18
enjoyment of the delivery method and satisfaction from it (Ryan & Deci, 2000), is what
increases the likelihood of the behavior (e.g., technology adoption). Six dimensions comprise
intrinsic motivation, as often measured by the Intrinsic Motivation Inventory (IMI; Ryan, 1982):
interest/enjoyment, perceived competence, effort/importance, pressure and tension, perceived
choice, and value/usefulness. SDT has served as the basis of understanding the adoption of
traditional (i.e., face-to-face) therapeutic interventions, mass media and social marketing
strategies, and eHealth web and computer interventions (Riley et al., 2011). SDT has been
applied to other technology-based interventions as well, in particular video game-based
behavioral change. Ryan et al. (2006) posited that video games tend to be high in
interest/enjoyment, and engaging with these fun video games is intrinsically motivating. Ryan et
al. (2006) also applied a subtheory of SDT called Cognitive Evaluation Theory (CET), which
states that competence and autonomy foster intrinsic motivation, further increasing learning and
behavioral change. They found that as participants advance through games, they gain a sense of
competence and autonomy, which enhances motivation to continue playing games and
experience positive outcomes like short-term well-being. Similar to perceived convenience
though, there is a dearth of research that measures mHealth app users’ motivation, especially in
comparison to existing computer-based interventions.
In summary, it is known that intrinsic motivation is a key component related to
behavioral change and positive outcomes, at least in face-to-face psychotherapy, web and
computer interventions, and even video game interventions. It is also known that perceived
convenience, one of mobile technology’s most often cited benefits, is key to technology
adoption. However, perceived convenience is infrequently measured, and if it is, it is generally
done qualitatively in the development stage or after the study has been completed. Past studies
MOBILE APP FOR ANXIETY 19
have not systematically and quantitatively measured these perceptions of new technological
interventions. In addition, user perceptions of convenience and intrinsic motivation have not
been examined in comparison to existing modalities of intervention, such as computer-based
interventions.
Significance and Innovation of the Proposed Research

Although there are a number of studies examining mHealth for anxiety disorders (e.g.,
Botella et al., 2011; Eonta et al., 2011; Flynn, Taylor, & Pollard, 1992; Gorini et al., 2010;
Pallavicini, Algeri, Repetto, Gorini, & Riva, 2009; Possemato et al., 2012; Vogel et al., 2012), a
literature review of PsycINFO, the Journal of Medical Internet Research (JMIR), and Google
Scholar publications yielded very few studies examining the efficacy of a mobile app for anxiety
disorders. The following study was conducted in order to test a mobile app for anxiety, which
uses the CBT technique that has gained the most empirical support for symptom improvement,
cognitive reappraisal. Testing efficacy is key because a user “who has an unsatisfactory
experience with an ineffective app may be less likely to seek further treatment that stands to
alleviate their symptoms”, or they may feel that additional help is not necessary because of
beliefs that the ineffective app is sufficient treatment (Price et al., 2014, p. 433). This study
addressed past methodological limitations by including a no-treatment waitlist control group and
a computer-based website comparison group. Including a waitlist control group strengthened the
design by being in a better position to rule out regression to the mean and variability of mood
across time. In addition, to directly examine benefits due to a mobile app modality in comparison
to an already existing intervention modality, this study compared the efficacy of the CBT
technique delivered via an app versus a website. This is important because we know that
computer-based CBT is effective, compared to being on a waitlist. Although mHealth has gained
MOBILE APP FOR ANXIETY 20
increasing popularity, it hasn’t yet consistently been demonstrated to be as effective or more
effective than existing interventions. A demonstration of the efficacy of an app compared to a
computer-based intervention could inform whether to invest in the dissemination of mobile
mental health apps. If this study demonstrates that mobile apps are effective, this could be
another step towards feasible solutions to the access problem in mental healthcare.
Additionally, user perceptions towards mHealth apps (i.e., perceived convenience,
intrinsic motivation) have not been quantitatively examined in the literature across the total time
in using the intervention and in comparison to existing modalities of intervention. While it is
certainly imperative to examine the efficacy of an app, it should also be a priority to assess
attitudes towards the app, such as perceived convenience, as that is one of the most often cited
benefits of mobile technology for mental health and one of the greatest barriers to seeking and
receiving treatment, along with intrinsic motivation, a known mechanism of behavioral change.
If user perceptions of convenience, for example, are significantly lower in one modality, this
could be examined and improved for future versions of the intervention. In addition, it could lend
to the development of dissemination guidelines. For example, it could very well be that the
mobile app is largely perceived as more convenient and motivating than the website, thus
lending support for the use of mobile apps with people suffering from these disorders. On the
other hand, it is also possible that there is wide variability in perceptions of convenience and
motivation, so that some people perceive a mobile app more positively, whereas others perceive
a computer-based website more positively. In this case, user fit might be the most appropriate
and effective. Measuring perceptions of the intervention modalities will lend information on how
to improve the interventions and to whom to disseminate these interventions.

MOBILE APP FOR ANXIETY 21
Overview

There is qualitative evidence to support mobile apps as a better delivery method of
psychological treatment than existing means, possibly due to convenience (e.g., Naughton et al.,
2013). This study was conducted in two parts. A pilot study first examined the basic usability
and feasibility of the mobile app and computer-based website interventions. The second part
consisted of a randomized controlled trial (RCT) to assess the efficacy of the app, compared to
the website and a waitlist, for reduction in anxiety symptoms. In addition, the RCT included an
examination of attitudes towards the mobile app and computer-based intervention as secondary
outcomes.
Study 1: Pilot Study

Methods

Participants and Design. Fifty-two college student volunteers were recruited at a large
private university in California. These participants provided their informed consent to
participate, filled out the Beck Anxiety Inventory (BAI), and then were interviewed to further
determine eligibility. Inclusion criteria were as follows: 1) self-reported and measure-confirmed
mild or moderate anxiety (BAI score = 8-25), 2) access to a smartphone device with an Internet
data connection and iOS or Android operating system and a computer with Internet access, 3)
ownership of current, working email address, 4) fluency in English, and 5) 18 years of age or
older. Potential participants at risk for suicide were excluded, as were those with psychosis,
bipolar disorder, eating disorder, and substance abuse or dependence based on the Mini
International Neuropsychiatric Interview (M.I.N.I.) Version 7.0.0 (Sheehan, 2015). Additional
exclusion criteria included involvement in any other form of psychological treatment and
unstable dosage of any psychotropic medication for the following six weeks. Should a potential
MOBILE APP FOR ANXIETY 22
participant have been at risk for suicide, he or she would have been advised to seek help from
their general practitioner, local emergency room, or an official emergency number, and the
relevant information and telephone numbers would have been provided. Those with severe
anxiety (i.e., BAI scores greater than 25) were also excluded from this study, as this was a new
experimental intervention. Those that scored in the severe range on the BAI and those that met
criteria for disorders that excluded them from the study were given referrals for nearby therapy
clinics.
Twenty-five participants met eligibility criteria. The size of this pilot sample exceeded
the general guideline of 10% of the final study size (e.g., Lackey & Wingate, 1998). Eligible
participants completed online questionnaires of demographic information and frequency of
mobile phone and computer use; anxiety symptoms; secondary outcomes of depression, work
and social functioning, and satisfaction with life; and user perceptions of convenience and
motivation. Time to complete these questionnaires was captured with the Qualtrics software.
Participants were randomly assigned to one of the two active conditions (mobile app or
computer-based website) and were instructed to complete one entry. They rated themselves on a
1-10 scale of anxiety within the intervention (app or website) before and after completing an
entry. After completion of the entry, participants answered online a basic usability questionnaire
and open-ended questions to gain feedback on the interventions. Participants were compensated
with Subject Pool points equating to one hour.
Measures. The following measures were used in both the pilot study and RCT. The
purpose of including all of the measures in the pilot study was to determine the feasibility and
acceptability of the questionnaires. In addition to the following measures, demographic
information was collected.
MOBILE APP FOR ANXIETY 23
Demographic Information. Participants were asked to identify their age and gender.
They were also asked the highest level of education they have completed: grade school/middle
school, some high school, high school diploma/GED, some college, associates degree, bachelors
degree, and post graduate degree. These were further collapsed into two categories for future
Chi-square analyses: some or all of high school completed and at least some college completed.
Participants were also asked to identify their family’s annual income with the following options:
less than $4,999, 5,000-14,999, 15,000-24,999, 25,000-34,999, 35,000-44,999, 45,000-54,999,
55,000-64,999, 65,000-74,999, and 75,000 or more. These were further collapsed into two
categories: less than $75,000 and $75,000 or more. Participants were additionally asked to
identify their ethnicity, with options including Asian-American, American Indian, Black (non-
Hispanic, including African American), Mexican-American/Chicano, Hispanic/Latino (non-
Mexican American), Pacific Islander, White (non-Hispanic/Latino), and Other (please specify).
Furthermore, participants were asked to identify how many hours per day they use their mobile
phone and computer.
Mini International Neuropsychiatric Interview Version 7.0.0 (M.I.N.I.; Sheehan, 2015).
The M.I.N.I. was used to determine whether interested participants met exclusion criteria for this
study. The M.I.N.I. is the “gold standard” for a brief diagnostic interview of current and lifetime
DSM disorders, for what was previously termed Axis 1 disorders. It has high interrater reliability
(κ = 0.88-1.00) and takes only about 15 minutes to administer.
Beck Anxiety Inventory (BAI; Beck, Epstein, Brown, & Steer, 1988; BAI during pilot
study,  = .60; BAI during RCT,  = .84). The BAI was used to confirm inclusion criteria of
mild to moderate anxiety. It is a 21-item self-report measure that includes cognitive, behavioral,
and physiological symptoms of anxiety (e.g., unable to relax, heart pounding or racing).
MOBILE APP FOR ANXIETY 24
Respondents rate the symptoms using a four-point Likert scale of the extent to which they were
bothered by each symptom over the past week (0 = not at all; 1 = mildly, it did not bother me
much; 2 = moderately, it was very unpleasant but I could stand it; 3 = severely, I could barely
stand it). The BAI has a maximum score of 63, with a score of 0-7 indicating minimal anxiety, 8-
15 indicating mild anxiety, 16-25 indicating moderate anxiety, and 26-63 indicating severe
anxiety (Beck & Steer, 1993). This measure is standardized, used often, and has good
psychometric properties (Beck et al., 1993).
Generalized Anxiety Disorder – 7 questions (GAD-7; Spitzer, Kroenke, Williams, &
Löwe, 2006; s = .81-.88). The GAD-7 is a seven-item self-report measure that is efficient and
valid for screening general anxiety symptoms and assessing their severity in practice and
research. Respondents rate how often they have been bothered by anxiety symptoms (e.g.,
feeling nervous, anxious or on edge) on a 0 (not at all) to 3 (nearly every day) scale. A total score
of 0-4 indicates minimal anxiety, 5-9 indicates mild anxiety, 10-14 indicates moderate anxiety,
and 15-21 indicates severe anxiety. This measure was shown to have good reliability and
criterion, construct, factorial, and procedural validity (Spitzer et al., 2006).
Within-Intervention Anxiety Ratings. Participants rated their level of anxiety on a scale
of 1 (no anxiety) to 10 (worst anxiety imaginable) before and after completing an entry. Global
ratings such as a 1-10 scale provide a convenient format for soliciting everyday judgments of
anxiety level and limit participant burden (Kazdin, 2003).
Patient Health Questionnaire-9 (PHQ-9; Spitzer et al., 1999; s = .72-.84). The primary
clinical outcome of interest in this study is anxiety symptoms. However, given the high
comorbidity of depressive and anxious symptoms (Lamers et al., 2011) and the efficacy of CBT
for both depression and anxiety (e.g., Beck, 2011), the RCT study also tracked depressive
MOBILE APP FOR ANXIETY 25
symptoms to examine if there is any difference at baseline between groups or change throughout
the duration of the study. Thus, participants completed the PHQ-9, a nine-item self-report
measure of depressive symptoms that is well-validated and widely used as a brief diagnostic and
severity measure. This measure has been selected by the comprehensive CBT rollout in the
United Kingdom as their measure of choice for depression, as well as by the DSM-5 task force as
a measure to assess the severity of depression. It measures each of the DSM criteria for MDD,
and scores range from 0 to 27. Participants rate the frequency of symptoms (e.g., little interest or
pleasure in doing things; feeling down, depressed or hopeless) over the last two weeks on a scale
of 0 (not at all) to 3 (nearly every day), with 1 = several days and 2 = more than half the days. A
total score of 0-4 indicates minimal depression, 5-9 indicates mild depression, 10-14 indicates
moderate depression, 15-19 indicates moderately severe depression, and 20-27 indicates severe
depression. It has been shown to have good sensitivity and specificity, as well as excellent
reliability and validity (Watts et al., 2013).
Work and Social Adjustment Scale (WSAS; Mundt, Marks, Shear, & Greist, 2002; s =
.77-.88). Subclinical anxiety has been shown to be related to impairment in work and daily
functioning (e.g., Karsten et al., 2013). Thus, the WSAS was included to examine any changes in
functional impairment during the RCT. The WSAS is a five-item questionnaire that assesses
subjective impairment in five areas: work, home management, social life, private leisure, and
relationships (e.g., “Because of my anxiety, my ability to form and maintain close relationships
with others, including those I live with, is impaired”). Respondents rate their level of agreement
with each item using a 0-8 point Likert-scale (0 = not at all, 2 = slightly, 4 = definitely, 6 =
markedly, 8 = very severely). Item scores are summed, with a possible range of 0-40. Higher
scores indicate greater functional impairment, with scores of 10 or greater indicating significant
MOBILE APP FOR ANXIETY 26
functional impairment. The WSAS was shown to have good reliability, validity, and sensitivity
(Mundt et al., 2002).
Satisfaction with Life Scale (SWLS; Diener, Emmons, Larsen, & Griffin, 1985; s =
.83-.86). Another secondary outcome measure of interest was global satisfaction with life,
measured by the SWLS. This is a five-item measure of global life satisfaction (e.g., “In most
ways my life is close to my ideal”; “So far I have gotten the important things I want in life”).
Respondents are asked to rate their level of agreement or disagreement with the statements based
on a 7-point Likert-type scale ranging from 1 (strongly disagree) to 7 (strongly agree). Scores
range from 5-35, with higher scores reflecting greater levels of satisfaction with life. A total
score of 5-9 indicates one who is “extremely dissatisfied”, 10-14 = “dissatisfied”, 15-19 =
“slightly dissatisfied”, 20 = “neutral”, 21-25 = “slightly satisfied”, 26-30 = “satisfied,”, and 31-
35 = “extremely satisfied” (Pavot & Diener, 1993). The SWLS is an often-used scale with strong
psychometric properties that provides for the assessment of subjective well-being. In support of
its validity, SWLS scores have been found to be positively correlated with other self-report
measures of subjective well-being, as well as external ratings of well-being (Pavot & Diener,
1993).
Perceived Convenience Questionnaire (PCQ; Yoon & Kim, 2007; s = .82-.94). The
PCQ is a four-item measure of perceived convenience (e.g., “I have access to the [app/website]
everywhere”) that uses a five-point Likert scale, with higher scores reflecting greater perceived
convenience. It was adapted for use with a mobile app and website by replacing “English
learning system” with “app” or “website”. It has been demonstrated to have good discriminative
validity and reliability (  = 0.91; Chang et al., 2013).
MOBILE APP FOR ANXIETY 27
Intrinsic Motivation Inventory (IMI; Ryan, 1982; s = .86-.86). The IMI is a 37-item
multidimensional measure of intrinsic motivation that assesses participants’ subjective
experience of a target activity. The six dimensions include: interest/enjoyment (IE; e.g., “I enjoy
doing this activity very much”), perceived competence (PCo; e.g., “I think I am pretty good at
this activity”), effort/importance (EI; e.g., “It was important to me to do well at this task”),
pressure and tension (PT; “I felt pressured while doing this”), perceived choice (PCh; “I believe I
had some choice about doing this activity”), and value/usefulness (VU; “I believe this activity
could be of some value to me”), yielding six subscale scores. Recently, a seventh subscale was
added to examine experiences of relatedness, bringing the total number of items to 45, but it was
not included in this study, as the validity of this subscale has not yet been established. It is also
not relevant to the study, as there is no involvement of an in-person healthcare provider. It has
been shown that the exclusion of specific subscales have no impact on the others, and the use of
specific subscales of the IMI are chosen based on the research questions of interest. Respondents
are asked to rate their level of agreement with the statements based on a 7-point Likert-type scale
ranging from 1 (not at all true) to 7 (very true). Twelve items are reverse coded, so that higher
scores reflect higher levels of each of the subscales. Although the items are organized by
subscale in Appendix D, the instructions for the IMI dictate that the items are randomly ordered.
Because the items within subscales overlap somewhat, randomizing their presentation makes this
less salient to participants. The items have been shown to be coherent and stable in factor
analyses across a variety of tasks, conditions, and settings. The IMI also has received strong
support for its validity (McAuley, Duncan, & Tammen, 1987).
Subjective Usability Scale (SUS; Brooke, 1986; s = .92-.94). The SUS provides a quick
tool to measure the usability of a range of products and services, including mobile applications
MOBILE APP FOR ANXIETY 28
and websites (e.g., “I needed to learn a lot of things before I could get going with this system”).
It is typically utilized after the participant has used the system that is being evaluated and before
any debriefing or discussion takes place. The SUS contains 10 items with five response options,
ranging from “strongly agree” to “strongly disagree”. Five items are reverse coded, so that higher
scores reflect greater usability. The advantage of using the SUS is that it provides cut-off scores
to determine whether the system has acceptable usability. From 500 studies, SUS scores of 68
and above appear to indicate at least average usability (i.e., a usable system; Sauro, 2011). The
SUS has become an industry standard and has been demonstrated to have reliable results with
small sample sizes and validity in discriminating between usable and unusable systems (Brooke
et al., 1986).
Qualitative feedback. Participants in the pilot study were asked open-ended questions
regarding their attitudes towards the interventions. The questions are based on the domains of the
SUS, and participants were asked what they thought of the ease of use, appearance, overall
functionality, helpfulness, and acceptability for people with anxiety. They were also asked how
they felt before, during, and after using the intervention. Finally, they were asked if they have
any feedback for the study in general.
Treatment Description
Mobile App Intervention. The mobile app is called PsyApp, created in conjunction with
Yuchi Deng, a student in USC’s Computer Science Master’s program. The app is supported by
iOS and Android operating systems. PsyApp is based on two main components: assessment and
intervention. The assessment portion of this app consists of rating one’s self on a scale of 1-10 (1
representing no anxiety and 10 representing the worst anxiety possible). The 1-10 scale was
chosen to limit burden, lower response rates, and attrition, as too many questions can increase the
MOBILE APP FOR ANXIETY 29
likelihood of missing data and dropout (Otto, Smits, & Reese, 2005). The intervention portion of
this app consists of practicing cognitive reappraisal to reduce anxiety, which has received
support in the literature for predicting reductions in anxiety (e.g., Olatunji, Cisler, & Deacon,
2010; Westen & Morrison, 2001). A written script is embedded in the app to guide the user (see
Appendix C). Users then rerate themselves on the 1-10 scale of anxiety. Although the second
part of this app is referred to as the intervention, there has been evidence to support assessment
or the tracking of symptoms for alleviating psychological distress (Christensen et al., 2002;
Graham et al., 2000; Proudfoot et al., 2010).
Users were given log-in information and were required to sign in to the app with their
username and password, in order to ensure privacy. The data from the mobile app was stored
anonymously on a secure database. This is a USC server with double-encryption (i.e., encryption
of transmitted and stored data). SSL certificates were used on the app and the server, and https
protocol secured connections from the app to the server.
Computer-Based Intervention. The PsyApp website appeared identical to the mobile app
and contained the same components of assessment and intervention. The website also required a
log-in, and data was stored on the secure database with double-encryption and https protocol.
Participants were required to use the website on a computer only, and the website was coded
such that it was not functional on a mobile phone or tablet device.
Data Analyses
Participant characteristics. Descriptive characteristics of the participants (i.e., age,
gender, years of education, income, race/ethnicity, frequency of mobile phone and computer use,
mean scores and range on anxiety symptoms, perceived convenience of and motivation for using
the interventions) will be reported.
MOBILE APP FOR ANXIETY 30
Usability. Usability scores (i.e., SUS scores) will be reported for the app and website
groups. An SUS score of 68 and above will demonstrate at least average usability (i.e., a usable
system). In addition, responses to open-ended questions on attitudes towards the intervention will
be examined. Examples that describe key points and themes will be pulled.
Feasibility of the interventions. A pilot study is not suitable for hypothesis testing, and
thus efficacy cannot be evaluated in a pilot study, especially given the small sample size (Leon,
Davis, & Kraemer, 2011). Thus, in lieu of t-tests or other analyses, the average change scores of
the 1-10 anxiety ratings will be reported for the app and website groups to examine whether
anxiety scores generally increased, decreased, or stayed the same after completing an entry. In
addition, responses to the open-ended questions on how participants felt before, during, and after
using the intervention will be examined. Examples that describe key points and themes will be
pulled.
Feasibility of the study. Time to complete questionnaires will be reported, along with
the length of time to recruit 20 participants. Finally, any “bugs” in the intervention or study
procedure were noted and fixed for the RCT, in conjunction with the developer.
Results

Enrollment. Fifty-two college student volunteers were consented for the study, and 25
participants met the eligibility criteria. Reasons for exclusion included insufficient anxiety
symptomatology (15, 55.6%), severe anxiety symptom severity (7, 25.9%), unstable dosage of
psychiatric medication (2, 7.4%), ongoing psychotherapy (2, 7.4%), and co-occurring disorder
meeting exclusion criteria (1, 3.7%). Of the 25 eligible participants, ten were randomized into the
app group, and fifteen participants were randomized into the website group. Seven participants’
MOBILE APP FOR ANXIETY 31
responses from the app and website were not captured in the database due to a bug in the app and
website.
Participant characteristics. The participants had a mean age of 22.42 years (SD = 3.82;
range = 18-34). Sixty eight percent were female. Nine participants (36%) were graduate students,
and the rest were undergraduate students. Reported annual family income varied greatly from
less than $4,999 to over $75,000. Forty percent were White, 36% were Asian/Asian American,
20% were Latino, and 4% were Black. The participants reported spending an average of 5.10
hours per day on mobile phones (SD = 4.38; range = 1-20) and an average of 6.56 hours per day
on computers (SD = 6.52; range = 1-24).
To be eligible for the study, the participants had to score in mild-moderate range (i.e.,
scores ranging from 8-25) on the Beck Anxiety Inventory (BAI). On the BAI, the participants
had an average score of 16.12 (SD = 4.94; range = 10-25). On the Generalized Anxiety Disorder-
7 (GAD-7) scale, participants had a mean score of 14.44 (SD = 3.83, range = 9-22). The average
Perceived Convenience Questionnaire score was 17.28 (SD = 2.61, range = 12-20). All four
items of this measure received an average rating of at least 4.24 (4 = agree, 5 = completely
agree), suggesting that the app and website had acceptable convenience. In the app group,
participants scored a mean of 18.22 on the PCQ (SD = 2.54), and participants in the website
group scored a mean of 16.75 (SD = 2.57). IMI subscale means ranged from 3.60 (Pressure and
Tension subscale) to 6.27 (Perceived Choice subscale; SDs = .72-1.43), showing at least
“somewhat true” agreement with the various subscales of the IMI.
Usability. A Subjective Usability Scale (SUS) score of 68 and above demonstrates at
least average usability (i.e., a usable system). The mean usability score on the SUS was 78.20
MOBILE APP FOR ANXIETY 32
(SD = 23.77; range = 0-100), with a median of 85.00. In the app group, the mean SUS score was
88.06 (SD = 8.18) and in the website group, the mean SUS score was 72.66 (SD = 27.88).
Examining the responses to the open-ended questions, all participants stated that the app
and website were helpful for anxiety, particularly in terms of self-assessment (e.g., “knowing
myself better”) and when people cannot afford a therapist. However, one participant stated that it
was only helpful for research on anxiety. Most participants stated that the app and website were
easy to use and acceptable for people with anxiety. The two participants that felt the intervention
was not easy to use stated, “I was confused by the first question because I did not know what to
expect. The examples were a good component,” and “I don’t think it should necessitate a
username and password”. The participant that was unsure about the intervention’s acceptability
stated, “I think it would depend on a case by case situation. Some would prefer to use a scale
method of answering questions. Whereas others would like to type out their answers and
elaborate”. In terms of the appearance, there was variability with some liking the blue color and
simplicity and others stating it was “average but serves the purpose.” Others stated, “It’s not
catchy or enticing, but I don’t think that it needs to be”. Finally, participants reported a range of
how long they could see themselves using the app from “not very long” to “life long”.
Feasibility of the interventions. The average 1-10 anxiety score prior to using the app or
website was 4.39 (SD = 1.85; range = 2-8) and after using the intervention was 4.28 (SD = 2.08;
range = 1-9). Seven participants’ scores decreased on the 1-10 anxiety ratings (the largest
decrease being 3 points on the scale); six participants’ scores remained the same; and five
participants’ scores increased (the largest increase being 2 points on the scale). Participants in the
app group alone averaged a score of 4.33 (SD = 2.34) prior to filling out a thought record and
3.67 (SD = 1.97) after filling out the thought record. Participants in the website group averaged a
MOBILE APP FOR ANXIETY 33
score of 4.42 (SD = 1.68) prior to filling out an entry and a score of 4.58 (SD = 2.15) after filling
out an entry. In the app group alone, three participants experienced a decrease in 1-10 anxiety
ratings, two participants’ ratings remained the same, and one experienced a one-point increase in
anxiety ratings. In the website group, four participants experienced a decrease in 1-10 anxiety
ratings, four participants experienced anxiety ratings remaining the same, and four participants
experienced an increase in anxiety ratings. In terms of the emotional process that users
commented on in an open-ended format, most said that they felt more “relaxed” or less anxious
after completing an entry (e.g., “I felt skeptical before, good during and good after,” “I felt
relaxed before, during I felt kind of challenged; after I felt a bit more relaxed”). An example of a
completed entry is included in Figure 1.
Feasibility of the study. The complete duration of one session with a pilot study
participant was one hour. The length of time to recruit 25 eligible participants was a little over
four months. There were some “bugs” in the intervention, namely in the Android app. The
formatting was such that Android users had difficulty viewing what they were typing (one user
expressed that this increased her level of anxiety) and also had difficulty finding the “next”
button. Another issue discovered in the pilot study was the loss of some data. Seven participants’
responses from the app/website were not captured in the database. These issues were resolved by
the developer prior to the RCT.
Discussion
Results from the pilot study indicated that all participants were able to complete an entry
on the app/website without instruction from human interaction. Approximately 39% of the
participants’ 1-10 anxiety ratings decreased after using the app/website, which was promising
given this was the first time having filled out such an entry. Although small, the overall mean
MOBILE APP FOR ANXIETY 34
anxiety rating decreased after completing an entry. Five participants’ increased anxiety scores
did not appear to be alarming, as it is often expected that there may be initial increases in
symptom scores, particularly when participants are being asked to confront their anxiety. In
addition, the largest increase was by only two points on a ten point scale. The app and website
also had high levels of perceived convenience and usability, although the standard deviation in
the SUS score indicates some variability in participants’ perceptions of the app and website. No
changes were made to the content, as participants appeared to benefit from the example given
and questions asked. No change was made to the login requirement, as this was a security
measure to protect participants’ data. The appearance of the intervention was kept the same as
much as possible with the transition to a native application (i.e., developed for use specifically on
an iPhone or Android device), as most felt that the simplicity and color were appropriate for an
anxiety intervention. If the majority of participants reported that the intervention was unhelpful,
not easy to use, or not functional, changes would have been made to the content and/or design.
Likewise, if they had experienced negative emotions for the entire duration of using the
intervention, changes would have been made to it. The change that was made to the app was to
utilize a native application (i.e., an application program developed for use on that specific
platform) in order to resolve the bugs found in the app, particularly in the Android version.
Study 2: Randomized Controlled Trial

Methods
Participants and Design. Participants were recruited through the Subject Pool at a large
private university in California (n = 83), advertisements on an online student announcement
board at that university (n = 36), Craigslist (n = 10), summer psychology courses (n = 4) and the
university’s clinic (n = 2). Interested participants provided their informed consent, with the
MOBILE APP FOR ANXIETY 35
understanding that they may be assigned to a waitlist control group, which would delay receiving
the intervention for six weeks. The inclusion and exclusion criteria remained the same as in the
pilot study. Interested participants were first screened online to determine if they met the five
inclusion criteria. Those that screened in via the online survey were then further screened via
phone by the primary investigator to determine if they met any of the exclusion criteria. Each
participant was randomly assigned via computer randomization to the mobile app, website, or
no-treatment waitlist control condition. The participant flow is also depicted in Appendix A.
The order of questionnaire administration is depicted in Appendix B. At pretest, all
enrolled participants completed baseline anxiety symptom questionnaires (i.e., BAI & GAD-7)
and secondary outcome measures (i.e., PHQ-9, WSAS, & SWLS) online. Participants in the
active treatment conditions (i.e., app & website) were additionally asked to complete measures
that assess user perceptions (i.e., PCQ & IMI) of the interventions. Those in the active treatment
conditions were then instructed to complete daily entries for six weeks. Participants in the active
treatment conditions received a daily reminder via text message to fill out an entry; they chose
the time in which it was convenient for them to receive the text message and complete the entry.
Weekly, participants in the active conditions received an email from the primary investigator, in
order to “check in” and ask if there were questions about the intervention. This was to minimize
dropout and reduction of effects, as past studies and meta-analysis have shown that provider
contact improves both of these (e.g., Christensen et al., 2002). At midtreatment (i.e., week 3) and
posttreatment, those in the active treatment conditions completed the same online questionnaires
of anxiety symptoms, secondary outcomes, and user perceptions via an emailed web survey link.
At midtreatment and posttreatment, the usability scale (SUS) was also included in the online
survey. For those in the waitlist control group, at midtreatment and posttreatment, participants
MOBILE APP FOR ANXIETY 36
completed only the symptom questionnaires (i.e., BAI, GAD-7, PHQ-9, WSAS, & SWLS)
online. Weekly, these participants also received an email from the investigator, thanking them
for their participation in the study, again to minimize dropout. After week 6, the participants in
the waitlist control condition were then given their choice of one of the two active interventions.
If questionnaires for participants in any condition had not been completed two days after receipt,
one email reminder was sent. After four days of no response, a phone call reminder was given.
The duration of the intervention was six weeks, consistent with the mode duration of mHealth
interventions for mental disorders. This is also consistent with the length of many current
evidence-based cognitive-behavioral treatments that are short-term in nature (e.g., 6-8 weeks).
Participants were compensated with Subject Pool points equating to 4.25 hours for the total
duration of the study.
Measures. As in the pilot study, demographic information was collected to assess
participant characteristics. Participants in the RCT were additionally asked what year in school
they were completing, as the majority of participants were students. In addition, information
about participants’ frequency of mobile phone and computer use was collected to assess previous
experience with technology.
The measures of the RCT remained the same as in the pilot study. As the diagnostic
measure, the M.I.N.I. was again used to determine if interested participants met exclusion criteria
for this study. The primary outcomes of anxiety symptoms were again measured with the BAI
and GAD-7 at pretest, midtreatment, and posttest. In addition, participants in the active
intervention conditions rated their 1-10 level of anxiety daily on a scale of 1 (no anxiety) to 10
(worst anxiety imaginable) prior to and after filling out an entry. Secondary outcomes were
measured for all participants: depressive symptoms measured by the PHQ-9, work and social
MOBILE APP FOR ANXIETY 37
adjustment measured by the WSAS, and satisfaction with life measured by the SWLS. In addition,
in the active treatment conditions, adherence was measured using amount of entries completed.
Measures of user perceptions of the active treatments again included the PCQ for perceived
convenience, IMI for intrinsic motivation, and SUS as a basic usability measure.
Data Analyses and Hypotheses
Baseline between-group comparisons. Differences in baseline demographic
characteristics (i.e., age, gender, education, income, race/ethnicity, frequency of mobile phone
and computer use) and pre-treatment symptom questionnaires (i.e., BAI, GAD-7, PHQ-9)
between the three conditions were examined using one-way analysis of variance (ANOVA) for
continuous variables (i.e., age, technology use, symptom questionnaires) and Chi-square tests for
categorical variables (i.e., level of education, income level, gender, race/ethnicity) that had
sufficient cell sizes. Differences between groups were not expected because participants were
randomized to condition. However, if differences were found, they would have been accounted
for in the analyses.
Missing data. Rate of attrition for each group are reported, along with an analysis of
differences in attrition between groups using the Chi-square test. If missing data had been
predicted by the outcome of interest (i.e., anxiety), missingness would have been controlled for
by using maximum likelihood estimation in a mixed models approach. In addition, participants
were oversampled to account for possible attrition (see power analysis section).
Aim 1. Is there a treatment effect? Is there a difference in efficacy between the
mobile app and website version? To assess for a treatment effect, a repeated measures
multivariate analysis of variance (MANOVA) was performed. MANOVA is typically used to
compare mean differences between independent groups for more than one dependent variable
MOBILE APP FOR ANXIETY 38
(Wilcox, 2011). Repeated measures was used because the outcome variables were measured at
pretest, midtreatment (week 3), and posttest. MANOVA was believed to be the best statistical
test in this case, as it can handle unequal covariation, and thus reduce chance for type I error
(Wilcox, 2011). Prior to running the repeated measures MANOVA, assumptions of
independence of observations, multivariate normality, and homogeneity of variances were
checked. The independent variable for this analysis was the format of intervention, with three
levels—mobile app, website, and no treatment waitlist control. The dependent variable was
anxiety symptoms (BAI, GAD-7) along with depressive symptoms (PHQ-9). Although a
reduction in depressive symptoms was not the intended outcome, depressive symptoms were
included in this analysis as they show high comorbidity with anxiety symptoms (Lamers et al.,
2011).
This repeated measures MANOVA was conducted to assess intervention effects on
psychological symptoms (BAI, GAD-7, & PHQ-9). Given the evidence base for self-monitoring
and cognitive reappraisal in reducing anxiety symptoms, it was expected that there would be an
interaction between group and time, such that anxiety symptoms would improve over time for
the two active interventions but not for the waitlist group. The author tested for a significant
difference between the app and website modalities on the anxiety symptom scores. It was
additionally explored whether or not depressive symptom scores change across time, as this was
an app designed for reduction in anxiety and not depression. Finally, patterns of change across
time in the 1-10 ratings of anxiety for the app and website groups were examined graphically.
Power analysis. In order to determine the sample size needed to obtain small (f²mult =
0.02), medium (f²mult = 0.15), and large (f²mult = 0.35) effect sizes (Cohen, 1977, 1988), an a
priori power analysis was conducted using G*Power and confirmed using a power analysis
MOBILE APP FOR ANXIETY 39
simulation. To examine the between and within interactions, a total sample size of 600 would
have been needed for a small effect size, 98 for a medium effect size, and 40 for a large effect
size, for a power of 0.8. A recent meta-analysis (Davies et al., 2014) showed that computer-
delivered interventions, in comparison to inactive controls, improve anxiety for university
students (pooled standardized mean difference = -0.56; 95% CI [-0.77 to -0.35], p < .001). In
addition, Andersson’s (2009) review of the literature on Internet-delivered CBT (iCBT)
compared to no treatment showed large effect sizes (d’s = .73 – 1.00) for panic disorder, social
anxiety disorder, and posttraumatic stress disorder. Therefore, we aimed for a conservative
medium overall effect size of f²mult = 0.15 (sample size of 98). Rates of attrition have been shown
to vary from 0 to 33% for mobile mental health app studies (e.g., Burns et al., 2011; Rizvi et al.,
2011; Watts et al., 2013). To account for the highest dropout rate of 33%, this study recruited at
least an additional 33 participants to achieve sufficient power, with the total number of
participants being 135.
Secondary outcomes. Separate repeated measures MANOVA were run to examine any
differences in secondary outcomes of interest, work and social adjustment (WSAS) and
satisfaction with life (SWLS). Assumptions of independence of observations, multivariate
normality, and homogeneity of variances were again checked. The independent variable was
format of intervention with three levels—mobile app, website, and no treatment waitlist control.
The dependent variables included work and social adjustment and life satisfaction. As we
anticipated a treatment effect, it was expected that there would be an interaction between group
and time, such that these functional outcomes would improve over time for the two active
interventions but not for the waitlist group. The author explored whether or not there was a
significant difference between the app and website modalities on these two scores.
MOBILE APP FOR ANXIETY 40
Aim 2. Is there a difference in convenience and motivation between active treatment
modalities? To explore potential differences in perceived convenience and motivation, another
repeated measures MANOVA was performed. Assumptions of independence of observations,
multivariate normality, and homogeneity of variances were checked. Because only the active
interventions received the convenience and motivation measures, the independent variable in this
case was format of intervention with two levels—mobile app and website. The dependent
variables were convenience measured by the PCQ and intrinsic motivation measured by the IMI.
The author examined whether there was an interaction between group and time (i.e., whether
PCQ and IMI were higher for one group than the other over time).
In addition to the MANOVA, the mean basic usability scores (SUS) were examined for
each group, to confirm whether or not they met the threshold score of 68 and above for a usable
system. In addition, these SUS scores were compared between the two active treatment groups
with an independent-samples t-test. To examine any differences in adherence between the two
active treatment modalities, an independent samples t-test was conducted to determine if there
were any differences in the total number of entries completed between the app and website
groups.
Results
Enrollment, participant background, and study attrition. Of the 439 people who
expressed interest in the study, 135 were enrolled. Reasons for exclusion are shown in Table 1,
with the most common reasons being insufficient anxiety symptomatology (118, 26.9%), lack of
response to further screening (87, 19.8%), and severe anxiety symptom severity (75; 17.1%).
Eligible participants were randomized via computer randomization to the app (n = 47), website
(n = 44), and waitlist (n = 44).
MOBILE APP FOR ANXIETY 41
The overall study sample had a mean age of 22.6 years (SD = 7.7, range = 18-59).
Seventy-five percent were female. The majority of participants (72.6%) had completed at least
some college education, and the majority (50.4%) reported a family annual gross income of
$75,000 or more. Forty-four percent were Asian American, 37% were White, 9% were Latino,
7% were Black, and 3% were biracial. The participants reported spending an average of 5.3
hours per day on mobile phones (SD = 4.0; range = 1-20) and an average of 5.8 hours per day on
computers (SD = 3.1; range = 1-20).
Differences in continuous demographic variables (i.e., age, hours of phone & computer
use, baseline symptoms) among the app, website, and waitlist groups were examined using a
one-way ANOVA (see Table 2). No differences were found among the three groups (ps = .16 -
.98). Chi-square tests were used to examine differences among the three conditions in
demographic variables of level of education, income level, and gender (see Table 3). The cell
sizes for race/ethnicity were insufficient (i.e., less than 5 for some racial/ethnic groups) for a Chi-
square analysis. There were no statistically significant differences between groups on these
variables (ps = .06 - .90).
The overall rate of attrition (i.e., withdrawal from study or lack of completion of
posttreatment questionnaires) was 18.5%. The rate of attrition did not differ by app (25.5%),
website (20.5%), and waitlist (9.1%) groups,   (2, N = 135) = 4.23, p = .12. Attrition was more
likely amongst male participants (31.3% in males versus 13.7% in females,   [1, N = 135] =
5.09, p = .02), but no other demographic variables or level of baseline symptoms appeared to be
significant (see Tables 4 and 5). However, nonresponders to midtreatment questionnaires only
were significantly less anxious than responders (Mnonresponders = 8.9, SDnonresponders = 5.1; Mresponders
= 11.7, SDresponders = 7.3; t[133] = 2.1, p = .04). Also, at midtreatment and posttreatment, more
MOBILE APP FOR ANXIETY 42
participants in the app and website conditions failed to complete the questionnaires (e.g., BAI,
GAD-7, PHQ-9) than in the waitlist condition (midtreatment: 31.9% of app group and 36.4% of
website group compared to 6.8% of waitlist group,   [2, N = 135] = 11.93, p = .003;
posttreatment: 42.6% and 47.7% compared to 13.6%;   [2, N = 135] = 13.17, p = .001).
Mood outcomes. A repeated measures MANOVA was used to assess the intervention
effects on BAI, GAD-7, and PHQ-9 scores across the beginning, middle, and end of the six
weeks. The main effects of group (F[6, 154] = .89, p = .50) and time (F[6, 74] = 1.50, p = .19)
were not significant. There was also no interaction effect, indicating no significant differences
between groups on mood symptom scores across time, F(12, 474) = .39, p = .97. Univariate tests
also showed no intervention effect on these measures (BAI: F[4, 158] = .47, p = .76; GAD-7:
F[4, 158] = .37, p = .83; PHQ-9: F[4, 158] = .50, p = .73). The mean BAI, GAD-7, and PHQ-9
scores at each time point were graphed for the three groups (see Figures 2-4 and Table 7). As can
be seen in these figures, in the app and website groups, there was a slight increase in BAI scores
at midpoint, with a decrease in scores by the end of the study, such that by the end, BAI scores
were lower than where they started at the beginning of the study. The waitlist group experienced
a continuous decrease in BAI scores across the duration of the study. For GAD-7, there was an
increase in GAD-7 scores at Time 2 for all three groups. At the end of the study, GAD-7 scores
dropped for the app and waitlist groups to the point of being lower than Time 1 scores, but for
the website group, the GAD-7 scores dropped minimally at the end of the study. For PHQ-9, all
three groups experienced an increase in depressive symptoms initially, with a small reduction by
the end of the study.
A repeated measures MANOVA was also run focusing only on the anxiety measures (i.e.,
BAI & GAD-7) across time. The main effect of group was again not significant, F(4, 158) = .85,
MOBILE APP FOR ANXIETY 43
p = .49), nor was the main effect of time, F(4, 77) = 1.43, p = .23. The interaction was also not
significant, indicating no significant differences between the three groups on BAI and GAD-7
scores across time, F(8, 320) = .38, p = .93.
Patterns of change across time in the 1-10 ratings of anxiety within the app and website
were also examined. Person-centering was used by subtracting each person’s mean and their
week 1 scores. The graphs (Figures 5a-b) show that participants’ scores fluctuate by
approximately +/- 2 points, with a decrease by the end of the six weeks (depicted as Week 6).
This was re-graphed to examine Week 1 to Week 5 and Week 1 to Week 6, excluding the
intervening periods (see Figures 6a-b & 7a-b). By examining Week 1 and 5 versus Week 1 and
6, it can be seen that the decrease in scores did not occur until the end of the study (Week 6).
To sum, there were no main effects of group and time or interaction effect, indicating no
significant differences between the groups on mood symptom scores across time. This was also
the case when examining anxiety symptoms alone. There were trends towards anxiety scores
increasing at midpoint for the intervention groups and eventually decreasing. An examination of
the 1-10 ratings of anxiety within the intervention indicated that a small decrease occurred only
at week 6, and not prior to that week.
Secondary outcomes. A repeated measures MANOVA was performed to assess
intervention effects on WSAS and SWLS scores across time. The main effect of group was not
significant, F(4, 158) = .76, p = .55), but the main effect of time was significant, F(4, 77) = 4.04,
p = .005. Univariate tests showed an effect of time on the SWLS only, F(2, 160) = 5.44, p = .005
(WSAS: F[2, 160] = 1.60, p = .21), with SWLS scores increasing across time (MT1 = 22.67, SDT1
= 6.22; MT2 = 24.22, SDT2 = 6.88; MT3 = 25.06, SDT3 = 6.24). The interaction was not significant,
indicating no significant differences between the three groups on WSAS and SWLS scores
MOBILE APP FOR ANXIETY 44
across time, F(8, 320) = 1.02, p = .42. Univariate tests additionally showed no intervention effect
on these measures (WSAS: F[4, 160] = .58, p = .68; SWLS: F[4, 160] = 1.79, p = .13). As can be
seen in Figures 8 and 9, for WSAS, all three groups experienced a slight increase in scores at
midpoint, with a small reduction by Time 3. To sum, there was no main effect of group or an
interaction effect. However, there was a main effect of time, driven by SWLS scores increasing
over time.
Attitudes toward technology. A repeated measures MANOVA was performed to assess
intervention effects on PCQ and IMI subscale scores (see Figures 10-16). The main effect of
group was not significant, F(7, 29) = 1.38, p = .25), but the main effect of time was significant,
F(14, 227) = 2.48, p = .03. Univariate tests indicated an effect of time on the following IMI
subscales only: IE (F[2, 70] = 3.97, p = .02); PT (F[2, 70] = 13.07, p < .001); PCh (F[2, 70] =
4.66, p = .01); and VU (F[2, 70] = 7.63, p = .001). There was no significant effect of time on the
PCQ (F[2, 70] = 1.81, p = .17) or the PCo (F[2, 70] = 0.74, p = .48) and EI (F[2, 70] = 0.83, p =
.44) subscales of the IMI. Overall, IE, PCh, and VU scores decreased across time, while PT
scores increased over time (see Table 8). The interaction was not significant, indicating no
significant difference between groups on these scores across time (F[14, 130] = 1.11, p = .36).
However, univariate tests showed an intervention effect on the PCQ only, F(2, 96) = 3.18, p =
.046 (IMI IE Subscale: F[2, 96] = .10, p = .91; PCo Subscale: F[2, 96] = 1.54, p = .22; EI
Subscale: F[2, 96] = .13, p = .88; PT Subscale: F[2, 96] = .74, p = .48; PCh Subscale: F[2, 96] =
.01, p = .99; VU Subscale: F[2, 96] = 1.10, p = .34). An exploratory independent samples t-test
was then utilized to compare the PCQ scores between the app and website groups at Time 1,
Time 2, and Time 3. These indicated that those in the app condition (Time 1: M = 16.91, SD =
3.64; Time 2: M = 16.71, SD = 3.85; Time 3: M = 16.38, SD = 4.03) had higher levels of
MOBILE APP FOR ANXIETY 45
perceived convenience than those in the website group (Time 1: M = 15.16, SD = 4.68; Time 2:
M = 13.04, SD = 5.28; Time 3: M = 13.05, SD = 4.99) at all three time points (Time 1: t(89) =
2.01, p = .048; Time 2: t(56) = 3.05, p = .004; Time 3: t(45) = 2.54, p = .015).
Mean SUS scores for the app and website groups at midpoint were 45.17 (SD = 32.11)
and 33.52 (SD = 27.49) respectively. Mean SUS scores for the app and website groups at
posttreatment were 46.29 (SD = 31.39) and 38.75 (SD = 28.65) respectively. An independent
samples t-test was used to compare the SUS scores between the app and website groups at Time
2 (t[56] = .15) and Time 3 (t[59] = .33). Neither of these were statistically significant (ps = .15-
.33).
To sum, there was no main effect of group that was found. However, there was a main
effect of time, driven by the IE, PT, PCh, and VU subscales of the IMI, such that IE, PCh, and
VU scores decreased across time while PT scores increased over time. In addition, there was no
interaction effect, although univariate tests showed an intervention effect on the PCQ only, with
app users experiencing higher perceived convenience than website users at all three time points.
Finally, mean usability scores for both the app and website groups were lower than in the pilot
study and lower than the recommended cutoff score of 68 or greater.
Active Treatment Group (App and Website) Adherence Analyses. Two examples of
completed thought records are included in Figures 18-19. The average number of entries
completed was 19.9 (SD = 14.3). See Figure 17 for the distribution of number of entries
completed. To examine any differences in adherence between the two active treatment
modalities, an independent samples t-test was conducted to determine if the total number of
entries completed differed for the app (M = 21.32, SD = 14.89) and website (M = 18.32, SD =
13.62) groups. There was no difference between the two groups in number of entries, t(89) =
MOBILE APP FOR ANXIETY 46
1.00, p = .32. There was also no significant correlation between the number of entries completed
and Time 1, 2 or 3 BAI, GAD-7, PHQ-9, WSAS, or SWLS scores, with the exception of Time 1
BAI (see Table 6). Number of entries completed and BAI at time 1 were significantly correlated,
r = .28, p = .007.
Correlations were also examined between adherence; user perceptions of convenience
and motivation; and usability (see Table 9). Adherence was only related to perceived
convenience at time 3 (r = .30, p = .04) and perceived competence at Time 3 (r = .37, p = .01).
Adherence, perceived convenience, and perceived competence were not significantly correlated
with anxiety symptoms at any of the time points (ps = .06-.70; see Table 10), with the exception
of the previously mentioned correlation between adherence and Time 1 BAI scores, r = .28, p =
.007.
Post-hoc analyses comparing frequent and infrequent app/website users. When
splitting the app and website users into “frequent users” (average of at least 4 entries per week)
and “infrequent users” (less than 4 entries per week), the frequent users (M = 12.68, SD = 6.01)
had higher initial BAI scores than infrequent users (M = 9.85, SD = 5.64) at the beginning of the
study only, t(89) = 2.28, p = .025. There were no differences between the frequent and infrequent
users in pretreatment GAD-7 or PHQ-9 or any of the midtreatment and posttreatment symptom
questionnaires (ps = .19-.64).
Discussion

In the randomized controlled trial, the repeated measures MANOVAs showed no
treatment effect of the app or website on mood symptoms or secondary functional measures,
although over the duration of the study, satisfaction with life scores increased. In other words,
the app and website users were functioning no differently than the waitlist control participants in
MOBILE APP FOR ANXIETY 47
terms of mood or functional impairment. While the main symptom analyses were not significant,
there was a pattern of symptom scores initially increasing somewhat as the app and website
participants were asked to face their anxiety, with an eventual decrease by the end of the study,
which is a pattern of symptom scores often seen in treatments such as prolonged exposure
therapy. There were also potential trends toward the website users not experiencing as much
symptom benefit as app users (e.g., GAD-7, SWLS).
One possible reason there was no treatment effect is that the full “package” of cognitive
behavioral therapy is typically more comprehensive than the cognitive restructuring that was
offered in this study. While cognitive restructuring alone can be effective for anxiety (e.g.,
Borkovec, Newman, Pincus, & Lytle, 2002), some studies (e.g., Borkovec & Costello, 1993;
Durham et al., 1994) suggest that the combination of cognitive work with behavioral
interventions such as exposures and relaxation training may be most powerful. Although
cognitive restructuring in particular was chosen for the purpose of this research study (i.e.,
feasibility of participants learning this technique without human interaction; to have the ability to
attribute any treatment effects to this intervention alone), it is possible that this study may have
been underpowered due to having a titrated intervention.
It is also possible that the intervention was not as robust because it did not involve much
training on cognitive restructuring, with the exception of one provided example. Reappraisal is a
technique that is thought to require metacognition (Wells, 2002), and it is possible that some
participants could have benefitted from more training than others. This is evidenced by the two
examples of thought records in Figures 18-19 and by comparing these two thought records with
an example thought record from the pilot study (Figure 1), where the pilot study participant
provided more detail than the two RCT participants. While it is clear that all participants were
MOBILE APP FOR ANXIETY 48
able to complete the cognitive restructuring exercise without face-to-face instruction, the quality
of the completed thought records varied, perhaps because of a lack of training or because of the
demands of participating in the RCT. It was also found that pressure/tension subscale scores
were increasing over time, perhaps because participants were facing their anxiety without a
complete understanding of how to best manage their anxious thoughts. The importance of more
training on cognitive restructuring is further highlighted by the correlation found in this study
between perceived competence and adherence. Cognitive Evaluation Theory (CET) posits that
competence and autonomy foster intrinsic motivation, further increasing learning and behavioral
change. While in this study, intrinsic motivation was not directly related to anxiety symptom
reduction, there was a relationship between perceived competence and number of entries
completed (i.e., adherence). Thus, if users did not feel they were competent at the task, perhaps
because of a lack of further coaching, they may have been less likely to adhere to the treatment.
At the same time, it is also possible that because users did not adhere to the study intervention,
they felt less competent. While it is a strength that this study actually tested the app and website
as stand-alone interventions, they may be more effective to offer as an adjunct to treatment
and/or further coaching by a health professional. It is also possible that this study, when isolating
the app and website without therapist contact, found results similar to recent studies (e.g., Enock
et al., 2014; Pham et al., 2016), where adherence and treatment effects of mobile interventions
were not pronounced.
Another possible explanation for no treatment effect is that the duration of the study may
not have been long enough for treatment effects to be fully realized. The 1-10 anxiety ratings
within the intervention showed a pattern of some reduction in anxiety scores by week six. This
was not evidenced even in week five. It is possible that the intervention takes at minimum a full
MOBILE APP FOR ANXIETY 49
six weeks to “kick” in, and if the participants continued to use the intervention for additional
weeks, more of a treatment effect would have been evidenced. However, if it also possible that
participants, knowing it was the end of the study, wanted to show their improvement and thus
provide lower scores at the end of the study.
Other factors that may have diluted a treatment effect are related to participant variability.
First, there is burden. More participants in the app and website conditions failed to complete the
Qualtrics surveys than the waitlist condition. The app and website conditions are indeed more
burdensome than the waitlist condition, as they require daily entries whereas the waitlist
participants do not have a daily participation requirement. It’s possible that participating in the
study as an app or website user was seen as too burdensome, contributing to a reticence to
continue participating in this study. There also appeared to be a difference in usability scores in
the pilot study and the RCT. It is possible that participating in the study for six weeks contributed
to perceptions of the app/website being too burdensome and thus less usable. At the same time, a
completion rate of 74.5-79.5% in the app and website groups is still rather high, compared to the
highest dropout rate of 33% evidenced in the literature review.
A second important factor is participant choice. It is possible that a treatment effect was
not found in this study because participants did not get to choose the modality or type of therapy
that was given to them. Indeed, it was found that overall, perceived choice subscale scores were
decreasing across time. The literature shows that treatment preferences can affect treatment
outcomes (e.g., Iacoviello et al., 2007), although this literature is also mixed (e.g., Van et al.,
2009). Responses in this study provided further support that participants having the ability to
choose the modality may be important. For example, overall, the app was perceived to be more
convenient than the website. However, certain individuals provided feedback that it was very
MOBILE APP FOR ANXIETY 50
difficult to type their responses on the app, showing a preference for a website version.
Understanding patient preferences can help explain why there is such wide variability in
sustained usage of mobile apps for some participants.
It is also possible that some users began to experience less anxiety, became less engaged
in the study, and dropped out because they were feeling better or stayed in the study with
minimal engagement, not continuing to experience as many gains as they could have with
sustained use. It appears as though those who are experiencing less anxiety are less engaged in
the study, particularly at Time 1 with app/website entries and Time 2 with the symptom
questionnaires. Perhaps those with less anxiety did not view the thought records and symptom
monitoring as being as relevant or necessary, compared to those with more anxiety. Indeed,
value/usefulness subscale scores decreased over time, and a number of participants gave
feedback that there were days when they felt completing an entry was not necessary because they
did not have an anxious thought. For cognitive restructuring to be most successful, a cognitive
distortion must exist. Thus, for those who had days without anxious thoughts, they could have
benefited from other intervention techniques within the app or website, such as behavioral
interventions.
Still, this study made a potentially important distinction between the app and website, in
that the app was perceived as more convenient than the website. As mentioned previously, we
are aware that computer-based CBT is effective. While this study does not show superiority of
the app over the website in terms of a reduction in anxiety, this study shows that the app
modality may be perceived as being more favorable due to its convenience. Understanding user
perceptions can be useful, as this study found a relationship between perceived convenience and
adherence to an intervention. While it would be premature to offer guidelines for dissemination
MOBILE APP FOR ANXIETY 51
with this single study, it does lend support for the use of mobile apps for anxiety that are proven
to be efficacious because of apps’ convenience and consequently the likelihood of a user to
adhere to the treatment intervention. At the same time, this univariate test finding should be
interpreted with caution, as the overall MANOVA that included perceived convenience and
intrinsic motivation was not found to be significant.
The design of this study also includes a number of limitations. Although careful
consideration had been given to the design to minimize shortcomings, some limitations are
unavoidable. First, the dependent variables were measured largely by self-report. Self-report
measures can bias outcomes in a number of ways (see Schwarz, 1999). Still, this study included
only well-validated and reliable measures with established norms that have been used with
college student populations. The measures also correlated as expected with one another and
across time (e.g., Time 1 BAI & T1 GAD-7: r = .49, p < .01; T2 GAD-7 & T2 SWLS: r = -.31, p
< .01; Time 1 BAI & Time 2 BAI: r = .54, p < .01; see Tables 6, 9, and 10) and showed high
internal consistency both in the pilot study and RCT. In addition, self-report is often the best,
most frequently used, and oftentimes only way to measure mood and anxiety symptoms. While
the simple 1-10 rating of anxiety was used within the intervention for its brevity, future studies
may also include an alternative measure within the app/website that has greater measurement
precision and can further capture or separate differences. Also, it may be beneficial for future
studies involving college student participants to tailor the WSAS to students or to find a better
work and social adjustment scale for them. This scale includes the verbiage “my ability to work”
which might be adjusted to be more relevant to students and their academic affairs (e.g., “my
ability to study”). In addition, some home management examples such as “looking after home or
children” may not be as relevant to the majority of college students.
MOBILE APP FOR ANXIETY 52
Second, this study examines user perceptions that have gained the most support in the
literature (i.e., convenience & motivation), but there may be other important perceptions for
future studies to assess such as engagement. From examining the number of entries completed, it
appears there may have been wide variability in engagement with the intervention. This may
help explain why the SUS scores were found to be lower in the RCT than in the pilot study. It is
possible that engagement became lower as time passed during the RCT, which may give insight
into the recommend length of use of the intervention or whether certain aspects of the
intervention need to be modified so as to maintain high engagement. In fact, the lowest item-
level mean score on the SUS was for the item asking if participants would like to use the
intervention frequently. Perhaps it was too uninteresting, as the lower IMI scale score and IMI-IE
scores decreasing over time, would suggest, or not engaging.
Finally, this study included largely a college student population of Asian and European
American background, which may limit the generalizability of the study. It has also been cited in
the literature that mobile-based interventions are often preferred in younger participants (e.g.,
Matthews, Doherty, Coyle, & Sharry, 2008), and thus future mHealth studies should include a
wider range of ages of participants to examine older people’s perceptions of apps and websites.
In addition, this study did not include a clinical sample due to the experimental nature of this
previously untested intervention. It is therefore possible that this study actually included
participants who were not as likely to seek or utilize this type of intervention. The study may
have found more pronounced effects if it included participants seeking self-help interventions;
certain underserved groups who are more likely to rely heavily on smartphones; people who are
waiting for specialty care; and/or those that are looking for relapse prevention techniques.
MOBILE APP FOR ANXIETY 53
In conclusion, this study did not find a treatment effect for the app or website. However,
it did find that participants overall appear to find an app modality more convenient than the
website equivalent. Future studies should begin by examining ways to increase the effectiveness
of this intervention, with a consideration towards the inclusion of the “full package” of CBT.
This should be done with caution, as traditional treatment programs, in particular manualized
treatments, are typically text heavy and not always easily transferrable to a mobile modality. The
content can become abbreviated (e.g., by using a 1-10 scale for anxiety), which can reduce
previously established validity of treatments. Therefore, if a full CBT package is included, it is
recommended that the validity of the mobile or website versions are validated prior to
conducting an RCT. Future studies should also incorporate the addition of feedback to
participants to help maintain participant accountability in these self-help types of interventions
and improve the completion of these interventions. Future studies can also consider using these
technological interventions as adjuncts to treatment. It may also be helpful to incorporate more
feedback from participants (e.g., beginning with case studies interviewing participants about
their experience using a technological intervention compared to usual care); incorporate
participant choice; and consider a longer duration of study. App development typically outpaces
the research, especially for anxiety apps compared to depression apps (Aitken & Lyle, 2015). By
continuing to study these interventions for their efficacy, we can assist patients and providers,
who may already be skeptical and pessimistic about these new interventions, make informed
decisions on which apps may be most effective for their presenting problem.

MOBILE APP FOR ANXIETY 54
References

Agarwal, N., & Agarwal, S. (2012). Subclinical anxiety: Presence and implications in
hypertensive patients. The Internet Journal of Cardiology, 10. Retrieved from
http://ispub.com/IJC/10/2/14197

Ameringen, M., Turna, J., Khalesi, Z., Pullia, K., & Patterson, B. (2017). There is an app for

that! The current state of mobile applications (apps) for DSM ‐5 obsessive ‐compulsive

disorder, posttraumatic stress disorder, anxiety and mood disorders. Depression and

Anxiety, 34, 526-539. doi: 10.1002/da.22657

Ahn, T., Ryu, S., & Han, I. (2007). The impact of Web quality and playfulness on user

acceptance of online retailing. Information & Management, 44, 263–275.
doi:10.1016/j.im.2006.12.008
Aitken, M., & Lyle, J. (2015). Patient adoption of mHealth: Use, evidence and remaining
barriers to mainstream acceptance. Parsippany, NJ: IMS Institute for Healthcare
Informatics.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders
(5th ed.). Arlington, VA: American Psychiatric Publishing.
Andersson, G. (2009). Using the Internet to provide cognitive behaviour therapy. Behaviour
Research and Therapy, 47, 175-180. doi:10.1016/j.brat.2009.01.010
Andrea, H., Bultmann, U., van Amelsvoort, L. G. P. M., & Kant, Y. (2009). The incidence of
anxiety and depression among employees-The role of psychosocial work characteristics.
Depression and Anxiety, 26, 1040-1048. doi:10.1002/da.20516
Ansari, T. L., & Derakshan, N. (2011). The neural correlates of impaired inhibitory control in
anxiety. Neuropsychologia, 49, 1146-1153. doi:10.1016/j.neuropsychologia.2011.01.019
MOBILE APP FOR ANXIETY 55
Arch, J. J., Wolitzsky-Taylor, K. B., Eifer, G. H., & Craske, M. G. (2012). Longitudinal
treatment mediation of traditional cognitive behavioral therapy and acceptance and
commitment therapy for anxiety disorders. Behaviour Research and Therapy, 50, 469-
478. doi:10.1016/j.brat.2012.04.007
Azjen, I., & Fishbein, M. (1980). Understanding attitudes and predicting social behavior.
Englewood Cliffs, NJ: Prentice-Hall.
Ajzen, I. (1991). The Theory of Planned Behavior. Organizational Behavioral and Human
Decision Processes, 50, 179-211. Retrieved from http://www.journals.elsevier.com/
organizational-behavior-and-human-decision-processes/
Atienza, A. A., & Patrick, K. (2011). Mobile health: The killer app for cyberinfrastructure and
consumer health. American Journal of Preventive Medicine, 40, S151-S153. doi:10.1016/
j.amepre.2011.01.008
Baker, T. L. (1994). Doing Social Research (2
nd
Ed.). New York: McGraw-Hill Inc.
Bang, M., Timpka, T., Eriksson, H., Holm, E., & Nordin, C. (2007). Mobile phone computing
for in-situ cognitive behavioral therapy. Studies in Health Technology Informatics, 129,
1078-1082. Retrieved from http://www.iospress.nl
Beck, A. T., Epstein, N., Brown, G., & Steer, R. A. (1988). An inventory for measuring clinical
anxiety: Psychometric properties. Journal of Consulting and Clinical Psychology, 56,
893-897. doi:10.1037/0022-006X.56.6.893
Beck, A.T., & Steer, R.A. (1993). Beck Anxiety Inventory Manual. San Antonio: Harcourt Brace
and Company.
Beck, J. S. (2011). Cognitive behavior therapy: Basics and beyond, Second Edition. New York:
Guilford.
MOBILE APP FOR ANXIETY 56
Borkovec, T. D., & Costello, E. (1993). Efficacy of applied relaxation and cognitive–behavioral
therapy in the treatment of generalized anxiety disorder. Journal of Consulting and
Clinical Psychology, 61, 611–619.
Borkovec, T. D., Newman, M. G., Pincus, A. L., & Lytle, R. (2002). A component analysis of
cognitive– behavioral therapy for generalized anxiety disorder and the role of
interpersonal problems. Journal of Consulting and Clinical Psychology, 70, 288–298.
Boschen, M. J. (2009). Mobile Telephones and Psychotherapy. The Behavior Therapist, 32, 168-
181. Retrieved from http://www.abct.org/Journals/?m=mJournal&fa=TBT
Botella, C., Breton-López, J., Quero, S., Baños, R. M., García-Palacios, A., Zaragoza, I., &
Alcaniz, M. (2011). Treating cockroach phobia using a serious game on a mobile phone
and augmented reality exposure: A single case study. Computers in Human Behavior, 27,
217-227. doi:10.1016/j.chb.2010.07.043
Bouchard, S., Gauthier, J., Laberge, B., French, D., Pelletier, M. H., & Godbout, C. (1996).
Exposure versus cognitive restructuring in the treatment of panic disorder with
agoraphobia. Behaviour Research and Therapy, 34, 213-224.
Bower, J. L., & Christensen, C. M. (1995). Disruptive technologies: Catching the wave. Harvard
Business Review, 73, 43-53. Retrieved from https://hbr.org
Brendryen, H., Drozd, F., & Kraft, P. (2008). A digital smoking cessation program delivered
through internet and cell phone without nicotine replacement (Happy Ending):
Randomized controlled trial. Journal of Medical Internet Research, 10, e51. doi:
10.2196/jmir.1005
Brendryen, H., & Kraft, P. (2008). Happy Ending: A randomized controlled trial of a digital
multi-media smoking cessation intervention. Addiction, 103, 478-484. doi:
MOBILE APP FOR ANXIETY 57
10.1111/j.1360-0443.2007.02119.x
Brooke, J. (1996). SUS-A quick and dirty usability scale. In P. W. Jordan, B. Thomas, B. A.
Weerdmeester, & I. L. McClelland (Eds.), Usability evaluation in industry, (189-194).
London: Taylor & Francis.
Brug, J., Campbell, M., & Van Assema, P. (1999) The application and impact of computer-
generated personalized nutrition education: A review of the literature. Patient Education
Counsel, 36, 145-156. doi:10.1016/S0738-3991(98)00131-1
Brug, J., Oenema, A., & Campbell, M. (2003). Past, present, and future of computer-tailored
nutrition education. American Journal of Clinical Nutrition, 77, 1028S-1034S. Retrieved
from http://ajcn.nutrition.org
Burns, M. N., Begale, M., Duffecy, J., Gergle, D., Karr, C. J., Giangrande, E., & Mohr, D. C.
(2011). Harnessing context sensing to develop a mobile intervention for depression.
Journal of Medical Internet Research, 13, e55. doi:10.2196/jmir.1838
Burns, N., & Grove, S. K. (2004). The practice of nursing research (5th Edition). Philadelphia:
W.B. Saunders Company.
Butler, A. C., Chapman, J. E., Forman, E. M., & Beck, A. T. (2006). The empirical status of
cognitive-behavioral therapy: A review of meta-analyses. Clinical Psychology Review,
26, 17-31. doi:10.1016/j.cpr.2005.07.003
Chang, C.-C., Tseng, K.-H., Liang, C., & Yan, C.-F. (2013). The influence of perceived
convenience and curiosity on continuance intention in mobile English learning for high
school students using PDAs. Technology, Pedagogy and Education, 22, 373-386. doi:
10.1080/1475939X.2013.802991
Christensen, H., Griffiths, K. M., & Korten, A. (2002).Web-based cognitive behavior therapy:
MOBILE APP FOR ANXIETY 58
Analysis of site usage and changes in depression and anxiety scores. Journal of Medical
Internet Research, 4, e3. doi:10.2196/jmir.4.1.e3
Clark, D. M., Ehlers, A., Hackmann, A., McManus, F., Fennell, M., Grey, N., ... & Wild, J.
(2006). Cognitive therapy versus exposure and applied relaxation in social phobia: A
randomized controlled trial. Journal of Consulting and Clinical Psychology, 74, 568-578.
doi:10.1037/0022-006X.74.3.568
Cohen, J. (1977). Statistical power analysis for the behavioral sciences. New York: Academic
Press.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. (2nd ed.). Hillsdale, NJ:
Erlbaum.
Compuware. (2013). Mobile apps: What consumers really need and want. Retrieved from
http://offers2.compuware.com/rs/compuware/images/Mobile_App_Survey_Report.pdf
Craske, M. G. (2010). Cognitive-behavioral therapy. Washington, D.C.: American
Psychological Association.
Dagoo, J., Asplund, R. P., Bsenko, H. A., Hjerling, S., Holmberg, A.,Westh, S., … Andersson,
G. (2014). Cognitive behavior therapy versus interpersonal psychotherapy for social
anxiety disorder delivered via smartphone and computer: A randomized controlled trial.
Journal of Anxiety Disorders, 28, 410–417. doi:10.1016/j.janxdis.2014.02.003
Davies, E. B., Morriss, R., & Glazebrook, C. (2014). Computer-delivered and web-based
interventions to improve depression, anxiety, and psychological well-being of university
students: A systematic review and meta-analysis. Journal of Medical Internet Research,
16, e130. doi:10.2196/jmir.3142
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology:
MOBILE APP FOR ANXIETY 59
A comparison of two theoretical models. Management Science, 35, 982-1003. doi:
10.1287/mnsc.35.8.982
Deane, F. P., Glaser, N. M., Oades, L. G., & Kazantzis, N. (2005). Psychologists’ use of
homework assignments with clients who have schizophrenia. Clinical Psychologist, 9,
24-30. doi:10.1080/13284200500048141
Deci, E. L., & Ryan, R. M. (1985). The general causality orientations scale: Self-determination
in personality. Journal of Research in Personality, 19, 109-134. doi:10.1016/0092-
6566(85)90023-6
Dennis, T. A., & O’Toole, L. J. (2014). Mental health on the go: Effects of a gamified attention-
bias modification mobile application in trait-anxious adults. Clinical Psychological
Science, 2, 576-590. doi:10.1177/2167702614522228
Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The Satisfaction with Life Scale.
Journal of Personality Assessment, 49, 71-75. doi: 10.1207/s15327752jpa4901_13
Dobson, D., & Dobson, K. S. (2016). Evidence-based practice of cognitive-behavioral therapy.
Guilford Publications.
Dunn, H., Morrison, A. P., & Bentall, R. P. (2002). Patients’ experiences of homework tasks in
cognitive behavioral therapy for psychosis: A qualitative analysis. Clinical Psychology
and Psychotherapy, 9, 361-369. doi:10.1002/cpp.344
Durham, R. C., Murphy, T., Allan, T., Richard, K., Treliving, L. R., & Fenton, G. W. (1994).
Cognitive therapy, analytic psychotherapy and anxiety management for generalised
anxiety disorder. British Journal of Psychiatry, 165, 315–323. doi:10.1192/bjp.165.3.315
Ebner-Priemer, U.W., & Trull, T. J. (2009). Ecological momentary assessment of mood
disorders and mood dysregulation. Psychological Assessment, 21, 463-475.
MOBILE APP FOR ANXIETY 60
doi:10.1037/a0017075
Enock, P. M., Hofmann, S. G., & McNally, R. J. (2014). Attention bias modification training via
smartphone to reduce social anxiety: A randomized, controlled multi-session experiment.
Cognitive Therapy and Research, 38, 200-216. doi:10.1007/s10608-014-9606-z
Eonta, A. M., Christon, L. M., Hourigan, S. E., Ravindran, N., Vrana, S. R., & Southam-Gerow,
M. A. (2011). Using everyday technology to enhance evidence-based treatments.
Professional Psychology: Research and Practice, 42, 513-520. doi:10.1037/a0025825
Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention and behavior. Reading, MA:
Addison-Wesley.
Fisher, P. L. (2006). The efficacy of psychological treatments for generalised anxiety
disorder. In G. C. L. Davey & A. Wells (Eds.), Worry and its psychological disorders:
Theory, assessment and treatment (359-377). West Sussex: John Wiley & Sons Ltd.
Flynn, T. M., Taylor, P., & Pollard, C. A. (1992). Use of mobile phones in the behavioral
treatment of driving phobias. Journal of Behavior Therapy and Experimental
Psychiatry, 23, 299-302. Retrieved from http://www.journals.elsevier.com/journal-of-
behavior-therapy-and-experimental-psychiatry
Franco, C. M. (2012). Client motivation, working alliance, and the use of homework in
psychotherapy. (Doctoral dissertation). Retrieved from ProQuest Dissertations and
Theses database.
Gagné, M. and Deci, E. L. (2005). Self-determination theory and work motivation. Journal of
Organizational Behavior, 26, 331–362. doi:10.1002/job.322
Gartner, Inc. (2010). Gartner highlights key predictions for IT organizations and users in 2010
and beyond. Retrieved from http://www.gartner.com
MOBILE APP FOR ANXIETY 61
Givens, J. L., & Tjia, J. (2002). Depressed medical students’ use of mental health services and
barriers to use. Academic Medicine, 77, 918-921. Retrieved from
http://journals.lww.com/academicmedicine
Gorini, A., Pallavicini, F., Algeri, D., Repetto, C., Gaggioli, A., & Riva, G. (2010). Virtual
reality in the treatment of generalized anxiety disorders. Annual Review of CyberTherapy
and Telemedicine, 8, 31-35. Retrieved from http://www.arctt.info
Graham, C., Franses, A., Kenwright, M., & Marks, I. (2000). Psychotherapy by computer: A
postal survey of responders to a teletext article. The Psychiatrist, 24, 331–332. doi:
10.1192/pb.24.9.331
Greenberg, P. E., Sisitsky, T., Kessler, R. C., Finkelstein, S. N., Berndt, E. R., Davidson, J. R.,
Ballenger, J. C., & Fyer, A. J. (1999). The economic burden of anxiety disorders in the
1990s. Journal of Clinical Psychiatry, 60, 427–435. doi:10.4088/JCP.v60n0702
Guay, F., Vallerand, R. J., & Blanchard, C. (2000). On the assessment of situational intrinsic and
extrinsic motivation: The situational motivation scale (SIMS). Motivation and Emotion,
24, 175-213. doi:10.1023/A:1005614228250
Gupta, S., & Kim, H.-W. (2007). The moderating effect of transaction experience on the decision
calculus in on-line repurchase. International Journal of Electronic Commerce, 12, 127–
158. doi:10.2753/JEC1086-4415120105
Gwynn, R. C., McQuistion, H. L., McVeigh, K. H., Garg, R. K., Frieden, T. R., & Thorpe, L. E.
(2008). Prevalence, diagnosis, and treatment of depression and Generalized Anxiety
Disorder in a diverse urban community. Psychiatric Services, 59, 641-648. doi:
10.1176/ps.2008.59.6.641
Harrison, V., Proudfoot, J., Wee, P. P., Parker, G., Pavlovic, D. H., & Manicavasagar, V. (2011).
MOBILE APP FOR ANXIETY 62
Mobile mental health: Review of the emerging field and proof of concept study. Journal
of Mental Health, 20, 509-524. doi:10.3109/09638237.2011.608746
Hassija, C. M., & Gray, M. J. (2010). Are cognitive techniques and interventions necessary? A
case for the utility of cognitive approaches in the treatment of PTSD. Clinical
Psychology: Science and Practice, 17, 112–127. doi:10.1111/j.1468-2850.2010.01201.x
Helbig, S., & Fehm, L. (2004). Problems with homework in CBT: Rare exception or rather
frequent? Behavioural and Cognitive Psychotherapy, 32, 291-301. doi:
10.1017/S1352465804001365
Heron, K. E. and Smyth, J. M. (2010), Ecological momentary interventions: Incorporating
mobile technology into psychosocial and health behaviour treatments. British Journal of
Health Psychology, 15, 1–39. doi:10.1348/135910709X466063
Hertzog, M. A. (2008). Considerations in determining sample size for pilot studies. Research in
Nursing & Health, 31, 180-191. doi:10.1002/nur.20247
Hofmann, S. G., & Smits, J. A. J. (2008). Cognitive-behavioral therapy for adult anxiety
disorders: A meta-analysis of randomized placebo-controlled trials. Journal of Clinical
Psychiatry, 69, 621-632. Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/
PMC2409267/
Hossain, M. M., & Prybutok, V. R. (2008). Consumer acceptance of RFID technology: An
exploratory study. IEEE Transactions on Engineering Management, 55, 316–328. doi:
10.1109/TEM.2008.919728
Iacoviello, B. M., McCarthy, K. S., Barrett, M. S., Rynn, M., Gallop, R., & Barber, J. P. (2007).
Treatment preferences affect the therapeutic alliance: Implications for randomized
controlled trials. Journal of Consulting and Clinical Psychology, 75, 194-198.
MOBILE APP FOR ANXIETY 63
doi:10.1037/0022-006X.75.1.194
International Telecommunication Union. (2015). The world in 2015: ICT facts and figures.
Retrieved from http://www.itu.int
Johanson, G. A., & Brooks, G. P. (2010). Initial scale development: Sample size for pilot studies.
Educational and Psychological Measurement, 70, 393-400. doi:
10.1177/0013164409355692
Johanssen R., & Andersson, G. (2012). Internet-based psychological treatments for depression.
Expert Review of Neurotherapeutics, 12, 861–870. doi:10.1586/ERN.12.63
Kahneman, D., Fredrickson, B. L., Schreiber, C., & Redelmeier, D. (1993). When more pain is
preferred to less: Adding a better end. Psychological Science, 4, 401-405.
doi:10.1111/j.1467-9280.1993.tb00589.x
Karsten, J., Penninx, B. W., Verboom, C. E., Nolen, W. A., & Hartman, C. A. (2013). Course
and risk factors of functional impairment in subthreshold depression and anxiety.
Depression and Anxiety, 30, 386–394. doi:10.1002/da.22021
Kazdin, A. E. (2003). Research design in clinical psychology (Vol. 3). Boston: Allyn and Bacon.
Kessler, R. C., Aguilar-Gaxiola, S., Alonso, J., Chatterji, S., Lee, S., Ormel, J., Ustün, T. B., &
Wang, P. S. (2009). The global burden of mental disorders: an update from the WHO
World Mental Health (WMH) surveys. Epidemiologia e Psichiatria Sociale, 18, 23–33.
doi:10.1017/S1121189X00001421
Kessler, R. C., Chiu, W.T., Demler, O., & Walters, E. E. (2005). Prevalence, severity, and
comorbidity of twelve-month DSM-IV disorders in the National Comorbidity Survey
Replication (NCS-R). Archives of General Psychiatry, 62, 617-627. doi:
10.1001/archpsyc.62.6.617
MOBILE APP FOR ANXIETY 64
Kessler, R. C., & Greenberg, P. E. (2002). The economic burden of anxiety and stress disorders.
Neuropsychopharmacology: The fifth generation of progress, 67, 982-992. Retrieved
from http://www.acnp.org/Docs/G5/CH67_981-992.pdf
Kessler, R. C., Walters, E. E., & Wittchen, H-U. (2004). Epidemiology. In R. G. Heimberg, C. L.
Turk, & D. S. Mennin (Eds.), Generalized Anxiety Disorder: Advances in research and
practice (29-50). New York: The Guilford Press.
Kihlstrom, J. F., Eich, E., Sandbrand, D., & Tobias, B. A. (2000). Emotion and memory:

Implications for self-report. In A. A.Stone, & J. S. Turkkan (Eds.), Science of self-report:

Implications for research and practice (pp. 81-99). Mahwah, NJ: Erlbaum.

Lackey, N., & Wingate, A. (1998). The pilot study: One key to research success. In P. J. Brink &
M. J. Wood (Eds.), Advanced design in nursing research (375-387). Thousand Oaks, CA:
SAGE Publications, Inc.
Lai, V. S., & Li, H. (2005). Technology acceptance model for internet banking: An invariance
analysis. Information & Management, 42, 373–386. doi: 10.1016/j.im.2004.01.007
Lamers, F., van Oppen, P., Comijs, H. C., Smit, J. H., Spinhoven, P., van Balkom, Anton J. L.
M., . . . Penninx, B. W. J. H. (2011). Comorbidity patterns of anxiety and depressive
disorders in a large cohort study: The Netherlands study of depression and anxiety
(NESDA). Journal of Clinical Psychiatry, 72, 341-348. doi:10.4088/ JCP.10m06176blu
Leahy, R. L., Tirch, D., & Napolitano, L. A. (2011). Emotion regulation in psychotherapy: A
practitioner’s guide. New York: Guilford Press.
Lee, M.-C. (2010). Explaining and predicting users’ continuance intention toward e-learning: An
extension of the expectation–confirmation model. Computers & Education, 54, 506– 516.
Leon, A. C., Davis, L. L., & Kraemer, H. C. (2011). The role and interpretation of pilot studies in
MOBILE APP FOR ANXIETY 65
clinical research. Journal of Psychiatric Research, 45, 626-629. doi:10.1016/j.jpsychires.
2010.10.008
Lin, K. M. (2011). E-Learning continuance intention: Moderating effects of user e-learning
experience. Computers & Education, 56, 515–526. doi:10.1016/j.compedu.2010.09.017
Longmore, R. J., & Worrell, M. (2007). Do we need to challenge thoughts in cognitive behavior

therapy?. Clinical Psychology Review, 27, 173-187. doi:10.1016/j.cpr.2006.08.001

Luxton, D. D., McCann, R. A., Bush, N. E., Mishkind, M. C., & Reger, G. M. (2011). mHealth
for Mental Health: Integrating Smartphone Technology in Behavioral Healthcare.
Professional Psychology: Research & Practice, 42, 505-512. doi:10.1037/a0024485
Mackenzie, C. S., Reynolds, K., Cairney, J., Streiner, D. L., & Sareen, J. (2011). Disorder-
specific mental health service use for mood and anxiety disorders: Associations with age,
sex, and psychiatric comorbidity. Depression and Anxiety, 29, 234-242. doi:
10.1002/da.20911
Malvey, D., & Slovensky, D. J. (2014). mHealth: Transforming healthcare. New York: Springer.
Matthews, M., Doherty, G., Coyle, D., & Sharry, J. (2008). Designing mobile applications to
support mental health interventions. In J. Lumsden (Ed.), Handbook of research on user
interface design and evaluation for mobile technology (635–656). Hershey, PA: IGI
Global.
McAuley, E., Duncan, T., & Tammen, V. V. (1987). Psychometric properties of the Intrinsic
Motivation Inventory in a competitive sport setting: A confirmatory factor analysis.
Research Quarterly for Exercise and Sport, 60, 48-58. doi:10.1080/02701367.1989.
10607413
Michie, S., Abraham, C., Whittington, C., McAteer, J., & Gupta, S. (2009). Effective techniques
MOBILE APP FOR ANXIETY 66
in healthy eating and physical activity interventions: A meta-regression. Health
Psychology, 28, 690-701. doi:10.1037/a0016136
Mitte, K. (2005). Meta-analysis of cognitive-behavioral treatments for generalized anxiety
disorder: a comparison with pharmacotherapy. Psychological Bulletin, 131, 785.
doi:10.1037/0033-2909.131.5.785
Mohr, D. C., Burns, M. N., Schueller, S. M., & Clarke, G. (2013). Behavioral intervention
technologies: Evidence review and recommendations for future research in mental health.
General Hospital Psychiatry, 35, 332-338. doi:10.1016/j.genhosppsych.2013.03.008
Mohr, D. C., Hart S. L., Howard, I., Julian, L., Vella, L., Catledge, C., & Feldman, M. D. (2006).
Barriers to psychotherapy among depressed and nondepressed primary care patients.
Annals of Behavioral Medicine, 32, 254-258. doi: 10.1207/s15324796abm3203_12
Mohr, D.C., Ho, J., Duffecy, J., Baron, K.G., Lehman, K.A., Jin, L., & Reifler, D. (2010).
Perceived barriers to psychological treatments and their relationship to depression.
Journal of Clinical Psychology, 66, 394–409. doi:10.1002/jclp.20659
Mohr, D.C., Noth, T. K., Lattie, E.G., Palac, H.L., Kwasny, M.J., Weingardt, K., . . . Schueller,

S.M. (2017). IntelliCare: An eclectic, skills-based app suite for the treatment of

depression and anxiety. Journal of Medical Internet Research, 19, e10.

doi:10.2196/jmir.6645

Mundt, J. C., Marks, I. M., Shear, M. K., & Greist, J. M. (2002). The Work and Social
Adjustment Scale: A simple measure of impairment in functioning. The British Journal
of Psychiatry, 180, 461-464. doi:10.1192/bjp.180.5.461
Naughton, F., Jamison, J., & Sutton, S. (2013). Attitudes towards SMS text message smoking
cessation support: A qualitative study of pregnant smokers. Health Education Research,
MOBILE APP FOR ANXIETY 67
28, 911-922. doi:10.1093/her/cyt057
Nunnally, J. (1978). Psychometric Theory. New York: McGraw-Hill.
Oenema, A., Brug, J., & Lechner, L. (2001). Web-based tailored nutrition education: Results of a
randomized controlled trial. Health Education Research, 16, 647-660. doi:10.1093/her/
16.6.647
Olatunji, B., Cisler, J. M., & Deacon, B. J. (2010). Efficacy of cognitive behavioral therapy for
anxiety disorders: A review of meta-analytic findings. Psychiatric Clinics of North
America, 33, 557-577. doi:10.1016/j.psc.2010.04.002
Otto, M.W., Smits, J. A., Reese, H.E. (2005). Combined psychotherapy and pharmacotherapy
for mood and anxiety disorders in adults: review and analysis. Clinical Psychology:
Science and Practice, 27, 572–581. doi:10.1176/foc.4.2.204
Pallavicini, F., Algeri, D., Repetto, C., Gorini, A., & Riva, G. (2009). Biofeedback, virtual reality
and mobile phones in the treatment of Generalized Anxiety Disorder (GAD): A phase-2
controlled clinical trial. Journal of CyberTherapy & Rehabilitation, 2, 315-327. Retrieved
from http://journalofcybertherapy.webs.com
Park, S. Y., Nam, M. W., & Cha, S. B. (2011). University students’ behavioral intention to use
mobile learning: Evaluating the technology acceptance model. British Journal of
Educational Technology, 43, 592-605. doi:10.1111/j.1467-8535.2011.01229.x
Pavot, W., & Diener, E. (1993). Review of the satisfaction with life scale. Psychological
Assessment, 5, 164-172. doi:10.1037/1040-3590.5.2.164
Persons, J. B., Burns, D. D., & Perloff, J. M. (1988). Predictors of dropout and outcome in
cognitive therapy for depression in a private practice setting. Cognitive Therapy and
Research, 12, 557-575. doi:10.1007/BF01205010
MOBILE APP FOR ANXIETY 68
Petty, R. E., & Cacioppo, J. T. (1981). Attitudes and persuasion: Classic and contemporary
approaches. Dubuque, IA: Wm. C. Brown.
Pham, Q., Khatib, Y., Stansfeld, S., Fox, S., & Green, T. (2016). Feasibility and efficacy of an
mHealth game for managing anxiety: “Flowy” randomized controlled pilot trial and
design evaluation. Games for Health Journal, 5, 50-67. doi:10.1089/g4h.2015.0033
Polit, D. F., & Beck, C. T. (2004). Nursing research: Appraising evidence for nursing practice
(7th Edition). Philadelphia: Wolters Klower/Lippincott Williams & Wilkins.
Ponniah, K., & Hollon, S. D. (2009). Empirically supported psychological treatments for adult
acute stress disorder and posttraumatic stress disorder: A review. Depression and
Anxiety, 26, 1086-1109. doi:10.1002/da.20635
Possemato, K., Kaier, E., Wade, M., Lantinga, L. J., Maisto, S. A., & Ouimette, P. (2012).
Assessing daily fluctuations in Posttraumatic Stress Disorder symptoms and substance
use with interactive voice response technology: Protocol compliance and reactions.
Psychological Services, 9, 185-196. doi:10.1037/a0027144
Price, M., Davidson, T., Andrews, J., & Ruggiero, K.J. (2013). Access, use and completion of a
brief disaster mental health intervention among Hispanics, African-Americans and
Whites affected by Hurricane Ike. Journal of Telemedicine and Telecare, 19, 70-74. doi:
10.1177/1357633X13476230
Price, M., Yuen, E. K., Goetter, E. M., Herbert, J. D., Forman, E. M., Acierno, R., & Ruggiero,
K. J. (2014). mHealth: A mechanism to deliver more accessible, more effective mental
health care. Clinical Psychology and Psychotherapy, 21, 427-436. doi:10.1002/cpp.1855
Proudfoot, J., Parker, G., Pavlovic, D.H., Manicavasagar, V., Adler, E., & Whitton, A. (2010).
Community attitudes to the appropriation of mobile phones for monitoring and managing
MOBILE APP FOR ANXIETY 69
depression, anxiety and stress. Journal of Medical Internet Research, 12, e64. doi:
10.2196/jmir.1475.
Riley, W. T., Rivera, D. E., Atienza, A. A., Nilsen, W., Allison, S. M., & Mermelstein, R.
(2011). Health behavior models in the age of mobile interventions: Are our theories up to
the task?. Translational Behavioral Medicine, 1, 53-71. doi:10.1007/s13142-011-0021-7
Rizvi, S. L., Dimeff, L. A., Skutch, J., Carroll, D., & Linehan, M. M. (2011). A pilot study of the
DBT coach: An interactive mobile phone application for individuals with Borderline
Personality Disorder and Substance Use Disorder. Behavior Therapy, 42, 589-600.
Retrieved from http://www.elsevier.com/locate/bt
Ryan, R. M. (1982). Control and information in the intrapersonal sphere: An extension of
cognitive evaluation theory. Journal of Personality and Social Psychology, 43, 450-
461. doi:10.1037/0022-3514.43.3.450
Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic
motivation, social development, and well-being. American Psychologist, 55, 68–78. doi:
10.1037/0003-066X.55.1.68
Ryan, R. M., Rigby, C. S., & Przybylski, A. (2006). The motivational pull of video games: A
self-determination theory approach. Motivation and Emotion, 30, 344-360. doi:
10.1007/s11031-006-9051-8
Sauro, J. (2011). A practical guide to the System Usability Scale: Background, benchmarks, &

best practices. Denver, CO: Measuring Usability LLC.

Schmidt, N. B., & Woolaway-Bickel, K. (2000). The effects of treatment compliance on
outcome in cognitive-behavioral therapy for panic disorder: Quality versus quantity.
Journal of Counseling and Clinical Psychology, 68, 13-18. doi:10.1037//0022-
MOBILE APP FOR ANXIETY 70
006X.68.1.13
Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist,
54, 93-105. doi:10.1037/0003-066X.54.2.93
Sheehan D.V., Lecrubier, Y., Sheehan, K.H., Sheehan, K., Amorim, P., Janavs, J., ... & Dunbar,
G..(1998). The Mini-International Neuropsychiatric Interview (M.I.N.I.): The
development and validation of a structured diagnostic psychiatric interview for DSM-IV
and ICD-10. Journal of Clinical Psychiatry, 59, 22–33. Retrieved from
https://sts.psychiatrist.com
Sheehan, D. (2015). M.I.N.I: Mini International Neuropsychiatric Interview English version
7.0.0 for DSM-5. Retrieved from David V. Sheehan, M.D., M.B.A.
Shiffman, S., Stone, A.A., & Hufford, M.R. (2008). Ecological momentary assessment. Annual
Review of Clinical Psychology, 4, 1–32. doi:10.1146/annurev.clinpsy.3.022806.091415
Smith, A. (2010). Mobile access 2010. Pew Internet & American Life Project, July 7, 2010.
Retrieved from http://www.pewinternet.org/Reports/2010/Mobile-Access-2010.aspx
Spaniel, F., Vohlídka, P., Hrdlicka, J., Kozeny, J., Novak, T., Motlová, L., . . . Hoschl, C. (2008).
ITAREPS: Information technology aided relapse prevention programme in
schizophrenia. Schizophrenia Research, 98, 312-317. doi:10.1016/j.schres.2007.09.005
Spitzer, R.L., Kroenke, K., Williams, J. B., & Löwe, B. (2006). A brief measure for assessing
generalized Anxiety disorder: the GAD7. Archives of Internal Medicine, 166, 1092–1097.
doi:10.1001/archinte.166.10.1092
Spitzer, R.L., Kroenke, K., Williams, J.B., & Patient Health Questionnaire Primary Care Study
Group. (1999). Validation and utility of a self-report version of PRIME-MD: the PHQ
primary care study. JAMA, 282, 1737–1744. doi:10.1001/jama.282.18.1737
MOBILE APP FOR ANXIETY 71
Stone, A. A., Shiffman, S., Schwartz, J. E., Broderick, J.E., Hufford, M.R. (2002). Patient
noncompliance with paper diaries. British Medical Journal, 324, 1193–1194. doi:
10.1136/bmj.324.7347.1193
Sucala, M., Cuijpers, P., Muench, F., Cardoș, R., Soflau, R., Dobrean, A., ... & David, D. (2017).

Anxiety: There is an app for that. A systematic review of anxiety apps. Depression and

Anxiety, 34, 518-525. doi:10.1002/da.22654

Sutton, C. S., & Dixon, D. N. (1986). Resistance in parent training: A study of social influence.
Journal of Social and Clinical Psychology, 4, 133-141. doi:10.1521/jscp.1986.4.2.133
Thornton, P., & Houser, C. (2002). M-learning: Learning in transit. In P. Lewis (Ed.), The
changing face of CALL: A Japanese perspective (229–243). The Netherlands: Swets
& Zeitlinger.
Van, H. L., Dekker, J., Koelen, J., Kool, S., van Aalst, G., Hendriksen, M., Peen, J., &
Schoevers, R. (2009). Patient preference compared with random allocation in short-term
psychodynamic supportive psychotherapy with indicated addition of pharmacotherapy for
depression. Psychotherapy Research, 19, 205-212. doi:10.1080/10503300802702097
Vivyan, C. (2011). Self help & therapist resources. Retrieved from http://getselfhelp.co.uk
Vogel, P. A., Launes, G., Moen, E. M., Solem, S., Hansen, B., Haland, A. T., & Himle, J. A.
(2012). Videoconference- and cell phone-based cognitive-behavioral therapy of
obsessive-compulsive disorder: A case series. Journal of Anxiety Disorders, 26, 158-164.
doi:10.1016/j.janxdis.2011.10.009
Wang, Y.-S., Wu, M.-C., & Wang, H.-Y. (2009). Investigating the determinants and age and
gender differences in the acceptance of mobile learning. British Journal of Educational
Technology, 40, 92–118. doi:10.1111/j.1467-8535.2007.00809.x
Wantland, D. J., Portillo, C. J., Holzemer, W. L., Slaughter, R., & McGhee, E. M. (2004). The
MOBILE APP FOR ANXIETY 72
effectiveness of web-based vs. non-web-based interventions: A meta-analysis of
behavioral change outcomes. Journal of Medical Internet Research, 6, e40. doi:
10.2196/jmir.6.4.e40
Watts, S., Mackenzie, A., Thomas, C., Griskaitis, A., Mewton, L., Williams, A., & Andrews, G.
(2013). CBT for depression: A pilot RCT comparing mobile phone vs. computer. BMC
Psychiatry, 13, 49. doi:10.1186/1471-244X-13-49
Wells, A. (2002). Emotional disorders and metacognition: Innovative cognitive therapy. John

Wiley & Sons.

Westen, D., & Morrison, K. (2001). A multidimensional meta-analysis of treatments for
depression, panic, and Generalized Anxiety Disorder: An empirical examination of the
status of empirically supported therapies. Journal of Consulting and Clinical Psychology,
69, 875-899. doi:10.1037//0022-006X.69.6.875
Whittal, M. L., Robichaud, M., Thordarson, D. S., & McLean, P. D. (2008). Group and
individual treatment of obsessive-compulsive disorder using cognitive therapy and
exposure plus response prevention: A 2-year follow-up of two randomized trials. Journal
of Consulting and Clinical Psychology, 76, 1003-1014. doi:10.1037/a0013076
Wilcox, R. (2011). Modern statistics for the social and behavioral sciences: A practical
introduction. Boca Raton: CRC Press.
Wittchen, H. U. (2002). Generalized anxiety disorder: Prevalence, burden, and cost to society.
Depression and Anxiety, 16, 162-171. doi:10.1002/da.10065
World Health Organization, Calouste Gulbenkian Foundation. (2017). Improving access to and
appropriate use of medicines for mental disorders. Geneva: World Health Organization.
Licence: CC BY-NC-SA 3.0 IGO.
MOBILE APP FOR ANXIETY 73
Wu, J. H., & Wang, S. C. (2005). What drives mobile commerce?: An empirical evaluation of
the revised technology acceptance model. Information & Management, 42, 719-729.
doi:10.1016/j.im.2004.07.001
Yoon, C., & Kim, S. (2007). Convenience and TAM in a ubiquitous computing environment:
The case of wireless LAN. Electronic Commerce Research & Applications, 6,
102–112. doi:10.1016/j.elerap.2006.06.009
Zorilla, E. P., Redei, E., & DeRubeis, R. J. (1994). Reduced cytokine levels and t-cell function in
healthy males: Relation to individual differences in subclinical anxiety. Brain Behavior,
and Immunity, 8, 293-312. doi:10.1006/brbi.1994.1028

MOBILE APP FOR ANXIETY 74
Tables
Table 1
Reasons for Exclusion in the Randomized Controlled Trial
n % of total
screened
Insufficient anxiety symptoms 118 26.9
Lack of response 87 19.8
Symptom severity 75 17.1
Lack of iOS or Android OS 6 1.4
Lack of smartphone 5 1.1
Lack of functioning email address 4 0.9
Lack of Internet data connection 3 0.7
Unstable dosage of psych. medication 2 0.4
Lack of computer w/ Internet 1 0.2
Lack of English fluency 1 0.2
Under 18 years old 1 0.2
Co-occurring exclusionary disorder 1 0.2

MOBILE APP FOR ANXIETY 75
Table 2
Descriptive Statistics and Results of One-Way ANOVA for Continuous Variables
n M SD F(2,132) p
Age .10 .91
App 47 22.89 8.83
Website 44 25.27 6.63
Waitlist 44 22.18 7.69
Hours per day on mobile phone .10 .91
App 47 5.46 4.73
Website 44 5.09 3.80
Waitlist 44 5.23 3.53
Hours per day on computer 1.85 .16
App 47 6.39 3.87
Website 44 5.14 2.55
Waitlist 44 5.81 2.71
BAI .06 .94
App 47 11.23 6.63
Website 44 10.75 5.13
Waitlist 44 10.84 8.59
GAD-7 .02 .98
App 47 6.32 4.30
Website 44 6.41 4.38
Waitlist 44 6.20 4.59
PHQ-9 .44 .65
App 47 5.87 4.04
Website 44 6.25 5.03
Waitlist 44 5.32 5.04
Note. BAI = Beck Anxiety Inventory; GAD-7 = Generalized Anxiety Disorder – 7 questions; PHQ-9 = Patient
Health Questionniare-9.

MOBILE APP FOR ANXIETY 76
Table 3
Descriptive Statistics and Results of Chi-Square Analyses for Categorical Variables

n’s
App Website Waitlist
 
df p
Highest level of education collapsed 4.84 2 .09
Some/all of high school 17 7 13
At least some college 30 37 31
Year in school collapsed .21 2 .90
Underclassmen (freshmen, sophomores) 22 20 19
Upperclassmen (including graduate students) 22 21 23
Income Collapsed 1.99 2 .37
< $75,000 26 22 17
$75,000 + 21 22 25
Gender 5.58 2 .06
Female 30 34 38
Male 16 10 6
MOBILE APP FOR ANXIETY 77
Table 4
Differences in Continuous Background Variables Amongst Those Who Did (n = 110) and Did
Not (n = 25) Complete the Study

M SD t 133 p
Age .34 .74
Completers 22.66 7.74
Non-completers 22.08 7.90
Hours per day on mobile phone .36 .72
Completers 5.32 4.11
Non-completers 5.00 3.83
Hours per day on computer .15 .88
Completers 5.81 3.06
Non-completers 5.70 3.54
BAI .92 .36
Completers 11.21 6.96
Non-completers 9.80 6.53
GAD-7 .59 .55
Completers 6.42 4.34
Non-completers 5.84 3.68
PHQ-9 .60 .55
Completers 5.70 4.57
Non-completers 6.32 5.26
WSAS .47 .64
Completers 10.30 8.27
Non-completers 11.16 7.93
SWLS .56 .57
Completers 22.82 6.04
Non-completers 22.04 7.05
Note. BAI = Beck Anxiety Inventory; GAD-7 = Generalized Anxiety Disorder – 7 questions; PHQ-9 = Patient
Health Questionnaire-9; WSAS = Work and Social Adjustment Scale; SWLS = Satisfaction with Life Scale.

MOBILE APP FOR ANXIETY 78
Table 5
Differences in Categorical Background Variables Amongst Those Who Did (n = 110) and Did
Not (n = 25) Complete the Study

n’s
Completers Non-
Completers
 
df p
Highest level of education collapsed .33 1 .57
Completed high school 29 8
Completed college 81 17
Year in school collapsed .05 1 .83
Underclassmen (freshmen, sophomores) 49 12
Upperclassmen (including graduate students) 54 12
Income Collapsed .63 1 .43
< $75,000 51 14
$75,000 + 57 11
Gender 5.09 1 .02
Female 88 14
Male 22 10
MOBILE APP FOR ANXIETY 79
Table 6

Correlations Between Number of Entries and T1, T2, and T3 Symptom Questionnaires

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13 14. 15. 16.
1. No. of entries - - - - - - - - - - - - - - - -
2. T1 BAI .28** - - - - - - - - - - - - - - -
3. T1 GAD-7 .04 .49** - - - - - - - - - - - - - -
4. T1 PHQ-9 .12 .30** .58** - - - - - - - - - - - - -
5. T1 WSAS .00 .35** .54** .62** - - - - - - - - - - - -
6. T1 SWLS .11 -.28** -.35** -.45** -.41** - - - - - - - - - - -
7. T2 BAI .15 .54** .39** .38** .40** -.26** - - - - - - - - - -
8. T2 GAD-7 .05 .28** .58** .54** .47** -.26* .62** - - - - - - - - -
9. T2 PHQ-9 .21 .38** .55** .75** .59** -.38** .62** .74** - - - - - - - -
10. T2 WSAS .15 .31** .50** .60** .82** -.33** .51** .56** .71** - - - - - - -
11. T2 SWLS -.10 -.30** -.29** -.50** -.48** .79** -.33** -.31** -.42** -.48** - - - - - -
12. T3 BAI .08 .51** .54** .47** .53** -.32** .62** .52** .61** .57** -.35** - - - - -
13. T3 GAD-7 .06 .35** .75** .51** .50** -.23* .46** .68** .65** .58** -.21 .73** - - - -
14. T3 PHQ-9 -.14 .24* .53** .65** .54** -.32** .52** .61** .75** .61** -.32** .69** .73** - - -
15. T3 WSAS .04 .28** .52** .54** .69** -.32** .42** .50** .58** .76** -.35** .60** .62** .63** - -
16. T3 SWLS .21 -.09 -.25* -.39** -.40** .65** -.23* -.38** -.36** -.40** .77** -.33** -.30** -.46** -.37** -
Note. T1 = Time 1 or beginning of the study; T2 = Time 2 or midpoint of the study; T3 = Time 3 or end of the study. BAI = Beck Anxiety Inventory; GAD-7 = Generalized Anxiety Disorder – 7
questions; PHQ-9 = Patient Health Questionniare-9; WSAS = Work and Social Adjustment Scale; SWLS = Satisfaction with Life Scale. *p < .05. **p < .01.
MOBILE APP FOR ANXIETY 80
Table 7
Means and Standard Deviations of Repeated Measures MANOVA Variables

Measure Modality Time 1 Time 2 Time 3
M SD M SD M SD
BAI App 12.69 6.49 13.15 8.28 11.12 8.07
Website 10.90 5.76 11.60 7.97 10.40 8.70
Waitlist 10.83 9.10 9.53 7.60 8.14 7.20

GAD-7 App 6.69 4.07 7.12 4.38 5.92 3.96
Website 6.50 5.16 6.90 5.41 6.75 5.89
Waitlist 6.11 4.66 6.25 5.44 5.25 4.84

PHQ-9 App 6.23 4.39 7.19 5.86 5.65 4.84
Website 6.30 5.30 7.55 4.81 6.65 6.06
Waitlist 4.94 4.72 5.17 5.06 4.72 4.61

WSAS App 9.23 7.70 10.69 9.63 10.42 10.38

Website 12.30 8.75 14.40 10.51 13.30 10.26
Waitlist 9.92 8.36 10.24 8.64 8.97 9.33

SWLS App 23.58 5.99 23.81 6.63 24.69 5.73
Website 23.85 5.07 23.10 6.23 24.55 5.61
Waitlist 23.27 6.97 25.32 6.92 26.43 6.45

PCQ App 17.57 2.96 17.43 2.93 16.24 4.10
Website 14.44 5.83 12.00 5.98 13.44 5.10

IMI-IE App 4.14 0.98 3.73 1.13 3.76 1.37
Website 4.04 1.15 3.57 1.64 3.57 1.50

IMI-PCo App 4.43 1.00 4.21 1.13 4.71 1.08
Website 4.68 1.09 4.55 1.43 4.50 1.43

IMI-EI App 4.72 1.24 4.45 1.20 4.19 1.13
Website 4.35 1.59 4.76 1.56 4.58 1.28

IMI-PT App 2.90 1.04 3.70 1.37 3.65 1.26
Website 2.81 1.18 3.94 1.27 3.91 1.30

IMI-PCh App 5.61 1.12 5.05 1.53 4.98 1.32
Website 5.41 0.95 5.01 1.56 5.07 1.41
MOBILE APP FOR ANXIETY 81

Note. BAI = Beck Anxiety Inventory; GAD-7 = Generalized Anxiety Disorder – 7 questions; PHQ-9 = Patient Health Questionniare-9; WSAS =
Work and Social Adjustment Scale; SWLS = Satisfaction with Life Scale; PCQ = Perceived Convenience Questionnaire; IMI = Intrinsic
Motivation Inventory; IE = Interest/Enjoyment Subscale; PCo = Perceived Competence Subscale; EI = Effort/Importance Subscale; PT =
Pressure and Tension Subscale; PCh = Perceived Choice Subscale; VU = Value/Usefulness Subscale.

Measure Modality Time 1 Time 2 Time 3
M SD M SD M SD
IMI-VU App 5.26 0.97 4.81 1.29 4.71 1.38
Website 4.92 1.40 4.53 1.95 4.33 1.84
MOBILE APP FOR ANXIETY 82
Table 8

Descriptive Statistics for the Effect of Time on IMI Subscales for the Total Sample

Note. Time 1 = beginning of the study; Time 2 = midpoint of the study; Time 3 = end of the study. IMI = Intrinsic Motivation Inventory; IE =
Interest/Enjoyment Subscale; PT = Pressure and Tension Subscale; PCh = Perceived Choice Subscale; VU = Value/Usefulness Subscale.

M SD
IMI-IE
Time 1 3.91 0.99
Time 2 3.61 1.31
Time 3 3.56 1.34
IMI-PT
Time 1 2.93 1.15
Time 2 3.63 1.33
Time 3 3.55 1.27
IMI-PCh
Time 1 5.33 1.28
Time 2 5.05 1.41
Time 3 5.06 1.34
IMI-VU
Time 1 4.83 1.34
Time 2 4.58 1.47
Time 3 4.70 2.49
MOBILE APP FOR ANXIETY 83
Table 9

Correlations Between Number of Entries, Convenience, Motivation, and Usability

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13 14. 15. 16.
1. No. of entries - - - - - - - - - - - - - - - -
2. T1 PCQ .07 - - - - - - - - - - - - - - -
3. T1 IMI-IE .14 .18 - - - - - - - - - - - - - -
4. T1 IMI-PCo -.04 .04 .32** - - - - - - - - - - - - -
5. T1 IMI-EI -.03 .14 .33** .36** - - - - - - - - - - - -
6. T1 IMI-PT -.08 -.03 .03 -.01 .23* - - - - - - - - - - -
7. T1 IMI-PCh .08 .11 .37** .18 .12 .01 - - - - - - - - - -
8. T1 IMI-VU .15 .28* .63** .28* .35** .02 .37** - - - - - - - - -
9. T2 PCQ .22 .49** .23 -.03 .21 .11 .17 .36** - - - - - - - -
10. T2 IMI-IE .06 .26* .54** .13 .33* -.04 .14 .48** .41** - - - - - - -
11. T2 IMI-PCo .03 .03 .15 .37* .14 -.01 .18 .37** .29* .43** - - - - - -
12. T2 IMI-EI .17 .10 .58** .39** .60** .26 .26 .58** .35** .57** .53** - - - - -
13. T2 IMI-PT -.17 -.08 .00 .10 .11 .40** -.03 -.04 -.30* -.30* -.28* .01 - - - -
14. T2 IMI-PCh -.03 .18 .47** .07 .29* .13 .49** .47** .33* .56** .44** .54** -.21 - - -
15. T2 IMI-VU .08 .28* .65** .22 .35* .03 .21 .69** .37** .82** .50** .58** -.21 .54** - -
16. T2 SUS -.04 -.03 .16 -.02 .27 -.02 -.22 .05 -.08 .01 -.06 .05 .00 -.20 .03 -

MOBILE APP FOR ANXIETY 84
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13 14. 15. 16.
17. T3 PCQ .30* .47** .23 .17 .14 .25 .01 .19 .69** .18 .03 .08 -.09 -.02 .26 .05
18. T3 IMI-IE .14 .16 -.69** .17 .23 -.35* .23 .61** .31* .74** .32* .52** -.27 .56** .73** .03
19. T3 IMI-PCo .37* .08 .38* .35* .02 -.18 .12 .41** .23 .34* .80** .35* -.30* .28 .43** -.01
20. T3 IMI-EI .16 .11 .52** .29 .57** -.02 .40* .47** .14 .37* .33* .76** .07 .31* .47** .00
21. T3 IMI-PT .09 -.17 .13 .31 .27 .27 .16 .20 --.30* -.16 .06 .28 .66** =.22 .04 .11
22. T3 IMI-PCh .07 .28 .40* .07 .17 .06 .66** .49** .38* .47** .33* .45** -.18 .83** .40** -.40**
23. T3 IMI-VU .10 .29* .67** .22 .31* -.03 .22 .79** .32* .72** .43** .42** -.14 .50** .90** -.11
24. T3 SUS -.03 .01 -.08 -.03 .12 -.02 .08 -.07 .08 -.12 -.11 -.07 .07 -.04 -.08 .58**

17. 18. 19. 20. 21. 22. 23. 24.
17. T3 PCQ - - - - - - - -
18. T3 IMI-IE .04 - - - - - - -
19. T3 IMI-PCo .20 .36* - - - - - -
20. T3 IMI-EI .00 .55** .28 - - - - -
21. T3 IMI-PT -.13 -.13 .03 .27 - - - -
22. T3 IMI-PCh .07 .43** .28 .37** -.17 - - -
23. T3 IMI-VU .17 .70** .41** .44** .08 .40** - -
24. T3 SUS .05 -.08 -.20 -.06 .10 -.12 -.07 -
Note. T1 = Time 1 or beginning of the study; T2 = Time 2 or midpoint of the study; T3 = Time 3 or end of the study. No. of entries = number of entries or adherence; PCQ = Perceived Convenience
Questionnaire; IMI = Intrinsic Motivation Inventory; IE = Interest/Enjoyment Subscale; PCo = Perceived Competence Subscale; EI = Effort/Importance Subscale; PT = Pressure and Tension
Subscale; PCh = Perceived Choice Subscale; VU = Value/Usefulness Subscale; SUS = Subjective Usability Scale. *p < .05. **p < .01.
MOBILE APP FOR ANXIETY 85
Table 10

Correlations Between Number of Entries, Convenience, Competence, and Anxiety Symptoms

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13
1. No. of entries - - - - - - - - - - - - -
2. T1 PCQ .07 - - - - - - - - - - - -
3. T1 IMI-PCo -.04 .03 - - - - - - - - - - -
4. T1 BAI .28** .11 -.07 - - - - - - - - - -
5. T1 GAD-7 .04 .01 .07 .49** - - - - - - - - -
6. T2 PCQ .22 .49** -.03 -.22 -.09 - - - - - - - -
7. T2 IMI-PCo .03 .03 .37** -.15 .19 .29* - - - - - - -
8. T2 BAI .15 -.01 .01 .54** .39** -.09 -.20 - - - - - -
9. T2 GAD-7 .05 .00 .01 .28** .58** -.11 -.06 .62** - - - - -
10. T3 PCQ .30* .47** .17 .06 -.17 .69** .03 -.12 -.09 - - - -
11. T3 IMI-PCo .37* .08 .35* .01 .15 .23 .80** -.03 -.01 .20 - - -
12. T3 BAI .08 -.24 -.17 .51** .54** -.26 -.20 .62** .52** -.17 -.15 - -
13. T3 GAD-7 .06 -.26 -.06 .35** .75** -.28 -.05 .46** .68** -.27 -.11 .73** -
Note. T1 = Time 1 or beginning of the study; T2 = Time 2 or midpoint of the study; T3 = Time 3 or end of the study. No. of entries = number of entries or adherence; PCQ = Perceived Convenience
Questionnaire; IMI = Intrinsic Motivation Inventory; PCo = Perceived Competence Subscale; BAI = Beck Anxiety Inventory; GAD-7 = Generalized Anxiety Disorder – 7 questions. *p < .05. **p <
.01.
MOBILE APP FOR ANXIETY 86
Figures

Figure 1

Example of a Thought Record from the Pilot Study

What anxious thought am I having?
School and two jobs will be overwhelming to handle

What evidence do I have that _____? What evidence do I have that the opposite is true?
I've missed orientation, social events, and two labs prior to starting school today.
I will be telecommuting for one job and in person for another working 40 hours
total while attending school.

My first class hasn't even started yet, and I have not missed a work assignment or
started my second job yet

Is there another point of view? Is there another explanation for _____?
I'm relating my grad school experience to my undergrad

It's possible that in the past school and forming social relationships at school has
caused me to feel anxious, nervous, etc.

What would I say to a close friend who was feeling this way? Is it different from what I
say to myself? Why is that?
I would tell them to relax, take things one step at a time.

This is different from what I would say to myself because I am more harsh on
myself

MOBILE APP FOR ANXIETY 87
Figure 2

Means of the Three Groups’ Beck Anxiety Inventory (BAI) Scores Across Time

MOBILE APP FOR ANXIETY 88
Figure 3

Means of the Three Groups’ Generalized Anxiety Disorder - 7 Questions (GAD-7) Scores Across
Time

MOBILE APP FOR ANXIETY 89
Figure 4

Means of the Three Groups’ Patient Health Questionnaire-9 (PHQ-9) Scores Across Time

MOBILE APP FOR ANXIETY 90
Figure 5a

Weekly Average Pre and Post 1-10 Scores of Anxiety for App and Website Users Across Six
Weeks with Person Centering by Subtracting Overall Averages

Figure 5b

Weekly Average Pre and Post 1-10 Scores of Anxiety for App and Website Users Across Six
Weeks with Person Centering by Subtracting Pre and Post Averages

MOBILE APP FOR ANXIETY 91
Figure 6a

Average Pre and Post 1-10 scores of Anxiety for App and Website Users at Week 1 and Week 5
with Person Centering by Subtracting Overall Averages

Figure 6b

Average Pre and Post 1-10 Scores of Anxiety for App and Website Users at Week 1 and Week 6
with Person Centering by Subtracting Overall Averages

MOBILE APP FOR ANXIETY 92
Figure 7a

Average Pre and Post 1-10 Scores of Anxiety for App and Website Users at Week 1 and Week 5
with Person Centering by Subtracting Pre and Post Averages

Figure 7b

Average Pre and Post 1-10 Scores of Anxiety for App and Website Users at Week 1 and Week 6
with Person Centering by Subtracting Pre and Post Averages

MOBILE APP FOR ANXIETY 93
Figure 8

Means of the Three Groups’ Work and Social Adjustment Scale (WSAS) Scores Across Time

MOBILE APP FOR ANXIETY 94
Figure 9

Means of the Three Groups’ Satisfaction with Life Scale (SWLS) Scores Across Time

MOBILE APP FOR ANXIETY 95
Figure 10

Means of the Two Intervention Groups’ Perceived Convenience Questionnaire (PCQ) Scores
Across Time

MOBILE APP FOR ANXIETY 96
Figure 11

Means of the Two Intervention Groups’ Intrinsic Motivation Inventory (IMI) –
Interest/Enjoyment (IE) Subscale Scores Across Time

MOBILE APP FOR ANXIETY 97
Figure 12

Means of the Two Intervention Groups’ Intrinsic Motivation Inventory (IMI) – Perceived
Competence (PCo) Subscale Scores Across Time

MOBILE APP FOR ANXIETY 98
Figure 13

Means of the Two Intervention Groups’ Intrinsic Motivation Inventory (IMI) – Effort/Importance
(EI) Subscale Scores Across Time

MOBILE APP FOR ANXIETY 99
Figure 14

Means of the Two Intervention Groups’ Intrinsic Motivation Inventory (IMI) – Pressure and
Tension (PT) Subscale Scores Across Time

MOBILE APP FOR ANXIETY 100
Figure 15

Means of the Two Intervention Groups’ Intrinsic Motivation Inventory (IMI) – Perceived Choice
(PCh) Subscale Scores Across Time

MOBILE APP FOR ANXIETY 101
Figure 16

Means of the Two Intervention Groups’ Intrinsic Motivation Inventory (IMI) – Value/Usefulness
(VU) Subscale Scores Across Time

MOBILE APP FOR ANXIETY 102
Figure 17

Distribution of Number of Entries Completed Over the Duration of the Study by App and Website
Participants

MOBILE APP FOR ANXIETY 103
Figure 18

First Example of a Thought Record from the Randomized Controlled Trial

What anxious thought am I having?
I have a lot of homework to do once I return back to LA tomorrow

What evidence do I have that _____? What evidence do I have that the opposite is true?
I have homework piled up from several days, and I haven’t started it yet

I have a couple of days before the deadline, so it’s not a big task

Is there another point of view? Is there another explanation for _____?
Yes, I may not be able to complete the full homework, but I may do some

What would I say to a close friend who was feeling this way? Is it different from what I
say to myself? Why is that?
I would tell him to enjoy this period of time and then work hard and focus on
studies. Same as I’m telling myself

MOBILE APP FOR ANXIETY 104
Figure 19

Second Example of a Thought Record from the Randomized Controlled Trial

What anxious thought am I having?
I had a weird dream during my nap

What evidence do I have that _____? What evidence do I have that the opposite is true?
I thought I missed my midterm tomorrow

I didn’t miss it

Is there another point of view? Is there another explanation for _____?
I could be anxious that I haven’t studied enough

What would I say to a close friend who was feeling this way? Is it different from what I
say to myself? Why is that?
To relax and go back over your notes another time

MOBILE APP FOR ANXIETY 105
Appendices

Appendix A: Consort Flowchart for Randomized Controlled Trial

Individuals applied for the study online (n = 439)
Unsuccessful Online Application (n = 214)
 Insufficient (n = 118) or severe anxiety
symptoms (n = 75) on BAI
 No smartphone, computer w/ Internet,
or email (n = 19)
 Not fluent in English (n = 1)
 Under 18 years of age (n = 1)

Individuals met inclusion (n = 225)
Individuals completed telephone interview with MINI (n = 138)
Unsuccessful Telephone Interview (n = 3)
 Suicidal risk
 Psychosis
 Bipolar Disorder
 Eating Disorder
 Substance Abuse/Dependence (n = 1)
 Involvement in other psychological tx
 Unstable dosage for 6 weeks (n = 2)
Unable to contact (n = 87)
Participants met all inclusion criteria and were randomized into T1, T2, or C (n = 135)
T1 – Mobile Group
(n = 47)
T2 – Computer Group
(n = 44)
C – Waitlist Control Group
(n = 44)

MOBILE APP FOR ANXIETY 106
Appendix B

Questionnaire Administration

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6
App &
Website
Groups

BAI X X X
GAD-7 X X X
1-10
anxiety X X X X X X X X X X X X X X X X X X X X
X
X X X X X X X X X X X X X X X X X X X X
X
PHQ-9 X X X
WSAS X X X
SWLS X X X
Adherence X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X
PCQ X X X
IMI X X X
SUS X X

Waitlist
Control
Group
BAI X X X
GAD-7 X X X
1-10
anxiety
PHQ-9 X X X
WSAS X X X
SWLS X X X
Adherence
PCQ
IMI
SUS
MOBILE APP FOR ANXIETY 107
Appendix C

Intervention Script

Use these questions to become aware of and examine anxious, unhelpful, automatic
thoughts. Be sure to answer each question and type in your responses.

What anxious thought am I having?
Example: I will fail my test.

What evidence do I have that _____? What evidence do I have that the opposite is true?
Example: What evidence do I have that I will fail my test?
I have not done the readings for the past five lectures.
What evidence do I have that I will not fail my test?
I have not yet failed any tests throughout my schooling.
There are still five days before the exam, and I can study.

Is there another point of view? Is there another explanation for _____?
Example: Is there another point of view?
It’s possible I won’t completely fail and get an F.
Is there another explanation for why I feel like I’ll fail?
Maybe it’s because this class is a required class for med school,
and I’m placing a lot of importance on it.

What would I say to a close friend who was feeling this way? Is it different from what I
say to myself? Why is that?
Example: I would tell them to do their best, and that there’s a lot of time to study.
This is different from what I would say to myself. I think I’m hardest on
myself.

Adapted from Leahy, Tirch, & Napolitano (2011) and Vivyan (2011)

MOBILE APP FOR ANXIETY 108
Appendix D

Measures

Generalized Anxiety Disorder – 7 questions (GAD-7)

Over the last 2 weeks, how often have you been bothered by any of the following problems?

Not at all Several
days
More
than
half the
days
Nearly
every
day
1. Feeling nervous, anxious or on edge 0 1 2 3

2. Not being able to stop or control worrying

0

1

2

3

3. Worrying too much about different things

0

1

2

3

4. Trouble relaxing

0

1

2

3

5. Being so restless that it is hard to sit still

0

1

2

3

6. Becoming easily annoyed or irritable

0

1

2

3

7. Feeling afraid as if something awful will
happen

0

1

2

3

MOBILE APP FOR ANXIETY 109
Patient Health Questionnaire-9 (PHQ-9)

Over the last 2 weeks, how often have you been bothered by any of the following problems?
Not at all Several More than Nearly
days half the days every day
1. Little interest or pleasure in doing things 0 1 2 3

2. Feeling down, depressed, or hopeless 0 1 2 3

3. Trouble falling or staying asleep, or sleeping too much 0 1 2 3

4. Feeling tired or having little energy 0 1 2 3

5. Poor appetite or overeating 0 1 2 3

6. Feeling bad about yourself—or that you are a failure or have 0 1 2 3
let yourself or your family down

7. Trouble concentrating on things, such as reading the 0 1 2 3
newspaper or watching television

8. Moving or speaking so slowly that other people could have 0 1 2 3
noticed. Or the opposite-being so fidgety or restless that you
have been moving around a lot more than usual

9. Thoughts that you would be better off dead, or of hurting 0 1 2 3
yourself

10. If you checked off any problems, how difficult have these problems made it for you to work,
take care of things at home, or get along with other people? (circle one)
Not difficult at all
Somewhat difficult
Very difficult
Extremely difficult

MOBILE APP FOR ANXIETY 110
Work and Social Adjustment Scale (WSAS)

People's problems sometimes affect their ability to do certain day-to-day tasks in their lives. To
rate your problems look at each section and determine on the scale provided how much your
problem impairs your ability to carry out the activity. This assessment is not intended to be a
diagnosis. If you are concerned about your results in any way, please speak with a qualified
health professional.

If you’re retired or choose not to have a job for reasons unrelated to your problem, write an “X”
here: ___

0 1 2 3 4 5 6 7 8
Not at all Slightly Definitely Markedly Very
severely

1. Because of my anxiety, my ability to work is impaired. ‘0’ means ‘not at all impaired’ and ‘8’
means very severely impaired to the point I can’t work. _____

2. Because of my anxiety, my home management (cleaning, tidying, shopping, cooking, looking
after home or children, paying bills) is impaired. _____

3. Because of my anxiety, my social leisure activities (with other people; e.g., parties, bars,
clubs, outings, visits, dating, home entertaining) are impaired. _____

4. Because of my anxiety, my private leisure activities (done alone; such as reading, gardening,
collecting, sewing, walking alone) are impaired. _____

5. Because of my anxiety, my ability to form and maintain close relationships with others,
including those I live with, is impaired. _____

MOBILE APP FOR ANXIETY 111
Perceived Convenience Questionnaire (PCQ)

1 = completely disagree, 3 = neutral (neither disagree nor agree), and 5 = completely agree

1. It is convenient for me to complete an entry by using the [app/website].
2. I have access to the [app/website] everywhere.
3. It would be convenient for me to complete an entry by using the [app/website].
4. I find the [app/website] to be convenient.

Adapted from Yoon and Kim (2007)

MOBILE APP FOR ANXIETY 112
Intrinsic Motivation Inventory (IMI)

For each of the following statements, please indicate how true it is for you, using the following
scale:

1 2 3 4 5 6 7
not at all somewhat very
true true true

Interest/Enjoyment (IE)
1. I enjoyed doing this activity very much
2. This activity was fun to do.
3. I thought this was a boring activity. (R)
4. This activity did not hold my attention at all. (R)
5. I would describe this activity as very interesting.
6. I thought this activity was quite enjoyable.
7. While I was doing this activity, I was thinking about how much I enjoyed it.

Perceived Competence (PCo)
8. I think I am pretty good at this activity. #1
9. I think I did pretty well at this activity, compared to other students. #2
10. After working at this activity for awhile, I felt pretty competent. #3
11. I am satisfied with my performance at this task. #4
12. I was pretty skilled at this activity. #5
13. This was an activity that I couldn’t do very well. (R) #6

Effort/Importance (EI)
14. I put a lot of effort into this. #1
15. I didn’t try very hard to do well at this activity. (R) #2
16. I tried very hard on this activity. #3
17. It was important to me to do well at this task. #4
18. I didn’t put much energy into this. (R) #5

Pressure/Tension (PT)
19. I did not feel nervous at all while doing this. (R) #1
20. I felt very tense while doing this activity. #2
21. I was very relaxed in doing these. (R) #3
22. I was anxious while working on this task. #4
23. I felt pressured while doing this. #5

Perceived Choice (PCh)
24. I believe I had some choice about doing this activity. #1
25. I felt like it was not my own choice to do this task. (R) #2
26. I didn’t really have a choice about doing this task. (R) #3
27. I felt like I had to do this. (R) #4
28. I did this activity because I had no choice. (R) #5
29. I did this activity because I wanted to. #6
MOBILE APP FOR ANXIETY 113
30. I did this activity because I had to. (R) #7

Value/Usefulness (VU)
31. I believe this activity could be of some value to me. #1
32. I think that doing this activity is useful for ______________________ #2
33. I think this is important to do because it can _____________________ #3
34. I would be willing to do this again because it has some value to me. #4
35. I think doing this activity could help me to _____________________ #5
36. I believe doing this activity could be beneficial to me. #6
37. I think this is an important activity. #7

MOBILE APP FOR ANXIETY 114
The System Usability Scale (SUS)

Please score the following ten items with one of five responses that range from
1 = Strongly Agree to 5 = Strongly disagree

1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a technical person to be able to use this system.
5. I found the various functions in this system were well integrated.
6. I thought there was too much inconsistency in this system.
7. I would imagine that most people would learn to use this system very quickly.
8. I found the system very cumbersome to use.
9. I felt very confident using the system.
10. I needed to learn a lot of things before I could get going with this system.

Abstract (if available)

Abstract Despite the existence of hundreds of anxiety-related mobile phone applications (apps), there are few studies to date testing the efficacy of an app for anxiety and comparing it to other platforms (e.g., mobile phone versus computer-based intervention). Thus, this study sought to develop and evaluate an app and computer-based website that teach evidence-based cognitive behavioral therapy techniques to participants with anxiety. ❧ The mobile app and website contain two components: assessment and intervention. Participants were asked to rate themselves on their level of anxiety and were guided through a cognitive reappraisal exercise. The website was identical to the mobile app and contained the same components of assessment and intervention, in order to compare the technology modality itself and not the content of the intervention. ❧ The app and website were initially examined in a pilot study. Results from 25 college students with measure-confirmed mild to moderate anxiety indicated that 72% of the participants’ anxiety ratings decreased or remained the same after using the app. The app and website also had high levels of perceived convenience and usability. A randomized controlled trial was then conducted by including a waitlist control group. Results from 135 participants with measure-confirmed mild to moderate anxiety indicated no interaction of group by time on anxiety symptoms, although there was a main effect for time on satisfaction with life and motivation scores. In addition, this study found higher perceived convenience in the app compared to the website, but no difference in perceived usability. Potential reasons for these results are discussed.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Internet-delivered eating disorder prevention: a randomized controlled trial of dissonance-based and cognitive-behavioral treatments

PDF

Identity, perceived discrimination, and attenuated positive psychotic symptoms among college students

PDF

Long-term blood pressure variability across the clinical and biomarker spectrum of Alzheimer’s disease

PDF

Values and persistence

PDF

Mobile self-tracking for health: validating predictors, effects, mediator, moderator, and social influence

PDF

Mixed methods investigation of user engagement with a smoking cessation app

PDF

The effects of math anxiety and low self-efficacy on students’ attitudes and interest in STEM

PDF

The effect of an educational fotonovela on the prevention of secondhand smoke

PDF

Offline social functioning and online communication: how social competence translates to an online context

PDF

Cognitive functioning following ovarian removal before or after natural menopause

PDF

Utilizing user feedback to assist software developers to better use mobile ads in apps

PDF

Renin-angiotensin system antihypertensive medicines with and without blood-brain-barrier crossing potential: effects on cognitive outcomes in the elderly

PDF

The influences of anxiety, coping, and social support on physical functioning among heart failure patients

PDF

Predictors of mHealth engagement in a mindful eating intervention for Type 2 diabetes

PDF

An examination of the association between spousal support and type 2 diabetes self-management

PDF

The cultural expression of depression in Asian Americans and European Americans

PDF

A sociocultural and developmental approach to intimate partner violence among a sample of Hispanic emerging adults

PDF

Reacting to the negative consequences of your decisions: an ATSS comparison of cognitive and affective reactions in older versus younger adults

PDF

Social norms intervention to promote help-seeking with depressed Asian and European Americans: A pilot study

PDF

Personalized normative feedback applied to undergraduates with problem drinking: a comparison with psychoeducation and an examination of cognitive-affective change mechanisms via the articulated ...

Asset Metadata

Creator Kim, Jean (author)

Core Title A mobile app for anxiety: an examination of efficacy and user perceptions

School College of Letters, Arts and Sciences

Degree Doctor of Philosophy

Degree Program Psychology

Publication Date 08/29/2017

Defense Date 08/09/2017

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag anxiety,cognitive behavioral therapy,mHealth,mobile application,OAI-PMH Harvest

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Lopez, Steven (committee chair)

Creator Email jeanmkim@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c40-424994

Unique identifier UC11264469

Identifier etd-KimJean-5696.pdf (filename),usctheses-c40-424994 (legacy record id)

Legacy Identifier etd-KimJean-5696.pdf

Dmrecord 424994

Document Type Dissertation

Rights Kim, Jean

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA