University of Southern California Dissertations and Theses

Feasibility, acceptability, and implementation context of a complex telerehabilitation intervention for post-stroke upper extremity recovery

(USC Thesis Other)

Feasibility, acceptability, and implementation context of a complex telerehabilitation intervention for post-stroke upper extremity recovery

PDF

Download

Open document

Flip pages

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content FEASIBILITY, ACCEPTABILITY, AND IMPLEMENTATION CONTEXT
OF A COMPLEX TELEREHABILITATION INTERVENTION
FOR POST-STROKE UPPER EXTREMITY RECOVERY
by
Miranda Rennie Donnelly
A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(OCCUPATIONAL SCIENCE)
May 2024
Copyright 2024 Miranda Rennie Donnelly

ii
Dedication
In memory of my dear friend Michaela Kathleen Connelly,
who taught me more with her gaze and smile
about the dignity and resilience of human life
than words ever could
(May 19, 1994 – November 29, 2005).
When we were children, I dreamed of redesigning the world for you.
I still do.

iii
Acknowledgments
Looking back, my decision to move across the country away from my family and friends
to pursue a PhD should have felt more daunting than it did. Yet, thanks to the unwavering
encouragement of my family, friends, and mentors, it has unfolded into a beautiful adventure.
An acknowledgements section feels inadequate to express my sincere gratitude for all those
who have been part of this journey, but please accept it as a starting point.
It has been an absolute privilege and joy to train under the mentorship of Dr. Sook-Lei
Liew. By her example, I have learned so much about being a thoughtful, ethical, and
compassionate scientist and leader. Lei, thank you for giving me the freedom and guidance to
explore methods and ideas that extend from your work. Your thoughtful support and trust in my
scholarly instincts have empowered me to find a niche in science where I feel fulfilled and
excited. Just as meaningful is how you have cheered me on in my personal aspiration of starting
a family. I am grateful for your authentic and holistic support of me and your friendship. I admire
and respect you, and being your mentee is a true honor.
I am also immensely grateful to the other members of my dissertation committee for their
thoughtful feedback and guidance: Dr. Stacey Schepens Niemiec for her advice in co-design
and technology development and Dr. Beth Pyatak for her expertise in translational science and
mixed methods. I additionally want to thank Dr. Monica Perez Jolles for introducing me to
Implementation Science and dedicating a great deal of time to growing my capacity in this area.
I also want to thank my mentors—and soon to be colleagues—at Towson University for
their guidance when I first considered a career in academia and support along the way, in
particular Dr. Jenna Yeager and Dr. Kate Eglseder who gave me the opportunity to be a
research assistant; the faculty who nominated me for an institute for future OT scientists, where
I started to envision my future as a scholar; and Dr. Amanda Jozkowski who first nudged me to

iv
connect with Lei. From my first day of OT school to now, as I prepare to begin as an Assistant
Professor at Towson University, my entire TU “family” has been influential in my growth.
Thank you, also to my NPNL lab mates who have been both collaborators and friends on
this journey, especially Dr. Octavio Marin-Pardo, Dr. Julia Juliano, Dr. Lily Rudolph-Ito, Bethany
Lo, Dr. Artemis Zavaliangos-Petropulu, Coralie Phanord, Stuti Chakraborty, Erendiz, and Mahir
Khan. You have made this experience extraordinary. To the research assistants and OTD
residents whom I have had the great pleasure of working with over the years: thank you for
helping me develop as a leader, for your many contributions to this work, especially Aisha,
Amisha, Barri, Emily, Grace, Jessica, Joanne, Julie, Kyle, Melanie, Renee, Rosie, and Kira. I
am also immensely grateful for the members of the NPNL Stroke Advisory Board, who have
made me a better scientist and leader: Armando, Barri, David, Erika, Jett, John, Kristal, Marcos,
Randy, thank you for your many contributions to this work and comradery.
I also would not have been able to complete this degree without the support of friends,
both near and far. Thank you, Merino Family, for being far more than James’ childcare
providers; you have become family to us, and your genuine love for our child has enabled me to
focus on my PhD studies knowing that he is being cared for by his “Aunty Dallas and Uncle
Frank.” To Dr. Emily Campi, who started as a fellow PhD student and quickly became a
cherished friend to my family. Thank you also, to my long-time friends who have accompanied
me and cheered me on through this journey, especially Amanda, Barbara, Olivia, Michelle,
Rachel, and Shelby.
I would like to especially thank my parents, Mike and Jody Rennie, for their selfless love
and encouragement throughout my whole life – even when it means moving across the country
from them! Mom and Dad, you have always gone above and beyond for me, from somehow
making it possible for me to do more after-school activities than there were days in the week to
enabling incredible educational and travel opportunities. I have made it to this point because of
your sacrifices and confidence in my ability to do anything I aspire to. Thank you to my siblings

v
Rachel and Bill for being role models of determination and excellence. To my in-laws, Mike and
Alison Donnelly, for all of your support and visits. To both of our families of origin and extended
families, thank you. I am blessed by you all.
I am deeply grateful for my husband, who, before we were even engaged, “chased me”
to Los Angeles in full support of my scholarly ambitions (he proposed 3 days before I started the
PhD program!). His enthusiasm for my personal and scholarly growth and career pursuits have
never dimmed. Sean, even as our lives have become fuller –having children, renovating a 90-
year-old home, starting a small business – your steadfastness in supporting me has only grown.
I see your love for me in the day-to-day rhythms of our family, as you make space for me to
think, write, and grow. Thank you for the beautiful life we are creating.
I also thank my children, both of whom came into my life during my PhD studies. James,
your endless curiosity, and the echoes in our home of “I wonder…” and “I have an idea,” which
you say with an admirable sense of confidence and pride, inspire me as a mom and a scholar to
keep wondering. Joanna, you have yet to make your appearance but your ‘kicks of
encouragement’ during long writing sessions bring a smile to my face and remind me of the
many joys I have in this life.
To each person who has been with me on this journey, whether mentioned here by
name or not, I thank you for your love and support.
Funding
I would like to gratefully acknowledge the funding sources that have supported me
personally and the REINVENT project: the Mrs. T.H. Chan Division of Occupational Science
and Occupational Therapy, The Margaret Rood Dissertation Research Award, The USC
Stevens Center of Innovation, the American Heart Association, the U.S. Army Research Office,
and the National Institutes of Health.

vi
Table of Contents
Dedication………………………………………………………………………………………………….ii
Acknowledgements………………………………………………………………………………………iii
List of Tables…………………………………………………………………………………………….viii
List of Figures…………………………………………………………………………………………….ix
List of Abbreviations………………………………………………………………………………….…..x
Abstract……………………………………………………………………………………………………xi
Chapter 1: Introduction…………………………………………………………………………………...1
Significance……………………………………………………………………………………….1
Overview and Specific Aims…………………………………………………………………….2
Background……………………………………………………………………………………….5
Stroke Recovery and Rehabilitation…………………………………………………...5
Common Pitfalls in the Translation of Complex Rehabilitation Technologies
to Clinical Practice……………………………………………………………..… …...11
Approaches, Frameworks, and Outcomes to Improve the Translation of
Complex Rehabilitation Technologies to Clinical Practice ………………..……….14
Present Studies………………………………………………………………………...25
Chapter 2: Acceptability and Usability of a Post-stroke Complex Telerehabilitation
Biofeedback Intervention Among Stroke Survivors Using it at Home…………………………...….26
Abstract………………………………………………………………………………………….26
Introduction……………………………………………………………………...………………27
Methods………………………………………………………………………………………….28
Results…………………………………………………………………………………………..33
Discussion…………………………………………………………………………………….…41
Chapter 3: Evaluating pre-implementation outcomes of an EMG-based stroke
rehabilitation intervention using a video demonstration survey methodology………………….…46
Abstract………………………………………………………………………………………….46
Introduction………………………………………………………………………………...……47
Methods………………………………………………………………………………………….49
Results…………………………………………………………………………………………..53
Discussion………………………………………………………………………………………65
Chapter 4: A new community advisory board model to strengthen community partnerships
in stroke rehabilitation research………………………………………………………………………..73
Abstract………………………………………………………………………………………….73
Introduction……………………………………………………………………………………...74
Methods………………………………………………………………………………………….77
Results…………………………………………………………………………………………..85
Discussion……………………………………………………………………………………….89
Conclusion………………………………………………………………………………………95

vii
Chapter 5: Discussion…………………………………………………………………………………..97
Summary of Key Findings…………………………………………………………………….98
Implications Across Studies………………………………………………………………….100
Innovative Approaches……………………………………………………………………….104
Limitations……………………………………………………………………………………..108
Future Directions………………………………………………………………………………108
Contributions to Occupational Science……………………………………………………...110
Conclusion……………………………………………………………………………………..110
References……………………………………………………………………………………………..112
Appendices……………………………………………………………………………………………..127
Appendix A: Questionnaire Constructs and Remote Training Protocol…………………127
Appendix B: Implementor Survey……………………………………………………………129
Appendix C: NPNL Stroke Advisory Board Mission, Vision, and Scope…………………138

viii
List of Tables
Table 2.1 Participant Demographics…………………………………………………………………..33
Table 2.2 Post-Study System Usability Questionnaire Scores by Item……………………………35
Table 2.3 Relationships Between Personal Characteristics and Perceived Usability
and Acceptability…………………………………………………………………….…………………..36
Table 2.4 Unified Theory of Acceptance and Use of Technology Item and Construct Scores…38
Table 3.1. Survey Respondent Characteristics………………………………………………………54
Table 3.2. Acceptability and Feasibility Scores by Practice Setting……………………………….57
Table 4.1 Advisory Board Activities……………………………………………………………………80
Table 4.2 Engagement Principles and Associated Levels of Engagement………………………..83
Table 4.3 Mean REST Condensed Version Scores by Engagement Principle - Quality Scale…85
Table 4.4 Mean REST Condensed Version Scores by Engagement Principle - Quantity Scale…87

ix
List of Figures
Figure 2.1 Tele-REINVENT Interface…………………………………………………………………30
Figure 4.1 Continuum of Partner Engagement in Research…………………………………………79

x
List of Abbreviations
ARAT, Action Research Arm Test
CAB, community advisory board
CER, community-engaged research
EMG, electromyography
EP, engagement principle
EPIS, Exploration, Preparation, Implementation, Sustainment framework
FIM, Feasibility of Intervention Measure
AIM, Acceptability of Intervention Measure
FMA, Fugl-Meyer Assessment
InfoQual, information quality
InterQual, interface quality
NPNL, Neural Plasticity and Neurorehabilitation Laboratory
OT, occupational therapy
OTs, occupational therapists
OTAs, occupational therapy assistants
PSSUQ, Post-Study System Usability Questionnaire
PT, physical therapy
PTs, physical therapists
PTAs, physical therapy assistants
REST, Research Engagement Survey Tool
SysQual, system quality
UCD, user-centered design
USC, University of Southern California
UTAUT, Unified Theory of Acceptance and Use of Technology

xi
Abstract
Advanced rehabilitation technologies can address access and practice gaps that
negatively affect the recovery potential and outcomes of stroke survivors living with severe
impairment. However, despite major advancements in rehabilitation technology, many of these
innovations fail to be integrated into clinical practice. This dissertation evaluates implementation
and design outcomes of a game-based electromyography (EMG) intervention among key
stakeholders in stroke rehabilitation, namely stroke survivors and therapists, in the early stage
of implementation planning. We found that Tele-REINVENT is perceived as acceptable and
feasible to stroke survivors and therapists, though there are various multi-level barriers to
uptake in clinical practice that need to be addressed through careful implementation planning.
We anticipate that applying these findings to the ongoing development and preparation of TeleREINVENT will result in greater patient confidence and ease of using the system at home for
high repetition movement practice and efficient clinician use within the fast-paced health care
model in which stroke therapy is delivered. Additionally, we set the foundation for a long-term
research-community partnership with stakeholders via a novel community advisory board
model. Our collaboration with the NPNL Stroke Advisory Board will continue to enhance the
design of ongoing pre-implementation work and ultimately support its translation into routine
post-stroke care. The overarching goal of this dissertation is to examine factors that will
influence the uptake of electromyography biofeedback into routine clinical practice in stroke
rehabilitation, from the perspective of stakeholders who will be users or implementors. There is
great value in engaging stakeholders in the design and implementation of technology-based
interventions, especially as we seek to translate neuroscientific findings from the laboratory into
innovations that can improve in the lives of stroke survivors.

1
Chapter 1: Introduction
Significance
High-technology solutions are ubiquitous in our daily lives, influencing the way we
participate in mundane occupations, engage with other people, and explore our environments.
From the moment we wake up, our smart phones alert us to our scheduled events and the
current weather so we can dress and prepare for the elements. Vehicles with lane departure
alerts keep us safe en route to our destinations. Online social networks connect us to people
and news from around the globe, bringing our awareness to everything from popular trends to
international political proceedings as they unfold in real-time. This era of digital technology and
high-powered computing has transformed the rhythms of ordinary life in extraordinary ways.
Technological solutions are being generated for nearly every human want, discomfort, and
need. Yet, humans continue to experience challenges and discomforts, and technologies often
fall short of their intended impact. This is even true—perhaps especially true—in health care,
where technological innovations abound, but patients still experience long-term disability,
illness, and suboptimal outcomes.
In my work as a clinical occupational therapist and occupational science researcher with
survivors of neurologic injury, I have observed time and again the hope and optimism that new
rehabilitation technologies excite, followed by the disappointment when these ‘solutions’
produce marginal or no benefits after significant investment of time and capital. These
observations, corroborated by scientific evidence, have led to my conviction as a therapist and
scientist that technology for the sake of innovation is not enough. For novel technologies to
radically transform human lives, they must cooperate with the drive and strength of the human
spirit. We must design technologies with a deep knowledge of human being and doing; the
meaning we ascribe to our occupations; the habits and routines that structure our lives; and the
resilience and adaptability that lead to human thriving. Occupational scientists have a wealth of

2
knowledge about these topics and could profoundly shift the way technological innovations
integrate with people’s lives. In fact, as scientists and therapists who often work with underresourced communities, I believe we have an ethical and social responsibility to influence the
development of health and rehabilitation technologies and endorse only those which promote
advancement toward and engagement in individuals’ chosen occupations.
In my current and future work, I am driven to use the power of occupational science
knowledge to influence the way health and rehabilitation technologies are conceptualized,
developed, evaluated, and implemented. In my dissertation work, I take a step in that direction,
by examining some of the many contextual, personal, and technical factors that influence the
uptake of rehabilitation technology into the daily lives and practices of stroke survivors and their
therapists. I intertwine theories and knowledge from occupational science, rehabilitation
science, implementation science, and user-centered design to begin to chisel away at some of
the complex barriers that influence the movement of efficacious interventions into clinical
practice and reveal opportunities for thoughtful and sustainable technological innovation in
stroke rehabilitation.
Overview and Specific Aims
Technology-based interventions have the potential to improve recovery outcomes among
stroke survivors living with complex, chronic symptoms (Cervera et al., 2018; Coscia et al.,
2019; Laver & Osborne, 2022) and reduce access gaps to post-stroke therapy services
(Towfighi & Ovbiagele, 2022; Verma et al., 2022). Novel technologies developed using principles
of motor learning and neural plasticity that provide higher dose, greater intensity, and
personalization of treatment show promising outcomes in efficacy and effectiveness trials.
However, there are a few major blockers to their use in clinical practice, which will be discussed
in greater depth in the sections that follow. One of these issues is a lack of planning for
implementation, which results in innovative therapies staying in the laboratory (Stephenson et
al., 2022). Part of the challenge of planning for the uptake and long-term sustainment of

3
complex rehabilitation technologies is that they are multi-level interventions, requiring support
and change from the provider and patient level through to organizational and system levels
(Bower et al., 2021; Morrow et al., 2021). Further, some recent innovations, including the one
examined in the present studies, are compatible with a telerehabilitation delivery method
(Cramer et al., 2019; Marin-Pardo, 2023; Swanson et al., 2023), which on one hand can
drastically reduce access barriers by providing high quality treatment in the home environment
with a remote provider (Laver & Osborne, 2022). On the other hand, telerehabilitation introduces
myriad additional contextual factors and possible barriers to successful implementation
(Stephenson et al., 2022).
Another issue with the current rehabilitation technology landscape is the paucity of
treatment options that are effective for stroke survivors with more severe arm impairment after
stroke. Many clinic-based and telerehabilitation interventions require a minimum threshold of
movement which cannot be achieved by this subgroup (Cramer et al., 2019; French et al., 2016;
Thieme et al., 2018; Winstein et al., 2003). By contrast, electromyography-based (EMG)
biofeedback is an innovative approach that has mostly been used in the research context and is
effective for sensorimotor recovery among stroke survivors with severe disability (Donoso
Brown et al., 2014; Giggins et al., 2013; Li, 2016). Muscle activity in the affected arm is
amplified and transformed into visual feedback, which encourages and enables high doses of
movement. Our previous work evaluating Tele-REINVENT, an EMG-biofeedback system, has
demonstrated that it can be used independently by stroke survivors at home and can lead to
improvements in motor control. This may also reduce care partner burden and access barriers,
opening doors for participation in remote rehabilitation (Marin-Pardo et al., 2022; Marin-Pardo et
al., 2021).
Despite the promise of high-technology interventions for stroke rehabilitation, like EMGbiofeedback, there is a research-practice gap preventing these innovations from reaching
patients who could benefit from them (Caughlin et al., 2020; Jones et al., 2020). This is due in

4
part to limited evidence directing their implementation in clinical practice (Stephenson et al.,
2022). In response, this dissertation leverages occupational science and implementation
science frameworks, as well as methods and constructs from other disciplines to understand the
implementation context, feasibility, and acceptability of a post-stroke EMG-biofeedback
telerehabilitation system.
Specifically, in this dissertation I aim to:
1. Evaluate the acceptability and usability of Tele-REINVENT among stroke
survivors using it at home.
2. Evaluate the pre-implementation feasibility and acceptability of Tele-REINVENT
among implementers.
3. Co-create knowledge with stakeholders for application within stroke
telerehabilitation research to increase the effectiveness of research and
implementation outcomes.
The overarching goal of this dissertation is to examine factors that will influence the
uptake of an electromyography biofeedback system into routine clinical practice in stroke
rehabilitation. The findings from this dissertation will inform the ongoing development and
implementation of Tele-REINVENT, a complex telerehabilitation intervention based on
neuroscientific advances about motor learning and recovery (Marin-Pardo et al., 2022; MarinPardo et al., 2021; Spicer et al., 2017; Vourvopoulos et al., 2019) into the daily lives of stroke
survivors. Additionally, this work has the potential to enhance the delivery of other innovative
post-stroke telerehabilitation services and reduce barriers to accessing high-quality therapy.
Understanding how people engage with complex rehabilitation technologies in the home is
necessary context for creating accessible, effective, and affordable therapy alternatives that use
technology to support recovery for people with chronic stroke. In the sections that follow, I will
present a synthesis of relevant literature and frameworks that inform this work.

5
Background
Stroke Recovery and Rehabilitation
Stroke is a leading cause of chronic disability in the United States, with approximately
610,000 people experiencing their first stroke each year (Tsao et al., 2023). Strokes can cause a
wide range of long-term impairments, depending on various factors such as the affected brain
region(s), promptness of emergency care, and age, among other known and unknown factors.
Despite costly treatment ($36.5 billion per year in the United States) and extensive research on
the prevention and treatment of stroke (Tsao et al., 2023), 90% of stroke survivors do not fully
recover (American Heart Association, 2019; Winstein et al., 2016). In fact, the rates of stroke
incidence and mortality continue to increase (Feigin et al., 2023; Tsao et al., 2023), and
increasingly so among young and middle-aged people (Feigin et al., 2023). Thus, there is a
serious need to identify effective, innovative treatments that can improve recovery outcomes
and quality of life for survivors of stroke.
Symptoms and Recovery. While the effects of stroke vary greatly between survivors,
there are certain impairments that are commonly seen. Stroke frequently causes altered
movement patterns that progress throughout the stages of recovery, often beginning with
flaccidity of the extremities on the contralesional side of the body and progressing to spastic and
synergistic movements. Hemiplegia and associated motor control challenges are common and
negatively impact independence in mobility and activities of daily living. Stroke survivors may
also experience altered somatosensory function, pain, increased falls, depression, vision
impairment, and other symptoms that negatively affect occupational participation (Hildebrand et
al., 2023; Winstein et al., 2016). Typically, most spontaneous recovery of function occurs within
the first six months after stroke, though experience-dependent recovery is still possible
(Kwakkel et al., 2003). Early return of voluntary motion or grip strength of the affected arm are
good signs of recovery, whereas more severe impairment in the acute phase of stroke can yield

6
a wider range of recovery outcomes in the chronic phase (Kwakkel et al., 2003; Prabhakaran et
al., 2008).
Standard Care in Stroke Rehabilitation. The scope of “standard” or “usual” care in
upper extremity sensorimotor stroke rehabilitation is surprisingly ambiguous, and has been
referred to as a “black box” (Arienti et al., 2022; Cahill et al., 2022). Even systematic reviews
auditing clinical care practices and definitions of usual care in comparative effectiveness studies
come short in terms of specific practices. Instead, broad categories of practices are listed (e.g.,
neuromuscular facilitation techniques, strengthening, range of motion, stretching, electrical
stimulation, sensory re-education), with little understanding of how they were manualized.
Additionally, intervention functions, forms, dosage, and frequency are poorly reported (Arienti et
al., 2022; Cahill et al., 2022).
Despite the lack of clarity about what standard stroke rehabilitation involves, there are
established clinical practice guidelines that ideally are informing therapists’ treatment planning.
Broadly, there is evidence that increasing the number of repetitions of movements can induce
movement retraining (Lang et al., 2015; Lohse et al., 2014) and that high repetition movement
training should be done in the context of functional activity (Hildebrand et al., 2023). However,
despite evidence that high numbers of repetitions are feasible in some settings, such as
inpatient rehabilitation (Waddell et al., 2014), the reality is that very small numbers of repetitions,
likely too few to amount to functional change, are achievable in standard practice (Lang et al.,
2009). This is in part due to the many goals of therapy that must be addressed concurrently in
relatively short therapy sessions during increasingly brief bouts of rehabilitation (Hildebrand et
al., 2023; Ottenbacher et al., 2014; Winstein et al., 2016). For example, post-stroke
occupational therapy can address remediation of symptoms, maintenance of existing function,
compensation to promote independence in daily life, prevention of subsequent stroke, and
health promotion (Hildebrand et al., 2023). Among survivors with more severe impairment,
compensation is prioritized over remediation because of the practical need to prepare patients

7
for community living (Barreca et al., 2003; Winstein et al., 2016). Because of this wide spectrum
of intervention approaches, paired with reimbursement structures that incentivize addressing
compensatory strategy training (Brown et al., 2022), it is no surprise that standard care does not
support the high dose and intensity of movement that is likely to yield clinically important
differences (Lang et al., 2009, 2016).
Gaps in post-stroke rehabilitation. Advancements in post-stroke rehabilitation have
led to functional improvements among stroke survivors but there are still many gaps in care.
Here I will present three major issues that are relevant to the proposed work. First, rehabilitation
services for stroke survivors in post-acute care settings are underutilized, despite substantial
evidence supporting improved functional outcomes (Ayala et al., 2018). Increasing the utilization
of outpatient services is a high-priority public health issue (Office of Disease Prevention and
Health Promotion, n.d.) and will require systematic changes to achieve. Factors such as remote
geographic location, impaired community mobility (CDC, 2020), limited or no access to
transportation, lack of social support, and low income contribute to access gaps and
underutilization of rehabilitation (Mahak et al., 2018), which are amplified among historically
underserved populations (U.S. Department of Health and Human Services, 2022) and people
with severe stroke (Marzolini et al., 2016; Teasell et al., 2018).
A second issue in stroke rehabilitation is the lack of effective treatment options for
individuals with moderate to severe hemiparesis. Evidence-based rehabilitation approaches that
support repetitive, task-specific practice (e.g., constraint induced movement therapy,
neuromuscular electrical stimulation, mirror therapy, exercise, robotics, and virtual reality
gaming) enhance motor outcomes (French et al., 2016; Thieme et al., 2018; Winstein et al.,
2016); however, these approaches require a minimum threshold of movement, so stroke
survivors with severe hemiparesis who have little to no functional movement in their affected
arm typically cannot benefit from them. Individuals with more severe motor impairment have
lower potential for motor recovery, and the absence of effective rehabilitation techniques

8
worsens the prognosis (Kwakkel et al., 2003; Winstein et al., 2016). This population also often
experiences greater access gaps, so there are additional unmet needs for treatments delivered
via telehealth and effective at-home therapy options that can be used between therapy sessions
or after discharge.
A third issue in stroke rehabilitation is a trend of failed translation of novel rehabilitation
technologies into routine clinical practice. Many innovative products leveraging principles of
neural plasticity and motor learning have been developed in laboratories; however, attempts to
translate them to clinical practice frequently fall flat. There are many factors influencing this
research-to-practice gap (Cahill et al., 2022), but a major one is a lack of planning for the
changes at the organizational and provider levels that are often needed to support successful
implementation and sustainment.
Novel therapy approaches. In response to the need for interventions that provide
higher dose and intensity of rehabilitation interventions, novel therapy approaches have been
developed to supplement standard therapy practices. Some of these novel approaches use
telerehabilitation paradigms and complex technologies.
Telerehabilitation. Telerehabilitation is an increasingly common and feasible alternative
to in-person rehabilitation to increase access to therapy and improve functional outcomes
among stroke survivors by providing effective, specialized treatment in the home environment
(Knepley et al., 2021; Laver & Osborne, 2022), often sooner post-stroke and more frequently,
both of which influence functional outcomes (Lang et al., 2016; Winstein et al., 2016).
Interventions delivered via telerehabilitation can yield comparable results to in-person
treatment (Caughlin et al., 2020; Cramer et al., 2019; Tchero et al., 2018), be more costeffective (Caughlin et al., 2020), increase the dose of rehabilitation (Cramer et al., 2021), and
enable more active patient engagement in therapy sessions (McLean et al., 2013). Many
telerehabilitation approaches utilize everyday technologies—mainstream devices people use in
daily life, such as phones, tablets, and computers (Malinowsky et al., 2011; Patomella et al.,

9
2013). Beyond their use for remote face-to-face therapy visits (Cason, 2017; Dahl-Popolizio et
al., 2020), everyday technologies can be used as rehabilitation tools, like using mobile
applications to train specific upper extremity movements (Jang & Jang, 2016) and native video
applications to record therapeutic activities in the clinic for at-home use to improve performance
in home exercise programs (Emmerson et al., 2017). While these and other telerehabilitation
paradigms benefit some stroke survivors, they typically require a minimum threshold of volitional
and independent movement, making them unusable for survivors with more severe
hemiparesis, like many standard care interventions.
Complex technology-based rehabilitation interventions. As opposed to everyday
technologies, complex technologies are specialized, tend to be less familiar to the average
consumer, and are characterized by the integration of various software and hardware
components (Jilke, 2021). In rehabilitation, complex technologies also tend to be part of
complex health interventions, which are characterized by the coordination of multiple interacting
components; change at the organizational level; behavior change among providers and
patients; measurement of multiple outcomes; flexibility in how the intervention is applied; and
adaptations to support patients who have multiple symptoms (Craig et al., 2008; Pérez Jolles et
al., 2019; Pfadenhauer et al., 2017). In neurorehabilitation, examples of complex technologybased interventions include brain computer interfaces (Ono et al., 2014), EMG biofeedback
(Kim, 2017), robotics (Chang & Kim, 2013), kinematic sensors (Dukes et al., 2013), and
immersive virtual reality (Laver et al., 2017).
Often, complex technologies in neurorehabilitation have adaptive behaviors or multiple
high-technology component; for example, a game-based rehabilitation system that introduces
task-specific movements graded to match patients’ baseline function (Fong et al., 2021) or a
serious gaming intervention leveraging kinematic sensors and virtual environments to provide
movement biofeedback (Dukes et al., 2013). Complex technologies can provide adaptive
experiences, high doses of movement, and amplified biofeedback of movement attempts,

10
creating opportunities for clients with severe hemiparesis to participate in effective, remote
rehabilitation that goes beyond passive or active-assisted range of motion techniques.
Additionally, telerehabilitation approaches that use complex technologies have the potential to
address a serious need for novel rehabilitation options for stroke survivors with severe
hemiplegia; however, there are unique barriers to using complex interventions in home
environments with patients with severe hemiparesis, which has stalled the translation of such
interventions to clinical settings. These issues will be discussed in more depth in a subsequent
section of this chapter. First, we will explore the complex technology-based intervention
examined in the present studies.
Electromyography (EMG) Biofeedback. One complex technology that can be effective
for improving motor control among chronic stroke survivors with severe hemiplegia is EMG
biofeedback (Donoso Brown et al., 2014; Kim, 2017; Lirio-Romero et al., 2021). After stroke,
damage to areas of the brain that produce and convey motor commands to the muscles result in
impaired coordination and muscle activation. Over time, repeated attempts to initiate movement,
met with dysfunctional or no movement, can result in learned non-use, wherein the brain stops
sending motor commands after repeated failures (Taub et al., 2006). In EMG-biofeedback
paradigms, muscle activity from the affected limb, as measured by EMG signals, is presented to
the stroke survivor through visual feedback to reinforce attempted movements and retrain brainmuscle connectivity. This form of feedback provides patients with real-time, extrinsic feedback
about their performance so they can learn to control their muscles, even in the absence of
visible movement (Giggins et al., 2013).
Tele-REINVENT. Tele-REINVENT is a novel EMG biofeedback intervention developed
by Liew and colleagues (Marin-Pardo et al., 2021) that provides real-time visual feedback via a
screen or virtual reality display in response to attempted arm movements. It can be used in
clinics and is also portable for use at home as part of a home exercise program or
telerehabilitation program. Tele-REINVENT and its non-portable precursor, REINVENT are

11
efficacious for improving quality of life, motor control of the hemiparetic arm, and
corticomuscular coherence of the affected side after moderate to severe stroke (Marin-Pardo et
al., 2021; Spicer et al., 2017; Vourvopoulos et al., 2019). Tele-REINVENT consists of a laptop
computer, low-cost EMG sensors, a custom interface, and adaptive games. EMG sensors are
positioned on the forearm to detect muscle activity during wrist movement attempts, even if
movement is not visible. An EMG calibration captures muscle activity in real time to adjust ingame thresholds to match users’ function each day. This is important for stroke survivors, for
whom motor control can vary within and between days depending on fatigue, spasticity,
exercise, temperature, stress, and other factors (Cheung et al., 2015). The EMG signals are
processed in real time and transformed into game activity on the screen. Thus, users can play
games with trace muscle activity from their hemiparetic arm without hands-on assistance from a
therapist.
The Tele-REINVENT EMG-biofeedback paradigm reinforces attempted movements,
even in the absence of complete or functional movements. Tele-REINVENT is a complex
technology because it integrates various hardware (e.g., EMG sensors, visual displays,
development board, laptop) and software components (e.g., custom scripts, game
environments, signal processing) into a specialized, novel product. While previous research has
found Tele-REINVENT to be safe and efficacious for improving motor outcomes and quality of
life metrics, more research is needed to identify facilitators and barriers to its use with stroke
survivors both in the clinic and at home.
Common Pitfalls in the Translation of Complex Rehabilitation Technologies to Clinical
Practice
Rehabilitation researchers have produced numerous complex rehabilitation
technologies—some designed for telerehabilitation—that are efficacious for improving
sensorimotor outcomes in stroke survivors with hemiparesis (Cervera et al., 2018; Coscia et al.,
2019; Laver et al., 2017; Laver & Osborne, 2022). However, translation and implementation

12
efforts for complex rehabilitation technologies, including those delivered via telerehabilitation,
often meet resistance from key stakeholders and fail to achieve the intended goal. Many factors
undermine the translation of evidence-based practices into health care settings, including the
complexities involved in developing, implementing, and evaluating technology for use in realworld contexts (Dopp et al., 2019; Rizzuto & Reeves, 2007; Stephenson et al., 2022).
Design/Development. Oftentimes, early stage health technology development focuses
on economic analysis, proof of concept, and manufacturer buy-in, rather than the needs of
stakeholders who will ultimately use the technology (e.g., members of the target population,
clinicians), causing innovation efforts to fail (Shah & Robinson, 2006; Smith et al., 2019).
Markiewicz and colelagues (2014) revealed that early-stage decision making in medical device
development is commonly motivated by competition and enthusiasm for innovation, rather than
on high-quality evidence and systematic assessment of the clinical need, context, and change
processes by which the intervention will lead to desired outcomes (Craig et al., 2008). Because
proof of concept is often prioritized over systematically addressing stakeholder needs, decisions
in early design and development phases can compromise the integrity of the intervention and
likelihood of successful implementation. Poor design practices result in wasted resources, and
most importantly, products that do not fulfill real needs (Markiewicz et al., 2014; Rizzuto &
Reeves, 2007; Smith et al., 2019).
Implementation. Despite evidence for the effectiveness of technology-based
rehabilitation interventions and their role in advancing the quality and access of post-stroke
therapy, there is limited evidence directing their implementation in the home and other realworld practice settings (Caughlin et al., 2020; Jafni et al., 2017; Stephenson et al., 2022). In
addition to design and development pitfalls that affect translation, there are also many
contextual factors and stakeholder needs that make rehabilitation technologies particularly
challenging to implement. For example, successful implementation regularly requires adaptation
to accommodate reimbursement structures at the federal level; organization-level policies;

13
device acquisition practices, infrastructure, and support at the clinic-level; clinician preferences
and skills; and patient access. Moreover, using novel technologies requires extensive clinician
training for set up, effective use, and troubleshooting. For paradigms that include complex
technologies like EMG or virtual reality, rehabilitation professionals may face wide knowledge
and skill gaps (Appleby et al., 2019). In a health care environment marked by burnout and
turnover, adding new digital tools and increasing training demands is a tall order that may be
met with resistance from clinicians (Reith, 2018).
There are additional factors that specifically influence device use in telerehabilitation,
such as state licensure and practice laws affecting the circumstances in which services can be
rendered (Mullangi et al., 2021; Turner & Etherton, 2022). Additionally, reimbursement for
telerehabilitation varies greatly between plans and sponsors. Before the COVID-19 pandemic,
few state Medicaid programs reimbursed for telerehabilitation services (Marzano, 2017). During
the pandemic, restrictions on telehealth were temporarily lifted, making way for telerehabilitation
reimbursement (Chan, 2021); however, as emergency measures continue to be deimplemented post-pandemic, the reimbursement landscape continues to change and will impact
who can receive these services and under what conditions. Understanding these evolving
contexts is vital for successful implementation.
Evaluation. Evaluating a complex intervention involves assessing efficacy,
effectiveness, and implementation processes (Craig et al., 2008), as well as contextual factors
of the system, organization, and service delivery situations. Additionally, usability and
accessibility evaluations and product-market fit are important for technology-based
interventions. Contrary to more traditional translational research practices, which follow a linear
path from efficacy to implementation (Weber, 2013), evaluation of these various outcomes and
intervention components should be tested iteratively and across all phases and in context, when
possible (Leppin et al., 2020).

14
Approaches, Frameworks, and Outcomes to Improve the Translation of Complex
Rehabilitation Technologies to Clinical Practice
This dissertation reflects the integration of knowledge and methods from various
disciplines, such as occupational science, implementation science, and user-centered design.
Here, I will briefly discuss the primary disciplines and approaches informing the present studies,
along with guiding frameworks from each.
Occupational Science. This work is first and foremost positioned in occupational
science and occupational therapy research. The use of rehabilitation technology is situated in
people’s daily life activities and routines, the environments in which they participate in those
activities, their interpersonal relationships, personal motivations, past experiences with
technology, and other personal factors. Technology use can be an occupation in its own right, a
tool supporting other occupations, or an extension of the self, depending on the technology and
how an individual uses it and ascribes meaning to the experience (Patomella et al., 2013;
Proffitt et al., 2019; Van Den Eede, 2014).
In addition to personal and social factors influencing technology-based intervention use
in stroke rehabilitation, there are also organizational- and system-level factors that influence
how and if people use rehabilitation technology (Juckett et al., 2020; Stephenson et al., 2022).
The holistic and multifaceted understanding of human occupation in context inherent to
occupational science is useful for examining such complex situations and contexts (Carlson et
al., 2014; Clark et al., 1991b; Cutchin & Dickie, 2013; Hocking, 2021). When technology is
designed and evaluated outside of the ‘situational whole’, crucial information about how it might
fit into people’s lives is absent. Decontextualizing technology use offers little insight into how it
supports or hinders individual personal goals and activities, so technology use should be
studied in the context of peoples’ everyday lives. Accordingly, participants in the present studies,
whether they have lived or professional experience of stroke, are conceptualized as individuals
who construct their daily routines around occupations, some of which affect their health and

15
well-being (Jackson et al., 1998), rather than as users of technology, healthcare providers, or
stroke survivors.
Occupational Science Analytic Framework: Situation-oriented analysis of digital
engagement in occupation. Madsen et al. (2021) developed an analytic framework to facilitate
a situationally oriented analysis of the use of digital technologies for engagement in occupation.
Drawing on a transactional perspective, the framework focuses on the situation-person whole
and assumes that everyday life is an experienced situation, where people and contexts are
highly interconnected and functionally related (Dickie & Cutchin, 2013). To summarize this
perspective, situations are either indeterminate (i.e., problematic, incoherent)—created when
change disrupts the effectiveness of existing habits or behaviors—or determinate (e.g.,
unproblematic, closed). When situations are problematic, creativity and action are used to
‘functionally coordinate’ the individual-situation whole and address the characteristics,
challenges, and needs that influence engagement. Occupational science theorizes that
occupation is a form of functional coordination (Dickie & Cutchin, 2013).
Madsen and team (2021) build on the transactional perspective of occupation,
suggesting that situations either facilitate or hinder the use of digital technologies for
occupational engagement. Further, digital technology use is situated in everyday life and is
influenced by personal and contextual factors. Habits in this view are seen as foundational to
coherent situations. Context is the configuration of situations, and ‘end-in-view’ is an evolving
drive toward functional coordination. Ultimately, engagement in occupation (in this framework,
via digital technologies) is how individuals transform indeterminate situations into determinate
situations and support stability in life.
Implementation Science. A pragmatic discipline, implementation science studies how
interventions become embedded within real-world contexts and uses that knowledge to
systematically integrate evidence-based interventions into routine community and clinical
practices (Leppin et al., 2020). This is achieved by identifying and addressing barriers and

16
facilitators to inform the development and tailoring of implementation strategies, which improve
the uptake of an intervention. Implementation science is related to, but distinct from,
translational science, which is more frequently discussed in occupational science (Clark &
Lawlor, 2009; Sainburg et al., 2017). Both disciplines contribute to bridging the researchpractice gap. Translational research is focused on turning basic science knowledge into clinical
interventions and establishing their effectiveness, whereas implementation research focuses on
and supports implementation and evaluation of outcomes in clinical and community settings
(Weber, 2013). Notably, implementation science approaches promote equity, access, and
partner engagement (Brownson et al., 2021; Hursting & Chambers, 2021; Pérez Jolles et al.,
2022), which are also priorities in occupational science research (Haywood et al., 2019;
Whiteford & Hocking, 2012). Ultimately, the goal of using implementation science approaches is
to improve the quality, effectiveness, and public impact of health services (Eccles & Mittman,
2006).
Because of the many factors influencing uptake of complex technology-based
interventions, there is a need for systematic and contextually-driven analysis of facilitators and
barriers to technology use and practical application of these findings (Craig et al., 2008; Rizzuto
& Reeves, 2007). For example, implementation science frameworks and methodologies can
facilitate investigation of the practicalities of implementation, such as how devices are funded
(e.g., self-pay, health insurance, grants), where they are used (e.g., outpatient clinics,
community centers, homes), what resources are required to implement the technology in these
venues (e.g., clinician skill, social support, physical space, therapy session duration), and
barriers to access (e.g., transportation, time constraints, patients’ cognitive or physical skills,
affordability). Context is understood to be dynamic, so throughout the phases of implementation
science research, context is frequently analyzed to uncover its multiple layers and interactive
factors as they relate to implementation (Aarons et al., 2011).

17
Additionally, in contrast to the unsystematic way in which many technology-based
interventions are developed, implementation science approaches typically start with rationalizing
the development or implementation of a new intervention with theory and empirical evidence
(Craig et al., 2008). To that end, it is important to define core functions and forms of the
intervention. Core functions refer to the integral and theory-informed aspects of the intervention
that facilitate change, and that need to be clearly defined a priori. The specific and customizable
strategies, steps, or actions, called forms, that will be used to carry out the core functions should
be developed (Lengnick-Hall et al., 2021; Pérez Jolles et al., 2019). Whereas core functions are
transferable across contexts and essential to an intervention’s mechanism of action, forms can
evolve, adapt, and be tailored to meet the unique needs of local contexts (Pérez Jolles et al.,
2019). Taken together, the concepts of functions and forms offer guidance on the assessment of
multi-level interventions’ fidelity and adaptations, which can then be linked to proximal and distal
outcomes. Through systematic, structured analysis of the implementation context,
implementation science approaches can guide early-stage technology innovation toward
successful uptake.
Implementation Framework: EPIS. Implementation science is known for using
comprehensive, novel and pragmatic frameworks, measures, and tools to drive successful
implementation efforts from the exploration of an evidence-based practice through its sustained
implementation in real-world settings (Aarons et al., 2011). We are using the Exploration,
Preparation, Implementation and Sustainment (EPIS) framework (Aarons et al., 2011) to guide
the implementation process for Tele-REINVENT. EPIS is a deterministic framework, which is
suitable for gathering information about the barriers and enablers that influence implementation
of Tele-REINVENT. EPIS addresses the processes and structures that are at play between
contextual factors, which is useful for understanding the practicalities of implementing
technology in the highly nuanced health care climate (Aarons et al., 2011). EPIS has four
phases: Exploration, Preparation, Implementation, and Sustainment. In each phase, the outer

18
system and inner organizational contexts are defined along with bridging and innovation factors
that are specific to the innovation being implemented (see EPISframework.com). In the TeleREINVENT project, we are in the Preparation phase. Previously, the research team identified
that EMG-biofeedback was an efficacious EBP for improving motor control post stroke. Working
with stakeholders, it was identified that a telerehabilitation version of REINVENT would enable
stroke survivors to access rehabilitation at home. The choice to develop and implement TeleREINVENT was made because (1) it is a therapy option for individuals for whom standard
treatment is not typically effective (Teasell et al., 2018), (2) it can bridge gaps for people who
have limited access to therapy (Verma et al., 2022), and (3) it is well-tolerated and enjoyed by
stroke survivors (Donnelly et al., 2023; Marin-Pardo et al., 2022). Now, we are identifying
barriers and facilitators to implementation, making plans to reduce barriers, optimizing the
system design, and developing implementation strategies that will facilitate use of TeleREINVENT. The research conducted in the exploration and preparation phases can also be
described as ‘pre-implementation’ work, which focuses on identifying the fit between an
intervention and its delivery setting (Alley et al., 2023; Ellis et al., 2020; Jafni et al., 2017). As the
term suggests, this occurs prior to a device being put into practice.
User-Centered Design Process. User-centered design (UCD) is an approach to
designing innovations grounded in an understanding of user characteristics and needs (Dopp et
al., 2019; Interaction Design Foundation, 2016). Just as analyzing context is important in
occupational science for understanding engagement in occupation and in implementation
science for understanding the dynamic factors influencing uptake and sustainment, analyzing
the context of system use is a necessary precursor to design.
In UCD, the process of design includes specifying user requirements, designing
solutions, and evaluating the design against the previously identified requirements. These steps
are iterative and span all stages of system development to ensure that the design of an
innovative technology meets the needs of the user through their entire experience (Interaction

19
Design Foundation, 2016). While the present studies do not explicitly design a rehabilitation
system, the findings from each study have provided crucial context and outcomes influencing
the ongoing design of Tele-REINVENT. Therefore, the UCD process was an important
framework for conceptualizing the present studies and bridging theory and interdisciplinary
practice.
Interdisciplinary Approaches. The present studies integrate knowledge and
frameworks from various disciplines, including occupational science, implementation science,
and design, among others. While each has its distinct contribution and role in the current work,
there are also areas of complementarity and overlap. In addition to context being a core concept
in all noted disciplines, engaging stakeholders is a key process in all three disciplines.
Engaging Stakeholders in rehabilitation innovation. The needs of patients and
clinicians are typically the catalyst for health technology innovation, and yet these groups are
frequently excluded from the development of interventions or products designed to meet these
needs (Shah & Robinson, 2006; Smith et al., 2019). End users are regularly not involved in the
innovation process until clinical effectiveness testing (Smith et al., 2019). However, before then,
significant expenditures of time, money, and effort have been dedicated to developing an
intervention or product without consideration of the implementation context and dynamic user
needs (Smith et al., 2019). Users are the primary stakeholders for health technologies (Wale et
al., 2021), and yet their exclusion has caused countless technology development projects to fail
(Bano & Zowghi, 2015; Shah & Robinson, 2006). This problem is amplified when older adults
and people with disabilities are the target users (Patomella et al., 2013).
As an alternative, co-creation involves bringing together diverse stakeholders to
exchange expertise, whether from lived or professional experience, in designing, planning,
testing, and implementing interventions that they will use or deliver (Pérez Jolles et al., 2022). In
rehabilitation technology, key stakeholders may include stroke survivors, families, care partners,
health care providers, and health insurers, to name a few. The value of co-producing knowledge

20
with stakeholders is prominent in both the occupational science and implementation science
(Moullin et al., 2020; Pérez Jolles et al., 2022; Phipps et al., 2016). However, there are not
always clear paths to including stakeholders, and in some cases stakeholder engagement fails
to achieve its intended purpose (Fischer et al., 2020). Moullin and coauthors (2020) recommend
developing partnerships with stakeholders early in the implementation process, before defining
the issue or developing research questions.
Stakeholders may be involved throughout the entire process as consultants or
collaborators, or at a single timepoint with deeper engagement, such as defining research
questions or administering the intervention (Edward et al., 2021; Harrington et al., 2018; Moullin
et al., 2020). Stakeholders can be meaningfully engaged in rehabilitation innovation in a variety
of ways, so the benefits of such engagement can also vary; however, engaging with key
stakeholders will generally lead to greater relevance of technology-based interventions to the
needs and contexts of the end-users (Dobe et al., 2022; Kerr et al., 2018; Shah & Robinson,
2006). Two such participatory approaches are community advisory boards and co-design, each
of which will be briefly discussed below.
Community Advisory Boards. Community advisory boards (CABs) are one approach to
support collaborations between researchers and the community they seek to serve with their
research. Some of the benefits of CABs include feedback on project plans and results; advice
on design, implementation, and communication with the target audience; and access to
community channels to enhance buy-in (Halladay et al., 2017; Vaughn et al., 2018). CABs have
been established at the institutional level, like the American Heart Association [AHA] (Roach et
al., 2021), and for specific projects (Haywood et al., 2019). Establishing CABs is an adaptable
approach to engage stakeholders in meaningful research activities to improve the relevance of
research for the target community.
Co-Design. Co-design is an umbrella term encompassing a range of participatory
methods and activities that engage stakeholders in the development of innovative products or

21
services. Co-design trades traditional researcher-participant dynamics for a collaborative and
democratic approach, elevating the voices of the end user or service recipient. Co-design can
involve brainstorming, consulting, testing, or disseminating, among other activities (Edward et
al., 2021; Harrington et al., 2018). A product may be considered co-designed because feedback
was solicited through a survey at a single timepoint, or because a stakeholder from the target
population is a co-investigator. While co-design is effective as an iterative process spanning the
entire development process, it can also be used during one or multiple phases (Edward et al.,
2021). Dobe et al. (2022) conducted a review of co-design in the stroke rehabilitation literature
and found that co-design methodologies are a new and expanding research space in stroke
rehabilitation, with community interventions being the primary setting for co-design.
There are many benefits of employing a co-design approach, but perhaps the most
compelling reason to collaborate with stroke survivors on developing health technologies is also
the most obvious: if a technology is intended to solve a problem experienced by stroke survivors
in daily life, the individuals who have the problem should have input about the solution. Codesign methodologies in health technology research empower individuals who are often the
patients to be active in their health care and participate as co-designers of health innovations
(Östlund et al., 2020). Additionally, interventions developed through co-design tend to be more
person-centered (Harrington et al., 2018), which is a key principle of health care service in the
United States and elsewhere. In essence, co-design can yield products and services that the
target population are more likely to use (Davidson & Jensen, 2013; Edward et al., 2021).
Outcomes. Just as the present studies draw on frameworks from various disciplines,
complementary outcomes from these disciplines are also used. Specifically, we examined
engagement; implementation outcomes including acceptability and feasibility; and the design
outcome of usability in the present studies, each of which will be briefly discussed below.
Engagement. Engagement is a foundational construct in occupational science, though
its exact definition is debatable. A scoping review of engagement in occupation-based literature

22
identified that engagement has been used in terms of “active involvement in occupation, finding
value and meaning, subjective experience of engagement, social and environmental
interactions, balanced engagement, and developing identity through occupation” (Black et al.,
2019, p. 275). As discussed previously in this chapter, engaging stakeholders is crucial for the
successful development and implementation of rehabilitation technology. In all three of the
present studies, we sought to engage stakeholders and elevate their perspectives as valuable
sources of knowledge. However, measuring engagement—whether engagement truly occurred
and to what extent—is another challenge.
Implementation Outcomes. A common pitfall in the translation of rehabilitation
technologies to clinical practice identified previously in this chapter is the failure to account for
the multi-level barriers that hinder technology-based intervention uptake. Implementation is
most successful when an intervention is not only evaluated with efficacy outcomes (e.g., patient
symptoms, function) and service delivery outcomes (e.g., safety, patient-centeredness,
timeliness), but also implementation outcomes, which measure the effects of implementation
processes. Proctor et al. (2011) described the following implementation outcomes: acceptability,
adoption, appropriateness, costs, feasibility, fidelity, penetration, and sustainability. Each of
these is theoretically and empirically justified and can be evaluated with practical measures.
Implementation outcomes provide a taxonomy for conceptualizing the effects of implementation.
Notably, they can be evaluated at any point in implementation, even during pre-implementation
work. A benefit of evaluating implementation outcomes during this phase is that barriers can be
identified while the intervention and system design are still open to changes.
The present studies evaluate acceptability and feasibility of Tele-REINVENT.
Acceptability (also referred to as acceptance) includes key stakeholders’ attitudes and
satisfaction with an innovation (e.g., a service, intervention, practice, or technology) after
firsthand experience with it. Content, complexity, comfort, delivery method, and trustworthiness
can influence the extent to which a stakeholder accepts an innovation (Proctor et al., 2011). For

23
example, engaging experiences, progress indicators, logistical flexibility, training, and
interactions with clinicians positively influence acceptability of post-stroke telerehabilitation
interventions (Caughlin et al., 2020; Y. Chen et al., 2020).
Feasibility is the extent to which an innovation can be realistically carried out within a
specific setting (Proctor et al., 2011). Even if an innovation is accepted by stakeholders and
meets a need, the resources available in the practice setting must meet the requirements of the
innovation for successful implementation. Feasibility studies in telerehabilitation systematically
test the practical elements of implementing an innovation in a particular setting so that
implementation is successful and sustainable (Craig et al., 2008; Cramer et al., 2021).
In early-stage rehabilitation innovation, measuring acceptability provides insight into the
(un)satisfactory features of an innovation so that resources can be directed toward facilitating
acceptable experiences (Stephenson et al., 2022). Similarly, measuring feasibility early ensures
that a new telerehabilitation practice can successfully be carried out in the home environment.
Usability. The major outcomes of UCD work are systems that are easy to use and meet
users’ needs. Usability, therefore, is a primary outcome of interest during the design process.
Usability is a property dependent on interactions among users, product(s), tasks, and contexts
(Lewis, 2014). It is typically examined using questionnaires inquiring about satisfaction with
features of an interface (e.g., organization of on-screen content); processes related to using a
system (e.g., software installation); emotional reactions to using a system; helpfulness of
documentation (e.g., online tutorials, manuals); the ease of learning to use the system; and the
extent to which the system can be used efficiently to complete tasks (Sauro & Lewis, 2016).
Usability is related to acceptability and feasibility. For example, usability focuses on the
intuitiveness or ease of use of a product, but an easy-to-use product may be more acceptable to
an end user and may be more feasible to implement in settings where efficiency and speed are
highly valued. Therefore, evaluating and subsequently improving the usability of a rehabilitation
technology can contribute to greater acceptability and feasibility.

24
Complementarity of Occupational Science and Implementation Science. The two
primary disciplines informing this work are occupational science and implementation science.
There are many strengths of implementation science that are compatible with the vision for
occupational science, from describing and measuring contextual factors to the practical
complexities of enacting change. For example, implementation science heavily emphasizes the
importance of context, “not just as a backdrop” but as “a constellation” (Damschroder et al.,
2009, p. 3) of interacting circumstances, characteristics, and variables that influence outcomes
at multiple levels (e.g., individual, community, organizational, system; Damschroder et al., 2009;
Nilsen & Bernhardsson, 2019). The multi-level characteristics of local contexts often impact both
the implementation of an intervention and its effectiveness. This view of context parallels with a
transactional perspective on occupational science, where the relationships between context and
individual are the focus, rather than on the distinction between the two (Cutchin & Dickie, 2013).
Another area of complementarity is the value of engaging stakeholder perspectives. Coproducing knowledge with stakeholders has become an anchor in occupational science (Aldrich
& Marterella, 2014; Baranek et al., 2020) and community-engaged research has improved
interventions in occupational therapy (Haywood et al., 2019). Similarly, collaborations with
stakeholders in implementation science are important for improving health equity, acceptability
of interventions, outcome relevance, and other important factors that enhance the overall
delivery of health care (Kerkhoff et al., 2022; Pérez Jolles et al., 2022).
Finally, occupational science and implementation science both leverage mixed methods
to evaluate constructs that require analysis at multiple levels (Bauer et al., 2015; Clark et al.,
1991a), such as engagement, acceptability, feasibility, and usability. Mixed methods provide
robust and complementary research tools to study these constructs and provide a more
comprehensive picture of the implementation context for post-stroke telerehabilitation
technologies. Moreover, my colleagues bring various disciplinary and professional backgrounds
to the Tele-REINVENT team, spanning neuroscience, biomedical engineering, game

25
development, occupational therapy, and occupational science. Mixed methods help us draw
from our diverse expertise and develop a more thorough interpretation of the findings.
Present Studies
The central goal of the present studies is to examine factors influencing the
implementation of Tele-REINVENT into routine stroke rehabilitation practice. In this chapter, I
described the current state of stroke rehabilitation and novel technology-based approaches
designed to improve post-stroke outcomes. Then, I explained factors undermining the
translation of these approaches to clinical practice, followed by approaches, frameworks, and
outcomes that can support translation.
In Chapter 2, I present a study evaluating the acceptability and usability of TeleREINVENT among stroke survivors using it at home. A version of this chapter was published in
the American Journal of Occupational Therapy with co-authors Octavio Marin-Pardo, MS, Aisha
Abdullah, MS, Coralie S. Phanord, BS, Amisha Kumar, Stuti Chakraborty, BS, and Sook-Lei
Liew, PhD, OTR/L (Donnelly et al., 2024). In Chapter 3, I share the findings from a survey
conducted with rehabilitation therapists about the feasibility and acceptability of Tele-REINVENT
based on a short demonstration video. In Chapter 4, I describe the development of a CAB to cocreate knowledge with stakeholders to increase the effectiveness of research and
implementation in stroke rehabilitation.
Finally, in Chapter 5, I synthesize findings from all three studies, describe the
innovations and research implications of this work, and suggest future directions. I also
emphasize the important role and perspective of occupational science in designing rehabilitation
products that will be used in people’s everyday lives and routines.

26
Chapter 2: Acceptability and Usability of a Post-stroke Complex
Telerehabilitation Biofeedback Intervention Among
Stroke Survivors Using it at Home
Abstract
Complex telehealth interventions can facilitate remote occupational therapy services and
improve access for people living with stroke and other chronic neurologic conditions. It is
important to understand factors that influence the uptake of these technologies. This preimplementation study explores the fit between electromyography (EMG) biofeedback and
telerehabilitation for stroke survivors to optimize EMG-biofeedback interventions and more
broadly, support other efforts to develop complex telerehabilitation interventions. Stroke
survivors (N=11; mean age = 54 years) used a game-based EMG-biofeedback intervention for
arm sensorimotor rehabilitation at home, delivered via telehealth. We conducted a mixed
methods analysis of usability and acceptability data collected with stroke survivors during a pilot
and feasibility study. Quantitative measures show high levels of perceived usability and
acceptability, supported by qualitative findings describing specific facilitators and barriers. Preimplementation studies can improve the design and relevance of complex telehealth
interventions. One major conclusion from this study is the influence of therapy providers on
acceptability and usability of complex telehealth interventions. This study supports the
development and implementation of complex telehealth interventions, which have the potential
to improve access to occupational therapy for clients living with chronic neurologic conditions.

27
Introduction
Complex telehealth interventions for neurologic recovery have the potential to facilitate
remote delivery of occupational therapy services and improve access for people living with
complex, chronic neurologic conditions (Davis et al., 2020; Laver & Osborne, 2022). While there
is no clear delineation between ‘simple’ and ‘complex’ interventions, characteristics of
complexity include (1) interacting components, (2) specialized patient or provider behaviors, (3)
a heterogeneous target audience, (4) multiple outcomes, and (5) adaptability of the intervention
(Craig et al., 2008). Applying these characteristics to neurologic rehabilitation, interventions
such as brain and muscle biofeedback, serious gaming, and functional electrical stimulation
could be described as complex. Interventions become increasingly complex when delivered at
home, through virtual therapy sessions, using multiple technologies, with remote data
monitoring, or any combination of these. The potential of complex telehealth interventions is
underrealized, in part because they are particularly challenging to develop and implement
(Stephenson et al., 2022).
There is a growing body of literature evaluating complex telehealth interventions for
neurologic conditions, including stroke, though many are small trials producing relatively weak
evidence (English et al., 2022). Even systematic reviews are inconclusive about the efficacy of
telerehabilitation, in part because of the variety of interventions and outcome variables (Laver et
al., 2020). While efficacy outcomes are vital to intervention research, early examination of
implementation outcomes are also important. Even highly efficacious complex interventions may
not be routinely adopted if there is a poor fit between the user(s), intervention, and delivery
context. Therefore, concurrent assessment of efficacy and implementation outcomes has been
proposed to support the successful embedding of interventions into service delivery (Curran et
al., 2012; English et al., 2022). The evaluation of fit between intervention and delivery context
prior to implementation is referred to as ‘pre-implementation’ (Kerkhoff et al., 2022).

28
Facilitators and barriers to implementing stroke rehabilitation interventions have been
well-described in the literature, and the field is moving toward developing targeted
implementation strategies that promote the sustainable uptake of innovative treatments into
occupational therapy practice (Juckett et al., 2020; Stephenson et al., 2022). An important step
is evaluating factors known to influence the uptake of complex telehealth interventions
(Skidmore et al., 2014). For instance, Broens et al. (2007) identified five determinants of
success of telemedicine: technology, acceptance, financing, organization, and policy/legislation.
The current pre-implementation study examines the first two of these, among stroke survivors.
The technology determinant includes support, training, usability, and quality. Acceptance (or
acceptability) includes the attitudes and perceptions of stakeholders toward the telehealth
technology, including its content, complexity, comfort, and other dimensions (Broens et al.,
2007; Proctor et al., 2011). The remaining determinants are not examined in this paper because
they require methods that leverage different stakeholder perspectives (e.g., cost analysis,
organizational assessment, policy analysis).
In this study, we evaluated the usability and acceptability of an electromyography (EMG)
biofeedback telehealth intervention for post-stroke upper extremity motor recovery (TeleREINVENT; Marin-Pardo et al., 2022). This study builds on previous pilot work examining the
acceptability of Tele-REINVENT through thematic analysis (N=4; Donnelly et al., 2023) by using
a mixed methods approach, including validated measures of usability and acceptability, with a
larger sample of users. The purpose of this pre-implementation work is to support (1)
identification of the fit between EMG biofeedback and telerehabilitation for stroke survivors (2)
optimization of EMG-biofeedback interventions, and more broadly, (3) other efforts to develop
complex telerehabilitation interventions for routine clinical practice.
Methods
The data reported here were collected between November 2021 and November 2022 as
part of a pilot and feasibility trial of Tele-REINVENT in which participants used the system at

29
home for 6 weeks through a combination of therapist-guided Zoom videoconference sessions
and participant self-led sessions. Here, we briefly describe the trial procedures to contextualize
the usability and acceptability findings. See Appendix A for additional protocol details and MarinPardo et al. (2022) for the technical specifications and efficacy findings. All study procedures
were conducted observing the ethical standards of the University of Southern California
Institutional Review Board and the revised (2013) Helsinki Declaration.
Participants
We sought to recruit stroke survivors in the chronic phase of recovery (> 6 months since
onset) with moderate to severe upper extremity hemiparesis, which we operationalized as
limited wrist extension and a minimum level of extensor EMG activity (i.e., hold 30% of a
prerecorded maximum voluntary contraction for ten, 4-second trials). English and Spanishspeaking individuals were eligible. Significant vision loss, receptive aphasia, hand contractures,
cognitive impairment (Montreal Cognitive Assessment [MoCA] < 20), secondary neurological
disease, current use of anti-spasticity medication, and current therapy treatment targeting arm
function were exclusion criteria.
We identified prospective participants via a stroke research database, flyers, and word of
mouth. Rolling recruitment was completed from October 2021 to October 2022. Prospective
participants completed a phone screening for criteria that could be assessed verbally, followed
by an in-person screening to test muscle activity and cognition.
Protocol
Participants returned to the laboratory within one week of the screening for baseline
clinical and physiological testing. Additionally, we oriented them to Tele-REINVENT (Figure 2.1)
and sent it home with them to complete 15, 1-hour sessions over 3 weeks. They returned to the
laboratory for midpoint testing and were invited to use Tele-REINVENT at home for 3 more
weeks if they completed ≥ 80% of at-home sessions and demonstrated improvement on at least

30
one measure (Action Research Arm Test [ARAT], Fugl-Meyer Assessment Upper Extremity
[FMA], or EMG task). Finally, participants returned to the laboratory for post-testing.
Figure 2.1
Tele-REINVENT Interface
Note. A) The components of the Tele-REINVENT interface: a laptop computer with a charger
and a wireless mouse, EMG sensors (enclosed in the blue cases), disposable electrodes,
alcohol wipes, and a foam roll and towel to assist with cushioning and positioning. B) A user
with EMG sensors over his wrist flexor and extensor muscles using the electrode pads provided.
The sensors are connected to the laptop via a USB cable. A game can be seen on the screen.
C) Tele-REINVENT has 3 game selections: Skeeball, Planet Jump, and Blinko. D) When a user
selects a game, an instruction screen appears, explaining the objective of the game and which
movements (wrist flexion or extension) correspond with which actions in the game. This image
shows the Planet Jump start screen informing the user that wrist flexion causes the avatar to

31
stop while wrist extension causes the avatar to jump. E) In Skeeball, wrist extension is used to
roll a ball up a lane into one of three bullseye rings. F) In Planet Jump, wrist extension is used to
jump the astronaut avatar over obstacles and wrist flexion to prevent collisions with volcanoes.
G) In Blinko, wrist flexion and extension are used to move a disk across the top of a vertical
game board, and when the timer runs out, the ball falls down the board until it lands on a point
value.
Outcome measures
Demographic data were collected during the first visit. During the post-testing visit,
participants completed validated questionnaires measuring perceived usability and acceptability,
and a semi-structured interview.
Quantitative Outcomes
Usability. Usability is dependent on interactions among users, the product(s), tasks, and
contexts. Perceived usability was assessed quantitatively using the Post-Study System Usability
Questionnaire Version 3 (PSSUQ; Sauro & Lewis, 2016). The PSSUQ is a reliable 16-item
questionnaire that covers five usability characteristics (quick completion of work, ease of
learning, high-quality documentation and online information, functional adequacy, and rapid
acquisition of productivity) and is usable with small samples. Each item is scored on a 7-point
Likert scale (1=Strongly Agree; 7=Strongly Disagree). Lower scores indicate greater
satisfaction. The PSSUQ produces an overall score and three subscales: System Quality
(SysQual), Information Quality (InfoQual), and Interface Quality (InterQual).
Acceptability. Acceptability is the stakeholders’ perception that the components of an
intervention are agreeable or satisfactory. Acceptability was assessed quantitatively using the
extended unified theory of acceptance and use of technology model (UTAUT; Chao, 2019),
which contains 31 items and measures 8 constructs that predict technology acceptance (effort
expectancy, performance expectancy, perceived enjoyment, satisfaction, trust, mobile selfefficacy, perceived risk, behavioral intention; see Appendix A for definitions). Each item is
scored on a 5-point Likert scale (1=Strongly Disagree; 5=Strongly Agree). Higher scores
indicate greater acceptance.

32
Qualitative Outcomes
Participants completed a semi-structured interview about their experience with TeleREINVENT (e.g., perceived benefits, preferences, facilitators, and barriers to use). Interviews
were conducted in a private room by a member of the research team (M.R.D., O.M.P., or C.S.P)
and typically lasted between 20 and 40 minutes. Questions were intentionally broad and
experiential to evoke stories and honest appraisals rather than asking specific questions about
usability and acceptability (see Donnelly et al., 2023 for interview guide). The interviewer did not
view participants’ questionnaire responses prior to conducting the interview. This allowed
participants to share what they perceived to be salient rather than prompting comments on
specific features or questionnaire constructs.
Interviews were audio recorded, transcribed verbatim in their original language, and
checked by a second reviewer. Participant quotes in this paper were only edited with brackets
for clarity, ellipses for brevity, or to remove identifiable information. The “accents” of non-native
English speakers (i.e., English as a second language) were not edited. Quotes from Spanishspeaking participants were translated to English.
Data Analysis & Interpretation
All statistical analyses were run in R (version 3.6.3; R Core Team, 2020). Descriptive
analysis was completed for demographic data. An overall PSSUQ score, subscale scores, and
mean scores for each item were calculated. An overall UTAUT score, means for each
construct, and mean scores for each item were calculated. Multiple linear regression tested if
personal characteristics (i.e., age, education, time since stroke, severity of impairment)
contributed to acceptability and usability. Interview transcripts were analyzed using PSSUQ and
UTAUT constructs as pre-defined codes. The complete transcripts were analyzed; however,
only segments that addressed the constructs were coded. Initial coding was conducted
independently by two reviewers using a word processor. Codes were then compared line-by-line
and consensus was reached to establish the final dataset. Concurrent triangulation was used to

33
corroborate the mixed methods findings; quantitative and qualitative data were collected
concurrently, analyzed, and reported separately, and integrated during interpretation.
Results
Eleven participants used Tele-REINVENT at home for up to 6 weeks. Two of the eleven
participants completed ≤ 3 weeks of the protocol but were included in this analysis because
their data are relevant to the constructs of interest. Demographic data are summarized in Table
2.1. Despite efforts to recruit females, all participants were males, which is a challenge noted in
the stroke literature (Carcel & Reeves, 2021).
Usability
Quantitative Outcomes
The PSSUQ overall score and subscale scores show that Tele-REINVENT is highly
rated for usability (Table 2.2, norms listed for reference). Multiple linear regression tested if
personal characteristics significantly predicted usability scores. None of the factors were
significantly associated with usability (Table 2.3), though this null result may be due to our small
sample. We also calculated the mean for each PSSUQ item to identify areas of relative strength
and weakness of Tele-REINVENT’s usability. Because a score of 1 indicates greater usability,
we defined items with scores ≥ 3 as high priority areas of improvement to optimize TeleREINVENT’s usability. This threshold yielded two InfoQual items (see boldface items in Table
2.2).
Table 2.1
Participant Demographics (N=11)
Characteristic n (%)
Age, M ± SD 54 ± 9.5
Sex
Male 11 (100)
Race
African American 3 (27.2)
Asian 2 (18.2)

34
White 2 (18.2)
Hispanic 2 (18.2)
Native Hawaiian 1 (9.1)
Other 1 (9.1)
Ethnicity
Hispanic or Latino 4 (36.4)
Not Hispanic or Latino 7 (63.6)
Years of Education, M ± SD
13.91 (2.5)
Chronicity (months since onset), M ± SD 75.91 ± 59.4
Handedness Prior to Stroke
Left 3 (27.2)
Right 8 (72.7)
More Impaired Side
Left 8 (72.7)
Right 3 (27.2)
Baseline FMA, M ± SD
29.27 ± 14.3
Baseline ARAT, M ± SD
16.82 ± 15.4
Note. FMA = Fugl-Meyer Assessment of the Upper Extremity; ARAT = Action Research Arm
Test

35
Table 2.2
Post-Study System Usability Questionnaire Scores by Item
Item M SD
1 Overall, I am satisfied with how easy it is to use this system 2.00 0.67
2 It was simple to use this system 2.10 0.99
3 I was able to complete the tasks and scenarios quickly using this system 2.50 1.08
4 I felt comfortable using this system 1.60 0.70
5 It was easy to learn to use this system 1.70 0.95
6 I believe I could become productive quickly using this system 2.10 1.29
7 The system gave error messages that clearly told me how to fix
problems 4.20 1.48
8 Whenever I made a mistake using the system, I could recover easily
and quickly 3.80 1.69
9 The information (such as online help, on-screen messages, and other
documentation) provided with this system was clear 2.30 1.42
10 It was easy to find the information I needed 2.60 1.43
11 The information was effective in helping me complete the tasks and
scenarios 2.40 0.97
12 The organization of information on the system screens was clear 2.40 1.43
13 The interface of this system was pleasant 1.90 1.29
14 I liked using the interface of this system 2.20 1.32
15 This system has all the functions and capabilities I expect it to have 2.80 1.40
16 Overall, I am satisfied with this system 2.10 1.60
Scale (scale scoring rule; scale norma
)
System Quality (average Items 1–6; 2.80) 1.95 0.68
Information Quality (average Items 7–12; 3.02) 2.91 1.04
Interface Quality (average Items 13–15; 2.49) 2.36 1.22
Overall (average Items 1–16; 2.83) 2.43 0.91
Note. Scores closer to 1 indicate greater usability. Items with mean scores ≥ 3 (indicating areas
for improvement) are in bold. The PSSUQ produces an overall score and three subscales.
Scale norms are provided for reference. PSSUQ = Post-Study System Usability Questionnaire
Version 3 a Sauro & Lewis (2016)

36
Table 2.3
Relationships Between Personal Characteristics and Perceived Usability and Acceptability
Usability(N = 11, R2 = 0.375) Acceptability
(N = 11, R2 = 0.457)
Predictor b SE 95% CI p
Value
b SE 95% CI p Value
Age -0.05 0.05 -0.18 to 0.08 0.35 0.01 0.03 -0.07 to 0.10 0.69
Education 0.05 0.16 -0.36 to 0.47 0.75 0.02 0.11 -0.27 to 0.31 0.87
Chronicity 0 0.01 -0.02 to 0.02 0.91 0 0.01 -0.01 to 0.02 0.46
FMA -0.05 0.08 -0.26 to 0.16 0.57 0.02 0.06 -0.13 to 0.16 0.74
ARAT 0.05 0.07 -0.13 to 0.23 0.50 -0.04 0.05 -0.16 to 0.08 0.46
Note. Summary statistics from multiple linear regression to test associations between personal
characteristics and usability and acceptability scores. The sample size (N), R2
, b, SE, 95% CI,
and p value for all predictors are reported. FMA = Fugl Meyer Assessment of the Upper
Extremity (baseline); ARAT = Action Research Arm Test (baseline).
Qualitative Outcomes
System Quality. Most participants described a learning process to become productive
with the system. They noted that sensor set up and calibration were tedious initially, and the
system did not respond well to their attempted movements. The intervention involves various
complex tasks (e.g., calibrating EMG sensors), but by the end of the study, most perceived the
system as simple, even “basic”. Some attributed these changes to learning how the system
processes movements:
“Learning the hand movements of what was most effective…moving the, either the ball
or…making the little figures jump… Sometimes I would have overt movements to where
instead of keeping the wrist just going at a steady up and down, I would sometimes
tense up and turn to the sides. So, when I turned the hand to the sides, I think that’s
when I wasn’t getting good reaction with the games.”
As participants became comfortable with the intervention, the provider remotely adjusted game
pace and sensitivity settings. For example, as one participant mastered Skeeball,
“[the researchers] were able to speed it up a little more…so that was a good
adjustment…the slower speed was definitely good for the first couple weeks because
you’re still trying to adjust to the game. But then after, you know, you start to—you
attempt to progress at the game and it’s like ‘okay, you know, let’s do this a little quicker
now.’”
Information Quality. Substantial evidence showed that the researchers—rather than
the system notifications, video tutorials, or manual—were the primary information sources and

37
facilitators of usability. Participants noted that when they encountered issues, the system did not
offer clear guidance on the nature of the issue or how to proceed. Early in the trial, the research
team was particularly involved in supporting participants as they acclimated to the system, and
the information provided during those interactions was perceived as high quality. Only one
participant reported using the manual, and none mentioned the tutorial videos.
Interface Quality. Participants wanted more durable sensor cases with built-in
placement cues and identified features they expected the games to have, such as the ability to
resume a level, view a previous high score between sessions, or slow the speed of Planet Jump
for more processing time.
Acceptability
Quantitative Outcomes
The UTAUT scores indicated that Tele-REINVENT was highly acceptable among stroke
survivors (Table 2.4). Multiple linear regression tested if personal characteristics significantly
predicted acceptability scores. None of the factors were significantly associated with
acceptability (Table 2.3).
We also calculated the mean of each UTAUT item to identify specific barriers and
facilitators to acceptability. A score of 5 indicates greater acceptability, so we defined items with
scores < 4 as high priority areas of improvement to optimize Tele-REINVENT’s acceptability.
This threshold yielded 7 items across five subscales (see boldface items in Table 2.4).

38
Table 2.4
Unified Theory of Acceptance and Use of Technology Item and Construct Scores
Constructs Items M SD
Learning how to use REINVENT is easy for me 4.45 0.69
My interaction with REINVENT would be clear and
understandable 4.18 0.87
Effort Expectancy I find REINVENT easy to use 4.36 0.50
It is easy for me to become skillful at using REINVENT 3.82 1.08
I would find it easy to get REINVENT to do what I want
it to do 3.55 0.82
Performance
Expectancy
Using REINVENT would improve my performance 4.36 0.50
Using REINVENT increases my chances of achieving
goals that are important to me 4.27 0.79
Using REINVENT would allow me to accomplish
rehabilitation tasks more quickly 4.18 0.75
Using REINVENT would enhance my effectiveness in
rehabilitation 4.00 0.89
Perceived Enjoyment
I find using REINVENT enjoyable 4.27 0.79
The actual process of using REINVENT is pleasant 4.18 0.75
I have fun using REINVENT 4.00 1.26
Satisfaction
I was very content with REINVENT 3.82 0.60
I was very pleased with REINVENT 4.27 0.79
I was satisfied with REINVENT's efficiency 4.00 1.18
I felt delighted with REINVENT 3.91 0.94
Overall, I was satisfied with REINVENT 4.09 1.14
Trust
I believe that REINVENT is trustworthy 4.36 0.81
I trust in REINVENT 4.27 0.79
I do not doubt the honesty of REINVENT 4.09 1.04
Even if not monitored, I would trust REINVENT to do the
job right 4.09 0.70
REINVENT has the ability to fulfill its task 4.27 0.65
Self-efficacy
I am confident using REINVENT even if there is no one
around to show me how to do it 3.82 1.08
I am confident using REINVENT even if I have never used
such a system before 4.27 0.65
I am confident using REINVENT even if I have only the
software manuals for reference 4.09 0.70
Perceived Risk
I think using REINVENT puts my privacy at risk 4.27 0.65
Using REINVENT exposes me to an overall risk 4.09 0.83
Using REINVENT will not fit well with my self-image 3.82 1.33
Behavioral Intention
Assuming I had access to REINVENT, I intend to use it 4.45 0.69
Given that I had access to REINVENT, I predict that I
would use it 4.36 0.67
I plan to use REINVENT in the future 3.82 0.87
Construct
Effort Expectancy 4.07 0.43
Performance Expectancy 4.20 0.65
Perceived Enjoyment 4.15 0.85

39
Satisfaction 4.02 0.84
Trust 4.22 0.66
Self-efficacy 4.06 0.47
Perceived Risk 4.06 0.65
Behavioral Intention 4.21 0.69
Note. Mean score for each item. Scores closer to 5 indicate greater acceptability. Items with
mean scores < 4 (areas of improvement) are denoted in bold. The UTAUT produces one overall
score, and we calculated the mean for each construct.
Qualitative Outcomes
Effort Expectancy. Tele-REINVENT was perceived as easy to use; however, as
previously noted, there was a learning curve, “The first time was a little difficult because I had to
get used to how to connect everything…because it’s…a system that I’m not used to” but, “as
time went on it got easier and easier.” Even participants who had limited experience using
computers found Tele-REINVENT more effortless to use than they expected.
Performance Expectancy. Participants perceived that Tele-REINVENT supported their
rehabilitation goals. Tele-REINVENT encouraged them to focus on controlling the movement
and relaxation of their affected arm. Most participants perceived some functional improvement
(e.g., grasping handles/knobs more easily) and felt it elicited greater awareness of their arm.
Perceived Enjoyment. Tele-REINVENT was widely enjoyed, particularly when the
games were perceived to be an appropriate challenge. Earning a better score or leveling up
were particularly motivating and led to enjoyment. Blinko is probability-based, and while some
participants enjoyed the animations and movement practice, the scoring did not reflect
performance, making it less motivating. Technical issues and a limited selection of games
somewhat detracted from enjoyment.
Satisfaction. Participants were overall satisfied to try a novel rehabilitation technology,
play games using their affected arm, interact with the researchers, and perceive improvements.
In contrast, the challenges of using such a system were dissatisfying, like when sensors were
placed incorrectly, “you have to start the program over again.”

40
Trust. Participants did not discuss trust. We hypothesize that the informed consent
process contributed to high levels of trust among participants.
Perceived Risk. There were no data about privacy risks. However, one participant
perceived troubleshooting system issues as risky, “you don't know what might happen if…you
don't know what you're doing...I'd rather just stick to what I know and just take notes.”
Self-Efficacy. The self-efficacy findings were twofold. First, participants exuded pride as
they described their game achievements. Many were eager to share that they could now use
the system mostly independently, even when technical issues arose. It was evident that TeleREINVENT provided opportunities to skillfully use their affected arm and participate in
rehabilitation without hands-on assistance. By contrast, participants sometimes doubted if they
had the skills needed to use the system. For example, some experienced sessions where they
felt they could not control their arm and doubted their own abilities, “I was getting frustrated
because no matter how I told the brain to send the signal, it didn’t do it the first few times.” Most
participants emphasized that virtual support from the researchers helped build confidence and
was a valuable part of the intervention.
Behavioral Intention. Most participants said they would continue using Tele-REINVENT
if given the opportunity. Alternatively, one participant who stopped participating in the study after
3 weeks described his intention to not use Tele-REINVENT, noting that it takes “practice,
practice, practice…I really didn’t have the chance to practice like I wanted to…I just don’t have
time.” He also added that “glitches” are “a deterrent. To me, it’s just like, you know what? I’ll do
this later...and of course, technology is technology, but if it was a smoother session, it would
encourage more activity.” While there was interest in continued Tele-REINVENT use, there are
factors that should be addressed to support such use.

41
Discussion
We evaluated the usability and acceptability of an EMG-biofeedback telehealth
intervention among stroke survivors to optimize its design, as both usability of technology and
acceptability are determinants of successful telemedicine implementation (Broens et al., 2007).
We triangulated the findings to identify opportunities to improve Tele-REINVENT and generate
broader insights for developing complex telerehabilitation interventions.
Tele-REINVENT scored favorably for technology usability. The PSSUQ is not typically
interpreted using normative scores; however, in the absence of PSSUQ data from a similar
intervention, the norms (listed in Table 2.3) can be a reference (Sauro & Lewis, 2016). Notably,
the PSSUQ scores for Tele-REINVENT exceed the norms. We identified InfoQual items that
were rated less favorably than the other items. The PSSUQ authors report that InfoQual tends
to perform worse than the other subscales, in part because creating informative, actionable
error messages is challenging (Sauro & Lewis, 2016). Despite the relative weakness, the
InfoQual scores for Tele-REINVENT are remarkably positive, even though participants reported
challenges addressing errors. The qualitative data suggest that much of the information that
facilitated usability of the system came from interactions with the researchers, not from the
system itself. We hypothesize that the disparity between high PSSUQ scores and critique in the
interview data may be due to different framings of “the system.” While the software system did
not give clear messages to guide recovery from mistakes, participants had easy and quick
access to support, which may have led to the interpretation of “the system” as a broader entity
encompassing the human and technological components of the intervention. This framing is
useful to consider in complex telehealth interventions, where the patient, provider, technology,
context, and tasks each play a role in the intervention.
Similarly, qualitative findings about acceptability were nearly inseparable from
interactions with the team. The qualitative data suggest that the providers reduced technical
barriers that affected effort expectancy, perceived enjoyment, self-efficacy, and satisfaction. For

42
example, guided sessions developed participants’ confidence and skills. This can be seen
quantitatively as well; the item measuring confidence using a novel system was scored high, but
the item measuring independent use of the system had a lower score. Researcher interactions
were a relative strength of the intervention, even beyond the benefits of technical support;
however, the level of support provided in this study may not be feasible in clinical environments.
Therefore, as implementation efforts progress, repeat usability testing in a realistic clinical
environment is warranted.
We also identified relationships spanning constructs on both questionnaires. For
example, when participants encountered issues they could not solve (a reflection of SysQual
and InfoQual), they perceived the system as more difficult to use (effort expectancy) and
sometimes questioned if the issue was caused by their own lack of skill (self-efficacy). Given
that both perceived usability and acceptability are widely accepted as determinants of success
of technology implementation (Broens et al., 2007; Chao, 2019), it is not surprising that the
usability and acceptability findings are closely linked.
Personal characteristics can influence the uptake of complex telehealth interventions.
These associations are important to identify so that interventions can be adapted for diverse
groups. For example, one study found that educated and resourced young-old, white females
were more likely to use health technology than their peers who were older, male, and/or of
different races. (Hung et al., 2020), revealing disparities that can be addressed methodically. In
our study, the personal characteristics we tested had no significant explanatory value for
acceptability or usability. This is promising, as older adults with chronic, severe stroke and less
education have more gaps to accessing care (Ayala et al., 2018; Teasell et al., 2018). Future
work with a larger sample should further evaluate if age, education, time since stroke, and
stroke severity influence implementation outcomes. .
In addition to identifying relative strengths and weaknesses of Tele-REINVENT, we
extracted insights to more broadly support other pre-implementation efforts to develop complex

43
telerehabilitation interventions for use in clinical practice. For each of the 5 ‘lessons learned’, we
include examples of how we have applied (or plan to apply) them, and the findings of this study,
to optimize Tele-REINVENT.
1. Include stakeholders early and often in the design and implementation planning
of complex telehealth interventions. There has been a rise in literature describing the
benefits of co-creation and pragmatic tools for engaging stakeholders (Kerkhoff et al., 2022;
Pérez Jolles et al., 2022). Though not discussed here, our group often engages stroke survivors
in the design of interventions; for example, they have consulted on wearable sensor design,
game development, and alpha testing. While stroke survivors were participants in this study, we
prioritized using knowledge from lived experience to influence enhancement of the system.
2. Identify and enable the distinct value of the provider as both a clinician and
facilitator of technology use. The intervention provider is an important component of complex
telehealth intervention delivery, as seen in our study and others (Caughlin et al., 2020; Neibling
et al., 2021). However, skilled therapy providers can quickly become technical support when
using technology-based interventions. We identified the primary way that providers add value to
the Tele-REINVENT is by personalizing the intervention. For example, participants benefitted
from feedback about positioning and technique and adjustments to game settings. We
strategized ways to enable that value and reduce barriers. For example, in the version of TeleREINVENT tested in this study, game settings were adjusted in configuration files. We have
now developed a clinician interface for all setting adjustments. We also identified that
troubleshooting consumes valuable participant-provider contact time, so we also focused on
reducing the impact of technical issues, discussed more below.
3. Differentiate user error from system error. We observed that uncertainty about the
source of a problem quickly led to frustration, low self-efficacy, and dissatisfaction. However,
when participants knew they triggered an error (e.g., by placing a sensor incorrectly), the
problem was manageable, and they could act. Similarly, if they realized that a game was not

44
reading their EMG signal, they could restart the software. Providing clear, actionable error
messages equips users with information they need to proceed. Based on our findings, we
developed simple, jargon-free, and actionable error messages for common issues.
4. Develop implementation strategies to address common barriers. Technical
issues are ubiquitous in rehabilitation technology, whether an intervention is elegant or
bootstrapped; however, they do not have a detrimental effect on acceptability (Caughlin et al.,
2020; Donnelly et al., 2023; Neibling et al., 2021). In fact, implementation strategies that
address inevitable barriers can enhance usability and acceptability. In addition to a brief inperson orientation and virtual sessions, we provided a printed user manual with simple wording
and photos. We also embedded tutorial videos into the system. In the interview, only one
participant mentioned the manual, and no one mentioned the videos. However, they all reported
technical challenges. This provided evidence that the implementation strategies we developed
were not the right fit for our users—that is, they clearly preferred personal (live) consultations
versus written guides or asynchronous videos. We intend to co-design more effective and
sustainable implementation strategies with both stroke survivor and clinician stakeholders, given
that live consultations and training may not be feasible in real therapy contexts.
5. Embed progress tracking and social features. Gamified experiences can be
motivating, and this has been leveraged in many rehabilitation technologies. However, game
mechanisms and preferences may differ by target population. For example, of the 3 TeleREINVENT games, most participants found the levels in Planet Jump and scores in Skeeball to
be most motivating. Blinko users emphasized that they were motivated by using their affected
arm, not the score. Our previous, preliminary work examining the acceptability of TeleREINVENT (Donnelly et al., 2023) and the current study suggest the following for future game
development: (a) preserve achievements between sessions (i.e., all-time high scores), (b) use
scoring schemes that directly reflect performance, (c) gamify multiple rehabilitation goals, and
(d) develop options to compete (e.g., leaderboard) and communicate with other users.

45
Limitations & Future Directions
This study used inexpensive, user-focused, and replicable methods to assess preimplementation usability and acceptability. This pragmatic approach allowed quick iterations of
intervention design. Despite the benefits, these methods do not capture the full depth and
breadth of usability and acceptability, so additional testing in future stages of this intervention is
warranted. Additionally, both the small sample size and exclusion criteria may limit the
generalizability of the usability and acceptability findings. We gained valuable perspective from
stroke survivors, and future work is underway to assess pre-implementation outcomes among
other stakeholders, including therapists, as they are important decision makers and influencers
of rehabilitation technology uptake (Chen & Bode, 2011). Finally, future work should examine
financing, organizational, and policy factors to support future implementation of REINVENT
(Broens et al., 2007). This pre-implementation work will support embedding the system into
routine clinical practice in the future.
Conclusions
This study contributes to an emerging body of literature examining the use of complex
telehealth interventions with survivors of neurologic injury. Our findings and ‘lessons learned’
underscore the value of occupational therapists in delivering telehealth interventions and
opportunities for telehealth to enable remote access to therapy. Implementation outcomes, such
as acceptability and usability, are often reserved for late-stage intervention research; however,
evaluating these outcomes pre-implementation can ensure that complex interventions meet the
needs and requirements of its intended users and delivery context. This is important, as
complex telehealth interventions can open doors for survivors of neurologic injury who face
barriers to accessing occupational therapy and would benefit from technology-enabled therapy
at home.

46
Chapter 3: Evaluating pre-implementation outcomes of an EMGbased stroke rehabilitation intervention using a video
demonstration survey methodology
Abstract
Many promising novel rehabilitation devices have been developed to address gaps in
post-stroke recovery outcomes; however, there is a trend of failed translation from the laboratory
to uptake in real clinical settings (Hughes et al., 2014; Morrow et al., 2021). This gap is due in
part to a lack of understanding of the multilevel contextual barriers that affect uptake of new
technology-based interventions into post-stroke care and lack of planning for implementation
support. Additionally, clinicians are often left out of design and development processes, despite
their privileged knowledge about patients’ needs and the health care context (Celian et al.,
2021; Smith et al., 2019). Thus, to enhance the design and implementation planning of an
electromyography biofeedback intervention (Tele-REINVENT), we evaluated pre-implementation
acceptability and feasibility among rehabilitation therapists. Occupational and physical therapy
practitioners (N=55) viewed a video demonstration of the device and completed brief measures
of acceptability and feasibility, as well as open-ended questions about barriers and supports for
using Tele-REINVENT in practice. Overall, therapists rated Tele-REINVENT as acceptable and
feasible. Qualitative findings include anticipated challenges to using Tele-REINVENT and
supports that may be needed to successfully use it in practice. The findings from this study will
inform the ongoing design and implementation planning of Tele-REINVENT. Additionally, groups
developing new devices for stroke rehabilitation may benefit from these findings as they seek to
understand some of the contextual and personal factors that influence the uptake of technology
in clinical practice.

47
Introduction
There is an abundance of literature describing the development and efficacy of
innovative rehabilitation devices leveraging virtual reality, robotics, and wearable technology for
post-stroke motor recovery (Everard et al., 2022; Laver et al., 2017; Mehrholz et al., 2018).
However, very few make it to market or are adopted in the clinical settings for which they were
intended (Hughes et al., 2014; Morrow et al., 2021). Health technology innovation has been
shown to be most successful when clinical needs are clearly identified, stakeholders (e.g.,
patients, clinicians, administrators, payors) are included in solution design, and clinical use is
simulated from the start of development (Smith et al., 2019). In practice, however, early-stage
development often focuses on proof of concept and economic analysis, excluding key
stakeholders and causing innovation efforts to fail (Markiewicz et al., 2014; Shah & Robinson,
2006; Smith et al., 2019).
Clinicians have an irreplaceable role in health technology development, as they are both
end-users of devices adopted in clinics and key influencers of technology uptake among patient
end-users for devices used in daily life (Celian et al., 2021; Morrow et al., 2021; Smith et al.,
2019). Including clinicians early in development can improve the relevance of devices to fill real
gaps in patient care, reduce the costs spent iterating on unsuitable prototypes, and improve the
feasibility of devices given their contextual knowledge about patient care and health systems
(Smith et al., 2019). Therefore, consulting rehabilitation therapists in the development of stroke
(tele)rehabilitation devices has become an increasingly common practice (Feldner et al., 2019;
Lu et al., 2011; Simpson et al., 2021). Additionally, understanding therapists’ preferences and
factors influencing their decision-making to adopt new technologies has become a priority
(Bower et al., 2021; Caughlin et al., 2020; Celian et al., 2021; C. C. Chen & Bode, 2011; Elnady
et al., 2018; Morrow et al., 2021).

48
One critical period for rehabilitation therapist involvement in device development is preimplementation, which is focused on identifying the fit between an intervention and a target
setting. As the term suggests, this occurs during the planning stage of implementation, prior to a
device being put into practice. Pre-implementation includes examination of the context,
capacities, and barriers of the setting; identification of adaptations that improve the fit; and
development of implementation strategies (e.g., processes, supports) to overcome barriers (Ellis
et al., 2020; Goodrich et al., 2020). To this end, pre-implementation work often examines
implementation outcomes, such as acceptability, appropriateness, feasibility, and fidelity as
preconditions for successful service delivery (Proctor et al., 2011; Weiner et al., 2017).
Acceptability and feasibility are particularly useful metrics to assess in early implementation
processes as they are indicators of later adoption (Proctor et al., 2011). In the context of
rehabilitation technology development, acceptability is the perception among stakeholders that
the technology-based intervention is agreeable and satisfactory, particularly in terms of the
content, complexity, and comfort. The criteria for evaluating acceptability are based on personal
preferences, needs, and expectations. For example, two clinicians who work in the same facility
may have very different assessments of acceptability (Proctor et al., 2011; Weiner et al., 2017).
Feasibility describes how successfully a technology-based intervention can be carried out in a
specific setting given available resources (e.g., effort, money, time) and has practical criteria
(Proctor et al., 2011; Weiner et al., 2017). Evaluating these outcomes early—even prior to using
the intervention in real clinical practice—can enhance long-term outcomes (Proctor et al., 2011).
Purpose
In this study, we examine the pre-implementation acceptability and feasibility of TeleREINVENT, an electromyography (EMG) biofeedback intervention for post-stroke sensorimotor
recovery (Marin Pardo et al., 2021, 2022) among implementers (rehabilitation therapists) after
they watched a brief demonstration video of the system. We also gathered insights about
possible challenges and supports to using Tele-REINVENT in practice. The findings of this study

49
will inform implementation planning for Tele-REINVENT, including identifying contexts in which
Tele-REINVENT may be most feasible and developing preliminary plans to support the uptake
and sustained use of Tele-REINVENT in those contexts.
Methods
We used a concurrent triangulation design and a novel video demonstration approach to
examine the pre-implementation feasibility and acceptability of Tele-REINVENT. A mixed
methods design is appropriate for this study, as there are benefits to both distilling these
constructs into quantifiable numerical scores (e.g., to track the effects of implementation
processes over time) and describing the nuances and contextual factors that are likely to affect
future uptake of Tele-REINVENT. Quantitative and qualitative data were collected concurrently,
analyzed independently, and integrated during interpretation.
Intervention: Tele-REINVENT
Tele-REINVENT is a game-based EMG biofeedback system designed to improve upper
extremity sensorimotor function among stroke survivors with moderate to severe hemiparesis
(Marin-Pardo et al., 2022; Marin-Pardo et al., 2021). Trace muscle activity of the hemiparetic
forearm flexor and extensor muscles is detected by surface EMG signals and used to control
game play on a computer screen or in a virtual reality headset, providing real-time visual
feedback of muscle activation attempts. Tele-REINVENT decreases synergistic movement
patterns and promotes active and repetitive use of the hemiparetic arm. Unlike standard care
treatments, Tele-REINVENT responds to attempted movement, strengthening pathways
between the brain and the target muscles even in the absence of visible movement. TeleREINVENT is portable and designed for use in therapy clinics or patients’ homes, either through
telerehabilitation or as part of a home exercise program for high repetition movement retraining
(Marin-Pardo et al., 2022; Marin-Pardo et al., 2021).

50
Participants
We recruited occupational therapy (OT) practitioners (occupational therapists [OTs] and
occupational therapy assistants [OTAs]) and physical therapy (PT) practitioners (physical
therapists [PTs] and physical therapy assistants [PTAs]) who are licensed in any of the 50
United States and have professional experience treating stroke survivors. Participants were also
required to be at least 18 years of age, have normal or corrected vision, and speak English.
Recruitment was conducted by word of mouth and digital flyers, which were distributed via
online groups and pages intended for OT and PT practitioners. The flyers included a QR code
for direct access to the Information Sheet, which was embedded in a Qualtrics survey (Provo,
UT). Participants self-screened for eligibility, and if eligible, continued to the survey.
Procedures
Participants were asked to view a video demonstration of Tele-REINVENT. The unlisted
YouTube video was approximately 3 minutes long and was embedded in a Qualtrics survey for
ease of viewing. The video, which can be viewed here, included a description of how the system
works, who it can benefit, its technological components, and research supporting its use. It also
integrated videos of stroke survivors using the device in multiple settings—in-clinic, at home
independently, and at home via telerehabilitation. After viewing the video, participants were first
prompted to provide feedback on Tele-REINVENT based on the “video demonstration, your
experience, and your work context…for these questions, assume that Tele-REINVENT is free to
use.” Participants then responded to two open-ended questions: “What would make it
challenging for you to use Tele-REINVENT with patients?” and “What support would you need to
use Tele-REINVENT with patients?” Then, participants were asked if they would want to use
Tele-REINVENT in their practice, and to provide their rationale. Next, participants completed two
brief quantitative measures of acceptability and feasibility. Finally, participants provided
information about themselves (e.g., profession, years in practice) and their work context (e.g.,
practice setting, employer characteristics). All responses were required, save for one optional

51
field for participants to share any other thoughts or feedback about implementing TeleREINVENT in their practice. For the full survey content, see Appendix B. Upon completion of the
survey, participants were automatically directed to a separate survey to collect their name,
email, and state in which they were licensed so we could verify their license and compensate
them with a digital retail gift card while keeping their responses anonymous. The entire survey
process took approximately 15 minutes.
Quantitative Outcomes
The acceptability of Tele-REINVENT among implementors (therapists) was evaluated
using the Acceptability of Intervention Measure (AIM; Weiner et al., 2017). The perceived
feasibility of Tele-REINVENT was evaluated with the Feasibility of Intervention Measure (FIM;
Weiner et al., 2017). Both the AIM and FIM are valid and reliable measures consisting of 4
affirmative statements about the construct of interest and a 5-point Likert scale ranging from
“Completely disagree” (1) to “Completely agree” (5).
Qualitative Outcomes
Through the open-ended survey questions, we sought to identify (1) potential barriers to
implementing Tele-REINVENT, from the perspective of the implementor; (2) supports that might
be needed to successfully implement Tele-REINVENT; and (3) factors influencing a therapist’s
preliminary interest in using Tele-REINVENT.
Data Analysis and Interpretation
We calculated AIM and FIM scores for each participant by taking the mean of all four
items on each measure. Cut-off scores for interpretation are not available for these measures.
Therefore, we interpreted AIM and FIM scores using 4 points as a minimum threshold, as
scores ≥ 4 indicate agreement with the positively worded measure items. AIM scores ≥ 4
suggested high acceptance of Tele-REINVENT and scores < 4 suggested low acceptance.
Similarly, FIM scores ≥ 4 suggested high perceived feasibility of Tele-REINVENT and scores < 4
suggested low perceived feasibility. We also divided participants into 2 groups for each measure

52
(AIM < 4 and AIM ≥ 4; FIM < 4 and FIM ≥ 4) and reported frequencies for sample characteristics
as well as analyzed the qualitative data from each group to examine if there were any unique
challenges or supports anticipated by these groups for whom Tele-REINVENT was perceived as
less acceptable and less feasible.
We analyzed the informational content of the open-ended survey data using manifest
qualitative content analysis (Forman & Damschroder, 2008). The responses to each required
open-ended question were grouped for analysis, resulting in 3 distinct data subsets for analysis:
challenges, supports, and factors influencing interest. For data collected from the optional
prompt (i.e., additional thoughts or feedback), we coded each response as either a challenge,
support, or factor influencing interest, and included those data in the corresponding analysis.
Inductive coding was done in three stages. During preliminary coding, we identified
words, phrases, and passages that may be particularly important to the research questions and
wrote initial descriptive codes. Some responses addressed multiple topics, and these were each
coded as separate data points. Second, during the initial interpretation phase, we grouped data
points with similar descriptive codes into clusters to identify patterns and test preliminary
conclusions. Third, we synthesized the data in each category, wrote descriptive and interpretive
summaries, and used these to draw conclusions about the data and name the core topic areas.
Finally, we conducted conclusion verification by reviewing the raw data to identify evidence that
supported or refuted our conclusions (Forman & Damschroder, 2008). Aside from a few
“negative cases,” which we report in the results, we found that our conclusions were sound.
Additionally, the responses we received were relatively brief and collected from rehabilitation
therapists who share a similar professional taxonomy. Therefore, many of the same words and
phrases were noted across responses and were consistently coded in the same manner.
Throughout all phases of analysis and interpretation, we used data displays (tables, charts) to
organize data and recognize patterns.

53
During interpretation of the qualitative data, we calculated additional frequencies about
the characteristics of the respondents and their work contexts to explore whether there were
common characteristics between respondents who identified similar challenges and supports.
These frequencies are integrated with the qualitative results.
Results
Fifty-five rehabilitation therapists (OT, OTA, PT, PTA) viewed the video demonstration of
Tele-REINVENT and completed the survey between October 16, 2023, and January 17, 2024.
Characteristics of the sample are described in Table 3.1. To summarize, the sample was
approximately half OT practitioners (50.9%) and half PT practitioners, though few assistants
(OTA, PTA) participated (n=3). Most respondents were staff therapists (65.5%) who had been
practicing for ≤10 years (70.9%) and were between the ages of 20 and 39 (74.6%).
Quantitative
The mean AIM score across all participants was 4.21 (SD=0.51) and the mean FIM
score across all participants was 4.05 (SD=0.51). The scores from our sample straddle that
threshold, with 70.9% of participants scoring feasibility as ≥ 4 on the FIM and 81.8% scoring
acceptability as ≥ 4 on the AIM, suggesting that most respondents perceived Tele-REINVENT to
be both acceptable and feasible based on the video demonstration. To further analyze this
divide, we report descriptive statistics in Table 3.1 for participants who scored < 4 and ≥ 4 on
both measures. Notably, 62.5% of FIM ratings < 4 and 80% of AIM ratings < 4 were made by PT
practitioners. Because of the largely unequal group sizes for other characteristics, our analysis
of the differences between these groups was limited.

54
Table 3.1.
Survey Respondent Characteristics
Characteristic
Full
Sample FIM < 4 FIM ≥ 4 AIM < 4 AIM ≥ 4
(N=55) (n=16) (n=39) (n=10) (n=45)
n % n % n % n % n %
Profession
OT 27 49.1 6 37.5 21 53.8 2 20.0 25 55.6
OTA 1 1.8 0 0.0 1 2.6 0 0.0 1 2.2
PT 25 45.5 8 50.0 17 43.6 6 60.0 19 42.2
PTA 2 3.6 2 12.5 0 0.0 2 20.0 0 0.0
Years in practice
< 1 – 5 26 47.3 10 62.5 16 41.0 4 40.0 22 48.9
6 – 10 13 23.6 1 6.3 12 30.8 2 20.0 11 24.4
11 – 15 3 5.5 0 0.0 3 7.7 1 10.0 2 4.4
16 – 20 6 10.9 3 18.8 3 7.7 2 20.0 4 8.9
21 – 25 2 3.6 0 0.0 2 5.1 1 10.0 1 2.2
> 25 5 9.1 2 12.5 3 7.7 0 0.0 5 11.1
Highest educational level
Associate’s degree 3 5.5 2 12.5 1 2.6 2 20.0 1 2.2
Bachelor’s degree 4 7.3 2 12.5 2 5.1 0 0.0 4 8.9
Master’s degree 20 36.4 3 18.8 17 43.6 3 30.0 17 37.8
Entry-level clinical
doctorate 14 25.5 4 25.0 10 25.6 2 20.0 12 26.7
Post-professional clinical
doctorate 13 23.6 4 25.0 9 23.1 2 20.0 11 24.4
PhD, ScD, EdD (or
equivalent) 1 1.8 1 6.3 0 0.0 1 10.0 0 0.0
Current professional role
Staff therapist 36 65.5 11 68.8 25 64.1 6 60.0 30 66.7
Staff Therapist with
leadership role (e.g., Lead
Clinician, Team Lead)
14 25.5 4 25.0 10 25.6 2 20.0 12 26.7
Clinical Leadership (e.g.,
Manager, Director) 3 5.5 1 6.3 2 5.1 2 20.0 1 2.2
Instructor 2 3.6 0 0.0 2 5.1 0 0.0 2 4.4
Age (years)
20 – 29 21 38.2 5 31.3 16 41.0 2 20.0 19 42.2
30 – 39 20 36.4 6 37.5 14 35.9 4 40.0 16 35.6
40 – 49 7 12.7 3 18.8 4 10.3 3 30.0 4 8.9
50 – 59 3 5.5 0 0.0 3 7.7 1 10.0 2 4.4
60+ 4 7.3 2 12.5 2 5.1 0 0.0 4 8.9
Geographic region of United
States
West: AK, CA, CO, HI, ID,
MT, NV, OR, UT, WA, or
WY
18 32.7 7 43.8 11 28.2 4 40.0 14 31.1

55
Southwest: AZ, NM, OK,
or TX 2 3.6 2 12.5 0 0.0 0 0.0 2 4.4
Midwest: IA, IL, IN, KS,
MI, MN, MO, ND, NE, OH,
SD, or WI
9 16.4 3 18.8 6 15.4 2 20.0 7 15.6
Northeast: CT, DC, DE, MA,
MD, ME, NH, NJ, NY, PA,
RI, or VT
24 43.6 3 18.8 21 53.8 4 40.0 20 44.4
Southeast: AL, AR, FL, GA, KY, LA, MS, NC, SC, TN, VA, 2 3.6 1 6.3 1or WV2.6 0 0.0 2 4.4
Note. Frequencies for the full sample are bolded. Frequencies for subgroups (AIM < 4, AIM ≥
4, FIM < 4, FIM ≥ 4) are also noted. OT = occupational therapist; OTA = occupational therapy
assistant; PT = physical therapist; PTA = physical therapy assistant.
Qualitative
For open-ended questions about challenges and supports, responses often consisted of
lists of brief phrases and words (e.g., “Training on use of equipment, how to trouble shoot if
issues arise”). However, we include direct quotes from survey responses in this manuscript
where possible and where quotes enhance the understanding of our conclusions. For the
question asking for therapists’ rationale for wanting (or not wanting) to use Tele-REINVENT,
there were often longer responses with more nuance, so more quotes are used to describe the
conclusions. In all cases, where quotes are used, they have been minimally edited only to
correct misspellings or write out medical abbreviations (e.g., changing “IP” to “inpatient
rehabilitation” or “UE” to “upper extremity”).
Anticipated Barriers or Challenges to Implementing Tele-REINVENT.
We sought to identify potential barriers or challenges to implementing Tele-REINVENT
with stroke survivors, from the viewpoint of prospective implementors (rehabilitation therapists).
Barriers and challenges were categorized into 3 core topics: factors of the practice setting; skills
and characteristics of the clinician-patient dyad; and access.
Practice Setting. Factors of the practice setting in which survey respondents work were
perceived as potential barriers to implementation. Here, practice setting refers to both the level
of care (e.g., acute, inpatient, or outpatient rehabilitation) and the infrastructure specific to each
facility. One barrier that therapists noted was the appropriateness of Tele-REINVENT for

56
addressing the therapy goals of the practice setting. For example, a hospital-based therapist
expressed concerned that Tele-REINVENT might be too difficult for acute patients. Other acute
and inpatient rehabilitation therapists mentioned that Tele-REINVENT may not be the priority
intervention because of the outcomes valued by payors, “For the stroke patients with moderatesevere defects, therapists may choose to practice some compensatory strategies for these selfcare activities to improve outcomes scores. It is a great time to introduce this tool to the patients
but the length of the stay in an inpatient rehabilitation facility could be a concern as well.”
Another participant noted, “Where I work, [inpatient rehabilitation], I likely wouldn’t implement
this until later on in the patient's stay. I usually like to start with functional electrical stimulation to
assist with motor recovery and coordination first.” Notably, of the 31 respondents who work in
outpatient rehabilitation and/or home health, none reported barriers related to the
appropriateness of Tele-REINVENT for the level of care and only one reported a possible barrier
related to reimbursement using it as a therapy service in the clinic. Multiple noted that it would
be useful as a home exercise program to complement other in-person approaches. To
corroborate these findings, we calculated mean AIM and FIM scores by practice setting to
identify practice settings (levels of care) in which Tele-REINVENT was perceived to be more
feasible and acceptable (Table 3.2). We found that acceptability was highest among therapists
working in home health, community-based, and sub-acute rehabilitation settings. Feasibility was
rated highest among therapists in home health and community-based practice settings.
Interestingly, although qualitatively, outpatient rehabilitation emerged as a level of care with
fewer barriers to using Tele-REINVENT, the AIM and FIM scores were comparable to other
settings where more barriers were identified (i.e., acute, inpatient, and sub-acute rehabilitation).
Six PT practitioners described that their scope of practice within their practice setting
would be a barrier to using Tele-REINVENT. For instance, “As a PT in acute rehabilitation
setting, since gait and transfer scores are the major outcomes for insurance…the time of

57
applying this tool in this setting will be limited for me personally” and, “As a PTA, I love this but,
in my setting, OT usually addresses upper extremity motor coordination.”
Table 3.2.
Acceptability and Feasibility Scores by Practice Setting
Practice Setting n AIM score ± SD FIM score ± SD
Acute 20 4.24 ± 0.47 4.06 ± 0.47
Inpatient 27 4.11 ± 0.46 4.05 ± 0.44
Outpatient 26 4.21 ± 0.59 4.08 ± 0.59
Home health 9 4.58 ± 0.45 4.19 ± 0.51
Community-based 3 4.67 ± 0.38 4.25 ± 0.66
Telehealth 5 4.25 ± 0.43 4.05 ± 0.54
Veterans Affairs 3 3.92 ± 0.14 3.92 ± 0.14
Academia 9 4.17 ± 0.56 4.28 ± 0.46
Sub-acute* 4 4.63 ± 0.48 4.06 ±0.66
Note. AIM and FIM scores calculated by practice setting. Because respondents could select
multiple practice settings, the groups are not mutually exclusive, and the sum of observations
(N=106) exceeds the number of respondents (N=55).
*Sub-acute was not an option on the survey but was noted in the open-ended “other” field by
multiple participants so we included it in the analysis.
Another practice setting factor was the infrastructure, including the physical space,
digital tools, and staffing. For example, multiple acute and hospital-based therapists commented
that having easy access to the equipment is a necessity and that not having a dedicated
physical place to use Tele-REINVENT with patients may be a barrier. Others noted the flexibility
of using it in a gym space or on the bedside tables in patient rooms since Tele-REINVENT is
portable. Additionally, the lack of digital infrastructure in the clinic such as a telehealth platform
or reliable wireless internet would be barriers for some therapists to using it as a
telerehabilitation intervention.
Respondents also mentioned the barrier of needing support and buy-in from their
managers or administration to use Tele-REINVENT in practice. A few related concerns were the
relative benefit over other technologies on the markeIt and already in the clinic. Specifically,
potential challenges to adopting Tele-REINVENT could be the limited scope of only addressing

58
wrist and forearm movements, and the quality of any outcome data or clinical reports produced
by the system. To examine whether there were differences in acceptability and feasibility among
respondents who have leadership roles, we calculated the average AIM and FIM scores for two
groups: staff therapists (n=36) and therapists with any leadership role (n=17). The staff therapist
group had a mean AIM score of 4.23 (SD=0.52) and FIM score of 4.02 (SD=0.50) whereas the
leadership group had a mean AIM score of 4.13 (SD=0.47) and FIM score of 4.01 (SD=0.44),
showing that at least in our sample, leaders did not have substantially different scores.
Qualitatively, the responses from therapists in leadership roles were also not substantively
different than those of staff therapists.
Finally, the time constraints were perceived as a possible barrier, across professions and
level of care. In particular, there was a concern that the time to set up, calibrate, troubleshoot,
and educate patients on the use of Tele-REINVENT may consume too much time from already
limited session lengths, noting that spending less time on active intervention has implications for
patient outcomes and reimbursement. One respondent summarized the concerns of many:
“Set up—it always seems really easy but in my experience, there are always glitches,
nuances, and caveats. In my clinic we've tried 2 virtual reality systems and 1
biofeedback. All of them had so many set up tech problems that it made them much less
useful from a dosing/intensity standpoint and a feasibility for maximizing schedule and
reimbursement (aka utilizing the whole session for something billable).” To that end, one
participant noted, “This seems more effective if patients could use this system outside of
therapy sessions while focusing on other aspects of rehabilitation during sessions.”
Skills and Characteristics of the Therapist-Patient Dyad. Some of the skills and
personal characteristics of the clinicians and patients were perceived as potential barriers to
using Tele-REINVENT in the context of a clinician-patient treatment dyad. Therapists frequently
noted a lack of familiarity with aspects of the technology, such as using EMG sensors and
customizing the device for each patient, as well as a lack of certainty in identifying eligible
patients with trace muscle activation. Regarding the patient’s skills and characteristics,
therapists were concerned that the severity of symptoms would be a challenge to using TeleREINVENT, including balance, arm function, cognition, attention, apraxia, and vision.

59
Additionally, patients’ technology proficiency and buy-in to a game-based novel technology
might be a barrier. Specifically, they were concerned that some patients and care partners,
especially older adults, would have difficulty using a computer and placing EMG sensors,
especially if used at home as part of a home exercise program or via telehealth.
Access. Potential access barriers were a concern for therapists, both for themselves
and the patients they serve. Aside from the basic technological proficiency needed to use TeleREINVENT, they were concerned that accessing technology support may be challenging, and
that usability issues common to rehabilitation devices would render the system inaccessible for
either themselves or their clients. In terms of other access issues in the clinic, a potential barrier
is compatibility of the Tele-REINVENT hardware and software with existing devices in the clinic
(e.g., laptops) and clinic policies (e.g., cybersecurity measures, infection control practices,
patient data storage protocols). For patients, additional concerns included access to reliable
wireless internet and a computer at home for carryover of Tele-REINVENT from the clinic to
home; and affordability, including out-of-pocket costs and insurance coverage.
Anticipated Supports for Successful Implementation of Tele-REINVENT
Building on the perceived challenges of implementing Tele-REINVENT, we sought to
learn from therapists about the supports they anticipate needing to successfully implement TeleREINVENT in their practice. The proposed supports were categorized into 4 core topics: initial
training; long-term technology support; administrative and infrastructure support; and additional
empirical evidence and product features.
Initial Training. Approximately half of all respondents explicitly mentioned training or
education as a necessary support for using Tele-REINVENT. Specific formats for therapist
training included in-person in-service presentations and protected time to gain hands-on
experience with the system prior to using it with patients. Additionally, therapists anticipated
needing dedicated time to conduct patient and care partner training, particularly if TeleREINVENT were to be used as part of a home exercise or telehealth program. Therapists also

60
specified content areas they would need addressed as part of training, including EMG sensor
placement; hardware cleaning and maintenance; setting up user accounts; adjusting sensitivity
and other parameters; and storing and retrieving patient data. Additionally, therapists identified a
need for training on how to best use Tele-REINVENT in the context of treatment planning with
specific patients. For example, one therapist noted a need to learn “cuing strategies for complex
patients including those with increased spasticity or communication difficulties” and another
inquired, “How long would a patient benefit from the given programs before they would max out
the software settings?” Additionally, strategies for being efficient with system set-up and
promoting long-term compliance with Tele-REINVENT were noted as important topics for
training.
Long-term Technology Support. In addition to the initial training, long-term technology
support was a common need noted by therapists. Respondents recommended specific formats
for support that they would find useful, including a phone number and email address they could
contact in case of issues, a frequently asked questions documents to share with patients,
written instructions to guide sensor placement, a troubleshooting manual, and short instructional
videos addressing common technical issues. The convenience and real-time access to these
supports for clinicians, patients, and care partners were important.
Administrative and Infrastructure Support. Additional supports were identified that
would need to be provided by the clinic. First, having the physical space and tools to use TeleREINVENT were needed, which in some cases included square footage to have TeleREINVENT set up and stored, and in other cases, a dedicated computer for Tele-REINVENT so
clinicians would not need to use their own work laptops to run the intervention. Multiple
therapists also described the need for support personnel such as transport and therapy
technicians to bring patients to the system and facilitate set up, especially in fast-paced settings
like skilled nursing facilities, to make the most of short treatment times. Finally, despite the
prompt to assume that Tele-REINVENT was free to use, financial supports were evidently a

61
salient concern for clinicians. Many respondents mentioned the need for financial support to buy
the system and related equipment both for in-clinic and at-home use, especially for lowerincome patients and those receiving disability income.
Additional Empirical Evidence and Product Features. A final support topic identified
in our analysis was the need for additional research and product features. Having more
empirical evidence to support the use of Tele-REINVENT in practice was a need identified by
five respondents. Specifically, “research supporting statistically significant outcomes with
standardized measures and carryover into functional performance/goals” and “data from
success stories of patients who used the software during recovery” were noted as useful for
justifying the adoption of Tele-REINVENT. To that end, therapists wanted the system to generate
additional outcome data and clinical reports so they could more easily track and document
patient progress over time. Finally, some of the PT practitioners surveyed wanted to see the
development of a lower extremity version of Tele-REINVENT, as they saw value in the
intervention approach and technology, but do not address upper extremity hemiparesis in their
practice.
Therapist Interest in Using Tele-REINVENT
Finally, in addition to the anticipated challenges and supports for implementation, we
wanted to know at a basic level if rehabilitation therapists would want to use Tele-REINVENT in
their practice. Participants were required to select “yes” or “no” regarding their desire to use
Tele-REINVENT and provide rationale for their selection. Fifty respondents (90.1%) selected
“yes”. Notably, all five respondents who selected “no” were PT practitioners. The rationales for
both selections are described here. Therapists’ responses were categorized into 3 core topics:
potential benefits for the target population; the intervention mechanism and approach; and
technological innovation.
Potential Benefits for the Target Population. Therapists were very satisfied with the
potential benefits of Tele-REINVENT for their patients, and specifically the target population—

62
stroke survivors with moderate to severe hemiparesis. They found it to be appropriate for
addressing motor function goals. For example, one therapist noted, “I see lots of patients with
moderate to severe hemiparesis that I think would greatly benefit from a tool that can detect
EMG and use it to encourage and direct movement” and another said that the system “provides
greater opportunity for patient engagement regardless of the severity of impairment.” It was also
perceived to address practical needs of this population, such as providing a motivating at-home
therapy option. For example, one therapist described how they might use Tele-REINVENT in
their practice:
“I think this would be a good tool for patients who are running low on therapy visits or
discharging soon and would be motivated to get more repetitions in at home. I would
potentially trial this during a couple sessions for them to determine if they found it
enjoyable/motivating and wanted to consider for home use.”
Another participant also noted, “It provides a new solution to rehab in an otherwise challenging
field for rehabilitation. I live and work in Ohio where often an appropriate treatment center for
neurorehabilitation is very far.”
In contrast to the perceived benefits, one PT justified their disinterest in using TeleREINVENT in their practice because it was incompatible with the skills of their patients, “I work
with patients with severe cognitive deficits and/or older age who… have a hard time figuring out
how to login to MyChart despite instructions given in person in advance and a take-home ‘how
to’ sheet… This seems like it would add another complicating factor for those individuals.”
Intervention Mechanisms and Approach. Related to the patient benefits, many
therapists identified specific evidence-based mechanisms underlying the design and use of
Tele-REINVENT that were appealing and aligned with their treatment approaches. While related
to the previous core topic, the intervention mechanisms are distinct from the patient benefits
because they were highlighted as features that clinicians specifically find appealing in new
interventions and techniques. For example, it was appealing that Tele-REINVENT applies motor
learning and neuro re-education principles including the “ability for patient[s] to do high

63
repetition exercises and…pick up EMG activity without visible movement.” Specifically, the
virtual biofeedback was an attractive feature, even when compared with other EMG biofeedback
systems, in part because “the real-time visual feedback system as well as the gamification
makes the activity more meaningful for patients.” Similarly, another therapist reported, “It is
difficult to find helpful tools for patients with moderate to severe hemiparesis. The biofeedback is
a helpful way for a patient to know they are working on functional use” and another said, “It's a
way for patients to really see the work they are doing to address their hand function.”
Despite the overall satisfaction with the underlying mechanisms of Tele-REINVENT, two
concerns arose about the intervention approach. First, one concern from an OT was that TeleREINVENT was not occupation-based, though other participants highlighted that online or video
game playing can be a meaningful occupation. A second concern was that the system may not
require enough clinician involvement to be justifiable, “I would not use this as a predominant
intervention within my in-clinic appointments because the patient can learn to do it
independently and at some point, this would not be considered a skilled intervention.” Despite
these concerns, both respondents indicated interest in using Tele-REINVENT in their practice.
Technological Innovation. Overall, Tele-REINVENT was perceived as an appealing,
novel, non-traditional, evidence-based, technological intervention. Multiple participants had
similar sentiments to one OT who said, “I love high technology interactive games as a form of
therapy to increase functional use of a hemiparetic arm that doesn’t seem like traditional
therapy/exercise.” Multiple respondents highlighted that set up appeared more approachable
and easier than other technology-based interventions, such as one participant who noted, “We
have similar technology including the Bimanual Arm Trainer and the Mind Motion Go. The TeleREINVENT system looks more up to date and easier to set up” and another who said, “Great
[home exercise program] and easy set up. I have used [functional electrical stimulation] bikes
and other [neuromuscular electrical stimulation] devices that can take a long time to set up.”
One participant even compared Tele-REINVENT to other EMG-triggered games, “We’ve been

64
using EMG triggered gaming for years, but it’s nice to have virtual visual feedback of the
movement vs just a stimulus triggered activation of an unrelated action.”
Respondents also identified that Tele-REINVENT innovatively addresses a challenging
area of rehabilitation by providing an option that increases independence as opposed to passive
or active-assisted approaches, “The technology allows for clinicians to be hands-free during
active time, allowing them to focus on aspects of treatment beyond active-assisted range of
motion.” Tele-REINVENT was also perceived to fill a gap in some clinics, “Our facility doesn’t
have any kind of equipment that uses biofeedback, and I anticipate this would be beneficial for
patients who require more real-time and visual feedback.” Another major benefit of this
technology noted by therapists is that it tracks progress over time and can be used at home to
continue recovery beyond the clinic.
Negative Cases. Of the five respondents who indicated they would not want to use TeleREINVENT in their practice, the rationale of four were counted as “negative cases,” meaning
that they did not align with or counterbalance the other conclusions drawn from this analysis.
Two of the “no” responses were because the respondent does not treat arm impairment.
Another noted that it would be too difficult to use Tele-REINVENT in combination with other
therapeutic tasks because of their high caseload. The final respondent selected “no” because
they have no decision-making power and would need manager approval.
Summary. One therapist summarized the rationale of many when they said, “A system
like Tele-REINVENT has great potential since the feedback is in real time to a population of
patients that were probably told they have achieved maximal level of recovery. More important
is the great psychological impact the system can have over individuals that greatly depend on
caregivers for simple tasks. Small achievements in muscle control and limb movement can
provide a pathway to great overall strides in function and independence.”

65
Discussion
We examined the acceptability and feasibility of Tele-REINVENT among OT and PT
practitioners based on their impressions after viewing a demonstration video. Participants
assessed Tele-REINVENT on brief, validated measures of acceptability and feasibility. They
also provided qualitative feedback about anticipated challenges to using Tele-REINVENT in their
practice and supports they foresee needing to use the system successfully. The findings from
this study will inform implementation processes, such as designing appropriate supports and
identifying contextual factors that may lead to greater uptake and sustainment of TeleREINVENT in real practice settings.
Rehabilitation therapists are both the implementers of Tele-REINVENT and end-users, in
addition to the clients themselves. Therefore, if Tele-REINVENT is to be successfully
implemented in clinical practice, therapists’ perspectives must be highly valued in design and
implementation processes. Unfortunately, clinicians are often excluded from these early
processes, and their involvement is limited to participating in effectiveness trials, after a device
has been developed (Smith et al., 2019). However, there are major benefits to seeking clinician
input earlier in the process, “Given the privileged position and immersion they have in patient
care, the clinician is in a unique position to understand the pain points… they function as a
valuable resource in the [multidisciplinary team] by providing insight into the feasibility of a
solution given their knowledge of its context in the cycle of care of the patient” (Smith et al.,
2019, p. 7). In this study, we learned from stroke rehabilitation therapists’ unique perspectives
far in advance of the clinical trial phase by collecting acceptability and feasibility data during preimplementation assessment.
Acceptability, Feasibility, Appropriateness
Most respondents rated Tele-REINVENT as acceptable and feasible. Of note, Proctor et
al. (2011) observed that implementation outcomes are interrelated in complex ways and Weiner
et al. (2017) showed that while the FIM and AIM are highly correlated, they are best represented

66
as distinct constructs from an empirical perspective. Acceptability is dynamic and will likely
change over time in different phases of implementation (Proctor et al., 2011). Feasibility will also
likely change as the system design evolves to better fit with the implementation context, which is
a goal of the pre-implementation phase. Therefore, assessing these constructs throughout
implementation will be useful for detecting change and adapting accordingly.
The qualitative data provided insights into acceptability and feasibility, though we elected
to frame the open-ended questions in terms of ‘challenges’ to implementation and ‘supports’
needed to use Tele-REINVENT in practice, as these are easily understood, pragmatic terms that
cover multiple constructs. Overall, our findings reflected the interconnectedness of acceptability,
feasibility, and appropriateness (the perceived fit of an innovation within a specific practice
setting or problem space; Proctor et al., 2011). An intervention may be seen as a good fit for
addressing the goals of stroke survivors in a specific practice setting (appropriateness) but may
be too time-consuming or difficult to use with patients (feasibility) or, may not align with the
practice preferences of the therapist (acceptability) (Proctor et al., 2011; Weiner et al., 2017).
We saw in our analysis that the appropriateness of Tele-REINVENT for specific settings was an
important factor, even among therapists who found it acceptable and feasible. If any one of
these implementation outcomes is rated poorly, the uptake of Tele-REINVENT may be
suboptimal. Overwhelmingly, Tele-REINVENT was perceived as a promising innovation in stroke
rehabilitation with potential benefits to an underserved segment of the population.
Barriers to Implementing Tele-REINVENT
Potential barriers to using Tele-REINVENT included factors of the practice setting; skills
and characteristics of the clinician-patient dyad; and access. The appropriateness of TeleREINVENT to address patient goals in different levels of care was as an important factor in the
prospective use of Tele-REINVENT. Similarly, suitability for the setting has previously been
identified as a factor in clinician decision-making to use a new stroke rehabilitation device (Chen
& Bode, 2011). In many settings, reimbursement is based on functional outcomes (e.g., ADL)

67
versus impairment reduction outcomes, so clinicians in those settings are less likely to adopt a
technology focused on the latter if there is not a relative advantage over ADL-focused treatment
(Celian et al., 2021), a concern that was also reflected in our findings. Additionally, a common
concern was the time that may be required to use complex rehabilitation devices, which is
incompatible with the fast-paced therapy delivery context (Celian et al., 2021). Tele-REINVENT
was initially designed for use by chronic stroke survivors at home, and our findings about the
anticipated barriers to using it in acute, hospital-based settings seem to confirm our original
speculation that this intervention would be a better fit for later stages of rehabilitation. Therefore,
future study of the implementation context for Tele-REINVENT will focus on outpatient
rehabilitation, home health, and community-based settings.
Personal attributes of the patients and clinicians have also been reported as potential
barriers to the uptake of stroke technologies and telerehabilitation approaches, including
clinician self-efficacy (Celian et al., 2021), patient acceptance and interest (Chen & Bode), and
patient proficiency with technology use (Stephenson et al., 2022), all of which were also noted in
our study. While clinicians in our survey reported concerns about patient buy-in and technology
proficiency, our previous work evaluating Tele-REINVENT with stroke survivors showed high
levels of acceptance; independent use of the system at home after training and guided use; and
a desire to integrate it into their rehabilitation (Donnelly et al., 2023; Donnelly et al., unpublished
[Chapter 2]). However, our previous work with stroke survivors was conducted with individuals
who had no or mild cognitive impairment and no or corrected vision deficits, so our prior results
are not generalizable to patients with more severe cognitive or visual deficits, which was a
concern of our survey respondents. In the future, as we study the effectiveness of TeleREINVENT among stroke survivors with more severe and complex poststroke symptoms, we
will also examine the role of care partners for supporting its uptake and sustained use.
Finally, our survey revealed potential access barriers to using Tele-REINVENT, such as
the availability of computers and reliable internet in patients’ homes, if used as part of a home

68
program or telehealth. While there is ample evidence in the stroke neurorehabilitation literature
about access barriers for patients (Y. Chen et al., 2020; Neibling et al., 2021; Stephenson et al.,
2022), our study showed important access issues for clinicians, including the compatibility of
Tele-REINVENT with existing digital tools and practices.
Given these findings about anticipated barriers, there is a need to focus on (1) equipping
clinicians with the supports to achieve mastery using Tele-REINVENT with patients, (2)
evaluating usability and clinical outcomes among users with more complex cognitive and
sensory impairments, and (3) adapting Tele-REINVENT to fit within clinic policies and digital
infrastructures, as we seek to address barriers to uptake in clinical practice.
Implementation Strategies
We sought to identify potential implementation strategies that would be appealing to
clinicians. Implementation strategies are the specific methods or techniques that are used to
support the adoption and sustainment of interventions in clinical practice—referred to as the
“‘how to’ component of changing healthcare practice” (Proctor et al., 2013, p. 1). A few examples
of implementations strategies are checklists, practice guidelines, training workshops, clinical
supervision, and changing physical structures.
The most mentioned supports from our data were Initial training and long-term technical
support. Suggestions for initial training were specifically for in-services, time for hands-on
practice, and written or audiovisual education materials. Indeed, clinician training, manuals, and
technical support are useful and common strategies used in allied health professions, and
stroke rehabilitation specifically (Murrell et al., 2021; Stephenson et al., 2022). While these are
important, the stroke rehabilitation literature endorses that single training sessions for complex
interventions do not provide clinicians enough time to develop work efficacy and mastery, which
are crucial for clinician acceptance, adoption, and optimal use in their practice (Caughlin et al.,
2020; Celian et al., 2021). This discrepancy between our findings and the literature is not
surprising, as implementors (i.e., clinicians, administrators) tend to underestimate the time

69
required to integrate a new practice successfully and fully into routine care (Caughlin et al.,
2020). Based on our data and recommendations from the literature, we propose that a series of
workshops and troubleshooting sessions, through which staff members develop roles and
responsibilities related to implementation (e.g., “champions”), in addition to longer-term
technical support and educational outreach, may be a useful approach to support clinician
confidence and efficacy (Caughlin et al., 2020; Celian et al., 2021). Additional training for
patients and care partners could also reduce the burden on clinicians and promote adoption at
home (Caughlin et al., 2020; Stephenson et al., 2022). Other barriers we identified in our study
may be addressed through evidence-based treatment protocols and care pathways to enhance
clinician decision-making and uptake (Caughlin et al., 2020; C. C. Chen & Bode, 2011).
The extensive training and support that will likely be needed toIIly implement and sustain
a complex, technology-based intervention like Tele-REINVENT will require administrative
support and organizational readiness for change (Caughlin et al., 2020; Celian et al., 2021).
Organizational readiness for change is the extent to which the members are prepared for the
psychological and behavioral modifications needed to achieve successful change (Weiner et al.,
2008). An organization that is more ready for change is likely to be prepared to enable learning
and practice time with Tele-REINVENT; make changes to the digital and physical infrastructure;
and adapt the intervention to meet the requirements of local policies and practices. Partnering
with health care administrators and information technology staff is vital for facilitating the
logistics of implementing a new intervention (Caughlin et al., 2020; Celian et al., 2021). In our
study, we observed concerns from therapists that their administrators may be resistant to
introducing a new device to the clinic. However, the responses of the 17 therapists that held
some leadership role revealed no substantive quantitative or qualitative differences from the
staff therapist responses. This suggests that further research should be done with clinic
managers, administrators, and other organizational leaders to identify barriers to adoption of
new devices and readiness for change, beyond feasibility and acceptability. Additionally, it will

70
be critical for clinical leadership to communicate their endorsement and support of TeleREINVENT use during the implementation process.
Not surprisingly, there was complementarity in the core topics identified as challenges
and supports; most of the proposed supports were in response to anticipated challenges.
However, we found it important to analyze them separately because assessing barriers and
facilitators is a precursor to selecting appropriate implementation strategies that will lead to
sustainable practices (Murrell et al., 2021). The anticipated supports identified by therapists in
our study aligned with implementation strategies previously used in stroke rehabilitation, as
reported by Juckett et al. (2020) and Murrell et al. (2021). However, as noted here, these
supports are likely not enough to facilitate organizationally supported, sustainable uptake of
Tele-REINVENT. A lingering question is whether the supports identified in this study were
suggested because they are the supports with which clinicians are familiar (e.g., in-service
presentations are common in clinical settings), or because they are perceived as truly the best
supports for implementing Tele-REINVENT, specifically.
Video Demonstration Approach
In this study, we demonstrated Tele-REINVENT in a short video that was embedded in
an online survey. In addition to the usefulness of the data we collected, a major benefit of this
approach was the ability to quickly gather data (median survey response time was 13 minutes,
including viewing the video) from respondents across practice settings and professions, in
different regions of the United States, with different levels of experience and education.
The video demonstration was pre-recorded, so respondents could provide feedback at a
time convenient to them. Additionally, this approach was cost-effective, in that it did not require
the resources that live demonstrations incur. As far as we can tell from the literature, an
asynchronous video demonstration has not been used to evaluate pre-implementation
outcomes among therapists for rehabilitation devices. We found that presenting Tele-REINVENT

71
on video provided a concise depiction, while containing enough information for clinicians to
provide insightful feedback.
A drawback of the video demonstration is that clinicians did not get hands-on experience
with Tele-REINVENT, so they only viewed the device through the content we developed to share
with them. While we sought to present a fair view of the intervention, we did not show any
technical difficulties, which are ubiquitous in rehabilitation technology (Neibling et al., 2021).
Additionally, therapists could not ask clarifying questions to members of the research or design
team prior to giving feedback. Therefore, some of the barriers and supports described by
respondents have already been addressed by the system design, though they were not
addressed in the video. However, even if already addressed, the responses provided insight into
what clinicians value and prioritize in the development of devices and will inform the marketing
and educational materials we develop in the future. Despite these drawbacks, we demonstrated
that a brief asynchronous video demonstration could elicit valuable data about preimplementation outcomes with findings that are well-supported by existing literature.
Limitations
Our recruitment materials specifically invited rehabilitation therapists to give feedback on
a new stroke technology. This may have deterred prospective participants who are averse to
technology-based interventions, resulting in the absence of valuable input from this segment of
the therapist population. Another limitation of our study was the use of multiple select questions
(e.g., “What are your work settings?”) rather than forced response questions. This question
design was selected to make the survey quicker to complete while capturing all options that
applied to the respondent. However, in the case where therapists work in multiple levels of care
or at multiple facilities, we were unable to parse out which settings they had in mind when
responding to the open-ended questions about Tele-REINVENT. Finally, our findings showed
that appropriateness was a key area of concern for clinicians, so future studies evaluating pre-

72
implementation outcomes might include a measure such as the Intervention Appropriateness
Measure (Weiner et al., 2017).
Conclusion
Rehabilitation therapists have pragmatic insights about using technology-based
interventions amidst the demands and constraints of clinical practice, and these perspectives
are vital to the successful development and implementation of new interventions. In this study,
we examined pre-implementation outcomes of Tele-REINVENT and learned about perceived
barriers and supports to using it in practice through the insights of OT and PT practitioners. We
used a novel video demonstration approach to elicit first impressions from a diverse sample of
therapists. We found that Tele-REINVENT was overall perceived as feasible, with factors
specific to the practice setting, the skills and characteristics of the clinician-patient dyad, and
access issues being the primary anticipated challenges. We also found that initial training,
longer-term availability of technical assistance, administrative buy-in, and additional research
evidence were anticipated supports needed to use Tele-REINVENT successfully. Finally,
therapists overall rated Tele-REINVENT as acceptable, which was corroborated with feedback
about the appeal of potential benefits for the target population; the intervention mechanism and
approach; and technological innovation. As the development and implementation planning for
Tele-REINVENT progress, practitioners will continue to play a crucial role in understanding and
overcoming the challenges to sustainable uptake of Tele-REINVENT.

73
Chapter 4: A new community advisory board model to strengthen
community partnerships in stroke rehabilitation research
Abstract
The potential benefits of community-engaged research for improving the relevance and
reach of research are well described; however, there are practical challenges to establishing,
growing, and evaluating these partnerships. The community advisory board (CAB) approach
has become increasingly common for supporting both large institutions (e.g., National Advisory
Board on Medical Rehabilitation Research; National Institutes of Health, 2023) and specific
research projects (Halladay et al., 2021). In addition to these approaches, there are
opportunities, not yet described in the literature, for CABs to be long-term partnerships at an
intermediate level, such as in a research laboratory. We established a CAB for the Neural
Plasticity and Neurorehabilitation Laboratory at the University of Southern California, composed
of stroke survivors, their care partners and loved ones, rehabilitation therapists, researchers,
and students. We sought to (1) infuse diverse perspectives of stroke recovery across all projects
and phases of research in the laboratory, and (2) bidirectionally mobilize knowledge between
people with lived experience of stroke and professionals to advance rehabilitation research,
even beyond the laboratory. Here we discuss the foundation of our CAB, including pragmatic
elements, Board activities, our evaluation of Board engagement, and lessons learned to-date.
We propose that establishing CABs for long-term collaborations across research projects can
elevate the voices of stakeholders, strengthen community engagement and access to research,
and enhance the translation and implementation of important scientific findings into real-world
contexts.

74
Introduction
Including community members and other non-academic partners in research is a best
practice in the health and social sciences to better address health-related issues (Sheridan et
al., 2017). Community-engaged research (CER) values the perspectives of people with lived
experience and other ‘insider’ expertise about a population, phenomenon, or health issue of
interest as important sources of knowledge that can improve research processes and outcomes
(Aldrich & Marterella, 2014, Vaughn, 2018). In stroke rehabilitation research, insiders may
include stroke survivors; service providers like physicians, therapists, and social workers; the
social networks of stroke survivors, such as family members, friends, care partners, and peers;
representatives from community organizations, like advocacy groups, charities, and faith-based
organizations; and, community leaders, such as local health administrators. CER is rooted in an
ecological view of health, recognizing that health is largely determined by social, environmental,
and lifestyle conditions, and therefore, community members must have a voice in health
improvement initiatives for such efforts to be effective (Howe & Briggs, 1982; McCloskey et al.,
2011).
There is a continuum of CER models and approaches, the selection of which depends
on the goals, structure, and context of a specific community-academy collaboration. A range of
terms is used in the CER literature to describe community member partnerships and
engagement models (Vaughn et al., 2018), such that it can be challenging to compare, choose,
and implement the ‘right’ evidence-based CER approach. Despite the inconsistency in
terminology, CER generally enhances the relevance of research questions to community needs,
reduces inequities in research and practice, increases transparency of research processes,
accelerates implementation of findings, and improves the sustainability of evidence-based
practices (Halladay et al., 2017; Staley, 2009; Vaughn et al., 2018). To that end, community
engagement is now required for many funding agencies and mechanisms (e.g., NIH, PCORI;
Sheridan et al., 2017). A recent review of CER in neurorehabilitation research reported that the

75
two primary community engagement activities used in stroke intervention studies were study
planning and developing culturally tailored education materials (Boden-Albala et al., 2023).
There is a particular need for CER in stroke rehabilitation research, as many structural,
sociocultural, and demographic factors influence health equity, recovery outcomes, and access
to care (Alawieh et al., 2018; Boden-Albala et al., 2023).
Community Advisory Boards
The Community Advisory Board (CAB) model is one approach to CER that invites
community members and non-academic partners to provide guidance, support, and feedback to
academic investigators on an ongoing basis (Vaugh et al., 2018). CABs can be an important
bridge between researchers and the community to build trust and align research with community
priorities. As with all CER approaches, CABs are not a ‘one size fits all’ model. Most CABs
described in the literature could be categorized as project boards or institutional boards. For
example, Halladay et al. (2017) describe a project-specific CAB that was established to advise
during the pre-award phase to increase investigators’ understanding of the community. This and
other ‘project’ boards tend to have fixed timelines and focus on specific, grant-funded project
aims. CABs that support an institution, such as the North Carolina Translational and Clinical
Sciences Institute Equity in Research Community & Patient Advisory Board (established in
2021) and the National Advisory Board on Medical Rehabilitation Research (established in
1999), provide long-term guidance on programming, policies, and practices of an institution,
rather than focusing on a specific project (National Institutes of Health, 2023; North Carolina
Translational and Clinical Sciences Institute, n.d.).
In either project or institutional boards, CABs and academic investigators can partner
with each other to different degrees. Determining who has decision-making power, which roles
and responsibilities each member assumes, and to what extent the Board is hands-on with
research activities are important considerations for establishing a CAB. These decisions will
depend, in part, on CAB members’ commitment to the goals and processes, their capacity to

76
engage in research activities, the investigators’ commitment to meaningfully engage with the
CAB, and mutual trust among all partners (Goodman & Sanders Thompson, 2017).
Benefits and Challenges of Community Advisory Boards
In a review of CER models, Vaughn et al. (2018) reported benefits of CABs including
more empowered communities with greater research capacities, better understanding of
community needs, improved research methodologies, and broader dissemination. Despite the
many possible benefits of CABs to develop genuine and meaningful partnerships, there are
challenges to implementing them successfully. First, there is a risk of tokenism, with CABs
becoming “empty ritual[s] of obtaining stakeholder feedback” (Goodman & Sanders Thompson,
2017, p. 486). Similarly, Hatcher et al. (2011) noted that “Engagement efforts will flounder in the
absence of transparency and reciprocity in the engagement process” (p. 100). In contrast to
symbolic engagement, Goodman & Sanders Thompson (2017) encourage ongoing process
development for communication, decision-making, vision casting, and change management,
which are vital to meaningful engagement. It is worth remarking that creating a CAB culture of
transparency and reciprocity requires significant investments of time and logistical effort on the
part of the investigators.
A related challenge of establishing and sustaining CABs is evaluating engagement
among community partners. Because meaningful engagement is core to CER, it is important
that community partners see the value and impact of their involvement in research and for
investigators to monitor the quality of the research partnership. However, evaluating
engagement is a challenge noted across CER approaches (Luger et al., 2020; MacQueen et al.,
2015). To support investigators in characterizing engagement, Goodman & Saunders Thompson
(2017) classified engagement between academic and community research partners on a
continuum. Starting from lower levels of community participation to equal engagement with
academic investigators, these classifications include outreach and education; consultation;
cooperation; collaboration; and partnership (see Figure 4.1 for definitions). Each level describes

77
the association between the quality and quantity of engagement and expected outcomes. While
these classifications apply broadly to CER, they provide language that may be useful for
classifying CAB engagement and creating shared expectations between all partners.
Purpose
The purpose of this manuscript is to describe the development of a CAB based in the
University of Southern California (USC) Neural Plasticity and Neurorehabilitation Laboratory
(NPNL) and evaluate the CAB’s engagement six months after its founding. Notably, the NPNL
Stroke Advisory Board is not a project or institution board. Alternatively, the Board has a shared
vision for advising and disseminating stroke research within and beyond the laboratory’s
projects. To that end, we propose the value of intermediate level CABs, as opposed to the
project or institutional CABs that are most often described in the literature.
Methods
Developing the NPNL Stroke Advisory Board
The NPNL Stroke Advisory Board was established within the USC NPNL, which is
directed by the last author. The overall mission of the laboratory is “to enhance neural plasticity
in a wide population of individuals in order to improve their quality of life and engagement in
meaningful activities.” Research conducted in the laboratory seeks to characterize and predict
changes in neural plasticity, enhance neural plasticity and recovery, and personalize the use of
plasticity-inducing interventions. Since the laboratory’s founding in 2015, ‘insiders’ with lived
experience of stroke, post-stroke care partnering, and professional experience with stroke
rehabilitation have been key to enhancing the laboratory’s research. The NPNL Stroke Advisory
Board was founded in early 2023 to nurture the relationships between these individuals and the
laboratory and develop processes to enable meaningful engagement among all research
partners. Specifically, we sought to (1) infuse diverse perspectives of stroke recovery across all
projects and phases of research in the laboratory, and (2) bidirectionally mobilize knowledge

78
between people with lived experience of stroke and professionals to advance rehabilitation
research, even beyond the laboratory.
Advisory Board Members
NPNL Stroke Advisory Board members (hereafter referred to as ‘board members’) were
recruited from a network of research participants, consultants with lived experience of stroke,
care partners of survivors, and clinicians with whom the laboratory had previously collaborated.
They had all previously consented to receiving future communications from the laboratory about
research opportunities. We contacted them via phone or email to describe the purpose of the
CAB and invite them to a launch meeting to learn more and meet other prospective members.
Additional board members joined after the launch meeting.
At its founding in March 2023, our CAB consisted of seven members. Four members are
stroke survivors and bring their expertise from lived experience of stroke. Additionally, they each
bring unique relevant professional experience, such as software engineering, assistive
technology, emergency medical care, secondary education, athletic coaching, stroke advocacy,
and research consultation. Two members are a spouse/partner to a stroke survivor and have a
care partnering role. In addition to their experience supporting a stroke survivor, each brings
relevant professional experience: one as a professional care partner to individuals with various
diagnoses, and one as a licensed social worker. Finally, one member is an occupational
therapist specializing in neurorehabilitation who has been a research collaborator on many
projects. Since gathering the data reported in this manuscript, two more stroke survivors have
joined the Board, one of whom also has academic training in occupational therapy. Thus far, the
Board has experienced a 100% retention rate and continues to grow.

79
Figure 4.1 Continuum of Partner Engagement in Research
Note. These levels were developed by Goodman et al. (2020) in the REST Implementation Study. They are summarized
here for ease of understanding the study results.

80
Board Structure and Activities
When we founded the NPNL Stroke Advisory Board, we asked that all members commit to a
minimum of two Board meetings per year, with the expectation that they would also participate
on ad hoc committees and projects. This approach was taken to (1) encourage members to
engage in projects and Board activities that aligned best with their interests, strengths, and
availability; and (2) minimize the potential burden of participation for board members. Board
members have participated in a variety of research-related activities ranging from advising on
recruitment methods to developing dissemination products. Table 4.1 summarizes all major
activities of the Board from March 2023 through December 2023, including who participated, the
timeline, frequency, duration, and target level of engagement. Some activities, such as the
stroke symptom awareness video, were conceptualized by the board members and facilitated
by academic partners, whereas other activities, such as the sensor co-design workshop, were
conceptualized by the academic partners and carried out with the board members.
Table 4.1
NPNL Stroke Advisory Board Activities
Activity &
Description Participants
Timeline
(all events
in 2023)
Frequency &
Duration
Target Level of
Engagement
Information Meeting
Prospective Board
members gathered inperson along with
NPNL members to
connect with each
other, discuss the
purpose and role of
the Advisory Board,
and ask questions.
Board
members,
NPNL
members
March One, 2 hr
meeting
Outreach &
Education
Virtual Board Meetings
Discussed and
planned CAB
activities, such as the
stroke symptom
awareness video and
advising sessions.
Identified possible
Board
members, 1–2
NPNL
members
April,
August 2, 1–1.5 hr
meetings Cooperation

81
directions for future
advocacy and
translational projects.
Stroke Symptom
Awareness Video
Board members
competed in a local
knowledge
mobilization challenge
to develop a short
awareness video
showcasing the lived
experience of stroke
onset for people at risk
of stroke.
5 Board
members, 4
NPNL
members, 2
external
volunteers
May –
January
Ad hoc meetings
(approximately 1-
2 hr each)and
filming sessions
(approximately. 3-
4 hr each)
Collaboration
Research
Dissemination Videos
Board members were
featured in two videos
to disseminate
research findings.
2 Board
members May, August
1 working session
with each Board
member, 2 hr
each
Outreach &
Education
Advisory Board
Retreat
Reflected on the
Board’s first 6 months
and refined the Board
purpose, vision, and
scope.
7 Board
members, 2
NPNL
members
October 1, 2.5 hr meeting Cooperation
Open Advising
Session
The Board provided
feedback to advisees
from the NPNL and
other research
laboratories about
specific barriers they
were facing in their
projects.
7 Board
members, 2
NPNL
members, 3
advisees
October 1, 1.5 hr session Consultation
Sensor Co-design
Workshop
The Board discussed
the design of an EMG
biofeedback sensor
and recommended
improvements in a
hands-on prototyping
workshop.
7 Board
members, 3
NPNL
members
December 1, 3 hr workshop Consultation
Note. Activities are listed in chronological order. The blue dashed line marks the administration
of the Research Engagement Survey Tool Condensed Version, discussed below. All activities

82
above the line were completed prior to collecting the Survey response. No Advisory Board
activities took place during the data collection period.
Evaluating Community Advisory Board Engagement
It is important to all research partners (academic and community) in our laboratory that
the Board be a meaningful source of engagement that enriches research processes and
outcomes. Therefore, we are striving to evaluate the engagement of community partners
regularly, both as a form of monitoring the partnership and stimulating discussion about the
direction of the Board.
Outcome Measure
We evaluated research engagement for the NPNL Stroke Advisory Board using the
Research Engagement Survey Tool (REST) Condensed Version, which examines the level of
engagement among non-academic stakeholders in a research partnership (Goodman et al.,
2021). The condensed REST has nine items that measure seven engagement principles (EPs)
(See Table 4.2 for a list of EPs). Each item is scored on two, five-point Likert scales: one to
measure the quantity (how often) and a second to measure the quality (how well) these
principles are followed in the research partnership. Scores closer to 1 indicate lower quantity or
quality of engagement. These scores can be used to calculate the overall quantity and quality of
engagement, and to align survey responses with Goodman and Sanders Thompson’s (2017)
engagement classifications (education/outreach, consultation, cooperation, collaboration,
partnership; See Figure 4.1 for definitions). These scores and classifications are useful for
describing engagement across a continuum, monitoring the partnership over time, and
comparing engagement across and between projects. Additionally, as the Board continues to
engage with research, we will be able to correlate research outcomes with community
engagement to support the effectiveness of our CAB model (Goodman et al., 2017, 2021). For
this first administration of the REST with the NPNL Stroke Advisory Board, we hypothesized that
the overall REST scores would align with engagement at the consultation level, because most

83
board activities during the period under evaluation were based on researcher requests for
feedback from CAB members. Ultimately, the researchers were responsible for applying that
feedback to research projects.
Table 4.2
Engagement Principles and Associated Levels of Engagement
Engagement principle (EP)
Highest possible level of
engagement
How a response of ‘Excellent’ or
‘Always’ for a REST item
corresponding to the EP would be
classified.
EP1. Focus on community perspectives and
determinants of health Collaboration
EP2. Partner input is vital Collaboration
EP3. Partnership sustainability to meet goals and
objectives Partnership
EP4. Foster co-learning, capacity building, and cobenefit for all partners Collaboration
EP5. Build on strengths and resources within the
community or patient population Cooperation
EP6. Facilitate collaborative, equitable partnerships Partnership
EP7. Involve all partners in the dissemination process* Partnership
EP8. Build and maintain trust in the partnership Partnership
Note. These engagement principles were developed by Goodman et al. (2020). They are listed
here for ease of understanding the study design and results.
*The REST Condensed Version does not include a question for EP7.
Procedures
Six months after founding the NPNL Stroke Advisory Board, board members were invited
(but not required) to complete the REST. To create an opportunity for honest reflection, the
board members completed the survey anonymously, either online via Qualtrics (Qualtrics,
Provo, UT), or on a paper version. Data collected on the paper surveys were input to Qualtrics
by a research assistant who did not know the board members to ensure that the authors would
not be able to identify the respondent by their handwriting. Responses were collected over the
span of one month.

84
Analysis & Interpretation
We analyzed the condensed REST responses using the instructions provided by the
measure authors (https://wp.nyu.edu/collegeofglobalpublichealth-goodman_mle_lab/rest/). To
summarize, we first calculated EP-specific scores, then we calculated the mean of the EPspecific scores to get overall quantity and quality scores. Next, we aligned item responses with
the 5 categories of engagement. Goodman and Sanders Thompson (2017) previously identified
the highest possible level of engagement for each EP (e.g., how a response of ‘Excellent’ or
‘Always’ would be classified), noted in Table 4.2.
Initial interpretation of the findings was conducted by the first author. Then, board
members were invited to conduct member-checking via Zoom. In this hour-long session, the first
author presented the EP-specific scores, overall scores, and the alignment of responses with
the levels of engagement. Screen-shared PowerPoint slides with large font were used to show
the results. The lowest EP-specific scores were bolded on the slide and verbally interpreted to
the board members as relative strengths and weaknesses of the Board processes. Additionally,
board members were introduced and oriented to the levels of engagement. All board members
then participated in a discussion about the results and interpretation of scores. The goal of this
member-checking session was to stimulate collective examination of the engagement in the
partnership, reflection of our Board processes, and planning for process improvement.
During the member-checking session, all research partners determined that the first
author would draft a document combining the meeting notes from both the member-checking
session and the prior Board retreat to capture the mission, vision, and scope of the NPNL
Stroke Advisory Board. The document was then distributed to all board members for
asynchronous iterative review and critique until an updated document was agreed upon by all
board members. There is a shared understanding that this living document will evolve alongside
the Board.

85
Results
Research Engagement Survey Tool Responses
Responses to the REST Condensed Version were collected between October 2 and
October 31, 2023. All seven Board members completed the survey.
For the quality of engagement scale (Table 4.3), where scores closer to 5 represent
‘excellent’ quality, and scores closer to 1 represent ‘poor’ quality, the highest EP-specific scores
(mean > 4.00) reflected high levels of mutual respect, trust, and openness to ideas among all
research partners (EP6a, EP8a, EP8b). The lowest EP-specific scores (mean < 4.00) showed
relative weaknesses in setting shared goals and timelines (EP3, EP6b), and building on the
strengths and resources of the community population (EP5), though these scores were overall
reflective of ‘good’ or ‘very good’ quality engagement. We calculated an overall quality score
(mean = 4.02 ± 0.7), which indicates that the EPs were perceived to have been followed very
well in the NPNL Stroke Advisory Board.
Table 4.3
Mean REST Condensed Version Scores by Engagement Principle – Quality Scale
Engagement Principle (EP)
Mean
Score ±
SD
N (%) by Likert Response
Poor Fair Good Very
Good Excellent
EP1: The focus is on problems
important to the community. 4.00 ± 0.8 0 (0%) 0 (0%) 2 (29%) 3 (43%) 2 (29%)
EP2: All partners assist in
establishing roles and related
responsibilities for the
partnership.
4.00 ± 1.0 0 (0%) 1 (14%) 0 (0%) 4 (57%) 2 (29%)
EP3: Community-engaged
activities are continued until
the goals (as agreed upon by
all partners) are achieved.
3.86 ± 0.7 0 (0%) 0 (0%) 2 (29%) 4 (57%) 1 (14%)

86
EP4: The partnership adds value
to the work of all partners. 4.00 ± 0.8 0 (0%) 0 (0%) 2 (29%) 3 (43%) 2 (29%)
EP5: The team builds on
strengths and resources within
the community or patient
population.
3.71 ± 0.8 0 (0%) 1 (14%) 0 (0%) 6 (86%) 0 (0%)
EP6a: All partners’ ideas are
treated with openness and
respect.
4.57 ± 0.8 0 (0%) 0 (0%) 1 (14%) 1 (14%) 5 (71%)
EP6b: All partners agree on the
timeline for making shared
decisions about the project.
3.86 ± 0.7 0 (0%) 0 (0%) 2 (29%) 4 (57%) 1 (14%)
EP8a: The partnership’s
processes support trust among
all partners.
4.14 ± 0.9 0 (0%) 0 (0%) 2 (29%) 2 (29%) 3 (43%)
EP8b: Mutual respect exists
among all partners. 4.57 ± 0.8 0 (0%) 0 (0%) 1 (14%) 1 (14%) 5 (71%)
Note. For each item, respondents were prompted to “Please rate how well the partners leading
the research do each of the following.” The response “Poor” represents a score of 1 and
“Excellent” represents a score of 5.
For the quantity of engagement scale (Table 4.4), where scores closer to 5 represent
that an EP is ‘always’ observed, and scores closer to 1 represent that an EP is ‘never’ observed,
all EP-specific scores were greater than 4.00 and the overall quantity score was 4.49 points
(SD=0.5). This indicates that the EPs were perceived to have been followed often in our CAB.
Of note, lower quality or quantity of engagement scores do not necessarily reflect poorly on the
partnership. Rather, lower scores are associated with less stakeholder participation on the
stakeholder engagement spectrum (Goodman & Sanders Thompson, 2017).

87
Table 4.4
Mean REST Condensed Version Scores by Engagement Principle - Quantity Scale
Engagement
Principle (EP)
Mean
Score ±
SD
N (%) by Likert Response
Never Rarely Sometimes Often Always
EP1: The focus is on
problems important to
the community.
4.33 ± 0.5 0 (0%) 0 (0%) 1 (17%) 2 (33%) 3 (50%)
EP2: All partners assist
in establishing roles
and related
responsibilities for the
partnership.
4.43 ± 0.5 0 (0%) 0 (0%) 0 (0%) 4 (57%) 3 (43%)
EP3: Communityengaged activities are
continued until the
goals (as agreed upon
by all partners) are
achieved.
4.43 ± 0.5 0 (0%) 0 (0%) 0 (0%) 4 (57%) 3 (43%)
EP4: The partnership
adds value to the work
of all partners.
4.43 ± 0.8 0 (0%) 0 (0%) 1 (14%) 2 (29%) 4 (57%)
EP5: The team builds
on strengths and
resources within the
community or patient
population.
4.29 ± 0.8 0 (0%) 0 (0%) 1 (14%) 3 (43%) 3 (43%)
EP6a: All partners’
ideas are treated with
openness and respect.
4.71 ± 0.5 0 (0%) 0 (0%) 0 (0%) 2 (29%) 5 (71%)
EP6b: All partners
agree on the timeline
for making shared
decisions about the
project.
4.57 ± 0.5 0 (0%) 0 (0%) 0 (0%) 3 (43%) 4 (57%)
EP8a: The
partnership’s
processes support trust
among all partners.
4.86 ± 0.4 0 (0%) 0 (0%) 0 (0%) 6 (86%) 1 (14%)

88
EP8b: Mutual respect
exists among all
partners.
4.71 ± 0.5 0 (0%) 0 (0%) 0 (0%) 2 (29%) 5 (71%)
Note. For each item, respondents were prompted to “Please rate how often the partners leading
the research do each of the following.” The response “Never” represents a score of 1 and
“Always” represents a score of 5. The total number of responses for EP1 is 6 because one
respondent selected “Not Applicable”.
Finally, we aligned the responses for each EP to the 5 categories of engagement. For
the quality scale, the responses suggest that on average, the NPNL Stroke Advisory Board
activities can be classified as 1.6% outreach and education, 11.1% consultation, 17.5%
cooperation, 63.5% collaboration, and 6.3% partnership. For the quantity scale, the responses
align with 1.6% outreach and education, 9.7% consultation, 3.2% cooperation, 70.8%
collaboration, and 14.7% partnership.
As described in the Methods, all research partners reviewed these findings and
discussed their perceptions of the relative strengths and weaknesses of the engagement. The
primary findings from the conversation were a relative strength of a strong sense of mutual
respect and trust within the group. One board member commented, “I like that the two highest
scores are basically ‘all partners are treated with respect and openness’...that just speaks
volumes to how much everyone brings to the table.” We also identified a relative weakness of
needing clearer short-term and long-term goals and timelines. While there have been many
ideas for how the Board can be a bridge between research and stroke survivor communities, we
had yet to define specific goals and timelines to facilitate execution. The Board discussed these
strengths and weaknesses to take strategic next steps. Board members iterated on a
conceptual model to frame the NPNL Stroke Advisory Board’s vision, which we anticipate will
guide and improve group goal setting. The Board requested that the first author draft a
document with the refined mission, vision, scope, and goals (Appendix C). Review of the
document was conducted asynchronously, and minor changes were made until all Board
members agreed, as described in the Methods.

89
Discussion
In this paper we describe the early development of the NPNL Stroke Advisory Board, an
intermediate level board (i.e., not project or institution based) and evaluate engagement after six
months of participation in research activities. When carried out thoughtfully, CABs can be a
bridge between researchers and community members to enhance research processes and
outcomes. However, there is a risk of CABs becoming symbolic of community engagement,
rather than authentic partnerships between the community and academy. One way to defend
against symbolic engagement is to evaluate engagement between partners regularly and
reflexively. We have chosen to administer the REST Condensed Version twice per year as a
quick but value-packed quantitative measure of the quality and quantity of engagement for the
NPNL Stroke Advisory Board. We use these data to fuel honest, reflective discussions that we
believe will enrich our Board’s engagement and help us carry out our vision more effectively.
The twice-yearly evaluation rhythm aligns with Kubicek and colleagues’ (2016) recommendation
and facilitates the co-learning process that is essential to thriving collaborations (Sufian et al.,
2011).
Six months after establishing the NPNL Stroke Advisory Board, seven board members
completed the REST Condensed Version and reviewed the results as a group. The board
members rated partner engagement favorably (all average scores and item responses > 3),
suggesting high quality and quantity of adherence to engagement principles. Encouragingly, the
scores from our small sample are comparable to those reported by Goodman et al. (2020) in the
REST Implementation Study Report (n = 177). When we aligned the EP scores to levels of
engagement, most scores were reflective of ‘collaboration’ between academic and community
partners. However, as noted in Table 4.2, the target level of engagement for most Board
activities was less than collaboration (e.g., consultation, cooperation). Upon review of the data
in context of Board activities and through member-checking discussions, we identified that for
five of the seven board members, the stroke symptom awareness video constituted most of their

90
hours of participation on the Board (approximately 20 hours per board member). As noted in
Table 4.2, this activity was initiated by board members. Therefore, when completing the survey
measure, the quality and quantity of engagement in this specific activity weighed more heavily in
their assessment than the other activities. Because most Board activities have targeted
consultation and cooperation, we anticipate that future REST scores may align at those levels.
We set this lower target level of engagement because the Board is operating under a novel CAB
approach and is in the early stages of group forming and research capacity building. To that
end, we anticipated that the researchers would play a more prominent role in guiding most
Board activities. As the Board continues to develop, we hope to progress toward higher levels of
engagement between academic and community partners.
An Intermediate Level Community Advisory Board
As opposed to project or institutional boards, we sought to establish a board based in an
academic research laboratory that could operate flexibly at the local level and advise across
many research projects. Additionally, we wanted a board that could embark on advocacy and
dissemination activities as they saw necessary, without the constraints of a specific project.
Specifically, the NPNL Stroke Advisory Board has a vision of supporting translational research
and dissemination between research laboratories and the community. To our knowledge, such a
model for CABs has not been described in the literature. Toolkits for CABs provide guidance for
developing project-specific boards (e.g., Kubicek et al., 2016), but we have not found examples
of CABs that are long-standing, local partnerships, that advise and participate ‘hands-on’ in
research across multiple projects and with multiple investigators. The closest examples we
identified in the literature are community-based organizations that partner with universities and
other community agencies. One such exemplar is Healthy African American Families, which is a
community organization that developed a community-academic council to coordinate the efforts
across multiple research and training programs at three academic institutions to address a
variety of issues that disproportionately affect communities of color (Wells et al., 2006). By

91
contrast, our board is smaller, situated within an academic institution, and focuses on
connecting groups and individuals with lived and professional experience of stroke to advance
rehabilitation research and access.
Within the first year of establishing the NPNL Stroke Advisory Board, we identified a few
relative strengths and weaknesses of the intermediate level CAB model through the review of
the REST Condensed Version results and the discussions that followed. First, our Board can
advise on a range of projects and initiatives because we are not confined to a specific grantfunded project or to the agenda of a larger institution. This allows us to experiment with
disrupting traditional research paradigms and researcher-community dynamics. However, a
related challenge we are navigating is evaluating the ever-evolving ‘asks’ we make of the Board
as investigators and the ‘tasks’ that the Board chooses to take on. Despite our hopes of
disrupting some of the embedded research processes, we are still operating within existing
constraints and cultural knowledge of ‘how things are done’ in research. As a Board, we are
intentionally being reflexive about our strengths, interests, capacities, and expertise. As
investigators, we are also being reflexive about how our embedded knowledge affects our
perspectives, and being open to ideas brought to us by the Board that are different than
research norms. Our REST survey results suggest that our intermediate level CAB may support
higher levels of engagement (e.g., collaboration), which is a promising finding as we seek to
enhance engagement between researchers and the community.
A second strength is how energized and empowered our board members are to impact
the health of the community. The flexibility of this model allows them to bring diverse and
creative ideas to the table and act on the ideas that are feasible and best meet the needs of the
stroke survivor community. A challenge of this flexibility has been determining who has final
decision-making authority and how we evaluate feasibility. As academic researchers, we are
often evaluating the feasibility of an idea based on our own time availability, other demands and
priorities of academia, and the resources needed to carry out a project suggested by the board

92
members. At times, board members have had differing (perhaps more optimistic) expectations
of time and resources. However, through ongoing meetings, activities, and retreats, we continue
to explore the feasibility of ideas and are experiencing increasing alignment between all
partners. One formative experience for this was during the stroke symptom awareness video
project. We sought to connect with local stroke support groups and community organizations to
identify possible collaborations and paths for eventual dissemination of our awareness video.
Many board members and researchers attempted to build these connections by attending local
events and reaching out to contacts—time and energy consuming relationship building—but
many of those attempts were not successful. As a group, we experienced some of the
challenges of doing advocacy and dissemination work, which has led to better alignment of
expectations of each other and the Board’s overall output, as well as a narrowing of the Board’s
scope of practice. We anticipate this will be an evolving process in response to the needs of the
stroke research and survivor communities.
A third strength of this model is the diversity of valuable expertise from lived and
professional experience that our Board brings to rehabilitation research. Investigators who have
been advised by our Board see the value of these perspectives in their work. However, a
challenge has been making the most of these advising relationships by supporting the diversity
of education level and prior research exposure (or lack thereof). For example, some of our
board members have prior research consultation experience, and others have only engaged in
research as a participant. Additionally, some have graduate degrees and others have a high
school education. All our board members make valuable contributions regardless of these
sociodemographic and personal factors; however, it does require us as investigators to be
acutely aware of power dynamics, build capacity for research advising where possible, and
identify ways of communicating and gathering that support the inclusion of all members. We are
currently exploring ways to build research capacity on our board to improve the relevance of

93
feedback during advising sessions, while also elevating the valuable perspective of lived
experience, regardless of research capacity.
Our Board is particularly proud of a fourth strength, which is the depth of meaningful
relationships the board members and NPNL investigators have formed with each other. More
than once, board members have commented in a meeting, “You all are like family to me.” We
believe this trust, respect, and concern for each other will lead to a long-term, sustainable
partnership. However, a related challenge is determining the appropriate balance of structure to
facilitate these relationships without suppressing the vibrant culture that is organically emerging.
For example, determining how strictly we set and adhere to meeting agendas and how we
delineate social gathers from ‘business’ meetings are two challenges the academic partners
have been facing as we seek to cultivate group norms.
Finally, from the academic research partner perspective, two of the major difficulties of
developing an intermediate level board have been the time investment and role switching
required to facilitate Board activities. For example, as the primary academic research partner
facilitating the board, the first author has taken on additional roles of event planner and
executive producer of a short film, both of which have required additional professional
development outside of the existing roles and responsibilities typical of an academic researcher.
For the long-term sustainability of this Board and other long-term community partnerships,
having staff or workload effort dedicated to CAB facilitation will be necessary (Halladay et al.,
2017). Without dedicated resources, the intermediate level CAB approach could fall prey to
symbolic participation and lead to attrition and burnout for all research partners. In the case of
our Board, we had access to student volunteers and paid research assistants to support the
logistical coordination of Board activities. Additionally, we received 2 small grants (less than
$3,000 in total) to support initiation of the Board. Most of these funds were allocated for activity
materials and parking permits and stipends for Board members. Despite the intention to
compensate Board members at a modest rate for attending meetings ($15 per hour), many

94
Board members refused to accept stipends, preferring to be volunteers at the service of the
stroke community. Moving forward, we will need to acquire additional funding to support Board
activities. A challenge of the intermediate level CAB will be the fluctuating and sometimes
absent funding, in contrast to project-based CABs that are funded for the full extent of the
project. One potential option for sustaining the Board in the future is to request honorariums
from investigators seeking advice from the Board. Facilitating an intermediate level CAB with
adequate support may greatly enhance the science of the academic research partners and its
relevance to the community; however, there are practical barriers to doing so that must be
addressed for sustainable uptake of this CAB approach
Based on our experience developing the NPNL Stroke Advisory Board and the resources
currently available for developing CABs, we are finding that some of these strengths and
challenges may be unique to the intermediate level CAB. For example, CAB toolkits recommend
that researchers have clear roles and goals prior to assembling a CAB (Kubicek & Robles,
2016; Silberberg et al., 2011). Had we done this, we may not have faced some of the challenges
of determining decision-making authority and scope of practice. However, we intentionally
created very few boundaries in advance, with the goal of empowering the board to develop an
identity, scope of practice, and shared vision. As a result, we have an empowered Board that is
building strong relationships with each other and with other research and community partners.
We see this as a strength that is uniquely facilitated by the intermediate model.
Moving forward, our Board is engaging in advising sessions with academic advisees
within the NPNL and other research groups. Additionally, we are working to connect siloed
community-based stroke support groups to clinical resources and research opportunities.
Through these research activities, we are also striving to improve our processes, such as
testing different practices for meeting and communicating.

95
Limitations
Overall, our evaluation of our intermediate level CAB using the REST Condensed
Version was informative; however, we did encounter a few limitations using it for our CAB
context. First, not all partners fit squarely into the categories of academic or non-academic
partners. Rather, our Board represents a spectrum of research experience (e.g., a clinical
faculty member, a freelance research consultant). Second, because each Board member
chooses which projects and activities in which they want to invest their time, each person has
participated in different activities to varying extents, so the activities and interactions they had in
mind when completing the survey were different. Therefore, the levels of engagement aligned to
their REST responses were quite variable. Finally, despite our best efforts to create an
opportunity for honest, anonymous feedback, our Board is small (seven members), so it is
possible that the survey responses were influenced by a perception that they would be
identifiable, or by power dynamics that are unfortunately inherent in the investigator-respondent
relationship. Despite these limitations, the condensed REST provided a quick but robust way to
examine our engagement. Reviewing the results as a group elicited rich conversation that has
advanced our Board’s vision and productivity. In the future, as we build capacity as a Board, we
hope to have Board members administer and interpret the REST with less involvement from the
investigators to hopefully spur even more honest appraisals of the CAB engagement.
Conclusion
We propose that establishing intermediate level CABs for long-term collaboration across
research projects can empower community members to have an influential voice in research;
strengthen community-research partnerships and access to research; and enhance the
translation of scientific findings into real-world contexts. We share context for the development
of the NPNL Stroke Advisory Board, results from our self-evaluation, and some benefits and
challenges we have identified thus far to implementing intermediate level CABs. We hope that
other academic and community partners will see the value of intermediate level CABs and

96
expand upon this model. With a greater evidence base for this CAB model, we can work to
identify practices that lead to success and adaptations that support use of this model in local
contexts, with the goal of enhancing the relevance of research questions, study design,
outcomes, and dissemination products to improve the health and wellness of communities.

97
Chapter 5: Discussion
The present studies address gaps in post-stroke rehabilitation that prevent stroke
survivors from receiving high-quality therapy that could transform their engagement in
occupations. Briefly, post-stroke rehabilitation services in the United States are underutilized
beyond acute care, in part due to issues accessing care, whether because of geographic,
physical, financial, transportation, awareness of service, or other barriers (Ayala et al., 2018;
Mahak et al., 2018). Additionally, when stroke survivors can access care, there is a lack of
effective treatments for those with moderate to severe hemiparesis, and especially among those
in the chronic phase (> 6 months post stroke; Teasell et al., 2018; Winstein et al., 2016). When
minimal recovery, as measured by clinical outcomes, is noted in the chronic phase, patients
often have reduced eligibility for therapy services, which yields poor outcomes—a vicious cycle
that results in long-term disability (Pereira et al., 2012; Teasell et al., 2018). There are also few
treatment options that can be used to address the unique access needs of this population, such
as treatments delivered via telerehabilitation or those that can be used independently at home
between therapy sessions or upon discharge from therapy. Of the treatment options that have
been developed to address such needs, there is a trend of failed translation into routine clinical
practice (Hughes et al., 2014; Morrow et al., 2021). Even a cursory review of literature shows
many rehabilitation technologies developed to address gaps in stroke rehabilitation (Everard et
al., 2022; Laver et al., 2017; Mehrholz et al., 2018); however, attempts to implement them into
practice have been undermined by multidimensional contextual barriers and a lack of planning
and support for implementation.
To advance the science of stroke rehabilitation technology and its application in clinical
practice, this dissertation examined factors that are likely to influence the uptake of an
electromyography biofeedback system (Tele-REINVENT; Marin-Pardo et al., 2021) into routine
post-stroke clinical practice. We did this by evaluating the acceptability and usability of Tele-

98
REINVENT among stroke survivors using it at home (Chapter 2), evaluating the feasibility and
acceptability of Tele-REINVENT among prospective implementors (clinicians) based on a
demonstration video (Chapter 3), and co-creating knowledge with stakeholders to increase the
effectiveness of stroke telerehabilitation research (Chapter 4).
The findings from the present studies will inform the ongoing development and
implementation planning for Tele-REINVENT and support its integration into the daily lives of
stroke survivors. This work also has the potential to inform the design and delivery of other
innovative post-stroke telerehabilitation services and reduce barriers to accessing high-quality
therapy. Understanding how stroke survivors engage with complex telerehabilitation
technologies in the home is necessary for creating effective and usable therapy alternatives for
underserved groups. Additionally, understanding the situation-specific needs and decisionmaking of clinicians and other key implementors is important for sustained uptake in practice.
Summary of Key Findings
In Chapter 2, we reported our findings from a study of stroke survivors who used TeleREINVENT in their homes for six weeks with remote support from the research team. Efficacy
outcomes from this trial were previously reported by Marin-Pardo et al. (2022). In the present
study, we reported acceptability and usability findings, identifying that Tele-REINVENT is highly
rated for both based on validated quantitative measures, supported by qualitative findings about
stroke survivors’ experience using Tele-REINVENT. We identified through the interviews that
researchers played a crucial role as technology partners in reducing technical barriers
(Schepens Niemiec et al., 2023), which likely inflated the scores for certain subscales (i.e., effort
expectancy, perceived enjoyment, satisfaction) and contributed to greater adherence to the
intervention. This is supported by self-efficacy and perceived skills scores, suggesting that ease
of use may hinge on the support of a research team member. When implemented in clinical
practice, this role would land on therapists, which is not feasible or sustainable (Bower et al.,
2021; Morrow et al., 2021).

99
In Chapter 3, we built on the previous findings to identify possible barriers and supports
to using Tele-REINVENT in real clinical settings, from the perspective of rehabilitation therapists.
These clinicians play a dual role: they are both the implementors (service providers) of the
intervention as well as end users of the technology, in that they will use the whole system with
patients in the clinic and the clinician interface if used via telerehabilitation or as a home
exercise program. Our findings show barriers and supports that highlight the
interconnectedness of acceptability and feasibility. Briefly, barriers included factors of the
practice setting; characteristics of the clinicians and patients; and access issues. Supports
included training, technical support, infrastructure, and more research evidence. Additionally, we
observed that the appropriateness of Tele-REINVENT was an important factor influencing
clinicians’ assessment of the system, even among those who found it highly acceptable and
feasible. Despite the identified barriers, which are addressable through strategic design
adaptations and implementation planning, Tele-REINVENT was perceived as a promising
innovation in stroke rehabilitation with benefits for an underserved group.
In Chapter 4, we continued to prioritize engaging important stakeholders and users of
stroke rehabilitation research by developing the NPNL Stroke Advisory Board. Notably, this
Board uses a novel approach that is more flexible than a project-based CAB or institutional
CAB. We sought to quantitatively evaluate the engagement of the Board as a way of beginning
the process of monitoring engagement over time and determining best practices for this novel
approach to CABs. Our findings showed overall high quality and quantity of engagement
between academic partners and Board members, classified as collaborative, reflecting
adherence to evidence-based engagement principles. These findings suggest that our model of
CAB can enable high levels of engagement given the resources available in our laboratory and
university context.

100
Implications Across Studies
While each study in the current work addresses a unique aspect of the development and
uptake of stroke rehabilitation technologies, various common themes emerged across studies
that have implications for the ongoing work translating Tele-REINVENT into clinical practice and
future work designing and implementing complex stroke rehabilitation technologies for use in
everyday life.
Stakeholders offer valuable insights in the design and pre-implementation of complex
rehabilitation interventions.
Across all three studies presented in this dissertation, the perspectives of stakeholders
were highly valued as sources of knowledge about the situations in which technology-based
interventions are used and how various contextual factors and individual needs influence
uptake. To this end, we believe that stakeholder involvement in development, implementation,
and evaluation are foundational to any project that aims to integrate useful technologies into
people’s daily lives. Across the studies, we gathered pragmatic information about the use of
technology at home and in therapy clinics from the perspectives of the implementors (therapists)
and intervention beneficiaries (stroke survivors).
Each study engaged stakeholders to different extents, based on the aims of the study
and desired outcomes. In Chapter 2, stroke survivors used Tele-REINVENT for six weeks,
gaining a deep understanding of the system and how it fit into their everyday routines. They
provided feedback and insights throughout, and at the end of the study, participated in a long
form interview to discuss the nuances of using the system in their daily life situations, the
contextual factors that affected use, and the system’s potential role in their recovery goals. For
additional qualitative findings about the meaning and contexts underlying the acceptability of
Tele-REINVENT, refer to our phenomenological analysis (Donnelly et al., 2023). In Chapter 3,
clinicians had only brief exposure to the system, viewing a three-minute video about TeleREINVENT and responding to a survey (approximately 10 minutes). This brief point of contact

101
with clinicians was appropriate for gathering the pre-implementation data we sought to collect at
the current phase of development and implementation planning. Additionally, this approach
enabled us to collect data from more participants across a diversity of professional experiences,
as opposed to using a focus group or in-person demonstration methodology, which would have
included fewer participants, been more time consuming, and likely yielded similar data or
perhaps a depth of data not warranted by the research questions.
Finally, in Chapter 4, we initiated longer-term relationships with stroke survivors, care
partners, and professionals to improve the quality and relevance of stroke rehabilitation
research through developing a community advisory board. For one activity, our Board
participated in a co-design workshop aimed at improving the EMG sensor design for TeleREINVENT. They offered design suggestions based on their previous experiences with health
and rehabilitation technologies, developed hypotheses, and tested some of these with hands-on
prototyping. Importantly, this workshop was part of a larger effort to engage with stakeholders in
research, so the relationships and research capacity we had been developing through other
activities laid a foundation for mutual learning, contrasting with more traditional researcherparticipant dynamics.
Clinicians are keystones of successful implementation of rehabilitation technologies.
Another theme that emerged across studies and in the existing literature is the vital role
of rehabilitation therapists in successful implementation of rehabilitation technologies (Caughlin
et al., 2020; Y. Chen et al., 2020; Neibling et al., 2021). While this finding may seem intuitive, the
exclusion of therapists in development and implementation processes is shockingly common
and leads to the downfall of potentially effective technologies. We noted both in the literature
and the present studies that intervention providers are important gatekeepers in the uptake of
rehabilitation technology in clinics, as well as influencers of the rehabilitation-related purchasing
decisions of their patients (Celian et al., 2021; Elnady et al., 2018; Morrow et al., 2021). At the
same time, because they are often the personal connection point between patients and

102
rehabilitation technologies, skilled therapy providers can rapidly become technical support
specialists, which adds to therapist burnout and detracts from their unique value (Celian et al.,
2021; Curtis et al., 2023; Morrow et al., 2021).
We observed the important role of therapists for both technical support and skilled
delivery of Tele-REINVENT in Chapter 2. Stroke survivors relied more heavily on support from
researchers than the written and audiovisual supports provided. In many cases, they overlooked
the support materials we provided and contacted a member of the research team at the onset of
a problem. In clinical practice, this would likely translate to a reliance on therapists for support,
creating undue burden on them. We also identified that the primary way providers can bring
distinct value to Tele-REINVENT service delivery is by further personalizing the intervention
using their clinical observations (in addition to the personalized biofeedback provided by the
system) and remotely adapting and grading game settings to be appropriately challenging for
each patient. Similarly, clinicians who viewed the demonstration video and provided feedback
via the survey expressed concerns that troubleshooting with patients would occupy too much
valuable time in their limited treatment sessions and detract from their ability to provide skilled,
reimbursable services. As a result of these findings, we are orienting the design of
implementation strategies toward enabling therapists to work at their highest, most skilled
levels. In part, this will be achieved by reducing technical barriers and enhancing the quality and
accessibility of support, though additional investigation is needed to identify the specific
mechanisms by which support will be provided (e.g., live technician assistance, artificial
intelligence chat bot, interactive troubleshooting guide)
Clinicians and stroke survivors identified related and complementary factors influencing
the feasibility, acceptability, and usability of Tele-REINVENT
A third observation across studies was the complementarity of stakeholder knowledge
from different perspectives (i.e., clinicians or stroke survivors). While both clinicians and stroke
survivors offered distinct feedback that was unique to their role in rehabilitation, connections

103
between these perspectives were apparent. Previous studies show that patients often report
greater satisfaction and fewer barriers to using a new intervention than clinicians (Caughlin et
al., 2020; Janssen et al., 2020). In the current studies, we used different quantitative measures
to examine acceptability of Tele-REINVENT among stroke survivors and clinicians, given that
they had different experiences with the technology, so we cannot compare scores directly.
However, both groups rated Tele-REINVENT as highly acceptable on their respective measures.
Qualitatively, clinicians reported more barriers to using Tele-REINVENT than stroke survivors,
likely because of their institutional knowledge about health care and their specific facility. We
also suspect that during the study with stroke survivors, assistance from a provider (the
researcher) reduced the quantity and scope of barriers experienced by the stroke survivor as
compared to if Tele-REINVENT had been administered in a real clinical setting. Despite the
difference between these groups, the barriers identified by the stroke survivors were
corroborated by clinicians, and vice versa.
For example, as discussed above, just as therapists were concerned about spending too
much time troubleshooting technical issues, stroke survivors identified that interactions with a
provider were imperative to their success with the system, in part because of their ability to
provide technical support. Therapists highlighted the need for quality support, especially
technical, for both them and their patients, and we observed that tutorial videos and written
manuals were not appealing alternatives to live support among stroke survivors. Therefore, our
findings from both studies suggest a need for higher quality and more tailored messaging within
the system and supplemental support resources to intercept technical support interactions
between patients and clinicians. This not only should enhance patient outcomes but also make
it more feasible to use in the clinic and increase acceptability and appropriateness.
Another finding was the importance of self-efficacy for both patients and providers.
Among stroke survivors, high self-efficacy in game performance motivated continued use of the
system; however, their self-efficacy was reduced when they perceived that their lack of technical

104
proficiency was the underlying cause of technical issues. Therapists expressed concerns that
some patients, especially older adults, and their care partners, would have difficulty navigating
the software and using EMG sensors, especially if they were to use it at home. While therapists
did not get hands-on experience with Tele-REINVENT, some noted their own lack of confidence
or familiarity with aspects of the technology, most notably, EMG, as this is not yet a mainstream
rehabilitation practice. Some also doubted their ability to adapt Tele-REINVENT for patients with
complex symptoms. The self-efficacy of patients and providers influences the uptake of
rehabilitation technology (Bower et al., 2021) so facilitating self-efficacy will be an important goal
of implementation strategies for Tele-REINVENT.
Finally, across all three studies, stakeholders wanted Tele-REINVENT to be flexible for
use across a diversity of users and contexts. Board members in the co-design workshop
emphasized the importance of EMG sensors that could easily be donned by survivors with
spasticity and contractures, without making it more cumbersome for survivors without those
qualities. Similarly, therapists reported that it was important for them and their administrators for
Tele-REINVENT to be usable with a diversity of patients, such as those with more severe
symptoms, advanced age, low technical proficiency, and low-income. In both the study with
stroke survivors and the clinician survey, the ability to use Tele-REINVENT in various contexts,
including at home independently or via telehealth or in the therapy clinic was highly acceptable.
Innovative Approaches
The current studies used an innovative approach to therapy: a portable, telehealthenabled, EMG-biofeedback system to improve sensorimotor outcomes among an underserved
population—chronic stroke survivors with moderate to severe hemiparesis. The innovation of
the technology has been previously reported (Marin-Pardo, 2023; Marin-Pardo et al., 2022;
Marin-Pardo et al., 2021). The innovative contributions of the current studies are primarily
methodological, as we seek to move innovative technology into routine clinical practice. Notably,
we (1) evaluated implementation outcomes concurrently with efficacy outcomes, (2) used an

105
asynchronous ‘demo’ video to gather pre-implementation outcome data, and (3), developed an
intermediate level CAB to infuse first-hand lived and professional experiences of stroke into our
work.
Evaluating implementation outcomes concurrently with efficacy outcomes
Clinical translational research has often followed a linear path, progressing through a
spectrum from efficacy research in the laboratory to effectiveness research in clinical settings, to
implementation research (Khoury et al., 2007). Even though the intervention under evaluation is
the same in these stages, the goals of research are quite different. In rehabilitation research,
efficacy studies focus mostly on examining functional outcomes for patients and are concerned
with maintaining internal validity. Effectiveness research typically focuses on generalizability and
external validity. Both efficacy and effectiveness studies are controlled, though effectiveness
studies to a slightly lesser extent (Zettler et al., 2010). By contrast, implementation research
focuses on processes that will support the uptake of interventions into routine practice by
providers and systems (Eccles & Mittman, 2006). All three phases are important; however,
treating them as distinct phases has negative implications for the ultimate success of translation
and implementation efforts; if we do not consider implementation when designing efficacy and
effectiveness studies, then an intervention may not end up fitting the clinical setting for which it
was intended (Leppin et al., 2020).
In response to a traditional, linear approach to clinical research, Curran et al. (2012)
propose ‘effectiveness-implementation hybrid designs,’ wherein effectiveness and
implementation outcomes are studied concurrently to facilitate faster and more successful
translation of clinical findings into routine clinical practice. One of these designs, called Hybrid
Type 1, involves testing the effectiveness of a clinical intervention while gathering information on
its delivery and/or potential for real-world implementation (Curran et al., 2012). In the current
work, we adapted the Hybrid Type 1 study design to evaluate efficacy outcomes concurrently
with implementation outcomes. To that end, while conducting laboratory-based efficacy trials of

106
Tele-REINVENT, we were also evaluating implementation outcomes as part of the preparation
phase of the EPIS implementation framework. Implementation outcomes such as acceptability
and feasibility are often reserved for later stages of implementation research; however,
evaluating these outcomes before effectiveness or implementation research can help ensure
that complex interventions address the needs of the intended user and fit with the intended
delivery context. Because of this timing, we can make substantial changes to Tele-REINVENT
between efficacy and effectiveness trials if justified by our pre-implementation analysis.
Using an asynchronous video demonstration to gather pre-implementation data
In Chapter 3, we collected pre-implementation data about Tele-REINVENT after showing
OT and PT practitioners a video ‘demo’ of its use and features. To our knowledge, this is the first
study to use an asynchronous video ‘demo’ to capture clinician perspectives on a rehabilitation
device during implementation planning. Previously, written vignettes have been used to depict
the use of an intervention and elicit clinician perspectives ( Evans et al., 2015). We anticipated
that our video would provide a clearer depiction of Tele-REINVENT than written content and
would be more engaging for participants.
The primary benefit of this approach was the ability to efficiently collect data from a
diverse sample of OTs, OTAs, PTs, and PTAs who practice in different settings and regions of
the United States. Our sample also represented a wide range of years in practice and education
level. The demonstration video was delivered asynchronously as part of the online survey,
allowing clinicians to watch and provide feedback at their convenience, and over multiple survey
sessions if their schedule required it. We particularly wanted the survey to be feasible for
clinicians in fast-paced settings with minimal free time. Additionally, the asynchronous video
demonstration approach eliminated the human resources and funds required to coordinate a
nationwide tour of live in-person or virtual demonstrations.
Despite the many benefits of this novel approach, one drawback is that clinicians did not
get hands-on experience with Tele-REINVENT. Their perceptions of the system were informed

107
by the content we shared with them. Therefore, some of the barriers and supports described by
respondents were already addressed by the system design, even though they were not explicitly
discussed in the video. Given how early in the implementation process we conducted this study,
we do not see this as a major limitation; in fact, it gives insight into therapists’ priorities and will
inform the support materials we develop in the future.
Overall, we found that an asynchronous video ‘demo’ of Tele-REINVENT was a feasible
way to elicit insights from therapists about factors influencing future uptake of Tele-REINVENT.
As evidence to support our approach, we observed that many of our results were also
supported by other telerehabilitation technology studies. This suggests that a brief video
demonstration can yield valuable data that otherwise might be gathered using more time
consuming or expensive methods.
Developing an intermediate level community advisory board to inform telerehabilitation
research
A third methodological innovation presented in this work is the development of an
intermediate level advisory board to inform telerehabilitation research. Most CABs described in
the literature support either specific, grant-funded projects or large institutions. Instead, we
sought to form a CAB based in an academic research laboratory that could advise and
participate flexibly across various research projects, investigators, advocacy initiatives, and
dissemination efforts. Therefore, we developed the NPNL Stroke Advisory Board, whose
members envision it “as a resource hub connecting the [Southern California] stroke survivor and
caregiver community with the local stoke research community.” To our knowledge, this CAB
approach is an innovative hybrid of other CABs that has not been described previously.
Our intention with developing this new model of CAB was to facilitate long-term
collaborations between academic and community partners and allow these partners to influence
the trajectory of research processes, rather than conforming to existing project plans or
institutional initiatives. Our initial self-evaluation as a Board revealed multiple benefits and

108
complementary challenges of this model, discussed in greater detail in Chapter 4. Briefly, our
Board members feel empowered to improve the health of the community, and the flexibility of
this CAB model allows us to embark on a variety of projects and experiment with new
researcher-community dynamics. Our Board members represent a diversity of lived and
professional experience, and together have formed a collaborative partnership with meaningful
interpersonal relationships. Related, we have been challenged by the constraints of embedded
research processes, determining the Board’s scope of practice, establishing decision-making
and other group processes, and supporting research capacity in Board members.
The NPNL Stroke Advisory Board is in its early days, but we hope that by presenting this
innovative model, it will spark interest among other investigators to establish intermediate level
CAB that address the unique skills and interests of their community. As we and other groups
experiment with this model, we can develop an evidence base of practices that support
meaningful engagement, adaptations for local contexts, and outcomes that demonstrate
effectiveness of these CABs.
Limitations
In addition to the limitations identified for each study, the findings from the current work
may not be generalizabile beyond Tele-REINVENT. This is primarily due to the many
technology-specific factors that influenced ratings and descriptions of pre-implementation
outcomes. This limitation is not unique to Tele-REINVENT—it is a reality of most research
examining technology-based interventions. However, taken with the findings of other post-stroke
telerehabilitation technologies, the current work may be useful for anticipating and planning for
possible contextual barriers, facilitators, and implementation supports.
Future directions
The current work lays the foundation for ongoing preparation of Tele-REINVENT for
implementation. We identified barriers and support gaps at the administration, organization, and
system levels through the lens of therapists. We anticipate that many of the supports needed for

109
successful implementation and sustainment of a complex, technology-based intervention like
Tele-REINVENT will need to come from within organizations (Caughlin et al., 2020; Celian et al.,
2021). However, to fully understand these and their impact on implementation, future studies
should explore these contexts, barriers, and supports through expert interviews, analysis of
organizational factors, such as readiness for change and policies; and analysis of other local
and federal policies influencing the uptake of telerehabilitation technologies in post-stroke
rehabilitation. Based on the current work and future studies examining organizational and
system level contextual factors, a future step is to develop implementation strategies to
overcome identified barriers and leverage facilitators.
The preparatory work will eventually lead to implementation. We propose first piloting
Tele-REINVENT in an effectiveness-implementation study at two to three clinical sites. Prior to
implementation, we intend to evaluate organizational factors for the pilot sites, such as
measuring organizational readiness for change and qualities of the leadership staff. During this
trial, we would continue to evaluate implementation outcomes to monitor change over time and
in response to various implementation strategies. Ultimately, if any one of these implementation
outcomes is rated poorly, the uptake of Tele-REINVENT may be suboptimal. As clinicians get
hands-on experience with Tele-REINVENT in this phase, we also plan to evaluate their
perspectives as end users, not just implementors. These effectiveness-implementation studies
will support ongoing implementation planning to support broader uptake and eventual
sustainment of Tele-REINVENT in clinic-based and telerehabilitation practice.
Finally, we also intend to continue developing the NPNL Stroke Advisory Board by
testing new methods of engaging academic and community partners and evaluating
engagement. As we determine approaches that support engagement and influence research
outcomes, we hope to develop resources for other investigators and community members
seeking to employ a similar CAB model.

110
Contributions to Occupational Science
The present studies provide insights into the contexts and situations in which stroke
survivors and clinicians use technologies for rehabilitation. We observed how multilevel
contextual factors in both clinical and home settings can facilitate or hinder the use of
rehabilitation technology; clinician decision-making about using novel interventions in practice;
the role of self-efficacy in technology use; and the variety of ways transactions between
clinicians, patients, contexts, and technology features can unfold, among other findings that
support our understanding about how humans use technology to solve problems in everyday
life. We also demonstrated practical applications of this contextual, situated knowledge about
the use of technology to promote engagement in recovery-promoting occupations. To this end,
we employed implementation science approaches to evaluate the potential for Tele-REINVENT
to be implemented in real-world contexts. Further, we demonstrated the mutually beneficial
value of bridging occupational science, implementation science, and technology development.
We anticipate that the integrated perspectives and approaches from these disciplines will
enable more thoughtful technology designs and foster greater engagement in occupation in
ways that fit the routines and contexts in which people live.
Conclusion
This dissertation responded to gaps in post-stroke rehabilitation by examining the
implementation context and pre-implementation outcomes of Tele-REINVENT, an EMG
biofeedback system for stroke survivors with moderate to severe hemiparesis. Through these
studies, we moved Tele-REINVENT from proof-of-concept to an intervention that we are
preparing to implement into post-stroke clinical practice. While there is still more preparation to
be done prior to implementation, we hope that this work will demonstrate the importance of (1)
stakeholder engagement and (2) evaluation of implementation outcomes in all phases of
technology-based intervention development.

111
Ultimately, this work aims to move the needle forward in providing underserved stroke
survivors with potentially life changing access to high-quality therapy. One clinician in our study
eloquently observed:
A system like Tele-REINVENT has great potential since the feedback is in real time to a
population of patients that were probably told they have achieved maximal level of
recovery. More important is the great psychological impact the system can have over
individuals that greatly depend on caregivers for simple tasks. Small achievements in
muscle control and limb movement can provide a pathway to great overall strides in
function and independence.
The possibility of such outcomes warrants the soundest theoretical and neuroscientific
foundation and strategic implementation process so that we can ensure Tele-REINVENT has its
intended reach and improves the recovery and engagement of stroke survivors.

112
References
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. C. (2011). Advancing a conceptual model of evidencebased practice implementation in public service sectors. Administration and Policy in
Mental Health and Mental Health Services Research, 38(1), 4–23.
https://doi.org/10.1007/s10488-010-0327-7
Alawieh, A., Zhao, J., & Feng, W. (2018). Factors affecting post-stroke motor recovery: Implications
on neurotherapy after brain injury. Behavioural Brain Research, 340, 94–101.
https://doi.org/10.1016/j.bbr.2016.08.029
Aldrich, R., & Marterella, A. (2014). Community-engaged research: A path for occupational science
in the changing university landscape. Journal of Occupational Science, 21(2), 210–225.
Alley, Z. M., Chapman, J. E., Schaper, H., & Saldana, L. (2023). The relative value of preimplementation stages for successful implementation of evidence-informed programs.
Implementation Science, 18(1), 1–13. https://doi.org/10.1186/s13012-023-01285-0
American Heart Association. (2019). Rehab Therapy After a Stroke. https://www.stroke.org/en/lifeafter-stroke/stroke-rehab/rehab-therapy-after-a-stroke
Appleby, E., Gill, S. T., Hayes, L. K., Walker, T. L., Walsh, M., & Kumar, S. (2019). Effectiveness
of telerehabilitation in the management of adults with stroke: A systematic review. In PLoS
ONE (Vol. 14, Issue 11). Public Library of Science.
https://doi.org/10.1371/journal.pone.0225150
Arienti, C., Buraschi, R., Pollet, J., Lazzarini, S. G., Cordani, C., Negrini, S., & Gobbo, M. (2022).
A systematic review opens the black box of “usual care” in stroke rehabilitation control
groups and finds a black hole. European Journal of Physical and Rehabilitation Medicine,
58(4), 520–529. https://doi.org/10.23736/S1973-9087.22.07413-5
Ayala, C., Fang, J., Luncheon, C., Coleman King, S., Chang, T., Ritchey, M., & Loustalot, F. (2018).
Use of outpatient rehabilitation among adult stroke survivors - 20 states and the District of
Columbia, 2013, and four states, 2015. Morbidity and Mortality Weekly Report, 67(20),
575–578. https://www.cdc.gov/dhdsp/programs/
Bano, M., & Zowghi, D. (2015). A systematic review on the relationship between user involvement
and system success. Information and Software Technology 58, 148-69.
doi:10.1016/j.infsof.2014.06.011
Baranek, G. T., Frank, G., & Aldrich, R. M. (2020). Meliorism and knowledge mobilization:
Strategies for occupational science research and practice. Journal of Occupational
Science, 0(0), 1–13. https://doi.org/10.1080/14427591.2020.1824802
Barreca, S., Wolf, S. L., Fasoli, S., & Bohannon, R. (2003). Treatment interventions for the paretic
upper limb of stroke survivors: A critical review. Neurorehabilitation and Neural Repair,
17(4), 220–226. https://doi.org/10.1177/0888439003259415
Bauer, M. S., Damschroder, L., Hagedorn, H., Smith, J., & Kilbourne, A. M. (2015). An introduction
to implementation science for the non-specialist. BMC Psychology, 3(1).

113
https://doi.org/10.1186/S40359-015-0089-9
Black, M. H., Milbourn, B., Desjardins, K., Sylvester, V., Parrant, K., & Buchanan, A. (2019).
Understanding the meaning and use of occupational engagement: Findings from a scoping
review. British Journal of Occupational Therapy, 82(5), 272–287.
https://doi.org/10.1177/0308022618821580
Boden-Albala, B., Rebello, V., Drum, E., Gutierrez, D., Smith, W. R., Whitmer, R. A., & Griffith, D.
M. (2023). Use of community-engaged research approaches in clinical interventions for
neurologic disorders in the United States: A scoping review and future directions for
improving health equity research. Neurology, 101(7), S27–S46.
https://doi.org/10.1212/WNL.0000000000207563
Bower, K. J., Verdonck, M., Hamilton, A., Williams, G., Tan, D., & Clark, R. A. (2021). What factors
influence clinicians’ use of technology in neurorehabilitation? A multisite qualitative study.
Physical Therapy, 101(5), 1–9. https://doi.org/10.1093/ptj/pzab031
Broens, T. H. F., Huis in ’t Veld, M. H. A., Vollenbroek-Hutten, M. M. R., Hermens, H. J., van
Halteren, A. T., & Nieuwenhuis, L. J. . (2007). Determinants of successful telemedicine
implementations: A literature study. Journal of Telemedicine and Telecare, 13(6).
https://doi.org/https://doi.org/10.1258/135763307781644951
Brown, K., El Husseini, N., Grimley, R., Ranta, A., Kass-Hout, T., Kaplan, S., & Kaufman, B. G.
(2022). Alternative payment models and associations with stroke outcomes, spending, and
service utilization: A systematic review. Stroke, 53(1), 268–278.
https://doi.org/10.1161/STROKEAHA.121.033983
Brownson, R. C., Kumanyika, S. K., Kreuter, M. W., & Haire-Joshu, D. (2021). Implementation
science should give higher priority to health equity. Implementation Science, 16(1), 1–16.
https://doi.org/10.1186/s13012-021-01097-0
Cahill, L. S., Lannin, N. A., Purvis, T., Cadilhac, D. A., Mak-Yuen, Y., O’Connor, D. A., & Carey, L.
M. (2022). What is “usual care” in the rehabilitation of upper limb sensory loss after stroke?
Results from a national audit and knowledge translation study. Disability and Rehabilitation,
44(21), 6462–6470. https://doi.org/10.1080/09638288.2021.1964620
Carcel, C., & Reeves, M. (2021). Under-enrollment of women in stroke clinical trials: What are the
causes and what should be done about it? Stroke, 52, 452–457.
https://doi.org/10.1161/STROKEAHA.120.033227
Carlson, M., Park, D. J., Kuo, A., & Clark, F. (2014). Occupation in relation to the self. Journal of
Occupational Science, 21(2), 117–129. https://doi.org/10.1080/14427591.2012.727356
Cason, J. (2017). Telehealth is face-to-face service delivery. International Journal of
Telerehabilitation, 9(1), 77–78. https://doi.org/10.5195/ijt.2017.6225
Caughlin, S., Mehta, S., Corriveau, H., Eng, J. J., Eskes, G., Kairy, D., Meltzer, J., Sakakibara, B.
M., & Teasell, R. (2020). Implementing telerehabilitation after stroke: Lessons learned from
Canadian trials. Telemedicine and E-Health, 26(6), 710–719.
https://doi.org/10.1089/tmj.2019.0097

114
Celian, C., Swanson, V., Shah, M., Newman, C., Fowler-King, B., Gallik, S., Reilly, K.,
Reinkensmeyer, D. J., Patton, J., & Rafferty, M. R. (2021). A day in the life: A qualitative
study of clinical decision-making and uptake of neurorehabilitation technology. Journal of
NeuroEngineering and Rehabilitation, 18(1), 1–12. https://doi.org/10.1186/s12984-021-
00911-6
Cervera, M. A., Soekadar, S. R., Ushiba, J., Millán, J. del R., Liu, M., Birbaumer, N., & Garipelli,
G. (2018). Brain-computer interfaces for post-stroke motor rehabilitation: A meta-analysis.
Annals of Clinical and Translational Neurology, 5(5), 651–663.
https://doi.org/10.1002/acn3.544
Chan, A. H. (2021). Logistics of rehabilitation telehealth: Documentation, reimbursement, and
Health Insurance Portability and Accountability Act. Physical Medicine and Rehabilitation
Clinics of North America, 32(2), 429–436. https://doi.org/10.1016/j.pmr.2021.01.006
Chang, W. H., & Kim, Y.-H. (2013). Robot-assisted therapy in stroke rehabilitation. Journal of
Stroke, 15(3), 174–181. https://doi.org/10.1143/jjap.38.4868
Chao, C.-M. (2019). Factors determining the behavioral intention to use mobile learning: An
application and extension of the UTAUT model. Frontiers in Psychology.
https://doi.org/10.3389/fpsyg.2019.01652
Chen, C. C., & Bode, R. K. (2011). Factors influencing therapists’ decision-making in the
acceptance of new technology devices in stroke rehabilitation. American Journal of
Physical Medicine and Rehabilitation, 90(5), 415–425.
https://doi.org/10.1097/PHM.0b013e318214f5d8
Chen, Y., Chen, Y., Zheng, K., Dodakian, L., See, J., Zhou, R., Chiu, N., Augsburger, R.,
McKenzie, A., & Cramer, S. C. (2020). A qualitative study on user acceptance of a homebased stroke telerehabilitation system. Topics in Stroke Rehabilitation, 27(2), 81–92.
https://doi.org/10.1080/10749357.2019.1683792
Cheung, J., Rancourt, A., Poce, S. Di, Levine, A., Hoang, J., Ismail, F., Boulias, C., & Phadke, C.
P. (2015). Patient-identified factors that influence spasticity in people with stroke and
multiple sclerosis receiving Botulinum Toxin injection treatments. Physiotherapy Canada,
67(2), 157–166. https://doi.org/10.3138/ptc.2014-07
Clark, F., & Lawlor, M. C. (2009). The making and mattering of occupational science. In Willard &
Spackman’s Occupational Therapy: International Edition (pp. 2–14).
Clark, F., Parham, D., Carlson, M. E., Frank, G., Jackson, J., Pierce, D., Wolfe, R. J., & Zemke, R.
(1991a). Occupational science. 45(4).
Clark, F., Parham, D., Carlson, M., Frank, G., Jackson, J., Pierce, D., Wolfe, R. J., & Zemke, R.
(1991b). Occupational science: Academic innovation in the service of occupational
therapy’s f uture.
Coscia, M., Wessel, M. J., Chaudary, U., Millán, J. del R., Micera, S., Guggisberg, A., Vuadens,
P., Donoghue, J., Birbaumer, N., & Hummel, F. C. (2019). Neurotechnology-aided
interventions for upper limb motor rehabilitation in severe chronic stroke. Brain, 142(8),
2182–2197. https://doi.org/10.1093/brain/awz181

115
Craig, P., Dieppe, P., Macintyre, S., Mitchie, S., Nazareth, I., & Petticrew, M. (2008). Developing
and evaluating complex interventions: The new Medical Research Council guidance. Bmj,
337(7676), 979–983. https://doi.org/10.1136/bmj.a1655
Cramer, S. C., Dodakian, L., Le, V., McKenzie, A., See, J., Augsburger, R., Zhou, R. J., Raefsky,
S. M., Nguyen, T., Vanderschelden, B., Wong, G., Bandak, D., Nazarzai, L., Dhand, A.,
Scacchi, W., & Heckhausen, J. (2021). A feasibility study of expanded home-based
telerehabilitation after stroke. Frontiers in Neurology, 11.
https://doi.org/10.3389/fneur.2020.611453
Cramer, S. C., Dodakian, L., Le, V., See, J., Augsburger, R., McKenzie, A., Zhou, R. J., Chiu, N.
L., Heckhausen, J., Cassidy, J. M., Scacchi, W., Smith, M. T., Barrett, A. M., Knutson, J.,
Edwards, D., Putrino, D., Agrawal, K., Ngo, K., Roth, E. J., … Janis, S. (2019). Efficacy of
home-based telerehabilitation vs in-clinic therapy for adults after stroke: A randomized
clinical trial. Journal of the American Medical Association Neurology, 76(9), 1079–1087.
https://doi.org/10.1001/jamaneurol.2019.1604
Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012). Effectivenessimplementation hybrid designs: Combining elements of clinical effectiveness and
implementation research to enhance public health impact. Med Care, 50(3), 217–226.
https://doi.org/doi:10.1097/MLR.0b013e3182408812
Curtis, S., Sheehan, L., Buchman, E., & Bhattacharjya, S. (2023). Clinicians’ perspectives and
usage of rehabilitation technology: A survey. Disability and Rehabilitation: Assistive
Technology, 0(0), 1–8. https://doi.org/10.1080/17483107.2023.2284365
Cutchin, M. P., & Dickie, V. A. (2013). Transactional perspectives on occupation. In A. Cutchin,
Malcom P; Dickie, Virginia (Ed.), Transactional Perspectives on Occupation (Vol.
9789400744). Springer. https://doi.org/10.1007/978-94-007-4429-5
Dahl-Popolizio, S., Carpenter, H., Coronado, M., Popolizio, N. J., & Swanson, C. (2020).
Telehealth for the provision of occupational therapy: Reflections on experiences during the
COVID-19 pandemic. International Journal of Telerehabilitation, 2.
https://doi.org/10.5195/ijt.2020.6328
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009).
Fostering implementation of health services research findings into practice: A consolidated
framework for advancing implementation science. Implementation Science, 4(1), 1–15.
https://doi.org/10.1186/1748-5908-4-50
Davidson, J. L., & Jensen, C. (2013). Participatory design with older adults: An analysis of creativity
in the design of mobile healthcare applications. Proceedings of the 9th ACM Conference
on Creativity & Cognition, 114–123. https://doi.org/10.1145/2466627.2466652
Dobe, J., Gustafsson, L., & Walder, K. (2022). Co-creation and stroke rehabilitation: a scoping
review. Disability and Rehabilitation, 0(0), 1–13.
https://doi.org/10.1080/09638288.2022.2032411
Donnelly, M. R., Marin-pardo, O., Abdullah, A., Phanord, C., Kumar, A., Chakraborty, S., & Liew,
S. (2024). Pre-implementation analysis of the usability and acceptability of a poststroke
complex telehealth biofeedback intervention. American Journal of Occupational Therapy,

116
78(2), 1–12.
Donnelly, M. R., Phanord, C. S., Marin-Pardo, O., Jeong, J., Bladon, B., Wong, K., Abdullah, A., &
Liew, S. L. (2023). Acceptability of a telerehabilitation biofeedback system among stroke
survivors: A qualitative analysis. OTJR Occupation, Participation and Health, 43(3), 549–
557. https://doi.org/10.1177/15394492231153998
Donoso Brown, E. V., McCoy, S. W., Fechko, A. S., Price, R., Gilbertson, T., & Moritz, C. T. (2014).
Preliminary investigation of an electromyography-controlled video game as a home
program for persons in the chronic phase of stroke recovery. Archives of Physical Medicine
and Rehabilitation, 95(8), 1461–1469. https://doi.org/10.1016/J.APMR.2014.02.025
Dopp, A. R., Parisi, K. E., Munson, S. A., & Lyon, A. R. (2019). A glossary of user-centered design
strategies for implementation experts. Translational Behavioral Medicine, 9(6), 1057–1064.
https://doi.org/10.1093/tbm/iby119
Dukes, P. S., Hayes, A., Hodges, L. F., & Woodbury, M. (2013). Punching ducks for post-stroke
neurorehabilitation: System design and Initial exploratory feasibility study. IEEE
Symposium on 3D User Interfaces, 47–54. https://doi.org/10.1109/3DUI.2013.6550196
Eccles, M. P., & Mittman, B. S. (2006). Welcome to implementation science. In Implementation
Science (Vol. 1, Issue 1). https://doi.org/10.1186/1748-5908-1-1
Edward, H., Constantin, N., Ng, H., Radisic, A., D’Asti, A., Yule, A., D’Amore, C., Reid, J. C., &
Beauchamp, M. (2021). The use of co-design in developing physical activity interventions
for older adults: a scoping review protocol. JBI Evidence Synthesis, 20(2), 696–707.
https://doi.org/10.11124/JBIES-21-00061
Ellis, J., Band, R., Kinsella, K., Cheetham-Blake, T., James, E., Ewings, S., & Rogers, A. (2020).
Optimising and profiling pre-implementation contexts to create and implement a public
health network intervention for tackling loneliness. Implementation Science, 15(1), 1–11.
https://doi.org/10.1186/s13012-020-00997-x
Elnady, A., Ben Mortenson, W., & Menon, C. (2018). Perceptions of existing wearable robotic
devices for upper extremity and suggestions for their development: Findings from
therapists and people with stroke. JMIR Rehabilitation and Assistive Technologies, 20(5).
https://doi.org/10.2196/rehab.9535
Emmerson, K. B., Harding, K. E., & Taylor, N. F. (2017). Home exercise programmes supported
by video and automated reminders compared with standard paper-based home exercise
programmes in patients with stroke: A randomized controlled trial. Clinical Rehabilitation,
31(8), 1068–1077. https://doi.org/10.1177/0269215516680856
English, C., Ceravolo, M. G., Dorsch, S., Drummond, A., Gandhi, D. B. C., Halliday Green, J.,
Schelfaut, B., Verschure, P., Urimubenshi, G., & Savitz, S. (2022). Telehealth for
rehabilitation and recovery after stroke: State of the evidence and future directions.
International Journal of Stroke, 17(5), 487–493.
https://doi.org/10.1177/17474930211062480
Evans, S. C., Roberts, M. C., Keeley, J. W., Blossom, J. B., Amaro, C. M., Garcia, A. M., Stough,
C. O., Canter, K. S., Robles, R., & Reed, G. M. (2015). Vignette methodologies for studying

117
clinicians’ decision-making: Validity, utility, and application in ICD-11 field studies.
International Journal of Clinical and Health Psychology, 15(2), 160–170.
https://doi.org/10.1016/j.ijchp.2014.12.001
Everard, G., Declerck, L., Detrembleur, C., Leonard, S., Bower, G., Dehem, S., & Lejeune, T.
(2022). New technologies promoting active upper limb rehabilitation after stroke: an
overview and network meta-analysis. European Journal of Physical and Rehabilitation
Medicine, 58(4), 530–548. https://doi.org/10.23736/S1973-9087.22.07404-4
Feigin, V. L., Owolabi, M. O., Owolabi, M. O., Owolabi, M. O., Feigin, V. L., Abd-Allah, F., Akinyemi,
R. O., Bhattacharjee, N. V., Brainin, M., Cao, J., Caso, V., Dalton, B., Davis, A., Dempsey,
R., Duprey, J., Feng, W., Ford, G. A., Gall, S., Gandhi, D., … Zagożdżon, P. (2023).
Pragmatic solutions to reduce the global burden of stroke: a World Stroke Organization–
Lancet Neurology Commission. The Lancet Neurology, 22(12), 1160–1206.
https://doi.org/10.1016/S1474-4422(23)00277-6
Feldner, H. A., Howell, D., Kelly, V. E., McCoy, S. W., & Steele, K. M. (2019). “Look, your muscles
are firing!”: A qualitative study of clinician perspectives on the use of surface
electromyography in neurorehabilitation. Archives of Physical Medicine and Rehabilitation,
100(4), 663–675. https://doi.org/10.1016/j.apmr.2018.09.120
Fischer, B., Peine, A., & Östlund, B. (2020). The importance of user involvement: A Systematic
review of involving older adults in technology design. The Gerontologist, 60(7), e513-523.
10.1093/geront/gnz163
Fong, K. N. K., Tang, Y. M., Sie, K., Yu, A. K. H., Lo, C. C. W., & Ma, Y. W. T. (2021). Task-specific
virtual reality training on hemiparetic upper extremity in patients with stroke. Virtual Reality,
1. https://doi.org/10.1007/s10055-021-00583-6
Forman, J., & Damschroder, L. (2008). Qualitative content analysis. In L. Jacoby & L. A. Siminoff
(Eds.), Empirical methods for bioethics: A primer (1st ed., pp. 50–62). JAI Press.
French, B., Thomas, L. H., Coupe, J., Mcmahon, N. E., Connell, L., Harrison, J., Sutton, C. J.,
Tishkovskaya, S., & Watkins, C. L. (2016). Repetitive task training for improving functional
ability after stroke (Review).
https://doi.org/10.1002/14651858.CD006073.pub3.www.cochranelibrary.com
Giggins, O. M., Persson, M., & Caulfield, B. (2013). Biofeedback in rehabilitation. Journal of
NeuroEngineering and Rehabilitation, 10, 1. https://doi.org/10.1186/1743-0003-10-60
Goodman, M. S., Ackermann, N., Pierce, K. A., Bowen, D. J., & Thompson, V. S. (2021).
Development and validation of a brief version of the research engagement survey tool.
International Journal of Environmental Research and Public Health, 18(19).
https://doi.org/10.3390/ijerph181910020
Goodman, M. S., & Sanders Thompson, V. L. (2017). The science of stakeholder engagement in
research: classification, implementation, and evaluation. Translational Behavioral
Medicine, 7(3), 486–491. https://doi.org/10.1007/s13142-017-0495-z
Goodman, M. S., Thompson, V. L. S., Arroyo Johnson, C., Gennarelli, R., Drake, B. F., Bajwa, P.,
Witherspoon, M., & Bowen, D. (2017). Evaluating community engagement in research:

118
Quantitative measure development. Journal of Community Psychology, 45(1), 17–32.
https://doi.org/10.1002/jcop.21828
Goodrich, D. E., Miake-Lye, I., & Braganza, M. Z. (2020). The QUERI Roadmap for Implementation
and Quality Improvement. https://www.ncbi.nlm.nih.gov/books/NBK566216/
Halladay, J. R., Donahue, K. E., Sleath, B., Reuland, D., Black, A., Mitchell, C. M., Breland, C. E.,
Coyne-Beasley, T., Mottus, K., Watson, S. N., Lewis, V., Wynn, M., & Corbie-Smith, G.
(2017). Community advisory boards guiding engaged research efforts within a clinical
translational sciences award: Key contextual factors explored. Progress in Community
Health Partnerships: Research, Education, and Action, 11(4), 367–377.
https://doi.org/10.1353/cpr.2017.0044
Harrington, C. N., Wilcox, L., Connelly, K., Rogers, W., & Sanford, J. (2018). Designing health and
fitness apps with older adults: Examining the value of experience-based co-design. ACM
International Conference Proceeding Series, 15–24.
https://doi.org/10.1145/3240925.3240929
Hatcher, M., Warner, D., & Hornbrook, M. (2011). Managing organizational support for community
engagement. In CTSA Community Engagement Key Function Committee Task Force on
Principles of Community Engagement (Ed.), Principles of Community Engagement (2nd
ed., pp. 93–106).
https://www.atsdr.cdc.gov/communityengagement/pdf/PCE_Report_508_FINAL.pdf
Haywood, C., Martinez, G., Pyatak, E. A., & Carandang, K. (2019). Engaging patient stakeholders
in planning, implementing, and disseminating occupational therapy research. American
Journal of Occupational Therapy, 73(1), 1–9. https://doi.org/10.5014/ajot.2019.731001
Hildebrand, M. W., Geller, D., & Proffitt, R. (2023). Occupational therapy practice guidelines for
adults with stroke. The American Journal of Occupational Therapy, 77(5).
https://doi.org/10.5014/ajot.2023.077501
Hocking, C. (2021). Occupation in context: A reflection on environmental influences on human
doing. Journal of Occupational Science, 28(2), 221–234.
https://doi.org/10.1080/14427591.2019.1708434
Howe, M. C., & Briggs, A. K. (1982). Ecological systems model for occupational therapy. American
Journal of Occupational Therapy, 36(5), 322–327. https://doi.org/10.5014/ajot.36.5.322
Hughes, A. M., Burridge, J. H., Demain, S. H., Ellis-Hill, C., Meagher, C., Tedesco-Triccas, L.,
Turk, R., & Swain, I. (2014). Translation of evidence-based assistive technologies into
stroke rehabilitation: Users’ perceptions of the barriers and opportunities. BMC Health
Services Research, 14(1). https://doi.org/10.1186/1472-6963-14-124
Hung, L. Y., Lyons, J. G., & Wu, C. H. (2020). Health information technology use among older
adults in the United States, 2009–2018. Current Medical Research and Opinion, 36(5),
789–797. https://doi.org/10.1080/03007995.2020.1734782
Hursting, L. M., & Chambers, D. A. (2021). Practitioner engagement in implementation science:
Initiatives and opportunities. Journal of Public Health Management and Practice, 27(2),
102–104. https://doi.org/10.1097/PHH.0000000000001222

119
Interaction Design Foundation. (2016). What is user centered design (UCD)?
https://www.interaction-design.org/literature/topics/user-centered-design
Jackson, J., Carlson, M., Zemke, R., & Clark, F. (1998). Occupation in lifestyle redesign: The Well
Elderly Study occupational therapy program. American Journal of Occupational Therapy,
52(5), 326–336.
Jafni, T. I., Bahari, M., Ismail, W., & Radman, A. (2017). Understanding the implementation of
telerehabilitation at pre-implementation stage: A systematic literature review. Procedia
Computer Science, 124, 452–460. https://doi.org/10.1016/J.PROCS.2017.12.177
Jang, S. H., & Jang, W. H. (2016). The effect of a finger training application using a tablet PC in
chronic hemiparetic stroke patients. Somatosensory and Motor Research, 33(2), 124–129.
https://doi.org/10.1080/08990220.2016.1197117
Janssen, J., Klassen, T. D., Connell, L. A., & Eng, J. J. (2020). Factors influencing the delivery of
intensive rehabilitation in stroke: Patient perceptions versus rehabilitation therapist
perceptions. Physical Therapy, 100(2), 307–316. https://doi.org/10.1093/ptj/pzz159
Jilke, S. (2021). Measuring technological uncertainty and technological complexity: Scale
development and an assessment of reliability and validity. International Journal of
Innovation Science, 13(3), 381–400. https://doi.org/10.1108/IJIS-08-2020-0120
Jones, M., Deruyter, F., & Morris, J. (2020). The digital health revolution and people with
disabilities: Perspective from the United States. International Journal of Environmental
Research and Public Health, 17(381). https://doi.org/10.3390/ijerph17020381
Juckett, L. A., Wengerd, L. R., Faieta, J., & Griffin, C. E. (2020). Evidence-based practice
implementation in stroke rehabilitation: A scoping review of barriers and facilitators.
American Journal of Occupational Therapy, 74(1), 1–14.
https://doi.org/10.5014/ajot.2020.035485
Kerkhoff, A. D., Farrand, E., Marquez, C., Cattamanchi, A., & Handley, M. A. (2022). Addressing
health disparities through implementation science—a need to integrate an equity lens from
the outset. Implementation Science, 17(1), 1–4. https://doi.org/10.1186/s13012-022-
01189-5
Kerr, A., Smith, M., Reid, L., & Baillie, L. (2018). Adoption of stroke rehabilitation technologies by
the user community: Qualitative study. JMIR Rehabilitation and Assistive Technologies,
20(8). https://doi.org/10.2196/rehab.9219
Khoury, M. J., Gwinn, M., Yoon, P. W., Dowling, N., Moore, C. A., & Bradley, L. (2007). The
continuum of translation research in genomic medicine: How can we accelerate the
appropriate integration of human genome discoveries into health care and disease
prevention? Genetics in Medicine, 9(10), 665–674.
https://doi.org/10.1097/GIM.0b013e31815699d0
Kim, J.-H. (2017). The effects of training using EMG biofeedback on stroke patients upper
extremity functions. Journal of Physical Therapy Science, 29, 1085–1088.

120
Knepley, K. D., Mao, J. Z., Wieczorek, P., Okoye, F. O., Jain, A. P., & Harel, N. Y. (2021). Impact
of telerehabilitation for stroke-related deficits. Telemedicine and E-Health, 27(3), 239–246.
https://doi.org/10.1089/tmj.2020.0019
Kwakkel, G., Kollen, B. J., Van der Grond, J. V., & Prevo, A. J. H. (2003). Probability of regaining
dexterity in the flaccid upper limb: Impact of severity of paresis and time since onset in
acute stroke. Stroke, 34(9), 2181–2186.
https://doi.org/10.1161/01.STR.0000087172.16305.CD
Lang, C. E., Lohse, K. R., & Birkenmeier, R. L. (2015). Dose and timing in neurorehabilitation:
Prescribing motor therapy after stroke. Curr Opin Neurology, 28(6), 549–555.
https://doi.org/doi:10.1097/WCO.0000000000000256
Lang, C. E., MacDonald, J. R., Reisman, D. S., Boyd, L., Jacobson Kimberley, T., Schindler-Ivens,
S. M., Hornby, T. G., Ross, S. A., & Scheets, P. L. (2009). Observation of amounts of
movement practice provided during stroke rehabilitation. Archives of Physical Medicine and
Rehabilitation, 90(10), 1692–1698. https://doi.org/10.1016/j.apmr.2009.04.005
Lang, C. E., Strube, M. J., Bland, M. D., Waddell, K. J., Cherry-Allen, K. M., Nudo, R. J., Dromerick,
A. W., & Birkenmeier, R. L. (2016). Dose response of task-specific upper limb training in
people at least 6 months poststroke: A phase II, single-blind, randomized, controlled trial.
Annals of Neurology, 80(3), 342–354. https://doi.org/10.1002/ana.24734
Laver, K., Lange, B., George, S., Deutsch, J. E., Saposnik, G., & Crotty, M. (2017). Virtual reality
for stroke rehabilitation. Cochrane Database of Systematic Reviews, 11.
https://doi.org/10.1002/14651858.CD008349.pub4
Laver, K., & Osborne, K. (2022). Telerehabilitation in stroke. In Telerehabilitation: Principles and
Practice (pp. 43–57). https://doi.org/10.1016/B978-0-323-82486-6.00004-6
Leppin, A. L., Mahoney, J. E., Stevens, K. R., Bartels, S. J., Baldwin, L.-M., Dolor, R. J., Proctor,
E. K., Scholl, L., Moore, J. B., Baumann, A. A., Rohweder, C. L., Luby, J., & Meissner, P.
(2020). Situating dissemination and implementation sciences within and across the
translational research spectrum. Journal of Clinical and Translational Science, 4(3), 152–
158. https://doi.org/10.1017/cts.2019.392
Lewis, J. R. (2014). Usability: Lessons learned and yet to be learned. International Journal of
Human-Computer Interaction, 30(9), 663–684.
https://doi.org/10.1080/10447318.2014.930311
Li, X. (2016). Understanding upper extremity home programs and the use of gaming technology
for persons after stroke. Physiology & Behavior, 176(3), 139–148.
https://doi.org/10.1016/j.dhjo.2015.03.007.Understanding
Lirio-Romero, C., Torres-Lacomba, M., Gómez-Blanco, A., Acero-Cortés, A., Retana-Garrido, A.,
de la Villa-Polo, P., & Sánchez-Sánchez, B. (2021). Electromyographic biofeedback
improves upper extremity function: a randomized, single-blinded, controlled trial.
Physiotherapy (United Kingdom), 110, 54–62. https://doi.org/10.1016/j.physio.2020.02.002
Lohse, K. R., Lang, C. E., & Boyd, L. A. (2014). Is more better? Using metadata to explore doseresponse relationships in stroke rehabilitation. Stroke, 45(7), 2053–2058.

121
https://doi.org/10.1161/STROKEAHA.114.004695
Lu, E. C., Wang, R. H., Hebert, D., Boger, J., Galea, M. P., & Mihailidis, A. (2011). The
development of an upper limb stroke rehabilitation robot: Identification of clinical practices
and design requirements through a survey of therapists. Disability and Rehabilitation:
Assistive Technology, 6(5), 420–431. https://doi.org/10.3109/17483107.2010.544370
Luger, T. M., Hamilton, A. B., & True, G. (2020). Measuring community-engaged research
contexts, processes, and outcomes: A Mapping review. Milbank Quarterly, 98(2), 493–553.
https://doi.org/10.1111/1468-0009.12458
MacQueen, K. M., Bhan, A., Frohlich, J., Holzer, J., & Sugarman, J. (2015). Evaluating community
engagement in global health research: The need for metrics. BMC Medical Ethics, 16(1),
1–9. https://doi.org/10.1186/s12910-015-0033-9
Mahak, C., Shashi, Yashomati, Hemlata, Manisha, N., Sandhya, G., Dheeraj, K., Dhandapani, M.,
& Dhandapani, S. (2018). Assessment of utilization of rehabilitation services among stroke
survivors. Journal of Neurosciences in Rural Practice, 9(4), 461–467.
https://doi.org/10.4103/jnrp.jnrp_25_18
Malinowsky, C., Nygård, L., & Kottorp, A. (2011). Psychometric evaluation of a new assessment
of the ability to manage technology in everyday life. Scandinavian Journal of Occupational
Therapy, 18(1), 26–35. https://doi.org/10.3109/11038120903420606
Marin-Pardo, O. (2023). Development and Implementation of a Modular Muscle-Computer
Interface for Personalized Motor Rehabilitation After Stroke. University of Southern
California.
Marin-Pardo, O., Donnelly, M. R., Phanord, C. S., Wong, K., & Pan, J. (2022). Functional and
neuromuscular changes induced via a low-cost, muscle-computer interface for
telerehabilitation: A feasibility study in chronic stroke. Frontiers in Neuroergonomics, 3.
https://doi.org/10.3389/fnrgo.2022.1046695
Marin-Pardo, O., Phanord, C., Donnelly, M. R., Laine, C. M., & Liew, S. L. (2021). Development of
a low-cost, modular muscle–computer interface for at-home telerehabilitation for chronic
stroke. Sensors, 21(5), 1–15. https://doi.org/10.3390/s21051806
Markiewicz, K., Van Til, J. A., & Ijzerman, M. J. (2014). Medical devices early assessment
methods: Systematic literature review. International Journal of Technology Assessment in
Health Care, 30(2), 137–146. https://doi.org/10.1017/S0266462314000026
Marzano, G. (2017). Towards a new wave of telerehabilitation applications. Public Health Open
Access, 1(1). https://doi.org/10.23880/phoa-16000105
McCloskey, D. J., McDonald, M. A., Cook, J., Heurtin-Roberts, S., Updegrove, S., Sampson, D.,
Gutter, S., & Eder, M. (2011). Community engagement: Definitions and organizing
concepts from the literature. In CTSA Community Engagement Key Function Committee
Task Force on the Principles of Community Engagement (Second Edition) (Ed.), Principles
of Community Engagement (2nd ed., pp. 3–41).
https://www.atsdr.cdc.gov/communityengagement/pdf/PCE_Report_508_FINAL.pdf

122
McLean, S., Sheikh, A., Cresswell, K., Nurmatov, U., Mukherjee, M., Hemmi, A., & Pagliari, C.
(2013). The impact of telehealthcare on the quality and safety of care: A systematic
overview. PLoS ONE, 8(8). https://doi.org/10.1371/journal.pone.0071238
Mehrholz, J., Pohl, M., Platz, T., Kugler, J., & Elsner, B. (2018). Electromechanical and robotassisted arm training for improving activities of daily living, arm function, and arm muscle
strength after stroke. Cochrane Database of Systematic Reviews, 2018(9).
https://doi.org/10.1002/14651858.CD006876.pub5
Morrow, C. M., Johnson, E., Simpson, K. N., & Seo, N. J. (2021). Determining factors that influence
adoption of new post-stroke sensorimotor rehabilitation devices in the USA. IEEE
Transactions on Neural Systems and Rehabilitation Engineering, 29, 1213–1222.
https://doi.org/10.1109/TNSRE.2021.3090571
Moullin, J. C., Dickson, K. S., Stadnick, N. A., Albers, B., Nilsen, P., Broder-Fingert, S., Mukasa,
B., & Aarons, G. A. (2020). Ten recommendations for using implementation frameworks in
research and practice. Implementation Science Communications, 1(1).
https://doi.org/10.1186/s43058-020-00023-7
Mullangi, S., Agrawal, M., & Schulman, K. (2021). The COVID-19 pandemic— An opportune time
to update medical licensing opinion. JAMA - Journal of the American Medical Association,
181(3), 307–308. https://doi.org/10.1001/jama.2014.9809
Murrell, J. E., Pisegna, J. L., & Juckett, L. A. (2021). Implementation strategies and outcomes for
occupational therapy in adult stroke rehabilitation: A scoping review. Implementation
Science, 16(1). https://doi.org/10.1186/s13012-021-01178-0
National Institutes of Health. (2023). National Advisory Board on Medical Rehabilitation Research
(NABMRR).
Neibling, B. A., Jackson, S. M., Hayward, K. S., & Barker, R. N. (2021). Perseverance with
technology-facilitated home-based upper limb practice after stroke: A systematic mixed
studies review. Journal of NeuroEngineering and Rehabilitation, 18(43).
https://doi.org/10.1186/s12984-021-00819-1
Nilsen, P., & Bernhardsson, S. (2019). Context matters in implementation science: A scoping
review of determinant frameworks that describe contextual determinants for
implementation outcomes. BMC Health Services Research, 19(1), 1–21.
https://doi.org/10.1186/s12913-019-4015-3
North Carolina Translational and Clinical Sciences Institute. (n.d.). Equity in research community
& patient advisory board (CPAB).
https://tracs.unc.edu/index.php/services/engagement/cpab
Office of Disease Prevention and Health Promotion. (n.d.). Heart disease and stroke.
https://health.gov/healthypeople/objectives-and-data/browse-objectives/heart-diseaseand-stroke/increase-proportion-adult-stroke-survivors-who-participate-rehabilitationservices-hds-d05
Ono, T., Shindo, K., Kawashima, K., Ota, N., Ito, M., Ota, T., Mukaino, M., Fujiwara, T., Kimura,
A., Liu, M., & Ushiba, J. (2014). Brain-computer interface with somatosensory feedback

123
improves functional recovery from severe hemiplegia due to chronic stroke. Frontiers in
Neuroengineering, 7. https://doi.org/10.3389/fneng.2014.00019
Östlund, B., Fischer, B., Marshall, B., Dalmer, N., Fernandez-Ardévol, M., Garcia-Santesmases,
A., Lopez, D., Loos, E., Chang, F., Chen, X., Neven, L., Peine, A., Rosales, A., &
Kuoppamäki, S. (2020). Using academic work places to involve older people in the design
of digital applications. Presentation of a methodological framework to advance co-design
in later life. In Lecture Notes in Computer Science (including subseries Lecture Notes in
Artificial Intelligence and Lecture Notes in Bioinformatics): Vol. 12207 LNCS. Springer
International Publishing. https://doi.org/10.1007/978-3-030-50252-2_4
Ottenbacher, K. J., Karmarkar, A., Graham, J. E., Kuo, Y. F., Deutsch, A., Reistetter, T. A., Al Snih,
S., & Granger, C. V. (2014). Thirty-day hospital readmission following discharge from
postacute rehabilitation in fee-for-service medicare patients. Jama, 311(6), 604–614.
https://doi.org/10.1001/jama.2014.8
Patomella, A. H., Kottorp, A., & Nygård, L. (2013). Design and management features of everyday
technology that challenge older adults. British Journal of Occupational Therapy, 76(9),
390–398. https://doi.org/10.4276/030802213X13782044946229
Pereira, S., Graham, J. R., Shahabaz, A., Salter, K., N., F., Meyer, M., & Teasell, R. (2012).
Rehabilitation of individuals with severe stroke: Synthesis of best evidence and challenges
in implementation. Topics in Stroke Rehabilitation, 19(2), 122–131.
Pérez Jolles, M., Lengnick-hall, R., & Mittman, B. S. (2019). Core functions and forms of complex
health interventions: A patient-centered medical home illustration. Journal of General
Internal Medicine, 34(6), 1032–1038. https://doi.org/10.1007/s11606-018-4818-7
Pérez Jolles, M., Willging, C. E., Stadnick, N. A., Crable, E. L., Lengnick-Hall, R., Hawkins, J., &
Aarons, G. A. (2022). Understanding implementation research collaborations from a cocreation lens: Recommendations for a path forward. Frontiers in Health Services, 2.
https://doi.org/10.3389/frhs.2022.942658
Pfadenhauer, L. M., Gerhardus, A., Mozygemba, K., Lysdahl, K. B., Booth, A., Hofmann, B.,
Wahlster, P., Polus, S., Burns, J., Brereton, L., & Rehfuess, E. (2017). Making sense of
complexity in context and implementation: The Context and Implementation of Complex
Interventions (CICI) framework. Implementation Science, 12(1), 1–17.
https://doi.org/10.1186/s13012-017-0552-5
Phipps, D., Pepler, D., Craig, W., Cummings, J., & Cardinal, S. (2016). The Co-produced Pathway
to Impact Describes Knowledge Mobilization Processes. Journal of Community
Engagement and Scholarship, 9(1), 5.
Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013). Implementation strategies:
Recommendations for specifying and reporting. Implementation Science, 8(1), 1–11.
https://doi.org/10.1186/1748-5908-8-139
Proctor, E. K., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., &
Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions,
measurement challenges, and research agenda. Adm Policy Ment Health, 38(2), 65–76.
https://doi.org/10.1007/s10488-010-0319-7

124
Proffitt, R., Schwartz, J. K., Foreman, M., & Smith, R. O. (2019). Mass market technology research
and development. The American Journal of Occupational Therapy, 73(1), 1–6.
R Core Team. (2020). R: A language and environment for statistical computing.
Reith, T. P. (2018). Burnout in United States healthcare professionals: A narrative review. Cureus,
10(12). https://doi.org/10.7759/cureus.3681
Rizzuto, T. E., & Reeves, J. (2007). A multidisciplinary meta-analysis of human barriers to
technology implementation. Consulting Psychology Journal, 59(3), 226–240.
https://doi.org/10.1037/1065-9293.59.3.226
Roach, W. H., Bischoff, J. M., Dennis, B., Donalson, M., Dunbar, S. B., Hill, D., Ng-Osorio, J.,
Powers, S., & Wold, L. E. (2021). Lay stakeholders in science and research initiative: A
science advisory from the American Heart Association. In Circulation (Vol. 144, Issue 5,
pp. E96–E106). Lippincott Williams and Wilkins.
https://doi.org/10.1161/CIR.0000000000000999
Sainburg, R. L., Liew, S. L., Frey, S. H., & Clark, F. (2017). Promoting translational research among
movement science, occupational science, and occupational therapy. Journal of Motor
Behavior, 49(1), 1–7. https://doi.org/10.1080/00222895.2016.1271299
Sauro, J., & Lewis, J. R. (2016). Standardized usability questionnaires. In Quantifying the User
Experience (2nd ed.). Elsevier. https://doi.org/10.1016/B978-0-12-802308-2/00008-4
Shah, S. G. S., & Robinson, I. (2006). User involvement in healthcare technology development
and assessment: Structured literature review. International Journal of Health Care Quality
Assurance, 19(6), 498–513. https://doi.org/10.1108/09526860610687619
Sheridan, S., Schrandt, S., Forsythe, L., Hilliard, T. S., & Paez, K. A. (2017). The PCORI
engagement rubric: Promising practices for partnering in research. Annals of Family
Medicine, 15(2), 165–170. https://doi.org/10.1370/afm.2042
Simpson, L. A., Menon, C., Hodgson, A. J., Ben Mortenson, W., & Eng, J. J. (2021). Clinicians’
perceptions of a potential wearable device for capturing upper limb activity post-stroke: a
qualitative focus group study. Journal of NeuroEngineering and Rehabilitation, 18(1), 1–
10. https://doi.org/10.1186/s12984-021-00927-y
Skidmore, E. R., Dawson, D. R., Whyte, E. M., Butters, M. A., Dew, M. A., Grattan, E. S., Becker,
J. T., & Holm, M. B. (2014). Developing complex interventions: Lessons learned from a
pilot study examining strategy training in acute stroke rehabilitation. Clinical Rehabilitation,
28(4), 378–387. https://doi.org/10.1177/0269215513502799
Smith, V., Warty, R., Nair, A., Krishnan, S., Sursas, J. A., Da Silva Costa, F., Vollenhoven, B., &
Wallace, E. M. (2019). Defining the clinician’s role in early health technology assessment
during medical device innovation - A systematic review. BMC Health Services Research,
19(1), 1–14. https://doi.org/10.1186/s12913-019-4305-9
Spicer, R., Anglin, J., Krum, D. M., & Liew, S. L. (2017). REINVENT: A low-cost, virtual reality
brain-computer interface for severe stroke upper limb motor recovery. Proceedings - IEEE
Virtual Reality, 385–386. https://doi.org/10.1109/VR.2017.7892338

125
Staley, K. (2009). Exploring Impact: Public involvement in NHS, public health and social care
research. In October (Issue October).
Stephenson, A., Howes Id, S., Murphy, P. J., Deutsch, J. E., Stokes, M., Pedlow, K., &
Mcdonoughid, S. M. (2022). Factors influencing the delivery of telerehabilitation for stroke:
A systematic review. PLos ONE, 17(5). https://doi.org/10.1371/journal.pone.0265828
Swanson, V. A., Johnson, C., Zondervan, D. K., Bayus, N., McCoy, P., Ng, Y. F. J., Schindele, BS,
J., Reinkensmeyer, D. J., & Shaw, S. (2023). Optimized home rehabilitation technology
reduces upper extremity impairment compared to a conventional home exercise program:
A randomized, controlled, single-blind trial in subacute stroke. Neurorehabilitation and
Neural Repair, 37(1), 53–65. https://doi.org/10.1177/15459683221146995
Taub, E., Uswatte, G., Mark, V. W., & Morris, D. M. (2006). The learned nonuse phenomenon:
Implications for rehabilitation. Europa Medicophysica, 42(3), 241–255.
Tchero, H., Teguo, M. T., Lannuzel, A., & Rusch, E. (2018). Telerehabilitation for stroke survivors:
Systematic review and meta-analysis. Journal of Medical Internet Research, 20(10), 1–10.
https://doi.org/10.2196/10867
Teasell, R., Pereira, S., & Cotoi, A. (2018). The rehabilitation of severe stroke. Evidence-Based
Review of Stroke Rehabilitation.
Thieme, H., Morkisch, N., Mehrholz, J., Pohl, M., Behrens, J., Borgetto, B., & Dohle, C. (2018).
Mirror therapy for improving motor function after stroke. Cochrane Database of Systematic
Reviews, 2018(7). https://doi.org/10.1002/14651858.CD008449.pub3
Towfighi, A., & Ovbiagele, B. (2022). Health equity and actionable disparities in stroke: 2021
update. Stroke, 53(3), 636–642. https://doi.org/10.1161/strokeaha.122.035816
Tsao, C. W., Aday, A. W., Almarzooq, Z. I., Anderson, C. A. M., Arora, P., Avery, C. L., BakerSmith, C. M., Beaton, A. Z., Boehme, A. K., Buxton, A. E., Commodore-Mensah, Y., Elkind,
M. S. V., Evenson, K. R., Eze-Nliam, C., Fugar, S., Generoso, G., Heard, D. G., Hiremath,
S., Ho, J. E., … Martin, S. S. (2023). Heart disease and stroke statistics - 2023 update: A
report from the American Heart Association. In Circulation (Vol. 147, Issue 8).
https://doi.org/10.1161/CIR.0000000000001123
Turner, A. C., & Etherton, M. R. (2022). Utilization of telestroke prior to and following the COVID19 pandemic. Seminars in Neurology, 42(1), 3–11. https://doi.org/10.1055/s-0041-1742181
Van Den Eede, Y. (2014). Extending “extension”: A reappraisal of the technology-as-extension
idea through the case of self-tracking technologies. In D. M. Weiss, A. D. Propen, & C. E.
Reid (Eds.), Design, Mediation, and the Posthuman (pp. 151–172). Lexington Books.
Vaughn, L. M., Whetstone, C., Boards, A., Busch, M. D., Magnusson, M., & Määttä, S. (2018).
Partnering with insiders: A review of peer models across community-engaged research,
education and social care. Health and Social Care in the Community, 26(6), 769–786.
https://doi.org/10.1111/hsc.12562
Verma, A., Towfighi, A., Brown, A., Abhat, A., & Casillas, A. (2022). Moving towards equity with
digital health innovations for stroke care. Stroke, 53(3), 689–697.

126
https://doi.org/10.1161/strokeaha.121.035307
Vourvopoulos, A., Pardo, O. M., Lefebvre, S., Neureither, M., Saldana, D., Jahng, E., & Liew, S.-
L. (2019). Effects of a brain-computer interface with virtual reality (VR) neurofeedback: A
pilot study in chronic stroke patients. Frontiers in Human Neuroscience, 13, 1–17.
https://doi.org/10.3389/fnhum.2019.00210
Waddell, K. J., Birkenmeier, R. L., Moore, J. L., Hornby, T. G., & Lang, C. E. (2014). Feasibility of
high-repetition, task-specific training for individuals with upper-extremity paresis. American
Journal of Occupational Therapy, 68(4).
Wale, J., Thomas, S., Hamerljnck, D., & Hollander R. (2021). Patients and public are important
stakeholders in health technology assessment but the level of involvement is low – A call
to action. Research Involvement and Engagement, 7(1). https://doi.org/10.1186/s40900-
020-00248-9
Weber, G. M. (2013). Identifying translational science within the triangle of biomedicine. Journal of
Translational Medicine, 11(1), 1–10. https://doi.org/10.1186/1479-5876-11-126
Weiner, B. J., Amick, H., Lee, S. D., & Lee, S. D. (2008). Review measurement of organizational
services research and other fields. In Medical Care Research and Review (Vol. 65, Issue
4). https://doi.org/10.1177/1077558708317802
Weiner, B. J., Lewis, C. C., Stanick, C., Powell, B. J., Dorsey, C. N., Clary, A. S., Boynton, M. H.,
& Halko, H. (2017). Psychometric assessment of three newly developed implementation
outcome measures. Implementation Science, 12(1), 1–12. https://doi.org/10.1186/s13012-
017-0635-3
Whiteford, G. E., & Hocking, C. (Eds.). (2012). Occupational Science: Society, Inclusion,
Participation. Wiley-Blackwell.
Winstein, C. J., Miller, J. P., Blanton, S., Taub, E., Uswatte, G., Morris, D., Nichols, D., & Wolf, S.
(2003). Methods for a multisite randomized trial to investigate the effect of constraintinduced movement therapy in improving upper extremity function among adults recovering
from a cerebrovascular stroke. Neurorehabilitation and Neural Repair, 17(3), 137–152.
https://doi.org/10.1177/0888439003255511
Winstein, C. J., Stein, J., Arena, R., Bates, B., Cherney, L. R., Cramer, S. C., Deruyter, F., Eng, J.
J., Fisher, B., Harvey, R. L., Lang, C. E., MacKay-Lyons, M., Ottenbacher, K. J., Pugh, S.,
Reeves, M. J., Richards, L. G., Stiers, W., & Zorowitz, R. D. (2016). Guidelines for adult
stroke rehabilitation and recovery: A guideline for healthcare professionals from the
American Heart Association/American Stroke Association. In Stroke (Vol. 47, Issue 6).
https://doi.org/10.1161/STR.0000000000000098
Zettler, L. L., Speechley, M. R., Foley, N. C., Salter, K. L., & Teasell, R. W. (2010). A scale for
distinguishing efficacy from effectiveness was adapted and applied to stroke rehabilitation
studies. Journal of Clinical Epidemiology, 63(1), 11–18.
https://doi.org/10.1016/j.jclinepi.2009.06.007

127
APPENDIX A: QUESTIONNAIRE CONSTRUCTS AND REMOTE TRAINING PROTOCOL
Unified Theory of Acceptance and Use of Technology Questionnaire Constructs (Chao,
2019).
Effort Expectancy - the ease associated with using the system.
Performance Expectancy - an individual’s belief that the system helps to improve performance.
Perceived Enjoyment - the extent to which the activity of using a specific system is enjoyable,
aside from any performance consequences.
Satisfaction - general perceptions and attitudes of the experience, including features such as
support services.
Trust - perceptions or beliefs concerning reliability, trust, and integrity.
Self-efficacy - an individual’s belief that he/she possesses the aptitude and skills to succeed
when engaging in system-related tasks.
Perceived Risk - the likelihood of suffering a loss in the pursuit of using the system.
Behavioral Intention - the degree to which a person has formulated conscious plans regarding
whether to perform a specified future behavior related to system use.
Post-Study System Usability Questionnaire Constructs (Sauro & Lewis, 2016).
SysQual – ease of use, efficiency and productivity, comfortability, and simplicity of the system.
InfoQual – quality of system errors, ability to recover from errors, and the usefulness,
organization, and clarity of information provided by the system.
InterQual –any point of contact between the user and system (e.g., screens, sensors, games).
Remote Training Protocol
Participants were asked to complete 30, 1-hour, remote (at-home) training sessions with
Tele-REINVENT (Sessions 2-31). During the remote sessions, participants applied two
electromyography sensors to their hemiparetic forearm muscles, typically by themselves or
occasionally with help from a care partner. Then, they opened the Tele-REINVENT software on

128
the provided laptop computer and followed audiovisual prompts to complete a brief calibration.
This ensured that the system responded to participants’ muscle activity each day. Participants
then selected from a suite of arcade-style games that train isolated wrist flexion and extension
and discourage synergistic movements, which are common post-stroke. Approximately half of
the remote sessions were conducted via videoconferencing with a member of the research
team. The remaining sessions were conducted independently by the participants, though
members of the research team were typically available for ad hoc support via telephone. The
training protocol is described in additional detail by Marin-Pardo et al. (2022).

129
APPENDIX B: IMPLEMENTOR SURVEY
Link to video: https://www.youtube.com/watch?v=2BV0N8xcktU&feature=youtu.be

130
If yes, display:
If no, display:

131

132

133

134

135

136

137

138
APPENDIX C: NPNL STROKE ADVISORY BOARD MISSION, VISION, AND SCOPE
Mission
We bring our diverse experiences of stroke recovery to academic research with the goal of
enhancing the relevance of stroke research to the needs of our communities. To that end, we
connect individuals with lived experience of stroke and professionals to bidirectionally mobilize
knowledge.
Vision
We envision the NPNL Stroke Advisory Board as a resource hub connecting the SoCal stroke
survivor and caregiver community with the local stroke research community.
Scope
The NPNL Stroke Advisory Board engages in activities that promote knowledge sharing
between the stroke survivor/caregiver community and the stroke research community. As a
group with a presence in both communities, we are uniquely positioned to:
§ advise researchers conducting stroke recovery research,
§ support culturally relevant dissemination and uptake of research findings to communities
of stroke survivors/caregivers, and
§ encourage the stroke survivor community to participate in academic research.
To that end, the NPNL Stroke Advisory Board participates in projects such as:
1. Hosting advising sessions for academic investigators (faculty, students, staff)
conducting stroke recovery and rehabilitation research. This may take the form of
one-off consultation sessions/workshops/focus groups or longer-term consultative
services. In either case, the academic investigator (the advisee) presents specific
challenges or questions regarding any part of the research process to the Board
(advisors), including any necessary context for understanding the problem. Board
members review the information and provide recommendations based on their lived
experience and prior research experience.

139
2. Developing evidence-based educational resources to distribute within the stroke
survivor/caregiver community. Board members may identify research findings that
would be of use to the broader stroke community and develop educational materials that
are culturally relevant, including a plan for dissemination strategy. In other cases, an
academic investigator may present the Board with research findings and request
assistance in designing and disseminating culturally relevant educational materials. In
yet other cases, members of the stroke survivor community may approach the Board
and request information about a specific topic. The Board may address a wide range of
educational topics, such as stroke prevention, caregiver support resources, community
supports, rehabilitation opportunities, and new stroke therapies, to name a few. These
materials may be in the form of flyers, short videos, social media posts, or other forms as
determined by the Board and other relevant stakeholders.
3. Supporting recruitment efforts for academic stroke research studies. Board
members may connect academic researchers with community members who may be
prospective research participants. In some cases, there may be direct introduction, and
in other cases, Board members may share opportunities with peers in support groups,
community rehabilitation centers, community-based events, and other venues where
stroke survivors and their support networks gather.
4. Facilitating connection events between individuals with lived and professional
experience of stroke. The Board may gather other individuals with lived and/or
professional experience of stroke to exchange ideas, learn from each other, and build
connections. At the discretion of the Board and other relevant stakeholders, these
events may be virtual or in-person and one-time or recurring. Additionally, the specific
purpose, themes, and activities of each connection event will be determined by the
Board and other relevant stakeholders based on the needs of all communities
represented.

140
Structure
§ The Board is situated within the NPNL in the USC Division of Occupational Science and
Occupational Therapy. The Board may support some but not all activities of the NPNL
and may also support activities from other research laboratories conducting stroke
recovery research.
§ The Board convenes 2-3 times per year for official Board meetings, organized by the
Board facilitator and/or other members of the NPNL staff. During these meetings, we
evaluate opportunities and determine plans for the proceeding months.
§ Beyond the 2-3 official Board meetings, Board members participate in ad hoc
committees, projects, and activities at their discretion. Each member will choose how
they contribute to the Board efforts based on their availability, personal strengths, and
areas of interest.
§ The Board consults with other organizations and professionals as needed to advance
Board initiatives.

Abstract (if available)

Abstract Advanced rehabilitation technologies can address access and practice gaps that negatively affect the recovery potential and outcomes of stroke survivors living with severe impairment. However, despite major advancements in rehabilitation technology, many of these innovations fail to be integrated into clinical practice. This dissertation evaluates implementation and design outcomes of a game-based electromyography (EMG) intervention among key stakeholders in stroke rehabilitation, namely stroke survivors and therapists, in the early stage of implementation planning. We found that Tele-REINVENT is perceived as acceptable and feasible to stroke survivors and therapists, though there are various multi-level barriers to uptake in clinical practice that need to be addressed through careful implementation planning. We anticipate that applying these findings to the ongoing development and preparation of Tele-REINVENT will result in greater patient confidence and ease of using the system at home for high repetition movement practice and efficient clinician use within the fast-paced health care model in which stroke therapy is delivered. Additionally, we set the foundation for a long-term research-community partnership with stakeholders via a novel community advisory board model. Our collaboration with the NPNL Stroke Advisory Board will continue to enhance the design of ongoing pre-implementation work and ultimately support its translation into routine post-stroke care. The overarching goal of this dissertation is to examine factors that will influence the uptake of electromyography biofeedback into routine clinical practice in stroke rehabilitation, from the perspective of stakeholders who will be users or implementors. There is great value in engaging stakeholders in the design and implementation of technology-based interventions, especially as we seek to translate neuroscientific findings from the laboratory into innovations that can improve in the lives of stroke survivors.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Development and implementation of a modular muscle-computer interface for personalized motor rehabilitation after stroke

PDF

The effects of fast walking, biofeedback, and cognitive impairment on post-stroke gait

PDF

Learning reaching skills in non-disabled and post-stroke individuals

PDF

A framework for automated administration of post stroke assessment test

PDF

Exploring the intersection of occupational engagement and well-being in student musicians: A multi-method study

PDF

Exploring the effects of mindfulness on psychosocial factors for patients receiving hand therapy

PDF

Adoption and implementation of innovative diagnostic tools for Alzheimer's Disease: challenges and barriers in primary care

PDF

What do nursing home residents do? Exploring residents’ engagement in activities

PDF

Technology integration and its impact on 21st century learning and instruction: a case study

PDF

A system framework for evidence based implementations in a health care organization

PDF

Design and fabrication of ultrasound transducers: from single element to high frequency 2D array

PDF

Elements of a 1:1 computer laptop program in a Los Angeles County high school and implications for education leaders

PDF

Situated experiences: a qualitative study of day-to-day life and participation of adolescents and young adults with a spinal cord injury and their caregivers

PDF

A gap analysis of course directors’ effective implementation of technology-enriched course designs: An innovation study

PDF

Use of electronic health record data for generating clinical evidence: a summary of medical device industry views

PDF

Use of GIS for analysis of community health worker patient registries from Chongwe district, a rural low-resource setting, in Lusaka Province, Zambia

PDF

LGBT+Aging Immersion Experience: an innovative LGBT cultural competency course for healthcare professionals and students

PDF

Using a human factors engineering perspective to design and evaluate communication and information technology tools to support depression care and physical activity behavior change among low-inco...

PDF

Ubiquitous computing for human activity analysis with applications in personalized healthcare

PDF

A proposal for building envelope retrofit on the Bonaventure Hotel: a case study examining energy and carbon

Asset Metadata

Creator Donnelly, Miranda Rennie (author)

Core Title Feasibility, acceptability, and implementation context of a complex telerehabilitation intervention for post-stroke upper extremity recovery

School School of Dentistry

Degree Doctor of Philosophy

Degree Program Occupational Science

Degree Conferral Date 2024-05

Publication Date 04/03/2024

Defense Date 03/04/2024

Publisher Los Angeles, California (original), University of Southern California (original), University of Southern California. Libraries (digital)

Tag health technology,implementation science,OAI-PMH Harvest,Occupational Science,Rehabilitation,stroke,telehealth

Format theses (aat)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Liew, Sook-Lei (committee chair), Pyatak, Beth (committee member), Schepens Niemiec, Stacey (committee member)

Creator Email mdonnelly@towson.edu,mrennie@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC113865297

Unique identifier UC113865297

Identifier etd-DonnellyMi-12749.pdf (filename)

Legacy Identifier etd-DonnellyMi-12749

Document Type Dissertation

Format theses (aat)

Rights Donnelly, Miranda Rennie

Internet Media Type application/pdf

Type texts

Source 20240403-usctheses-batch-1134 (batch), University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu