University of Southern California Dissertations and Theses

Primary care physicians' experiences working within the patient-portal to improve the quality of patient care

(USC Thesis Other)

Primary care physicians' experiences working within the patient-portal to improve the quality of patient care

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Primary Care Physicians' Experiences Working Within the Patient-Portal to
Improve the Quality of Patient Care
By
Eric Fleming
A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
December 2020
Copyright 2020 Eric Fleming
ii
Dedication
I would like to dedicate this dissertation to my parents Jim and Jane Fleming, for never
holding me back from living my dreams and my twin brother Eron who has always been there
through the highs and lows that life gives you. To my husband, Keith, for putting up with me
during these past two and a half years. To Natalie, for reading and editing every one of my
assignments and dissertation, I could not have done it without you.

iii

Acknowledgments
Thank you to my extended family and friends in Los Angeles and Las Vegas, who
pushed me and encouraged me to finish the OCL program. The completion of this dissertation
would not have been possible without the love and support I received from all of you. To the
University of Southern California’s Rossier School of Education, I thank you for letting me
fulfill a lifetime dream of obtaining my doctorate.
Thank you to the OCL Cohort X. Without your encouragement and friendship, this would
not have been possible. To Laurie, my friend, mentor, and boss, it was a pleasure to experience
this journey with you.

iv

Table of Contents

Dedication ....................................................................................................................................... ii

Acknowledgments.......................................................................................................................... iii

List of Tables and Figures............................................................................................................. vii

Abstract ........................................................................................................................................ viii

CHAPTER ONE: OVERVIEW OF THE STUDY .........................................................................1
Introduction of the Problem of Practice ...............................................................................1
Organizational Context and Mission ...................................................................................2
Mission and Values ..................................................................................................2
Statistics ...................................................................................................................2
Organizational Performance Status/Need ............................................................................3
Related Literature .................................................................................................................4
Importance of the Organizational Innovation ......................................................................4
Organizational Performance Goal ........................................................................................6
Description of Stakeholder Groups ......................................................................................6
Stakeholder Groups’ Performance Goal ..............................................................................7
Stakeholder Group for the Study .........................................................................................7
Purpose of the Project and Questions ..................................................................................8
Methodological Framework .................................................................................................9
Definitions............................................................................................................................9
Organization of the Study ....................................................................................................9

CHAPTER TWO: REVIEW OF THE LITERATURE .................................................................11
Literature on the Problem of Practice ................................................................................11
Electronic Health Record .......................................................................................12
Patient-Portal..........................................................................................................12
COVID-19..............................................................................................................13
Physician Workload in the Digital Age .............................................................................14
Patient-Centered Care ............................................................................................16
Boundaries of Patient-Portal ..................................................................................17
Quality of Physician Work Processes ................................................................................18
Clerical Tasks.........................................................................................................18
Disruption of Physician Daily Workflow ..............................................................18
Physician Work-Life Balance ............................................................................................19
Patient-Portal is a Contributor ...............................................................................19
Quality of Physician Family Life ...........................................................................20
Clark and Estes Knowledge, Motivation and Organizational Influences Framework .......20
Stakeholder Knowledge, Motivation and Organizational Influences ................................22
Knowledge and Skills ............................................................................................22
Motivation ..............................................................................................................26

v

Organization ...........................................................................................................29
Organizational Culture ...........................................................................................29
Cultural Settings.....................................................................................................30
Cultural Models .....................................................................................................32
Conceptual Framework: The Interaction of Stakeholders’ Knowledge, Motivation, and
the Organizational Context ....................................................................................34
Conclusion .........................................................................................................................38

CHAPTER THREE: METHODOLOGY ......................................................................................39
Methodological Approach and Rationale ..........................................................................39
Sampling and Recruitment .................................................................................................40
Participating Stakeholders .....................................................................................40
Interview Sampling Strategy, Criteria, and Rationale ...........................................40
Explanation for Choices .........................................................................................41
Data Analysis .....................................................................................................................41
Documents and Data Collection ............................................................................42
Interviews ...............................................................................................................43
Credibility and Trustworthiness .........................................................................................44
Credibility Strategies .............................................................................................44
Ethics..................................................................................................................................46

CHAPTER FOUR: RESULTS AND FINDINGS .........................................................................49
Participating Stakeholders .................................................................................................49
Determination of Threshold ...............................................................................................51
Findings..............................................................................................................................51
Knowledge Findings ..............................................................................................51
Motivation Findings ...............................................................................................56
Organizational Findings .........................................................................................61
Historical Data ...................................................................................................................67
Data Analysis .........................................................................................................67
Synthesis ............................................................................................................................70

CHAPTER FIVE: DISCUSSION ..................................................................................................72
Purpose of the Project and Questions ................................................................................72
Research Questions ................................................................................................72
Recommendations for Practice to Address KMO Influences ............................................72
Knowledge Recommendations ..............................................................................72
Motivation Recommendations ...............................................................................75
Organization Recommendations ............................................................................79
Implementation and Evaluation Framework ..........................................................84
Organizational Purpose, Need and Expectations ...................................................85
Level 4: Results and Leading Indicators ................................................................85
Level 3: Behavior ...................................................................................................86
Level 2: Learning ...................................................................................................90
Level 1: Reaction ...................................................................................................92
Evaluation Tools ....................................................................................................93

vi

Data Analysis and Reporting .................................................................................94
Summary ............................................................................................................................94
Strengths and Weaknesses and the Approach ....................................................................96
Limitations and Delimitations............................................................................................96
Future Research. ................................................................................................................97
Conclusion .........................................................................................................................98

References ....................................................................................................................................101

Appendix A: Interview Protocols ................................................................................................111

Appendix B: Immediate Evaluation Tool ....................................................................................114

Appendix C: Immediate Evaluation Tool ....................................................................................116

Appendix D: Delayed Feedback Tool..........................................................................................118

vii

List of Tables and Figures

Table 1: Organizational Mission, Organizational Goal, and Stakeholder Performance Goal .........7
Table 2: Knowledge Influence, Knowledge Type, and Knowledge Influence Assessment ..........25
Table 3: Motivational fluence, and Motivational Influence Assessment ......................................28
Table 4: Assumed Organizational Influence, and Organizational Influence Assessment .............33
Figure 1: Conceptual Framework: Interaction of Stakeholder Knowledge and Motivation within
Organizational Cultural Models and Settings ........................................................36
Table 5: Primary Care Physician’s Demographics ........................................................................50
Table 6: Number of Portal Messages Sent by Primary Care Physicians .......................................69
Table 7: Assumed Knowledge, Motivation, and Organization Influences ....................................71
Table 8: Summary of Knowledge Influences and Recommendations ...........................................73
Table 9: Summary of Motivation Influences and Recommendations ...........................................76
Table 10: Summary of Organization Influences and Recommendations ......................................80
Table 11: Outcomes, Metrics, and Methods for External and Internal Outcomes ........................86
Table 12: Critical Behaviors, Metrics, Methods, and Timing for Evaluation ...............................87
Table 13: Required Drivers to Support Critical Behaviors ............................................................88
Table 14: Evaluation of the Components of Learning for the Program ........................................92
Table 15: Components to Measure Reactions to the Program1 .....................................................93

viii

Abstract
The purpose of this study was to conduct a needs analysis of the primary care physicians’
knowledge and skill, motivation, and organizational resources necessary to reach the
organizational performance goal of sending secure messages through the patient-portal.
The study utilized Clark and Estes gap analysis to determine whether the physicians have
adequate knowledge, motivation, and organization support to achieve the organization’s goal.
Ten primary care physicians were interviewed, providing the primary source of data, along with
historical data from the electronic health record. The data represented the number of secured
messages sent by the physicians per month and the hours the physicians spent in the electronic
medical record after their typical 10-hour workday. Findings indicate that the assumed
conceptual knowledge influence along with the self-efficacy and expectancy-value motivation
influences, and the cultural settings and models influences were either an asset or a need to the
stakeholder’s goal. Based on the findings and supported by literature and historical data,
solutions were offered to address the challenges. The study demonstrates how applying the
knowledge, motivation, and organizational framework help identify stakeholder’s needs to
provide quality care by sending secure messages through the patient-portal.
Keywords: patient-portal, primary care physicians, secured messages,
quality of patient care

1

CHAPTER: INTRODUCTION
Introduction of the Problem of Practice
The electronic patient-portal offers the potential for increased patient-physician
communication and improved clinical care beyond the office visit. The U.S. government
provides a clear definition of a patient-portal; it is a secure online website that gives patients
convenient 24-hour access to personal health information (PHI) from anywhere with an internet
connection (Kruse et al., 2015; Shah & Liebovitz, 2017; Zhong et al. 2020). The patient-portal
usage is associated with patient engagement, where patients use the portal for such functions as
messaging, obtaining laboratory results, making appointments, and requesting medication refills.
In 2009, less than 5% of hospitals utilized a web-based patient-portal, however since Congress
passed the Health Information Technology for Economic and Clinical Health Act (HITECH),
patient-portal adoption has gained considerable attention as it enables a secure means of
continuous patient-centered care (Bryan et al.,2020; Kruse et al., 2015; Shah & Liebovitz. 2017;
Zhong et al., 2020). The adoption of the patient-portal was integrated into federal meaningful
use metrics and is necessary for receiving financial incentives associated with implementing
electronic health record (EHR) (Chan et al., 2018). In 2017, the Medicare Access and CHIP
Reauthorization Act (MACRA) further targeted online patient engagement through the Quality
Payment Program utilizing patient portals (Kornacker, 2019). The undertaking of the patient-
portal and secured messaging is to improve patient outcomes, utilization, and patient experience.
Conversely, physician concerns regarding the patient-portal use, including fear of increased
workload, quality of physician work processes, and work-life balance, may contribute to the lack
of portal usage by the physicians (Arndt et al., 2017; Chan et al., 2008; Dyrbye et al., 2017; Guo
et al., 2017).

2

Organizational Context and Mission
St. Elsewhere Hospital (SEH) is located on the medical campus of the University of St.
Elsewhere (USE). St. Elsewhere is the pseudonym for both the university and the affiliated
Hospital. The Hospital is a medical enterprise, which strives for academic excellence, innovative
research, and advanced clinical care that attracts experts to teach and practice at the St.
Elsewhere School of Medicine (St. Elsewhere, 2020). SEH includes two acute care hospitals in a
metropolitan city in the southwestern part of the United States. In the late 2000s, U.S. News &
World Report ranked the Hospital among the Top 10 hospitals in America (St. Elsewhere, 2020).
Mission and Values
The SEH mission is to strive to be the leader in healthcare that is caring and stands for
equality and strength (St. Elsewhere, 2020). The community counts on SEH to be present in the
delivery of uncompromising healthcare with a promise to quality in clinical care, education, and
research. The SEH's values are to deliver valued healthcare through service quality (St.
Elsewhere, 2020).
Statistics
Throughout the health system, there are 800 inpatient beds and over 100 outpatient
practices. The 14,000 inpatients and over 400,000 outpatients per year are seen by 700 faculty
physicians who are supported by close to 1,400 nurses. The enterprise serves an ethnically and
diverse demographic population (St. Elsewhere, 2020). St. Elsewhere Hospital has a healthy
payer mix, which includes health maintenance organizations (HMOs) and preferred provider
organizations (PPOs) plus all government insurance plans. The balance of private and self-
paying patients is essential for Medicare, and Medicaid pays hospitals less than what it costs to
treat patients.

3

St. Elsewhere implemented a robust HER system in 2013, which includes and patient-
portal function, which allows an easy and convenient way for patients and physicians to
communicate with each other. As noted, the adoption of the HER was complicated for most
healthcare organizations (Chan et al., 2018) since it forced to change and created new processes
that highlighted accountability (Bryan et al.,2020; Kruse et al., 2015; Shah & Liebovitz. 2017;
Zhong et al., 2020).
Organizational Performance Status/Need
Numerous studies show primary care physicians (PCPs) spend more than one-half of
their workday, approximately 6-hours, interacting with the EHR and the patient-portal during
and after clinic hours (Arndt et al., 2017; Dyrbye et al., 2017; Guo et al., 2017; Hingle, 2016).
Physicians spend up to 90 minutes of charting time per day, which extends into their home life
(Robb, 2017). PCPs play an influential role in patients’ portals use, as their endorsement is an
essential factor in the adoption of these tools. The value that patients derive from the use of
portals will be directly affected by their providers’ attitudes and actions; encouragement to
activate an account can significantly increase use, whereas dismissing or passively ignoring a
patient’s potential interest in engaging via the portal can further impede the use (Shah &
Liebovitz, 2017). The literature suggests that because of the implementation of the EHR,
physicians have limited face-to-face time with patients; PCPs spent 44% of their workday in the
EHR doing clerical and administrative tasks, such as prescriptions, labs, and radiology orders
that account for 12.1% of their clinic hours (Arndt et al., 2017; Borg, 2017; Guo et al., 2017).
The EHR system includes patient-portals for instant and real-time access to communication with
providers, yet less than half (48%) of portal users have used their patient-portal to communicate
with their healthcare provider, and as a result, the volume of electronic messages received by

4

physicians is increasing at a rapid pace; in one health system, it has nearly tripled in 10-years
(Bryan et al., 2020). In 2016, only 26% of practicing physicians who used EHRs reported
connecting with patients directly using the patient-portal (Bryan et al., 2020). Physicians have to
correspond with their patients and staff daily, which can significantly affect their clinical practice
and how they deliver services to their patients (Bryan et al., 2020; Frankel, 2016; Karakose,
2014; Kruse et al., 2015).
Related Literature
The use of the patient-portal can enable collaborative care and communication, which
means that patients and providers can share timely and pertinent information such as test results,
changes in clinical status, medication changes, reminders for scheduled appointments, screening
tests, and preventive care (Van den Bulck et al., 2018; Shah & Liebovitz, 2017). The U.S.
implemented patient-portals as a way to meet the requirements for stage two of Meaningful Use,
of the Healthcare Information Technology for Economic and Clinical Health Act (HITECH), by
promoting patient involvement in health care delivery, which may lead to improved quality and
safety of care by enabling patients to spot errors in EMRs (Bryan et al., 2020; Dendere et al.,
2019). The patient-portal is known to improve the quality of and access to healthcare, plus it is a
positive influence on health care utilization. The patient-portal can be defined as an electronic
application through which patients can retrieve, manage and share their medical information, and
that of others for whom they are permitted, in a private, secure, and confidential environment
(Van den Bulck et al., 2018)
Importance of the Organizational Evaluation
The organization needs to implement a culture that provides PCPs with a more stable
means of communication and improved clinical care of their patients. The organization needs to

5

provide leadership, organizational culture, and financial means to navigate today’s digital age.
These factors likely not only facilitate the use of the patient-portal, but also distinguish
knowledge and skills that providers need to lead any meaningful systems change. The Centers
for Medicare & Medicaid Services (CMS), in 2012, encouraged the use of online secure
messaging by requiring that 5% of an Accountable Care Organization’s (ACO) patients send
secure messages to their provider before incentive payments can be collected. Therefore secure
messaging systems linked to the patient-portal are now a standard feature of most EHRs (Bryan
et al., 2020; Shah & Liebovitz, 2017). While almost half of the patients have utilized the portal’s
secured messaging feature, there has been less acceptance among physicians (Bryan et al., 2020).
In 2016, only 26% of practicing providers who used EHRs reported communicating with patients
directly using the patient-portal (Bryan et al., 2020). Despite these advantages, physicians have
voiced concern that patient-initiated messages may increase their workload while decreasing
productivity and compensation (Bryan et al., 2020). Likewise, over half of physicians currently
using patient-portals did not feel that patient messaging improved efficiency (Bryan et al., 2020).
The patient-portal can be a barrier to health care providers due to the increased workload
and disruption to existing clinical processes and staff roles, as well as concerns about
remuneration, data security, and liability (Ayre et al., 2019). Primary health care is unlike many
other medical specialties. It requires physicians to manage a large segment of the population for
preventative, routine, and chronic health needs. Often, patients requiring primary care have a
complex array of physical, emotional, and social concerns that may or may not be obviously
stated in the presenting problem, thus delivering effective primary care can be a daunting task.
The limited face-to-face time with patients as more time is spent documenting within the EHR
rather than talking with or examining the patient leads to dissatisfaction (Arndt et al., 2017; Guo

6

et al., 2017). Work-related stressors like inefficient work processes and environments such as
requiring physician-entry comprehensive documents and physician order entry and other tasks
not maximizing the time physicians spend working at the top of their licenses lead to physicians'
displeasure of work-life balance (West et al., 2018). Measures must be taken to improve the
quality of care for patients and assess the concerns physicians have regarding the increased
workload, work processes, and work-life balance. The goal of the patient-portal is to increase
patient-physician communication and improve clinical outcomes.
Organizational Performance Goal
As online patient engagement is supported by the CMS and requires eligible providers to
meet specific standardized of Meaningful Use criteria, SEH is starting to look at their
organizational performance goal on sending secure messages. By June 2025, all primary care
physicians will send a secure message to 80% of their panel through the patient-portal during the
reporting period. St. Elsewhere Hospital is engaging their primary care physicians and patients in
the management and coordination of their care for their patients. The PCPs developed the goal,
with the assistance from the Chair of Primary Care (CPC) and the Chief Nursing Officer (CNO).
Description of Stakeholder Groups
St. Elsewhere Hospital consists of many divisions, departments, and administrative
components that work together to complete the organization's goals. When looking at the PCP's
perspectives of the patient-portal and its effect on patient care, several stakeholder groups
worked closely on establishing this goal. The three principal stakeholders are the Chair of
Primary Care, the Chief Nursing Officer, and the primary care physicians themselves. The Chair
is the leader of the Department and provides stewardship to all primary care physicians. The

7

Chief Nursing Officer is the top-ranking nurse in the Ambulatory Services, with oversight of all
medical staff in the clinics, and the primary care physicians representing their needs and goals.
Stakeholder Groups' Performance Goal
Delivery of healthcare has shifted toward patient-centered care, and patient-portals have
become a vital health information technology that encourages patient engagement by providing
access to personal health information. St. Elsewhere Hospital wanted to expand their reach to
their patients and increase the number of secure messages for the organization. Table 1 provides
an explanation of the organizational mission, goals, and stakeholder performance goals.
Table 1
Organizational Mission, Organizational Goal and Stakeholder Performance Goal
Organizational Mission
The St. Elsewhere Hospital's mission is to strive to be the trailblazer in quality healthcare that is
compassionate, innovative, and stands for empowerment, integrity, collegiality, and vitality.
_____________________________________________________________________________
Organizational Performance Goal
By June 2025, all primary care physicians will send a secure message to 80% of their panel
through the patient-portal during the reporting period.
______________________________________________________________________________
Stakeholder Performance Goal
By June 2021, all primary care physicians will send a secure message to 30% of their panel
through the patient-portal during the reporting period.
______________________________________________________________________________
Stakeholder Group for the Study

8

The stakeholders for this qualitative study were the PCPs. Primary care is a staple in a
medical institution; it is the day-to-day healthcare option, which acts as the first contact and
primary point of continuity of care for patients. In academic medical institutions, the surgical
departments bring in the majority of the funding, but primary care feeds the institutions with a
steady stream of patients. St. Elsewhere Hospital is no different, for the PCPs work long hours,
and unlike their surgery partners, PCPs usually operate independently. The use of the patient-
portals offers the potential for increased patient-physician communication and improved clinical
care beyond the office visit. Actions must be taken to enhance the quality of care and patient
experience, which will increase the use of secure messages by the PCPs.
Purpose of the Study and Questions
The purpose of this study was to evaluate the PCPs knowledge and skill, motivation, and
organizational resources necessary to reach the organizational performance goal of sending
secure messages through the patient-portal. The evaluation began by generating a list of possible
needs and then moved to systematically assessing whether they are actual or validated needs.
While a complete evaluation would focus on all stakeholders, for practical purposes, the
stakeholder focused on in this evaluation were primary care physicians.
As such, the questions that guide this study were the following:
1. What is St. Elsewhere Hospital physician's knowledge and motivation related to the
goal of sending a secure message to 30% of their patient panel through the portal
during the reporting period, in 2021?

2. What is the interaction between St. Elsewhere Hospital's culture and context, and
physician knowledge and motivation to the goal of sending a secure message to 30%
of their patient panel through the portal during the reporting period, in 2021?

9

Methodological Framework
Clark and Estes (2008) gap analysis, a systematic, analytical method that helps to clarify
organizational and stakeholder performance goal to identify the gap between the actual
performance level and the preferred performance level within an organization, was chosen to
evaluate the performance of the PCPs utilizing the patient portal for ongoing patient care. The
assumed knowledge, motivation, and organizational needs impacted performance goal
completion based on personal knowledge and related literature. Influences were assessed using
interviews and literature review validated assumed performance gaps. Research-based solutions
were recommended and evaluated comprehensively.
Definitions
Electronic Health Record (EHR): The systematized collection of patient and population
electronically-stored health information in a digital format.
Patient-Portal: A website based online tool to help patients communicate with their physicians
and keep track of their appointments, test results, billing, and prescription renewals.
Organization of the Study
Five chapters are used to organize this study. This chapter provides the reader with the
key concepts and terminology commonly found in a discussion about the experiences of PCPs
utilizing the patient-portal for ongoing patient care. The organization's mission, goals, and
stakeholders, as well as the initial concepts of gap analysis adapted to needs analysis, were
introduced. Chapter Two provides a review of the current literature surrounding the use of the
patient-portal by PCPs. Topics of physician workload in the digital age, quality of physicians,
work processes, and physician work-life balance will be addressed. Chapter Three details the
assumed needs for this study as well as a methodology when it comes to the choice of

10

participants, data collection, and analysis. In Chapter Four, the data and results are assessed and
analyzed. Chapter Five provides solutions, based on data and literature, for addressing the needs
and closing the performance gap as well as recommendations for an implementation and
evaluation plan for the solutions.

11

CHAPTER TWO: REVIEW OF THE LITERATURE
The patient-portal was designed to be a tool to engage patients in their healthcare and
help them manage their health information. As the U.S. healthcare system rapidly adopted EHR
over the past decade, many with linked patient-portals, enthusiasm, and expectations for this new
technology as a means to engage and empower patients grew (Shah & Liebovitz, 2017). With the
growing use of the patient-portal, PCPs are spending much of their days performing tasks that do
not require their professional training. Primary health care is unlike many other medical
specialties; thus, delivering effective primary care can be a daunting task. This chapter first
reviews the literature on the EHR, specifically the patient-portal that is associated with physician
workload in the digital age, looking at the quality of physician work processes and physician
work-life balance. It examines the quality of the physician’s work processes regarding the
clerical task requirements, which disrupts the daily physician workflow and reviewing the
influence the patient-portal has on the quality of the physician’s family life and their work-life
balance. Then, the chapter explains Clark and Estes (2008) knowledge, motivation, and
organizational influences lens used in this study. Next, the chapter turns attention to defining the
types of knowledge, motivation, and organizational influences examined, and the assumed PCP’s
knowledge, motivation, and organizational influences on performance. The chapter ends with a
presentation of the conceptual framework guiding PCP experiences working with the patient-
portal.
Influences Working in the Digital Age
The expectation of required documentation of medical care within the patient-portal is
dramatically increasing physician work-load. Research states that labor-intensive documentation,
order entry, billing/coding, performance measurement, numerous patient portal-messages, lab

12

result inquiries, prescription renewal requests, and other administrative and regulatory
workplaces immense strain on the responsible physician’s complex tasks (Robb, 2017). Studies
have shown that patients express a desire for closer relationships with their providers; physicians
and patients are concerned that the patient-portal may threaten their preference for in-person
communication (Lyles et al., 2016). The implementation of the EHR has decreased the quality of
the face-to-face interactions the PCPs have with their patients (Alkureishi et al., 2016; Borg,
2017; Frankel, 2016). In a retrospective study at the University of Florida, although 75% of the
physicians felt that the EHR improved their efficiency, 92% experienced a decrease in efficiency
mainly due to disruption of workflow, increase in physician administrative work-load, and the
need to develop opportunities to bypass several limitations (Ewelukwa et al., 2018).
Electronic Health Record
The electronic health record (EHR) is a digital form of a patient’s chart, expanded for the
convenience of third parties. Like the traditional charts, they include e-prescription, clinical
decision support, and computerized physician order entry modules (Dyrbye et al., 2017; Robb,
2017). In the United States, ambulatory care services have been subject to dramatic pressures to
implement the use of the EHR over the past decade (Sinsky et al., 2016). In the context of rapid
change to move to the EHR, dissatisfaction with how physicians spend their time and skillsets
are used is widespread and growing (Sinsky et al., 2016). The utilization of health information
technologies and online resources like patient-portal is used to enhance care quality by
improving care access, efficiency, and chronic disease management.
Patient-Portal
The patient-portal is a tool within the EHR, which is typically accessed by patients
through a website and is the central component of patient engagement in healthcare. Research

13

had discovered that patients believed that secure messaging could boost their communication
with their PCPs and that patients were reassured with secure messaging as a useful way to
communicate with their care team (Hefner et al., 2019). Patient-portals have shown promise in
engaging individuals in self-management of chronic conditions by allowing patients to input and
track health information, facilitating communication between patients and providers, and
providing access to consumer-friendly information about diseases, in addition to serving as a
platform to view and schedule an appointment, and engage in secure communication with
providers (Chan et al., 2018; Sieck et al., 2017). Secure messaging is one of the most popular
features of patient-portals, and messaging volumes are growing exponentially, with one study
showing that the Department of Surgery was second only to the Department of Medicine in the
number of messages exchanged (Sulieman et al., 2020). Patients usually found messaging more
convenient and quicker than calling the clinic with questions and concerns, plus patients found
that when managing their chronic condition, messaging could reduce the need for frequent clinic
visits to discuss condition changes and updates (Hefner et al., 2019). As the use of messaging
increases, techniques to automate the analysis of messages may be critical to assist with triage,
message answering, or quantifying the care delivered through patient portals. Studies suggest
that secure messaging can facilitate access to care, improve patient satisfaction, and improve
health outcomes (Shah & Liebovitz, 2017; Sieck et al., 2017). Secure messaging is used to
promote care coordination between visits, address patient questions, handle routine health issues,
and help patients better manage their condition (Emont, 2011; Fact Sheet, 2012).
COVID-19 Pandemic
In December 2019, the novel coronavirus COVID-19 was initially identified as a case of
pneumonia in Wuhan, China, and has since become a global pandemic, affecting more than 150

14

countries around the world (Reeves et al., 2020). The World Health Organization (WHO)
declared the outbreak a pandemic on March 11, 2020, and on March 13, 2020, the Executive
Office of the President of the United States proclaimed the pandemic a national emergency
(Portz et al., 2020; Reeves et al., 2020). Healthcare systems were rapidly preparing and adapting
to increasing clinic demands by leveraging the capabilities of the EHR and patient-portals to
create virtual venues for meetings, day-to-day operations, and offering telemedicine visits for
their patients (Portz, 2020). With the increased social distancing and patient concerns about in-
person clinic visits, telehealth use has risen rapidly, and telehealth has the potential to provide
health care services a useful advance care planning tool during the pandemic and beyond.
Centers for Medicare & Medicaid Services reimbursement has rapidly expanded to cover
telehealth services such as nonemergent clinical visits, screening for COVID-19, and advance
care planning counseling (Portz et al., 2020). The patient-portal provides the ability to view
health information, and it supports patient-initiated entry of health information and electronic
interactions between patients and providers, which promotes engagement in care (Nouri et al.,
2020; Portz et al., & Reeves et al., 2020). Portals thus offer an incredibly accessible platform for
patients and care partners to learn about COVID-19 treatment options and communicate goals of
care with their providers.
Physician Workload in the Digital Age
The adoption of the patient-portal is an essential component of these national policy
efforts to reduce costs and improve the quality of care. The Health Information Technology for
Economic and Clinical Health (HITECH) Act of nearly $30 billion has been dedicated to
facilitating the adoption and meaningful use of health information technology (Giardina et al.,
2015; Shah & Liebovitz, 2017). Offering patients with electronic access to their medical

15

information and incentivizing patient engagement in health care as part of the meaningful use of
EHR is emphasized by the HITECH Act (Giardina et al., 2015). The Meaningful Use initiative
authorized financial incentive payments and penalties based on compliance with specific criteria.
Recent estimates have shown that over 75% of hospitals use at least a basic EHR, up from less
than 10% in 2008 (Shah & Liebovitz, 2017).
Today’s patients expect same-day access for face-to-face care during clinic hours and
rapid responses from their PCPs to telephone calls, patient-portal messages, laboratory results
inquires, and prescription renewal requests both during and after clinic hours. In a study of 142
family medicine physicians in a single system in Wisconsin, 44% of their physician’s workday
(157 minutes) was spent in the EHR doing clerical and other administrative tasks, which have
burdened the clinicians with dissatisfaction of the work-load in today’s digital age (Arndt et al.,
2017). Labor-intensive documentation, order entry, prior authorizations, billing/coding, other
administrative and regulatory work places immense strain on the responsible physician’s
complex tasks (Robb, 2017). Many physicians are spending 40% -55% of their time on EHR
documentation daily, and unfortunately, up to 90-minutes of digital charting time extends into
after-hours (Robb, 2017). In their current state, practicing physicians have claimed that the EHR
will occupy a lot of physicians’ time and pull attention away from their direct interactions with
patients and their personal lives (Hingle, 2016).
The approval of the HITECH Act and the Meaningful Use requirements have prioritized
patient engagement in health care through health information technology (Giardina et al., 2015).
While patients generally want access to their health information, very few are currently taking
advantage of it. It is expected that the number of patients using patient-portals will increase,
especially when PCPs move to more patient-centered care.

16

Patient-Centered Care
The British National Health Service had an expression “no decision about me, without
me,” emphasizing the significance of patient-centered care and shared decision making (Van den
Bulck et al., 2018). The increasing presence of technology in health care delivery, coupled with
consumer demand, contributes to a remarkable shift in traditional practices and a renewed
interest in patient empowerment as a means to advance this long-sought reform (Risling et al.,
2018). Patient-centered care is characterized as providing care that is respectful of, and reactive
to, individual patient preferences, and ensuring that patient values guide all clinic decisions (Van
den Bulck et al., 2018).
The World Health Organization (WHO) European Regional Office included
empowerment and patient-centered practice as critical elements in the Health 2020 report, a
follow-up on a previous WHO study on the effectiveness of empowerment to improve health
(Risling et al., 2018). Patient empowerment is increasingly present in today’s digital age. Patient
empowerment has been examined in conjunction with technological apps such as electronic
personal health records, patient web portals, and EHR (Risling et al., 2018). The importance that
people attribute to obtaining health information when using a patient-portal is partially predicted
by age and the level of patient empowerment, namely the importance of shared decision making
and questioning physicians’ decisions (Van den Bulck et al., 2018). To build on patient
empowerment and the patient-portal, many patients and caregivers living with chronic
conditions, such as cancer, diabetes, and kidney disease, indicated that they use their portal to
manage and keep track of their medical information (Giardina et al., 2015).

17

Patients may communicate electronically with their providers, access personal health
records, receive lab results, and request medication refills, but is there a limitation or time
consideration by when these patients have to hear from their providers.
Barriers of the Patient-Portal
There is an increased emphasis on understanding patient needs, and the issue is
particularly important, given the growing evidence that patient access to and use of portals are
linked to improved satisfaction and better health outcomes. There are boundaries around the
patient-portal that physicians and patients need to address, such as when patients communicate
several times a week, or even daily, about issues that are not time-sensitive or have already been
answered during appointments. Some patients use secure communication for medical visit
avoidance, circumventing a copayment, or incurring the cost of a visit (Bush et al., 2017). In
contrast, several physicians noted the avoidance might be reduced following upcoming
clarification regarding what constituted an e-visit and the ability to bill for such encounters (Bush
et al., 2017). Payment structures that accommodate technologically-mediated communications
between providers and patients via text messaging, emails, or virtual visits are instrumental in
eliminating providers' adoption barriers and should also be explored (Zhnog et al., 2018).
Another boundary of the patient-portal is health literacy among racial/ethnic minority
patients, moreover, the need for health literacy training or support due to the lack of confidence
in being able to independently interpret medical content presented on a portal without one-on-
one assistance (Lyles et al., 2016). An essential competency needed to make informed health
decisions, obtain and interpret health information, and improve quality of life is health literacy
(Wong et al., 2019). Computer literacy is needed by users to participate in contemporary society
due to the continuing expansion of electronic information and communication like patient-portals

18

(Wong et al., 2019) effectively. Consequently, physicians have to spend more time performing
clerical tasks instead of the hands-on healing they once encountered.
Quality of Physician Work Processes
Clerical Tasks
The patient portal has led to increased use of electronic communication among patients,
clinical staff, and physicians. In clinical practice today, a PCPs electronic inbox usually contains
a variety of messages from patients, other physicians, medical staff members, the pharmacy,
radiology, laboratory, and other departments (Lieu et al., 2019). The burden related to the
clerical tasks, which are now mandated by the federal government, is directly associated with
physician stress. The EHR systems are integral to patient care, but the amount of time a
physician spends on clerical tasks puts physicians at a higher risk for exhaustion (Guo et al.,
2017). It is essential to distinguish the multiple factors that are causing physicians to stress, and
the increased work-load clinicians experience due to the mandated clerical tasks while working
in the EHR (Arndt et al., 2017). Other influences include the inappropriate distribution of EHR
tasks to clinicians. Tasks such as technology-supported guidelines that have placed hard stops in
clinical workflow, healthcare workforce issues, and rapidly changing regulatory requirements, to
name a few (Arndt et al., 2017). As physicians are now required to complete all clerical tasks
within the EHR, including responding to patient inquiries and finalizing service billing promptly,
this burden to complete these clerical tasks disrupts the physician's daily workflow.
Disruption of Physician Daily Workflow
The EHR implementation has enlisted more demands on the physicians to complete all
the clerical tasks required through the EHR. The types of clerical tasks delegated to medical
assistants (MAs) are consistent with other clerical tasks that can be executed based on the

19

protocol (Lieu et al., 2019). However, the effectiveness of this approach differed depending on
the practice’s attributes, with several chiefs noting limitations, including variability in skill
levels, and turnover (Lieu et al., 2019). Physicians are now required to complete all clerical tasks
within the EHR, including responding to patient inquiries and finalizing service billing promptly.
This burden to complete these clerical tasks is adding to the physician’s dissatifaction of the job.
Audits of EHR data revealed that family physicians spend more than one hour of personal time
on computer tasks each day (Sinsky et al., 2016). The EHR’s implementation is enlisting more
demands on the physicians, which has entirely changed the natural flow of their clinics.
Conversely, this complexity, coupled with our culture’s demand for instant gratification
requiring primary care providers to always be on call for their patients, causes a decrease in
physician retention and disrupting that work-life balance.
Physician Work-Life Balance
Patient-Portal is a Contributor
The patient-portal contributes to physicians' increased workload due to the daily
documentation requirements that are now extending late into the evenings and on weekends and
disrupting their work-life balance. Physicians face many disruptions throughout their day; some
of these work process inefficiencies are order entry and documentation, along with unwarranted
work-loads like extended work hours and overnight call frequency (Dyrbye et al., 2017). Work-
home conflicts and other inadequacies include organizational climate factors like management
culture, lack of physician-nurse collaboration, and autonomy, deterioration in control, and
meaning at work, have all been associated with exhaustion among physicians (Dyrbye et al.,
2017). Roughly half (48%) of portal users have used their patient-portal to communicate with
their health care team (Bryan et al., 2020). However, although it has frequently been suggested

20

that patient messages have the potential to improve physician productivity by replacing
telephone calls and office visits, the literature conflicts on the extent to which portal messaging
can substitute for other clinical services, which can cause interruptions to the physician’s work-
life balance (Bryan et al., 2020). Exhaustion matters because it is associated with adverse
consequences on patient care. The quality of physician work processes and physician work-life
balance can be contributed to the patient-portal.
Quality of Physician Family Life
Statistical analyses of data from cross-sectional studies of physicians have reported
independent relationships between night or weekend calls, time spent at home on work-related
tasks, and work-home conflicts ( Dyrbye et al., 2017). A time and motion study of ambulatory
practices in four states showed that physicians who completed after-hours charting dedicated a
mean of 1.5 hours to after-hours work per day, with 59% of the time spent using an EHR (Sinsky
et al., 2016). Stress among primary care providers has been on the rise for over a decade due to
the changes in the government-regulated programs such as the Affordable Care Act (ACA),
Merit-based Incentive Payment System/Alternative Payment Model (MIPS/APM) and the EHR
(Krousel-Wood et al., 2017; Robb, 2017). It should be noted that portal access experienced an
average of four messages sent from patients to their physicians annually during the study period
(Zhong et al., 2018). If all 1300 panel patients turn into portal users, a weekly average of 113
messages is expected for a single provider (Zhong et al., 2018). Naturally, the relationship
between physician workload in the digital age, quality of physician work processes, and
physician work-life balance add to the experience of working in the patient-portal.
Clark and Estes’ (2008) Knowledge, Motivation and Organizational Influences Framework

21

Healthcare institutions and other organizations can examine performance gaps through a
systematic, analytic framework established by Clark and Estes (2008) that identifies gaps
between actual performance levels and performance goals, and organizational and stakeholder
performance goals. The use of the patient-portal to increase patient-physician communication
and improves clinical care beyond the office visit. The purpose of a gap analysis was to identify
whether PCPs have adequate knowledge, motivation, and organizational support to reach the
stakeholder’s goal. The first cause was knowledge. Krathwohl (2002) introduces four dimensions
of knowledge, which are factual, conceptual, procedural, and metacognitive. The second cause
was motivation, which was inherently cultural; individuals develop motivational beliefs from
others with whom they interact in various social contexts in the ecological functions they inhabit
(Clark & Estes, 2008). The four motivational influences particularly relevant to increase patient-
physician communication are 1) self-efficacy and competence beliefs; 2) attributions and control
beliefs; 3) goals; and 4) goal orientations. The third and final cause of the performance gap is in
the organizational influences on stakeholder performance, looking at cultural models and
settings. These influences may include work processes, resources, and workplace culture (Clark
& Estes, 2008).
Clark and Estes (2008) gap analysis was used to identify whether the St Elsewhere
physicians have adequate knowledge, motivation, and organization support to achieve the
stakeholder’s goal of sending a secure message to 30% of their patient panel through the portal
during the reporting period, in 2021. The first section will introduce the stakeholder’s
performance goal in the framework of knowledge and skills of the PCPs, then continue onto
what motivational influences play on behalf of the stakeholders to reach their goal, and finally
closing out this section by examining the organizational influences on achieving the

22

stakeholder’s goal. The presumed stakeholder’s knowledge, motivation, and organizational
influences on performance will be further examined through the methodology discussed in
chapter three.
Stakeholder Knowledge, Motivation and Organizational Influences
The three dimensions that will be reviewed are knowledge, motivation, and
organizational influences and how they may influence the problem of improving patient-
physician communication and patient care quality through the patient-portal. The patient-portal
provides access to information in the patient’s EHR and serves as a platform to view and
schedule appointments and engage in secure communication with providers (Sieck et al., 2017).
This literature review will focus on these two dimensions to better understand how the use of the
patient-portal can enable collaborative care and communications, so patients and providers can
share timely and relevant information to achieve quality health outcomes.
Knowledge and Skills
St. Elsewhere Hospital evaluated the strengths and needs of the PCPs to identify the
performance gaps that led to physicians' struggling to send secured messages from the patient-
portal to their patients and the cause of those needs. The purpose of the gap analysis was to
identify whether primary care physicians have adequate knowledge, motivation, and
organizational support to reach the stakeholder's goal. Clark and Estes (2008) introduced
strategies for identifying active elements in various types of performance solutions available to
close the gaps. Three critical factors were to be examined during the analysis process. The first
step in the analysis was to determine whether the physicians knew how to achieve their
performance goals and identify whether the physicians had the skills to reach those goals. The
second was to determine what motivated the physicians to keep moving and to achieve the stated

23

goal. The third and final step was an analysis of the organizational barriers, which were
conditions or processes that caused problems or barriers and prevented PCPs from sending
secure messages to their patients. What factors of the physician's knowledge, skills, and
motivation to achieve the goal and identification of the potential cultural settings and models of
the organizational barriers were impeding the physician's ability to achieve their goal? Physician
engagement may increase when they determine how to solve this problem and are better able to
perform their job responsibilities.
The gap analysis resolves to identify whether the physicians have adequate knowledge,
motivation, and organization support to achieve the organization's goal. Physicians at SEH
cannot just learn the what and how of the problem, they need to understand why or when these
problems occur. Patient-portals have shown promise in engaging individuals in self-management
of chronic conditions by allowing patients to input and track health information and exchange
secure electronic messages with their providers (Sieck et al., 2017).
To provide quality patient care through the patient-portal, the physicians at SEH should
understand the variances in their own knowledge. Krathwohl (2002) introduces four dimensions
of knowledge; they are factual, conceptual, procedural, and metacognitive. Factual knowledge is
known as facts and refers to knowledge of specific details, contexts, or domains (which includes
elements like terminology) and details that one must know to understand how to solve the
problem (Krathwohl, 2002; Rueda, 2011). Rueda (2011) describes conceptual knowledge as the
understanding of classifications, principles, theories, models, or structures pertinent to a
particular area. Individuals learn conceptual knowledge through reading, viewing, listening,
experiencing, or thoughtful and reflective mental activity (Krathwohl, 2002). Procedural
knowledge indicates how something is done, like driving a car, to knowing how to take a blood

24

pressure measurement. This knowledge can also refer to a method of inquiry, algorithms,
techniques, and particular methodologies that are required to accomplish specific activities
(Rueda, 2011). The last knowledge dimension is metacognitive knowledge, which refers to the
awareness of one's own cognition and particular cognitive processes. This type of knowledge
allows one to know when and why to do something (Rueda, 2011). Rueda (2011) explains that
metacognitive knowledge is a crucial aspect of strategic behavior in problem-solving, which
would allow individuals to consider contextual and conditional aspects of a given activity or
problem. A review of existing literature suggests that the knowledge influence, conceptual, is
particularly relevant to St Elsewhere's global goal of sending a secure message to 30% of PCPs
patient panel through the portal.
Conceptual Knowledge. Physicians need to understand the consequences of
communicating with their patients through the patient-portal. A study conducted by Kim and
Fadem (2018) focused on engaging patients to actively participate in their care as an effective
means to improve health outcomes. Therefore, patient-portals are essential pillar of patient-
centered care and engagement because they provide patients with easy access to personal health
information, and they encourage patients to understand and self-manage their conditions.
Additionally, they improve communication with providers through direct secure messaging,
which leads to a positive influence on patient-provider relationships, patient satisfaction and
patient participation in making decisions about their treatment (Kim & Fadem, 2018). Over the
past couple of years SEH has encouraged their PCPs to engage their patients through the patient-
portal by sending secure messages to their patients. The number of secure messages sent by SEH
physicians has been increasing over the past year. Bioinformatics experts have developed
interactive patient-portal programs to help persons living with diabetes and other chronic

25

diseases to use web-based systems to monitor their health (Zickmund et al., 2008). Studies have
shown that patients have accepted e-mail communication with their health care-providers,
providers, although it is less clear whether they are willing to learn to use the new bioinformatics
program (Zickmund et al., 2008). Research shows that, while some patients desire more medical
information than PCP provides, other patients suffer from information overload,
miscommunication, or misinterpretation of sensitive information, and a multitude of technical
issues state that patients are worried about reduced personal contact with their PCP (Alkire et
al.2020). Therefore, SEH physicians need to understand the consequence of using the patient
portal from different lenses; the technology needs to be convenient and straightforward for the
patient and provider, and the physician needs to ensure the patient has the same or similar
experience as if the patient is being seen in person. These concepts became particularly crucial
during the COVID-19 pandemic since PCPs were forced to rely on other communication forms,
such as the patient portal. The COVID-19 pandemic serves as a vivid reminder that their
communication and actions through the patient portal have important consequences to the
patient, which can directly affect their health.
Table 2 provides a summary of the knowledge type influence and how it will be
assessed.
Table 2
Knowledge Influence, Knowledge Type, and Knowledge Influence Assessment

Organizational Mission

St. Elsewhere Hospital's mission is to strive to be the trailblazer in quality healthcare that is
compassionate, innovative, and stands for empowerment, integrity, collegiality, and vitality.

Organizational Global Goal

26

By June 2025, all primary care physicians will send a secure message to 80% of their panel
through the patient-portal during the reporting period.

Stakeholder Goal

By June 2021, all primary care physicians will send a secure message to 30% of their panel
through the patient-portal during the reporting period.

Knowledge Influence Knowledge Type Knowledge Influence
Assessment

Physicians need to understand
the consequences of
communicating with their
patients through the patient-
portal.

Conceptual

Interviews

Motivation
The second dimension of this literature review focused on motivation. Motivation is
viewed as what gets humans going, keeps them moving, and tells them how much effort to spend
on their work, play, or relationships (Clark & Estes, 2008; Rueda, 2011). Research has shown
that there are three facets of motivated performance, and they are an active choice, persistence,
and mental effort (Clark & Estes, 2008). The active choice is when the individual begins to
pursue a goal actively, and the contrary to this action is to procrastinate, avoid, or delay taking
action. The second type is persistence, which means once the individual starts to push through to
the goal, they continue through the face of distractions, not allowing themselves to be distracted
by less critical goals (Clark & Estes, 2008). Clark & Estes (2008) state that the last type of
motivational performance is a mental effort, which accompanies choosing and persisting a
critical goal that needs adequate mental effort to complete this goal. One's own confidence
determines mental effort; those who lack confidence tend not to invest much mental effort into
tasks.

27

Motivation is inherently cultural; individuals develop motivational beliefs from others
with whom they interact in the variety of social contexts in the ecological functions they inhabit
(Clark & Estes, 2008). Two motivational influences particularly relevant to physicians
communicating and providing quality of care through the portal are 1) self-efficacy and 2)
expectancy-value.
Self-efficacy theory. According to the social cognitive theory (Usher & Pajares, 2009),
self-efficacy beliefs provide the foundation for human motivation, well-being, and personal
accomplishment. Primary care physicians need to feel confident in their ability to use the patient-
portal. Individuals form their self-efficacy beliefs by interpreting information primarily from
four sources: mastery experience, vicarious experience, social persuasions, and physiological
reactions (Clark & Estes, 2008; Usher & Pajares, 2009). Physicians are considered masters in the
field of healing arts, however, they may be less confident in their ability to master the EHR and
the patient portal. Therefore, providing ongoing training may help them master the use of the
patient portal. Furthermore, as more physicians become adept at using the patient portal, social
persuasion may motivate others gain confidence in using and developing work-flows and
shortcuts to navigate the patient-portal. Physicians need to find a way to motivate themselves and
find the confidence to work through their issues with the utilization of the EHR by enhancing
human accomplishment and well-being in countless ways.
Expectancy-value theory. Expectancy is an individual’s judgment of their capabilities,
and value is an individual’s belief about the importance of the reasons why they may engage in
specific tasks. According to expectancy-value theory, individuals’ expectations of being able to
perform relevant tasks or behaviors are essential determinants of their motivation (Oftedal et at.,
2010). For illustration, it has been suggested that individuals are more motivated to seek

28

challenging tasks, like exercise and meditation, when they value their experience of
accomplishing the task. Furthermore, Wigfield and Eccles (1992) believe that individuals’
expectations of being able to perform relevant behaviors are influenced by their perceptions of
support from others. As for the motivation influence to provide quality care through the portal,
the PCPs should see the value that the patient-portal adds to the complexity of care of their
patients. Not surprisingly, physicians play an influential role in patients’ use of the portal, as
their endorsement is a factor in the adoption of these tools (Shah & Liebovitz, 2017). The value
that patients receive from the use of the portal will be directly affected by the attitudes and
actions of their PCPs. Clinician engagement with patient-portals is essential to achieve and
sustain anticipated benefits to clinical outcomes.
Table 3 provides a summary of the two motivational type influences and how they will be
assessed.
Table 3

Motivational Influence, and Motivational Influence Assessment

Organizational Mission

St. Elsewhere Hospital's mission is to strive to be the trailblazer in quality healthcare that is
compassionate, innovative, and stands for empowerment, integrity, collegiality, and vitality.

Organizational Global Goal

By June 2025, all primary care physicians will send a secure message to 80% of their panel
through the patient-portal during the reporting period.

Stakeholder Goal

By June 2021, all primary care physicians will send a secure message to 30% of their panel
through the patient-portal during the reporting period.

Assumed Motivation Influence

Motivation Influence Assessment

29

Self-Efficacy –
Physicians need to feel confident in their
ability to master today's patient-portal.

Interview

Expectancy-Value –
Physicians should see the value that the
patient-portal adds to their complexity of care
of their patients.

Interview

Organization
The third and final step of the problem of practice is an analysis of the organizational
barriers, which are conditions or processes that cause problems or barriers and prevent or delay
patient-physician communication through the patient-portal. The third influence of this review
will focus on the organization. In all work settings, the employees know what, when, and why
they are supposed to do something to achieve the organization’s goals (Rueda, 2011). The
organization’s culture certainly filters and affects all efforts to improve performance, and
successful performance improvement will depend on taking the specific organizational culture
into account. Rueda (2011) states that many institutions strive to be organization-learning
institutions that refer to how organizations, not merely individuals, learn and adapt to challenges
and changes in the environment. Given the professional impacts, health care organizations have a
vested interest in cultivating physician engagement is suggested. This review of the
organizational influence focused on the cultural settings and models to better understand how St.
Elsewhere Hospital supported their PCPs in communicating with their patients through the
patient-portal and improved the quality of clinical care beyond the office visit.
Organizational Culture
The Merriam-Webster dictionary defines corporate culture as the set of shared values,
goals, attitudes, and practices that characterize an organization (Merriam-Webster, N.A.).

30

Culture is both a dominant force in performance and a difficult characteristic to identify and
influence (Rueda, 2011). Culture and cultural processes are often thought of as relating to
individuals rather than organizations, which makes it challenging to define and operationalize for
several reasons: 1) they are not always visible; 2) considered cultural knowledge (automated),
and not always transparent, so they are not easily accessible to individuals within an
organizational setting; and 3) they involve relative values (Rueda, 2011). Clark and Estes (2008)
state organizations develop different cultures over time, and that work culture is present in the
conscious and unconscious understanding of who they are, what they value, and how they do
what they do as an organization. SEH administration should strive to request all physicians work
in the patient-portal when communicating with their patients. Healthcare professionals have
become a threat to the ability of most healthcare organizations to distribute quality care and
effectively serve their communities. The core beliefs that characterize all organizational cultures
can guide decisions about goal selection and the processes and procedures used to achieve those
goals (Clark & Estes, 2008). For SEH to focus on its goal that all PCPs will send a secure
message to 80% of their panel through the patient-portal during the reporting period by June
2025, they should explore their cultural settings and models.
Cultural Settings
Cultural settings can impact behavior. They are also shaped by individuals and groups
who operate with cultural models that impact their behavior (Rueda, 2011). These settings can
help one think about the more visible aspects such as work settings, classrooms, and meeting
places, where these models develop and play out (Rueda, 2011). Cultural settings represent
visible, audible behavior patterns and norms. The literature suggests that secure messaging
through the patient-portal can facilitate access to care, improve patient satisfaction, and improve

31

health outcomes (Sieck et al., 2017). St. Elsewhere Hospital needs to examine their culture
settings to secure that all physicians work within the patient-portal to improve on their patient’s
outcomes.
A culture of learning. Peter Schein (2017) argues that organizational culture is a kind of
learning. It is a learned product of group experience, and its strength is a function of the
convictions of an organization's founders, the stability of the group or organization (Namada,
2018). Organizational learning occurs when any part of an organization’s units acquires
knowledge that the unit recognizes as potentially useful to the organization (Namada, 2018).
Looking back on the roll-out of the patient-portal, it is unclear if the organization provided
adequate training to the physicians. Primary care physicians have identified the patient-portal as
an essential factor in dissatisfaction with one’s EHR is associated with an intent to reduce
clinical work hours and leave the current practice. Learning within organizations is dependent
upon being able to transmit relevant information without distortion to enable high levels of
understanding and productive decision making (Namada, 2018). Organizations benefit from
operating under the premise of a learning culture and conversely to resistance to change.
Adapt to new practices and structures. Resistance to change refers to the action taken
by groups and individuals when they perceive that a change happening is a threat to them (Cinite
& Duxbury, 2018). St. Elsewhere Hospital, like many others, had to adapt to the patient-portal,
because of the American Recovery and Reinvestment Act (ARRA) passed by the U.S.
government in 2009, shifting the healthcare industry into the digital age (Barrett, 2017). This
legislation mandated that all healthcare facilities adopt a certified EHR and patient-portal system
by 2015 and achieve the standard for meaningful use (Barrett, 2017). Studies have shown that
the EHR can change physician autonomy (Barrett, 2017). With meaningful use benchmark

32

placed on physicians sending secure messages to their panel of patients, the day-to-day
operations changed for many PCPs. St. Elsewhere needs to look at the norms and activities of
their physicians and their adaptabilities to the digital age. Cultural settings can impact behavior
and are also shaped by individuals and groups (Rueda, 2011).
Cultural Models
Cultural models are the shared mental schema or normative identifications of how the
world works, or ought to work (Rueda, 2011). Culture models help define what is customary and
standard within the organization (Rueda, 2011). Academic studies on planned organizational
change have suggested that organizational communication variables also have an impact on
worker resistance to change and assimilating formal change into mainstream activities (Barrett,
2018). Cinite and Buxbury (2018) suggest that to support change and encourage others to do the
same, employees who feel that their organization is supportive of them, values their
contributions, and cares about their needs, interests, and well-being identify to the standards
within the organization.
Open communication and transparency. Healthcare in today’s digital age, having a
culture of open communication and transparency, keeps the physicians and administration
aligned. The implementation of the patient-portal was a massive undertaking for most healthcare
institutions, causing a potential risk of a culture of silence. The definition of a culture of silence
is a collective-level phenomenon of saying or doing very little in response to substantial
problems that face an organization (Rathert et al., 2019). This culture is also known as a lack of
communication within the organization. Studies have shown that culture in healthcare
institutions is designed to keep everything and everyone in silos (Barrett, 2017; Shanafelt &
Noseworthy, 2017). Sharing problems and communicating best practices across all physicians

33

and medical staff is not the norm. The EHR also interferes with interpersonal relationships
between the physician and their patients. The expectation that physicians document care
immediately requires clinicians to focus on the EHR rather than look at patients read their body
language or pay attention to their stories (Rathert et al., 2019). Therefore, encouraging a culture
of open communication and transparency is a cultural model that should be emphasized and
aligned wit the organizational mission of empowerment, integrity, collegiality, and vitality.
Table 4 discusses the assumed organizational influence and organizational influence
assessments.
Table 4
Assumed Organizational Influence, and Organizational Influence Assessment

Organizational Mission

St. Elsewhere Hospital's mission is to strive to be the trailblazer in quality healthcare that is
compassionate, innovative, and stands for empowerment, integrity, collegiality, and vitality.

Organizational Global Goal

By June 2025, all primary care physicians will send a secure message to 80% of their panel
through the patient-portal during the reporting period.

Stakeholder Goal

By June 2021, all primary care physicians will send a secure message to 30% of their panel
through the patient-portal during the reporting period.

Assumed Organizational Influence

Organization Influence Assessment

Cultural Setting Influence 1 –
The organization needs to develop resources
and training to support the patient-portal.

Interview

Cultural Setting Influence 2 –

Interview

34

The organization needs to adapt to new
practices and structures for the digital age.

Cultural Model Influence 1 –
The organization needs to develop a culture of
open communication and transparency.

Interview

Conceptual Framework: The Interaction of Stakeholders’ Knowledge, Motivation,
and the Organizational Context
The conceptual framework is a key part of a research study’s design; it is the structure of
concepts, assumptions, expectations, beliefs, and theories that support one’s research (Maxwell,
2013). The conceptual framework explains either through narrative or visual forms that illustrate
the relationships among the key factors, concepts, or variables (Maxwell, 2013). The conceptual
framework used in this study is Clark and Estes’ (2008) gap analysis. A gap analysis is a
systemic problem-solving approach to help improve performance and achieve organizational
goals. If a gap exists between actual performance and desired performance, a close examination
of the knowledge and motivation of stakeholders, and the organizational factors that influence
the stakeholder can identify the specific causes for the performance gap (Clark & Estes, 2008).
Taking the conceptual knowledge; the PCPs have regarding patient-center care in the
digital age, physicians must understand the benefits and consequences of using the patient-portal.
For some PCPs, it was a change in the way they delivered their services when the patient portal
was introduced, and they had to incorporate the digital platform into their daily practice. By
changing to the digital format, PCPs may feel they have mastered this new style of practicing
medicine. The inquiry is whether physicians are motivated to work in this new environment. The
inefficiency in the practice environment, including clerical burden, is a universal driver of
dissatisfaction (Shanafelt et al., 2017). The gap analysis resolves to identify whether the

35

physicians have adequate knowledge, motivation, and organization support to achieve the
organization's goal (Clark & Estes,2008). To give a visual of the conceptual framework, Figure 1
illustrates how St. Elsewhere Hospital’s cultural settings and models intertwine with the
knowledge and motivation to achieve the overarching goal.

36

Figure 1
Conceptual Framework: Interaction of Stakeholder Knowledge and Motivation within
Organizational Cultural Models and Settings

St. Elsewhere Hospital
Cultural Settings
Fostering a Culture of Learning
Adapt to New Practices and Structures

Cultural Models
Open Communication and Transparency

By June 2021, all primary care physicians will send a secure message to
30% of their panel through the patient-portal during the reporting period

Primary Care Physicians

Conceptual Knowledge

Motivation

Self-Efficacy

Expectancy-Value

37

St. Elsewhere Hospital’s organizational culture needed to support the achievement of the
goal, represented by the large blue circle in Figure 1, which encompasses the smaller green circle
with ease. St. Elsewhere Hospital’s culture of learning is the resistance to change, which
represents the action taken by groups that perceive that the change happening is a threat to them
(Cinite & Duxbury, 2018). For example, SEH had a division of physicians who resisted moving
onto the EHR until 2018, which was three years after the legislation mandated all healthcare
institutions adopt a certified EHR (Barrett, 2017). Barrett (2017) recognized that many aspects of
the physician's daily routine had changed dramatically during the past generation, adding another
layer of complexity to the management of healthcare; doctors are increasingly unhappy.
Resistance to change plays right into the research question of interaction between St. Elsewhere
Hospital’s culture and context, and physician knowledge and motivation to the goal by June
2021, that all PCPs will send a secure message to 30% of their panel through the patient-portal
during the reporting period.
The conceptual framework Figure 1 has a large blue circle that represents SEH and its
cultural settings and models. Culture of learning, along with new practices and structure, are
cultural settings, and below that is a culture of open communication and transparency, which
represents a cultural model. This model is a collective-level phenomenon of saying or doing little
in response to substantial problems that face an organization (Cinite & Dexbury, 2018). Within
this sizeable blue circle is a smaller green circle that represents the stakeholders of SEH, the
PCPs. The green circle represents the physician's knowledge and motivation influences, with
conceptual knowledge being present. Self-efficacy theory and expectancy-value theory represent
physician motivation. The smaller green circle is embedded inside, the more substantial blue
circle, for both circles represent the organization as a whole. There is a blue arrow connecting the

38

two circles to the orange box, which is St. Elsewhere Hospital’s organizational goal by June
2021, all primary care physicians will send a secure message to 30% of their panel through the
patient-portal during the reporting period.
Conclusion
The purpose of this study was to understand better how PCPs needed to open
communication with their patients and improve clinical care through the patient-portal. This
literature review examined adverse consequences on patient care, the quality of physician work
processes and physician work-life balance, and all factors related to working within the EHR and
the patient-portal. The influences of the physician’s knowledge, skills, and motivation to achieve
the goal and identification of the potential cultural settings and models of the organizational
barriers are impeding the physician’s ability to achieve their goal. The knowledge, motivation,
and organizational (KMO) conceptual framework model was used to understand the knowledge
of the stakeholders at SEH to reach the organization goal of 80% of all primary care physicians
sending a secure message to their panel through the patient-portal, by June 2025. In the next
chapter, the study’s methodological approach will be explained in detail.

39

CHAPTER THREE: METHODOLOGY
The purpose of the study was to investigate PCP's experiences working within the
patient-portal to open up patient-physician communication and improve clinical care beyond the
office visit. The study aimed to examine the knowledge, motivation, and organizational
influences that impede SEH from reaching its organizational goal of 80% of all PCPs sending a
secure message to 80% of their panel through the patient-portal during the reporting period
ending June 2025. The physician's concerns regarding patient-portal usages included fears of
increased workload, quality of physician's work processes, and work-like balance that
contributed to the lack of portal usage by the physicians.
The descriptive study conducted a series of interviews with primary care physicians at St.
Elsewhere. Questions included:
1. What is St. Elsewhere Hospital physician’s knowledge and motivation related to the goal
of sending a secure message to 30% of their patient panel through the portal during the
reporting period, in 2021?
2. What is the interaction between St. Elsewhere Hospital’s culture and context, and
physician knowledge and motivation to the goal of sending a secure message to 30% of
their patient panel through the portal during the reporting period, in 2021?

Methodological Approach and Rationale
The methodological approach that was used for this descriptive study was qualitative, and
the inductive approach goal was used to determine gaps in the organization based on the data.
The rationale for conducting a case study was an empirical inquiry that investigated PCP usage
of the patient-portal within a real-life context. Creswell and Creswell (2018) stated that a case
study was qualitative in design in which the researcher explores in depth a program or group of

40

people. The study is a descriptive and exploratory analysis of a group of physicians facing the
dilemma of working within the patient-portal to provide communication and quality patient care
while working in an academic institution.
Sampling and Recruitment
Participating Stakeholders
The stakeholders for this study were 10 PCPs. Primary care is a staple in a medical
institution; it is the day-to-day healthcare option, which acts as the first contact and primary point
of continuity of care for patients. In academic medical institutions, the surgical departments
bring in the majority of the funding, but primary care feeds the institutions with a steady stream
of patients.
Interview Sampling Strategy, Criteria, and Rationale
The interview sampling criteria for participation of the PCPs was:
1. All participants needed to be full-time physicians at St. Elsewhere Hospital.
2. All participants needed to be credentialed by the St. Elsewhere Medical Group.
3. All participants needed to be fully immersed in the ambulatory clinics and work in the
EHR.
The rationale for the interview criteria was that PCPs were full-time primary care physicians
at SEH, that worked in ambulatory clinics, and most importantly, worked in the government-
mandated EHR and patient-portal. The researcher wanted to understand the mindset of these
PCPs and drill deep into how they used the patient-portal. The research conducted on empirical
articles indicated that the implementation of the EHR was causing frustration with the PCPs
(Dyrbye et al., 2017; Shanafelt et al., 2016).

41

Criterion 1. All participants needed to be full-time physicians working at SEH in
primary care.
Criterion 2. All participants needed to be PCPs credentialed through both SEH and St.
Elsewhere Medical Group. The rationale for this criterion was that every participant had to be
able to admit patients into the hospital and see those patients at all off-site clinics.
Criterion 3. All participants were a PCP that was fully immersed in the ambulatory
clinical EHR. These physicians were able to give recommendations of knowledge, motivation,
and organizational solutions to working within the patient-portal for the ongoing care of their
patients.
Due to the COVID-19 crisis and unforeseen circumstances, one of the 10 physicians
interviewed (Amy) had to lower her full-time status to 60%. The researcher was informed of the
situation after all of the interviews were completed by the interviewer. Due to the low sample
size, the researcher chose to include Amy’s interview data in the study.
Explanation for Choices
The qualitative interview method was chosen because the researcher was looking for
more in-depth information that the physicians could provide through their thoughts, beliefs,
knowledge, and motivation of the implementation of the EHR and the patient-portal (Johnson &
Christensen, 2014). The researcher chose only this form of data collection because the number of
available PCPs was quite limited and wanted to conduct a loosely structured interview to get the
physicians to open up and explore the issues of working within the patient-portal.
Data Analysis
Qualitative research was based on the belief that people, in an ongoing fashion, construct
knowledge as they engage in, and make meaning of, an activity, experience, or phenomenon

42

(Merriam & Tisdell, 2016). The researcher was looking for the knowledge, motivation, and
organizational influences that PCPs have when experiencing working and communicating with
their patients through the patient-portal. Merriam and Tisdell (2016) further explain qualitative
researchers conducted necessary qualitative study was interesting in 1) how people interpret their
experiences; 2) how they construct their worlds; 3) what meaning they attribute to their
experiences; in other words, the general-purpose was to comprehend how people make sense of
their lives and their experiences. The type of qualitative research that was used in this study was
within a bounded system. The rationale for conducting a study was an empirical inquiry that
investigated physician experience working in the patient-portal within a real-life context.
Creswell & Creswell (2018) stated that a study was a qualitative design in which the researcher
explored in depth a program or group of people. It was a bounded system because the study
focused on PCPs who worked in outpatient or ambulatory clinics at SEH.
Documents and Data Collection
To support the interviews of the PCPs, the researcher obtained two types of historical
data from the current EHR. This data represented the usage of the EHR and the patient-portal by
the PCPs over the past two years. St. Elsewhere Hospital’s fiscal year runs from July through
June, FY2019, and FY2020. The first data set represented the number of portal messages sent by
the PCPs per month from the patient portal. The second data set represented the amount of time
the PCPs at SEH spent working in the EHR after the standard 10-hour workday. The time frame
of the captured data was from 6:00 PM to 6:00 AM. Due to the timing of the request, the data
from FY2020 only represented the first 10-months of the fiscal year. The researcher received the
data from the Information Technology (IT) Department, which maintained the data from the
EHR product called Cerner. Both sets of data embodied all 32 PCPs at SEH. The IT Department

43

could not separate the 10 physicians that participated in the interviews, so the data represents all
PCPs. The data was presented to the researcher with no identification of the PCP. The researcher
analyzed the data for the number of secure messages were sent per month, and the total among of
time PCPs spent in the EHR per month. The information was insightful to demonstrate the
knowledge and motivation the PCPs had regarding their usage of the patient-portal.
Interviews
Interview protocol. The qualitative interview method was chosen because the researcher
was looking for more personal information from the physicians, provided through their thoughts,
beliefs, knowledge, and motivation of working and communicating with their patients through
the EHR and patient-portal (Johnson & Christensen, 2014). A face-to-face interview was ideal
for the researcher to conduct a semi-structured interview. The interview criteria was 1) primary
care physicians were a full-time PCP at SEH: 2) that worked in ambulatory clinics: and 3) most
importantly, worked in the government-mandated EHR. The researcher sought to understand the
mindset of these PCPs and drilled deep into the experiences the PCPs had working in the patient-
portal. The interview questions focused on the physician’s knowledge of the impact of the
quality of care they could provide through the portal, and their understanding of the
consequences of communicating with their patients through the portal. The interview then moved
to ask what motivated the physicians to feel confident in their ability to master today’s portal and
the value in using the patient-portal. Finally, the researcher wrapped up the interview with what
influences the organization had to develop resources and training to support the patient-portal,
the need to adopt new practices and structures for the digital age, and if the organization
developed a culture of open communication and transparency. For the interview protocols, see
Appendix A.

44

Interview procedures. The interviews were conducted in May of 2020, just after the
collection of historical data was retrieved from the EHR. The researcher wanted the data to
determine how much time the PCPs actually spent in the EHR, and if they were really working in
the EHR after their daily clinics and on the weekends. The findings from the interviews were
analyzed using the analytic codes based on the assumed influences of the knowledge, motivation,
and organization. Open coding and tallying were used to identify themes and axial coding to
group and code physician’s responses representing their ideas, concepts, and beliefs (Maxwell,
2013; Merriam & Tisdell, 2016). A high threshold was set due to the small sample size;
therefore, the criteria for determining the validity of assumed influences were as follows:
assumed influences were validated as an asset when the threshold of 80% or more of the PCPs
gave responses confirming the correct influence. The themes receiving a confirming result lower
than 80% was considered a need or gap in the influence.
To safeguard the participants and protect the integrity of the data, an interviewer not
associated with the SEH Ambulatory Care unit, and who has no reporting relationship with or
supervisory authority over, the participants of the research study, was utilized. The researcher
selected the interviewer because she worked in research on the medical campus but not directly
with PCPs. Because of her research background, she had a working knowledge of the patient-
portal and clinics. The researcher had no relationship with the interviewer before the study. Due
to her current positions, the interviewer had acquired the SEH Human Subjects certification
through iStar and CITI training. The researcher provided the interviewer with an interview
protocol to guide the discussions and interview 10 PCPs. As indicated in the protocols, the
interviews were semi-structured; the questions were more flexible and was mixed with more or
less structured questions (Merriam & Tisdell, 2016). It gave the interviewer the freedom to use

45

probing questions to go more in-depth on the subject. All interviews were audio-recorded (with
participant’s verbal consent) and transcribed. The interviews were conducted via Teams
conference calls, so they were comfortable and in a private setting and lasted between 30 and 45
minutes. The independent interviewer de-identified the transcripts and except for the participant's
gender and years of service before providing the interview transcripts to the researcher.
Credibility and Trustworthiness
Positivists generally often question the trustworthiness of qualitative research, perhaps
because their concepts of validity and reliability cannot be addressed in the same way in
naturalistic work. In qualitative research, one of the assumptions is that reality is holistic,
multidimensional, and ever-changing; it is not a single, fixed, objective phenomenon waiting to
be discovered, observed, and measured as in quantitative research (Merriam & Tisdell, 2016). To
ensure credibility was one of the essential factors in establishing trustworthiness, the main
emphasis of a qualitative proposal was on how the researcher ruled out specific plausible
alternatives and threats to the interpretation and explanations. The researcher must provide a
clear argument that strategies described will adequately deal with the particular threats in
questions, in the context of the study being proposed (Maxwell, 2013). For this proposal, the
researcher provided three strategies to deal with the credibility of the study, and they were; 1)
rich data; 2) triangulation; and 3) respondent validation. These strategies were not to verify
conclusions but to test the validity of the conclusions and the existence of a potential threat to
those conclusions (Maxwell, 2013).
Credibility Strategies
Rich data. The researcher provided data that was detailed and varied enough that it
provided a full and revealing image of what was going on in the PCPs workday when working in

46

the patient-portal. The researcher audiotaped each interview and provided a verbatim transcript
of that interview (Maxwell, 2013). Besides the interview, the researcher obtained data from the
EHR on historical time-stamped entries of when the PCPs worked in the EHR after the allotted
ten-hour workday.
Triangulation. The researcher involved different methods, especially personal
knowledge, data analysis, and individual interviews, which are data collections strategies known
as triangulation (Shenton, 2004). The use of different methods in harmony compensated for their
individual limitations and utilized their own benefits (Maxwell, 2013). In this study, the
researcher used supporting data obtained from the EHR to provide a background to help explain
the attitudes and behaviors of the PCPs. At the same time, the researcher verified particular
details that the physicians had shared in their interviews. Individual viewpoints and experiences
could be verified against other physicians, and ultimately, a rich picture of the attitudes, needs, or
behavior of the physicians were constructed based on the contributions of a range of people
indicating triangulating via data sources (Shenton, 2004).
Respondent validation. A respondent validation or member check is the single most
important provision that can be made to bolster a study’s credibility (Maxwell, 2013; Shenton,
2004). Checks relating to the accuracy of the data take place at the end of the data collection
dialogues, or on the spot. The PCPs may be asked to read any transcripts of dialogues in which
they have participated; here, the emphasis should be on whether the physicians consider that
their words match what they intended (Shenton, 2004). In this study, each interview was auto-
recorded; the articulations themselves should at least have been accurately captured.
Ethics

47

This study intended to focus on the meaning and understanding of PCPs using the
patient-portal to increase patient-physician communication and improve quality medical care
through the portal. To explore the psyche of what causes PCPs to not want to communicate
through the patient-portal, the researcher needed to interview PCPs. The researcher ensured that
ethical principles were maintained throughout the study. To guarantee the safety of the
participants, a study proposal was submitted to the Institutional Review Board (IRB) at the
University of Southern California to ensure all ethical considerations concerning the human
subject were addressed. A core requirement of the IRB was for the researcher to obtain informed
consent from each participant of the study. Informed consent forms include documentation that
participation in the study was voluntary, the review of any potential risks, confirmation that all
the discussions were kept confidential, and acknowledgment that participants could withdraw
from the study at any time (Glesne, 2011; Rubin & Rubin, 2012).
Ethical responsibilities also explicitly included the role the researcher took in the study to
avoid any potential confusion or feeling of pressure to engage in the research. The researcher
used a pseudonym for both the university and the affiliated hospital. According to Rubin &
Rubin (2012), it is essential to ensure that the identity of the participants is not apparent through
the organizational description and other information provided for the study. The researcher was
currently working as an operations director for the university’s ambulatory clinics and did not
have authority over the physicians who participated in this study. To further safeguard the
participants and protect the integrity of the data, an interviewer not associated with the
ambulatory clinics, and who has no reporting relationship with or supervisory authority over the
participants of the research study, was utilized. The researcher ensured the information that the
participants provided was held at the highest level of confidentiality (Glesne, 2011). The

48

individual interviews for this study were audio-recorded, and permission forms were obtained
from the participants, as well as verbal consent before the interview.

49

CHAPTER FOUR: RESULTS AND FINDINGS
The purpose of the study was to evaluate the use of the electronic patient-portal that
increased patient-physician communication and improved clinical care beyond the office visit.
The patient portal supports core functionality such as secure messaging, appointment scheduling,
bill management, and access to select laboratory results. Chapter Four focuses on the research,
including physician interviews and time-stamped document analysis of their use of the patient
portal. The Clark and Estes (2008) conceptual framework was used to examine the primary care
physicians’ knowledge, motivation, and organizational needs and assets for utilizing the patient
portal for ongoing patient care.
The research questions were as follows:
1. What is St. Elsewhere Hospital physician’s knowledge and motivation related to the goal
of sending a secure message to 30% of their patient panel through the portal during the
reporting period, in 2021?
2. What is the interaction between St. Elsewhere Hospital’s culture and context, and
physician knowledge and motivation to the goal of sending a secure message to 30% of
their patient panel through the portal during the reporting period, in 2021?
The gap analysis will help prioritize the recommendation to close the gap (Clark & Estes, 2008;
Rueda, 2011).
Participating Stakeholders
The stakeholders of focus for this study were primary care physicians, and all
participation was voluntary. Thirty-two primary care physicians work in the ambulatory clinics
at SEH. An email invitation was sent to all 32 potential participants that met the criteria. Out of
the 32 potential participants, 51% were female, and 49% were male. Their answers and identity

50

would remain anonymous throughout each step of the process. A reminder email was sent
seven days after the initial email, and all interviews were conducted over a two-week period.
The participation rate for the interview was 31%. In March 2020, the COVID-19
(Coronavirus) pandemic escalated in the US, posing a technological challenge to practitioners,
who were forced to rapidly transition to telemedicine for many elements of clinical care (Fong et
al., 2020). Practitioners and non-practitioners alike were pushed to be innovative, and the priority
of the primary care physician changed, focusing on the welfare of their patients, therefore the
participation for the interviews was low. Each interview was audio-recorded, then transcribed
and analyzed. For the participation rate of 29% or 10 of the physicians, the demographics for the
individuals were 33% (6) females and 24% (4) males. In order to protect the anonymity of the
physicians, each physician was assigned a pseudonym. Primary care physician’s years working
in the ambulatory arena ranged from five to 21 years of ambulatory experience, as shown in
Table 5, by the demographics of the physicians and their years as a PCP.
Table 5
Primary Care Physician's Demographics
Pseudonym Gender
Years as
PCP
Amy Female > 10
Bob Male < 10
Jack Male < 10
Jane Female < 10
Janice Female < 10
Kim Female > 10
Mike Male < 10
Nancy Female < 10
Sandy Female > 10
Tom Male < 10

51

Determination of Threshold
The study applied two data sources; interviews, and document analysis. The informal-
structured interviews were used as the primary source of data to assess the assumed knowledge,
motivation, and organization influences. The 10 interviews were transcribed, coded, and used to
triangulate and validate the interview findings.
The criteria for determining the validity of assumed influences were as follows: assumed
influences were validated as an asset when the threshold of 80% or more of the PCPs gave
responses confirming the correct influence. The themes receiving a confirming result lower than
80% was considered a need.
Findings
The findings for this study revealed recurring themes that emerged from the interview
protocol as they related to the assumed influences. The study participants had varying years of
working as a PCP in the ambulatory setting, varying levels of experience working within the
EHR and, in particular, communicating with their patients through the patient-portal. Even
though their individual experience as physicians in ambulatory clinics had many differences,
their insight into using the patient-portal as a means for providing quality patient care offered
several parallels.
Knowledge Findings
The knowledge segment of the study focused on determining if the PCPs understood the
consequences of communicating with their patients through the portal. The interview items from
the knowledge influence were also structured to tie back to the desire of the stakeholder goal. In
the knowledge findings, one influence that came to light was conceptual knowledge and the

52

themes that resonated through the interviews. Conceptual knowledge had three themes,
management of messages, effective communicator, and an increase in productivity.
Management of messages. The knowledge influence focused on understanding the
consequences of communicating with their patients through the portal. This influence represents
conceptual knowledge, which was defined as knowledge of categories, classifications, principles,
generalizations, theories, models, or structures pertinent to a particular area (Rueda, 2011). By
definition, conceptual knowledge cannot be learned by repetition. The physicians were asked to
give examples of how the patient-portal provided better or worse communication between them
and their patients. Seventy percent of the physicians provided negative communications on the
first question. Kim expressed concerns about communicating through the portal by stating,
"What doesn't work is when patients abuse the portal by sending 21 issues and expect the
physicians to answer all of them in one day. The patient-portal is not the right place for patients
to ask about acute issues." Sandy examined her communication with what she cannot see:
The physician doesn't know what the patient sees from their side of the portal. Training
should be given to the physicians so they can explain exactly what their patients are
looking for. At times I really don't know if my patients have seen the message I sent.
Janice echoed the same thought by stating, "it would be helpful if there were some notification to
the provider that the patient has not opened your message." Mike, when talking about the
management of the messages, said the timing of the replies can be an issue:
The patient sends a portal message on a Friday afternoon at 4:00 pm. The message goes
to a pool first and it is not answered until late Monday afternoon. If this was an urgent
message this could be a bad outcome.

53

However, three physicians provided positive communication between their patients. Jack
said he felt the patient-portal provided better communication because "it makes it much more
accessible, and better than phone tag" with his patients. Tom expressed that when working with
his chronic patients through the portal, it provided better communications:
I'm seeing someone every three months, let's say for their diabetes, and they want to send
blood sugar logs you know, every couple of weeks. They can send me a message through
the portal every few weeks, and they can say here are my numbers.
Effective communicator. The physicians were asked whether their communication
through the patient-portal is an effective form of communication with their patients. All 10
physicians provided responses. Kim felt the patient-portal was an effective form of
communication, and she stated, “I have been told I’m very clear in my speech and my
communication. I make sure I’m right to the point, and I have developed templates for routine
questions.” Similarly, Bob provided a couple of examples of why he is capable, “I know I am
effective with my communication by the replies I receive from the patients. Thank you. That was
very helpful, or thank you for the very helpful explanation and things like that.” Sandy stated,
“by direct patient feedback or just by patients responding through the portal.” By contrast, Jack
knows when he needs to take another approach to his communication, “On the downside, every
now and then there is confusion and talking over the phone is necessary for the complicated
issues. One can try and do too much by the portal and avoid phone calls and in-person visits.”
Managing portal messages is accomplished by providing patient-centered care, which is
defined as delivering care that is respectful of, and responsive to, patient preferences, needs, and
values that guide clinical decisions (Van den Bulck et al., 2018). The majority of the physicians

54

referred back to their nurse triage or pool of nurses to manage their portal messages. For
example, Kim describes the way she manages her messages:
My messages go to a general pool of nurses. They manage the messages and then escalate
the urgent or medical needs to me. The nurses can fill out paperwork and refill requests. I
would delegate to others, but for the most part, I end up doing 98% myself. I try to do the
urgent requests between patients, complete the discharge summary and then the
submission of the billing and medication reconciliation takes time, leading to overworked
physicians.
Similarly, Mike uses triaging to manage his messages:
Essentially, the staff immediately sees the message and they'll already sent it to me. If it's
something they can't handle, or they don't know how to handle or whatever. If I initiate a
message, the message response will come back to me directly.
The patient-portal can be a lot for the physicians, but Jane manages her messages:
My care coordinator serves as a quick look through all my portal messages. If the
message is sent directly to me then I will answer them. I end up doing a lot of my portal
messaging after clinic, you know, between 5:00 PM - 7:00 PM.
By contrast, Bob “gives my patients specific instructions that the patient-portal is merely
a Monday through Friday, eight to five means of communication.” He very much sets the
expectation of how he will use the portal.
Increase in productivity. The physicians were asked if the patient-portal impacts
communication on productivity or workflow in their practice. All 10 providers provided
feedback on these questions, and 80% stated that the impact on communicating through the
portal disrupted productivity or workflow.

55

Mike described it as productivity, “It could take me one to two hours to respond to all of my
messages. That is probably the average range.” Kim has a similar experience with productivity:
Overall, day to day in the practice, it kills it. It's really hard, so I just leave my messages
to the end, unless they're urgent, so I'm done at a clinic at 5:00, but I'm on the portal till
about 6:00 pm or 7:00 pm.
Janice talks about the effect on efficiency in her clinic:
If you are trying to answer questions between visits, it could affect efficiency in your
clinic workflow because it is disruptive if you are getting messages between patients. I
check my messages at the beginning and at the end of the day.
During the conceptual knowledge session of the interviews, 40% of the providers spoke
of their concern regarding the financial aspect of working in the patient-portal. Jack felt that
working in the patient-portal made him more productive in terms of taking good care of his
patients, but "the downside is all these interactions on the portal, I do not get reimbursed for the
extra care I'm providing to my patients. So, the patients receive more care, better care, and we do
it for free." Nancy had the same sentiment regarding financial concerns:
Productivity in that I can take care of a lot more patients with the portal, however I'm not
getting paid for all the extra time I spend in the portal providing care, so one can say I'm
less productive financially for the organization.
Especially relevant to the increase in productivity, Bob stated:
It significantly increases productivity, but this is productivity that is not captured in how
many messages I have answered, so my chief doesn't know the time and energy I put out
in these messages. It also doesn't capture the productivity for billing purposes. My
productivity has significantly increased because you're basically, in a sense, doing a

56

patient billable visit through the portal. There is no compensation for the patient-portal,
but it has significantly increased productivity, from the standpoint of me addressing
problems treating problems, making diagnoses, and making recommendations.
The results of the conceptual knowledge influence from the interviews were that 70% of
the physicians provided negative comments regarding managing the messages through the
patient-portal. Conceptual knowledge must be learned by thoughtful reflective learning; all 10
physicians felt they were effective communicators through the patient-portal. Eighty percent of
the PCPs felt that the impact on communication through the patient-portal was a disruption to
productivity or workflow. The interview findings revealed that 50% of the interviewees
responded positively. To be considered an asset, their positive response is needed to meet or
exceed the 80% threshold; therefore, this appears to be a gap.
Motivation Findings
The study's motivation section concentrated on the physicians' need to feel confident in
their ability to master today's patient-portal and see the value that the patient-portal adds to the
complexity of care to their patients. The interview items from the motivation influences are also
structured to tie back to the aspirational stakeholder goal. In the motivation findings, two
influences were identified, and they were self-efficacy and expectancy-value motivation, and
each influence had themes that echoed through the interviews. Self-efficacy motivation had two
themes, and they were the physician’s ability to communicate and readjust the patient’s
expectations. The expectancy-value motivation also had two themes, access to care, and improve
communication.
Physician’s ability to communicate. The first motivation influence was focused on the
physician’s confidence in communication with their patients through the patient-portal.

57

Individuals form their self-efficacy beliefs by interpreting information primarily from four
sources: mastery experience, vicarious experience, social persuasions, and physiological
reactions (Clark & Estes, 2008; Usher & Pajares, 2009). During the interviews, the physicians
were asked to provide the level of confidence they have corresponding with their patients using
the portal. Nine physicians felt they were confident when corresponding with their patients. As a
response to this question, Jack stated, “nothing that comes to mind.” Amy felt very confident and
stated, “if I don’t feel confident, then I would pick up the phone and call them, which is pretty
rare, less than 10% of the time.” Janice stated:
If I feel the patient is not getting the meaning of my message, I will either set up an in-
person or a telemedicine visit. Sometimes it is really hard to explain things via the portal.
If it is something very complicated, or if it’s bad news or a difficult diagnosis, then I do
not like using the portal.
Bob was confident in his mastery of the portal:
I would say a high level of confidence because I think the people who use the portal have
some degree of sophistication to be able to use it. I can sort of gauge from their responses
if they get my meaning.
Tom indicated, “I have a high level of confidence that communication is effective to my patients.
I get most of the ultimate responses of gratitude and clarity from the patients.”
Readjust the patient’s expectations. The physicians were also asked if there was a time
when they did not feel confident using the portal. Individuals with higher self-efficacy, greater
belief in their own competence, and higher expectancies for positive outcomes will be more
motivated to engage in, persist at, and work hard at a task or activity (Rueda, 2011). The
outcome of this question was that 70% of the physicians provided examples of when they did not

58

feel comfortable communicating in the portal. All of these examples were what the patient was
doing or not doing. For example, Jane expressed her concerns:
When patients keep coming back with messages regarding the same question, it seems
that they are not getting what I am delivering over the portal. I know that I will need to
make a phone call or bring them back for a visit.
Amy responded with a concern when a mother who is anxious about her son's medical condition
that Amy did not think he had:
She was asking for repeat labs multiple times. I asked for them to see a specialist, and it
was going to take a few months. She kept coming back to me, saying, I want to do this. I
don't want to repeat the test. It's really hard to stop that cycle through the portal.
Nancy stated:
I never give bad news over the portal. I think people sometimes are trying to avoid a visit
by doing a portal message, so I tell them I'm sorry I can't take care of this over the portal.
Can you please make an appointment.
Sandy, Tom, and Bob felt differently about feeling confident in their ability to master the portal.
They felt there could be legal ramifications when corresponding with their patients. Sandy
affirmed her concerns by saying, "I don't feel confident making absolute statements on the portal.
Portal messages are recorded in the patient's medical chart. In the message, I will say this is
highly suggestive of this." Bob described his concerns regarding educational and medical literacy
by stating, "when someone does not have a high degree of medical literacy or even education,
you try to use appropriate language that people will understand, but there is still a concern." Tom
expressed his thoughts on language literacy by saying:

59

Some patients perhaps don't have English as their primary language, but they still try to
use the portal. Perhaps the messages aren’t quite getting through the way I wanted to, or
they are not fully understanding in the way I wanted to, they could write a message in
Spanish or any foreign language.
In all of these examples, the PCPs had to readjust the expectations of their patients, which
stemmed from higher self-efficacy, greater belief in their own expertise in mastering today’s
patient-portal.
The results of the self-efficacy motivation influence from the interviews were that 90% of
the physicians felt confident in their ability to communicate with their patients through the
patient-portal. All 10 PCPs felt that they had the motivation and confidence in their abilities to
defuse any situation their patients presented to them through the portal. The interview findings
revealed a combined score of 89% of the PCPs felt confident to master the patient-portal;
therefore, this influence is considered an asset.
Access to care. The second motivation influence focused on physicians seeing the value
that the patient-portal adds to the complexity of care of their patients. This influence represents
expectancy-value motivation. Eccles (2006) states that individuals are more likely to engage in
an activity when it provides value to them. All 10 of the physicians expressed that they see the
value in using the patient-portal. Amy stated, "Yes, because it allows me to communicate with
people more frequently, improves patient satisfaction, and my communication with patients. It
increases the number of touchpoints for them." Along those lines, Nancy viewed the value of the
portal the same, providing, "I think it's another way of communicating, and it's a way that you
could communicate asynchronously." Mike focused on the access to care the portal provides:

60

The access to care, which I think is a type of quality, has improved and therefore, I would
imagine the quality of care has improved. I would say that the EHR's various aspects, not
just the patient-portal, have improved. Medications are being reviewed from an electronic
health record versus handwritten every time, so the quality of care is the biggest chunk of
improvement.
Continuing in that same realm, Sandy stated, “Yes, the ability for them to see their labs, to
quickly communicate, just small questions that they have, that they may have forgotten to ask in
the office, or they might not ask.”
Improves communication. The physicians were asked if the patient-portal supported
chronic disease management and if the portal a valuable tool for their patients. Nine out of the 10
providers gave positive responses to the question regarding chronic disease management; Jack
provided a negative response. Tom was an advocate for the portal when it came to chronic
management stating, "Yes, chronic disease management is all about measuring things over time
and having the patients have the ability to send regular updates through the portal definitely
helps with chronic disease management." Amy felt similarly stating:
I feel it does because we can discuss small changes, make a sort of adjustment as we go,
and report back things like blood pressures and sugars. We can also make small
medication adjustments in-between visits without having to wait for a month, so the
portal is more proactive with patient care.
The physicians felt that the patient-portal was perfect for managing chronic illness. Janice
reflected, "yes, I think I support it, especially for my diabetic patients. I can refill medication and
send reminders for their eye exams. I can do all of this through the portal without them coming

61

in for a visit." However, Jack had not quite realized the potential that the portal has for chronic
management, stating, " I probably underutilize it for that purpose. "
The results of the expectancy-value motivation influence from the interviews were that
100% of the PCPs saw the value the patient-portal added to the care of their patients. Ninety
percent of the physicians felt that they have improved communication with their patients
regarding their chronic disease management. The interview findings presented a combined score
of 89% of the PCPs saw the value that the patient-portal adds to their complexity of care. This
influence is considered an asset.
Organizational Findings
The organizational portion of the study focused on the organization's need to develop
resources and training to support the patient portal, adapt to new practices, and develop a culture
of open communication and transparency. The findings are based out of one assumed cultural
setting and two assumed cultural models. The interview items from the organization's influences
are also structured to tie back to the aim of the stakeholder goal. In the organizational findings,
the assumed cultural setting had two themes, which were the quality of training and resources.
The assumed cultural settings had three themes, and they were patient portal enrollment,
promoting the portal, and communication.
Quality of training. The interview questions allowed the physicians to discuss their
impressions of how well the organization developed resources and training to support the
patient-portal. Under this cultural setting, two themes emerged from the interviews on the quality
of the training the organization provided the physicians. Sixty percent of the participants felt that
the organization provided no specific training on the portal. When asked if the organization
provided training, Sandy expressed, “not especially, and I would like to see more training.

62

Honestly offer a refresher course like every two years.” Several physicians felt the same. Janice
added, “ I actually never got trained on the portal, I just kind of figured it out by myself, so I
would say no.” Amy felt slightly different and stated, “it was part of my EHR training, and I
figured it out by talking to my colleagues, it’s not that complicated.” Jane and Nancy echoed the
same sentiment, saying they both did not recall any specific training on the patient-portal.
When questioned on what types of tools they felt they needed from the organization to
master the portal, Tom explained, "some basic training, best practices education would be nice
periodically and then of course notification of any significant updates would be beneficial."
Nancy stated, "it would be great if the portal could hook into our health maintenance
documentation, and it could send out messages to all of our patients saying it's time for your flu
vaccine, pap smear, and mammogram." Amy would like the organization to invest in the portal
so she could "do mass messages to our patients, like have a list of patients. It was a big problem
with the patient-portal that you can't create your own patient list and then email everybody on the
list.”
Resources. Besides asking about the organization’s training, the assumed influence
looked at the resources the organization has provided to support the patient-portal. Clark and
Estes (2008) state that culture plays a significant role in an organization. All 10 physicians
responded to the cultural setting, and 50% were in favor of the organization. Mike expressed his
concern with the lack of ongoing support stating, “support meaning manpower. For a long time,
we have needed more help in terms of clerical help and patient coordinators to help us go
through portal renewals. Help to call patients back or whatever it may be.” Tom reflected that he
felt the organization provided adequate training but stressed:

63

It would be nice to have higher-skilled staff to manage the message pools so physicians
are doing physician-level work and not bogging physicians down with work that can be
handled by people who can otherwise practice at the top of their license.
Jack echoed the same concern in his statement, “I feel that the organization needs to invest in
obtaining more staff to help screen the portal messages so we could get through them faster.”
Bob felt the organization needs a culture that supports and strengthens the practices stating:
The main tool I need would be the right resources (staffing), the groundwork that needs
to be done by getting the right complement of staff to field these portal messages. Nurses
with the appropriate experience could provide the patient advice and let me deal with
medical decision-making.
The results of the assumed cultural setting from the interviews were that 40% of the
physicians provided positive comments stating they felt the organization provided training to
support the patient-portal. For 50% of the PCPs, they felt there were adequate recourses
available to support the patient-portal. The interview findings revealed that 55% of the
interviewees responded positively to the cultural setting. To be considered an asset, their positive
response needed to meet or exceed the 80% threshold; therefore, this appears to be a gap.
Patient-portal enrollment. The assumed organizational influence was focused on the
organization's needs to adapt to new practices and structures for the digital age, representing the
cultural model. The physicians were asked to give examples of a new practice or structure the
organization introduced in the past year regarding the patient-portal. Half of the participants
responded that yes, they could provide examples regarding the portal within the last year. Tom
stated, "they started to do self-enrollment into the portal." Janice strengthened the theme by
responding, "there was a campaign to get patients to sign up for the portal, and it really made a

64

difference in communication within the organization." Jane recalled the organization had done
"small things like updated coding, information on shortcuts phrases. They put out educational
information specifically related to the current virus or isolation instructions.”
The physicians were also asked if they felt the organization had adjusted to the new
digital platform, and a little over half of the participants thought the organization had adjusted to
it. Janice agreed that the organization had supported the digital platform stating, "for such a large
organization, I think they have done an amazing job. The organization has been very nimble at
identifying staff that can help so that the physicians don't get overloaded and requiring people to
sign up." Mike expanded on the theme:
There's been attempts at marketing the portal. I mean, they made it easier to enroll.
Initially, the patient had to be invited to the portal. Now patients are given a phone
number, and they text the word Elsewhere Portal, and they are enrolled through their
phone. We have included instructions on how to join the portal in the depart summary.
Conversely, five physicians felt the organization had done nothing or very little to
welcome the digital platform. Tom felt “that the organization has not done well to get all
providers onto the platform. There are surgical services that are not using the patient-portal, and
it makes it difficult to coordinate services.” The primary care service needs to have all of the
faculty engaged on the patient-portal. The same thoughts came from Jack, stating:
It is frustrating when not everyone in the hospital is using the portal. It's frustrating when
you message another doctor within the portal, and you won't get a response. I would say
two-thirds of the specialties are not on the portal.
Due to the COVID pandemic, healthcare providers across the US have had to adapt to a digital
platform, and Kim felt, “I don’t think we have adjusted to the use of the portal. We have just

65

been thrown into it. It would have been healthier if there was a more structured time around the
portal.”
Promoting the portal. The second assumed organizational influence for a cultural model
was focused on the organization developing a culture of open communication and transparency.
For a successful change effort, solutions must consider and adapt to the organizational culture
(Clark & Estes, 2008). Regarding the culture of open communication and transparency, the
physicians at SEH indicated a couple of themes for this organizational influence. The physicians
were asked if the organization shifted its focus to promote the patient-portal to ensure active
participation. All physicians responded to this question, and 60% felt that the organization had
shifted its focus on participation. Both Sandy and Tom felt that the promotion to require all
patients to have an active portal account was a step in the right direction. Tom added, “having
patients enrolled at the time when they are registering” was a good idea. Bob explained that “the
organization encourages all physicians to check their portal messages on a daily basis, and they
are encouraging you to communicate with other physicians and patients through the portal.” The
hospital is promoting open communication between the providers and their patients.
However, as the hospital promotes open communications, the organization is not
promoting transparency, according to several responses during the interviews. Nancy stated:
Well, so they have done a lot. I don't think it works very well. They have tried to make it
so when patients check-in and they are asked if they would like to be on the portal and
they say yes, they are given a piece of paper on how to join. They need to invest in
educating our patients on how to use the patient-portal.

66

Kim also did not feel the organization had been fully transparent with all faculty, stating,
“clinicians have no choice, you need to be active on the portal, but there has been hesitation for
all providers to shift to the patient-portal.
Communication. The third theme coming out of the interviews regarding a culture of
open communication was how the organization tends to use email as the primary form of
communication to the faculty. Thirty percent of the PCPs felt that the organization had done an
adequate job of communicating to the providers. The physicians were asked how they felt about
the organization’s communications when making changes or upgrades to the patient-portal.
Sandy stated, “They have done an adequate job. The email form is the way they communicate
any changes or upgrades. I think that it is risky, as this information might get buried in your
email box.” Tom echoed that statement, “they do a fair job of communications and it is pretty
much through email messages when they update or make changes.” Both Nancy and Bob stated
that communication is through email, which most people will not read.
The final questions were focused on the organization’s communication regarding their
position on the innovation of the digital platform. All physicians responded, and not all felt the
organization had done a good job of communicating their position. Jack is not the best at reading
the mass emails that come out, so he might have missed it, declaring, “I have not sensed a clear
vision or digital direction of the organization.” Nancy stated:
The style of communicating is frustrating for those who would like to be able to
effectively use these things. It is frustrating because if you don't have everybody on
board, you can't have the system work with only half the people using it. It's an
ineffective system.

67

When reflecting on the innovation of the digital platform, Sandy declared, "is not something that
I think of, for the organization. I would like to see education for our patients on how to use the
patient-portal.” Bob is frustrated:
The physicians do not feel that they (physicians) are part of the decision-making process.
For example, when the organization picked the platform for the telemedicine rollout, the
physician had no say in which product we were going to use.
The results of the assumed cultural model from the interviews were that 50% of the
physicians felt the organization provided a new practice or structure within the past year
regarding the patient portal. Sixty percent of the physicians felt that SEH promoted the use of
the patient-portal. When it came to providing useful communication to the PCPs regarding their
position on the innovation and the digital platform, 30% of the PCPs felt the organization
provided communications. The interview findings revealed that 37% of the interviewees
responded positively to the cultural model. To be considered an asset, their positive response
needed to meet or exceed the 80% threshold; therefore, this appears to be a gap.
Historical Data
Data Analysis
The historical data retrieved from the IT Department represented the number of secure
messages sent by the PCPs to their patients during fiscal years 2019 and 2020 and also the
number of hours spent in the EHR after hours from 6:00 PM to 6:00 AM. The data in both
categories represented the first 10 months of each fiscal year, from July through April. Due to the
timing of the request, the full year of FY2020 was not available. Over the past two years, SEH
had a marketing campaign pushing to get their patients signed up to the patient-portal. This
campaign was referenced to by the physicians during their interviews. Janice stated, "there was a

68

campaign to get patients to sign up for the portal, and it really made a difference in
communication within the organization." Shown in Table 6, in FY2020 represented by the solid
orange line, the total number of messages (19,905) sent by the PCPs was 15% higher than
(14,844) messages in FY2019 represented by the solid blue line. The increase demonstrated the
PCPs are working more in the patient-portal than before, in FY2019. During the interviews,
Mike stated the patient-portal’s biggest problem was that it had become a second full-time job.
What was interesting about the data set was the time spent in the EHR does not correspond with
the number of messages sent. The time spent in the EHR after hours did increase by 25% from
FY2019 (dotted line in blue) to FY2020 (dotted line in orange). It appears, the PCPs are not
spending as much time after hours working in the EHR as they thought they had, or the increased
number of messages had not affected the efficiency of the response time to send out messages.
Several of the PCPs indicated their care-team handled many of the portal messages on their
behalf, which would explain the increase in messages sent.
When looking at the trend line for both years of after-hours time spent in the EHR, there
were very high and very low points that warranted further investigation. For the month of
February 2020, the amount of time spent in the EHR was at 22 hours. The COVID crisis did not
appear until March and April, so more research would need to be applied to understand the
increase. The pandemic began in March 2020, when many physicians started working from
home. The time spent in the EHR dropped rapidly, and it appears that with the closing of
ambulatory clinics, PCPs did not need to spend time after hours in the EHR, but further research
could help answer that question. What does stand out is the number of secure messages
skyrocketed due to the crisis. The summary of the data set is that portal messages are trending
up, and time spent in the EHR after hours pretty much mirrors the year before.

69

Table 6

Number of Portal Messages Sent by Primary Care Physicians
Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr
FY2019 1,235 1,125 1,150 1,343 1,243 1,112 1,375 569 1,350 1,426
FY2020 1,598 1,676 1,809 1,912 1,743 1,604 1,940 1,879 2,763 2,149
% Difference
23% 33% 36% 30% 29% 31% 29% 70% 51% 34%
Hours spent in the EHR After Hours by PCPs
Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr
FY2019 3:56:26 13:54:09 5:49:07 12:16:05 10:20:48 8:47:45 10:48:20 13:01:04 0:04:53 4:52:54
FY2020 15:57:39 9:24:08 22:48:05 0:35:41 19:49:24 14:42:51 19:46:59 22:00:04 12:53:54 1:50:05
% Difference
75% (-32%) 74% (-95%) 48% 40% 45% 41% 99% (-62%)

70

Synthesis
This chapter has presented the findings of the interviews and documented the EHR data,
as they related to the two research questions. The interviews were used as the primary source of
data to assess the assumed knowledge, motivation, and organization influences. For the assumed
influence results, see Appendix B.
The participant responses to the knowledge assumed influence that physicians need to
understand the consequences of communicating with their patients appears to be a need for the
physicians at SEH. Both themes for conceptual knowledge (79%) were the management of
messages and increased productivity, appearing to be a gap in the stakeholder's performance
goal. As shown in Table 7, the assumed knowledge influences conceptual knowledge has been
validated as a gap in the stakeholder's goal of sending a secure message to 30% of their patient
panel through the portal during the reporting period 2021.
Both the motivation assumed influence self-efficacy (89%) and expectancy-value (89%)
sections have shown that the physicians at SEH are motivated and feel confident in their ability
to master the portal and see the value that the portal adds to the complexity of care of their
patients. As shown in Table 7, the assumed motivation influences of self-efficacy and
expectancy-value are not a gap for the stakeholder's goal of sending a secure message to 30% of
their patient panel through the portal during the reporting period in 2021.
The physician's responses to the organization assumed influences were overwhelmingly
negative influences. The physicians (45%) did not feel the organization has developed resources
and training to support the portal nor adopted new practices and structure, and the PCPs (47%)
felt the organization had not developed a culture of open communication and transparency. As
shown in Table 7, the assumed organization influences cultural settings, and models have been

71

validated as a gap in the organizational goal of sending a secure message to 30% of their patient
panel through the portal during the reporting period in 2021.
Table 7
Assumed Knowledge, Motivation, and Organization Influences
Assumed Knowledge Influence Validated as a Gap?
Conceptual Knowledge
Physicians need to understand the consequences of
communicating with their patients through the patient-portal.
Validated

Assumed Motivation Influence Validated as a Gap?
Self-Efficacy Motivation

Physicians need to feel confident in their ability to master today's
patient-portal.
Not Validated
Expectancy-Value Motivation
Physicians should see the value that the patient-portal adds to
their complexity of care of their patients.
Not Validated

Assumed Organization Influence Validated as a Gap?
Cultural Setting

The organization needs to develop resources and training to
support the patient-portal.
Validated
Cultural Setting

The organization needs to adapt to new practices and structures
for the digital age.
Validated
Cultural Model

The organization needs to develop a culture of open
communication and transparency.
Validated

72

CHAPTER FIVE: DISCUSSION
Purpose of the Project and Questions
The electronic patient-portal offers the potential for increased patient-physician
communication and improved clinical care beyond the office visit. The patient-portal supports
core functionality such as secure messaging, appointment scheduling, bill management, access to
select laboratory results, and access to select electronic health record (EHR) data. The adoption
of the patient-portal was integrated into federal meaningful use metrics and is necessary for
receiving financial incentives associated with implementing the EHR (Chan et al., 2018). The
undertaking of the patient-portal and secured messaging is to improve patient outcomes,
utilization, and patient experience. Conversely, physician concerns regarding the patient-portal
use, including fear of increased workload, quality of physician work processes, and work-life
balance, may contribute to the lack of portal usage by the physicians (Arndt et al., 2017; Chan et
al., 2008; Dyrbye et al., 2017; Guo et al., 2017).
Research Questions

1. What is St. Elsewhere Hospital physician’s knowledge and motivation related to the
goal of sending a secure message to 30% of their patient-panel through the portal
during the reporting period, in 2021?
2. What is the interaction between St. Elsewhere Hospital’s culture and context and
physician knowledge and motivation to the goal of sending a secure message to 30%
of their patient-panel through the portal during the reporting period, in 2021?
Recommendations for Practice to Address KMO Influences
Knowledge Recommendations

73

Introduction. The knowledge influences in Table 8 represent the complete list of
assumed knowledge influences and their probability of being validated based on the most
frequently mentioned knowledge influences to achieving the stakeholder’s goal during informal
interviews. Individuals learn conceptual knowledge through reading, viewing, listening,
experiencing, or thoughtful and reflective mental activity (Krathwohl, 2002). As such, as
indicated in Table 8, it is expected that these influences have a high probability of being
validated and have a high priority for achieving the stakeholders' goal. Table 8 also shows the
recommendation for these highly probable influences based on theoretical principles.
Table 8
Summary of Knowledge Influence and Recommendation
Assumed Knowledge
Influence

Validated as
a Gap?
Yes, High
Probability
or No
(Y, HP, N)
Priority
Yes, No
(Y, N)
Principle and Citation Context-Specific
Recommendation
Physicians need to
understand the
consequences of
communicating with
their patients through
the patient-portal.
(D)
Y Y Information learned
meaningfully and
connected with prior
knowledge is stored more
quickly and remembered
more accurately because it
is elaborated with prior
learning (Schraw &
McCrudden, 2006)

Provide educational materials
so physicians can connect
new knowledge to prior
knowledge and to construct
meaning through their
communications with their
patients.

The knowledge influence was the need to understand the consequences of
communicating with their patients through the patient-portal. Conceptual knowledge was
selected to be further examined in light of supporting literature, and recommendations for
performance improvement.

74

Physicians need to understand the consequences of communicating with their
patients through the patient-portal. The interview findings showed that seven of the 10
physicians found that there were negative consequences to communicating through the patient-
portal, and eight of the 10 physicians felt the portal had negative consequences to managing the
messages on the productivity of their practice. In preparing the literature review, there is
evidence that shows primary care physicians need to gain conceptual knowledge when
communicating with their patients through the patient-portal. A recommendation embedded in
the information processing theory has been selected to close this conceptual knowledge gap, as
shown in Table 8. Information learned meaningfully and connected with prior knowledge is
stored more quickly and remembered more accurately because it is elaborated with prior learning
(Schraw & McCrudden, 2006). Physicians must conceptualize their communication with their
patients so that the patients understand and can follow the details of the communication and not
draw their own conclusions. The recommendation is to provide educational materials to help
physicians connect new knowledge to prior knowledge and to construct meaning (Schraw &
McCrudden, 2006) when communicating with patients through the patient-portal.
To better support, this recommendation is a study from Ohio State College of Medicine,
where Cynthia Sieck and her team examined the perspectives on secure messaging from
experienced ambulatory patient-portal users. Sieck et al. (2017) conducted 42 interviews, and
their findings revealed that information and training on the rules of engagement were needed for
communication within the patient-portal. Practice recommendations showed that patient-focused
information must be developed to set the tone for the rules of engagement and address issues
such as when secure messaging is appropriate, question topics that can be addressed via secure
messaging, what type of information to include in the messages, and how to understand the

75

information sent by the provider (Sieck et al., 2017). Patients are often confused with the model
of communication; rules of engagement are important for there are subjects and topics more
appropriate for the patient-portal vs. during telemedicine communications. In a study based in
2014 on the preference of the patients, the findings showed that physicians should consider the
timing of the release of sensitive test results, for instance, avoiding the release of test results just
before or during weekends or other off-hours if no staff are available to answer questions
(Giardina et al., 2015). The recommendation was made to standardize messages to develop
sound clinical practices for physicians when communicating test results in the patient-portal.
This should be developed and accompanied by strategies to help patients understand and manage
the information they receive (Giardina et al., 2015; Sieck et al., 2017). The evidence supports
that physicians need to understand the consequences of their communications to their patients
through the patient-portal.
Motivation Recommendations
Introduction. The motivation influences in Table 9 represent the complete list of
assumed motivation influences, and whether the influence is a priority to address in order to
achieve the stakeholder’s goal. The literature review supports the assumed motivational
influences. Motivation is viewed as what gets humans going, keeps them moving, and tells them
how much effort to spend on their work, play, or relationships (Clark & Estes, 2008; Rueda,
2011). Research has shown that there are three facets of motivated performance, and they are an
active choice, persistence, and mental effort (Clark & Estes, 2008). As such, indicated in Table
9, it is expected that these influences have no probability of being validated and a priority for
achieving the stakeholder's goal. Table 9 also shows the recommendation for these highly
probable influences based on theoretical principles.

76

Table 9
Summary of Motivation Influences and Recommendations
Assumed
Motivation
Influence

Validated as
a Gap
Yes, High
Probability,
No
(Y, HP, N)
Priority
Yes, No
(Y, N)
Principle and Citation Context-Specific
Recommendation

Physicians need to
be confident in their
ability to master
today’s patient-
portal.
(Self-Efficacy)

N

Y
Feedback and modeling
increases self-efficacy
(Pajares, 2006)
Learning and motivation
are enhanced when
learners have positive
expectancies for success
(Pajares, 2006)

Provide physicians with close,
concrete, and challenging goals
that allow the physicians to
experience success at the task
through modeling and feedback
opportunities.

Physicians should
see the value that the
patient-portal adds to
their complexity of
care of their
patients.
(Expectancy-Value)

N

Y

Individuals are more
likely to engage in an
activity when it provides
value to them. (Eccles,
2009).

Rationales that include a
discussion of the
importance and utility
value of the work or
learning can help learners
develop positive values
(Eccles, 2006; Pintrich,
2003).

Providing physicians with
model values, enthusiasm, and
interest in working in the
patient-portal will enhance their
care of their patients.

Provide physicians with the
rationales about the importance
and utility value of using the
patient-portal when caring for
their patients.

For this study, both motivation influences, self-efficacy, and expectancy-value were
found to be assets for the stakeholders. While these influences were not considered gaps, the
researcher felt it would be beneficial to monitor for changes and examine further supporting
literature and recommendations for performance improvement.
Physicians need to be confident in their ability to master today’s patient portal. All
10 of the physicians felt confident in their ability to master the patient-portal. They felt confident
in their patient’s ability to comprehend how they were communicating through the portal. The

77

literature review corroborates evidence that shows self-efficacy can promote motivation for
PCPs to master communication with their patients through the patient-portal. The interview
findings showed that three of the physicians were concerned about their patient’s education and
medical literacy. A recommendation advised by the self-efficacy theory is being proposed to
enhance their motivation to reach out to their patient that show signs of literacy and confusion.
Pajares (2006) declares that feedback and modeling increase self-efficacy. Communication with
primary care physicians is a concern of patients, as they often feel that their communication is
restricted to their face-to-face appointments, during which they often feel rushed and sometimes
intimidated (Bozan & Mooney, 2018). The recommendation to develop a process that supports
improved patient communication through feedback and modeling, with concrete goals, will
improve the patient and physician experience. This can be accomplished through the Patient
Feedback and Communication committee and the monitoring of the patient experience survey
scores.
To better support this recommendation, Michael, Dror and Karnieli-Miller (2019) study
focused on individual’s belief in their capability to apply high-quality communication with
patients through communication self-efficacy. The study focused on 653 medical students and
examined the associations between communication self-efficacy, communication attitudes,
empathy, attitudes, and the quality of patient-centered care (PCC). Self-efficacy has been found
to have a significant influence on behavior (Michael et al., 2019). The research suggested that
people tend to fear or even avoid what they think they cannot do. The results of this study
revealed that communication self-efficacy, communication attitudes, and empathy attitudes were
positively associated with PCC (Michael et al., 2019). A study at an academic center in Belgium
looked at health information needs, expectations, and attitudes toward PCC through the patient-

78

portal. Findings revealed the communication that patients wanted in the portal were as follows;
92.9% wanted timely test results, 84.1% wanted current medication, and 82.4% wanted past
medical visits, procedures, and surgery results (Van den Bulck et al., 2018). Physicians need to
be confident in their ability to master the patient-portal for patients who are demanding these
results to be posted.
Physicians need to see the value that the patient-portal adds to their complexity of
care of their patients. The interview findings showed that 100% of the PCPs felt that they saw
value in the access to care through the patient-portal, and 90% felt the portal improved their
communication and added to the complexity of care they provide. In preparing the empirical
research review, there was evidence to support that physicians should see the value that the
patient-portal brings to their ability to communicate with their patients. A recommendation
informed by utility value theory is being proposed to enhance the PCP's motivation to consider
the value their patients are receiving from the communication through the patient-portal and not
only their perspective. Eccles (2006) states that individuals are more likely to engage in an
activity when it provides value to them. Primary care physicians need to understand the value of
their participation in the patient-portal, not only for physicians to interact with their patients but
so that patients can access their medical records and medical exam results. The recommendation
is to provide the physicians with model values, enthusiasm, and interest in working in the
patient-portal, which is expected to enhance their care of their patients.
Patient-portal adoption is a new and growing field of study which shows evidence that
patient engagement improves health outcomes and reduces costs (Tavares & Oliveira,
2016). The adoption of patient-portals has become increasingly crucial for the delivery of
quality health care, including collaborations with health providers outside of clinical visits and

79

quick access to one’s personal health information (Irizarry et al., 2017; Tavares & Oliveira,
2016). A study on older adults and their adoption of the patient-portal as a tool for health care
engagement revealed that older adults needed to understand the value the patient-portal gives
them (Irizarry et al., 2017). In a study, the high and low health literacy groups felt that after they
received specific training on the benefits of the portal, they were more confident and understood
of the features of the portal (Irizarry et al., 2017). The study showed that older adults' willingness
to adapt to the patient-portal supported the recommendation of providing physicians with model
values, enthusiasm, and interest in working in the patient-portal, which added to their complexity
of care of their patients. A study by the University of Manchester supported the recommendation
of providing model values by offering a course in communication, which demonstrated that new
behavioral skills could be learned and integrated into the medical staff's daily routine (Bowman
et al., 1992). During the follow-up period, the acquired skills were not only maintained but also
further improved up to 37%, showing that training in communication skills gave value to their
performance of communication (Ammentorp et al., 2007; Bowman et al., 1992).
Organization Recommendations
Introduction. The organization’s influences in Table 10 represent the complete list of
assumed organization influences and their probability of being validated based on the most
frequently mentioned organizational influences to achieving the stakeholder’s goal during
informal interviews. The conceptual framework is a crucial part of a study’s design; it is the
structure of concepts, assumptions, expectations, beliefs, and theories that support the research
(Maxwell, 2013). The conceptual framework used in this study was Clark and Estes gap analysis.
A gap analysis is a systemic problem-solving approach to help improve performance and achieve
organizational goals (Clark & Estes, 2008). As such, indicated in Table 10, it is expected that

80

these influences have a high probability of being validated and have a high priority for achieving
the stakeholder’s goal. Table 10 also shows the recommendation for these highly probable
influences based on theoretical principles.
Table 10
Summary of Organization Influences and Recommendations
Assumed
Organization
Influence

Validated as
a Gap
Yes, High
Probability,
No
(Y, HP, N)
Priority
Yes, No
(Y, N)
Principle and Citation

Context-Specific
Recommendation
The organization
needs to develop
resources and
training to support
the patient-portal.
(Cultural Setting)

Y

Y
Effective change efforts
ensure that everyone has
the resources (equipment,
personnel, time, etc.)
needed to do their job and
that if there are resource
shortages, then resources
are aligned with
organizational priorities
(Clark & Estes, 2008).

Establish regular meetings with
the hospital and the Department
of Primary Care leadership to
discuss resources and training
for primary care providers on
effective ways to communicate
with their patients through the
patient-portal.
Work with the leadership of the
Department of Primary Care to
identify sources of funding for
training, equipment, and staff to
support the physicians working
in the EHR.

The organization
needs to adapt to
new practices and
structures for the
digital age.
(Cultural Setting)

Y

Y

Effective change begins by
addressing motivation
influencers; it ensures the
group knows why it needs
to change. It then addresses
organizational barriers, and
then knowledge and skills
needs (Clark & Estes,
2008).
Establish regular meetings with
the hospital and the Department
of Primary Care leadership to
discuss the EHR and provide
evidence on the benefits the
patients receive now that the
physicians are communicating
through the patient-portal.

The organization
needs to develop a
culture of open
communication and
transparency.
(Cultural Model)

Y

Y

Effective change efforts are
communicated regularly
and frequently to all key
stakeholders (Clark &
Estes, 2008)
Communicate the benefits of
the patient-portal to all
physicians and step out policies,
procedures, and expectations on
communicating through the
patient-portal.

81

The organization needs to adapt to new practices and structures for the digital age.
The interview findings showed that 50% of the PCPs felt the organization had promoted new
practices, like a campaign to enroll patients on to the portal. Seventy-five percent felt that the
organizations did promote the use of the patient-portal. In preparing the empirical research
review, there is evidence to support that today’s organizations must adapt to new practices to
compete in today’s digital age. A recommendation grounded in organizational leadership and
organizational change theory has been selected to close this gap. Effective change begins by
addressing motivation influencers; it ensures the group knows why it needs to change. It then
addresses organizational barriers, and then the knowledge and skills needed (Clark & Estes,
2008). Healthcare is continuously changing, and healthcare organizations need to adapt quickly
and ensure that physicians and medical staff are ready for the challenge. The recommendation is
to establish regular meetings with the hospital and the Department of Primary Care leadership to
discuss the EHR and provide evidence on the benefits the patients receive now that the
physicians are communicating through the patient-portal.
The needs and priorties change in the Healthcare setting, from new policies to new drug
treatment. The COVID-19 crisis is a prime example of introducing a new workflows and
structures to an existing clinic. Annals of Internal Medicine (2016) conducted a time and motion
study on physician time in ambulatory practice. The objective was to describe how the
physicians spent their time in the ambulance practice. Ambulatory care in the US has been
subject to dramatic burdens in the past decade due to cost cuts, the need to meet regulations, and
the transition to the EHR (Sinsky et al., 2016) The findings showed that physicians interacted
with the EHR 37.0% of their time when with their patients and 49.2% of their time within the
EHR and desk work (Sinsky et al., 2016). Providers who finalized after-hours diaries dedicated a

82

mean of 1.5 hours to after-hours work, with 59% of their time spent using the EHR and the
patient-portal (Sinsky et al., 2016). Due to the digital age, healthcare organizations need to
provide evidence to their physicians on the benefits their patients receive through the patient-
portal, because the physician’s daily workflows have been altered due to the digital age.
The organization needs to develop a culture of open communication and
transparency. The interview findings showed that 75% of the PCPs felt that the organization
promoted the use of the patient-portal. Conversely, 37% of the PCPs believed that the
organization lacked transparency in their communication style. The PCPs felt that the
communications should come in different forms and not only through an email. The physicians
felt the only form of communication the hospital use was email, and some of the physicians
never saw the communication. Empirical research has shown that healthcare organizations need
to establish a culture of open communication and transparency to accomplish their goals. A
recommendation that draws upon change theory has been selected to close this organizational
gap. Clark and Estes (2008) state that effective change efforts are communicated regularly and
frequently to all key stakeholders. The frequency with which a change mission is presented
throughout the organizational venues and meeting rooms have been found to generate a
considerable amount of energy around the change (Barrett, 2018). The recommendation is for the
hospital leadership to establish an internal marketing campaign to promote a culture of open
communication and transparency.
To better support, the recommendation, Ashley Barret from Baylor University (2018),
surveyed 345 employees in one healthcare organization looking at organizational change and
communication quality related to EHR implementation success, and EHR communication
quality. The results of their study revealed the key to overcoming EHR resistance regardless of

83

where it originates is communication (Barret, 2018). The benefits underlying the use of the EHR
must be highlighted to medical staff early in the implementation process, and the organization
needs to be honest about what the EHR can achieve and what roadblocks lie ahead (Barret,
2018). The recommendation that follows is to work with the marketing department to establish a
marketing campaign to promote a culture of open communication and transparency.
The organization needs to develop resources and training to support the patient-
portal. The interview findings showed that 40% of the PCPs felt that the organizations did not
provide adequate training of the patient-portal. Only nine physicians responded to the inquiries
regarding the development of resources for the patient-portal, which showed that 56% of the
comments were positive. In preparing the literature review, there was evidence that showed that
healthcare organizations need to develop resources and training to support their goals. A
recommendation justified in organizational leadership and organizational change theory has been
chosen to close this gap. Effective change efforts ensure that everyone has the resources
(equipment, personnel, time, etc.) needed to do their job and that if there are resource shortages,
then resources are aligned with organizational priorities (Clark & Estes, 2008). Healthcare
organizations should create printed educational materials that explain the patient-portal process
adapted for cultural and linguistic differences for the patients they serve (Kornacker et al.,
2019). The recommendation is to establish regular meetings with the hospital and the
Department of Primary Care leadership to discuss resources and training for primary care
providers on effective ways to communicate with their patients through the patient-portal.
Lewis University conducted a study that evaluated a program that focused on a 90-day
portal push marketing and re-education initiative for its patient-portal. Lewis University’s study
supported the recommendation and the study discovered that enhanced marketing and re-

84

education efforts increase portal registration numbers and use. The results revealed a substantial
increase from 25% to 82% of providers using the online patient-portal daily (Kornacker et al.,
2019). Before the portal push, only six of 24 providers were accessing the patient-portal link’s
daily utilization numbers, however after the portal push initiative, it showed an increase to 18 of
22 providers meeting CMS MU benchmark attestations (Kornacker et al., 2019). St. Elsewhere
Hospital administration should conduct regular meetings with the Department of Primary Care
leadership to discuss resources and training for PCPs on effective ways to communicate with
their patients when working within the patient-portal.
Implementation and Evaluation Framework
The implementation and evaluation framework that will be used to implement the plan is
the New World Kirkpatrick Model. Based on the original Kirkpatrick Model, the new version of
the World Kirkpatrick Model is an outcome-focused model that evaluates the outcomes of the
organization (Kirkpatrick & Kirkpatrick, 2016). The New World Kirkpatrick Model expands the
scope of the original model by adding concepts and process measures to enable educators to
interpret the results of the evaluation and to provide educational programs (Kirkpatrick &
Kirkpatrick, 2016). The new model starts at level four – different results, then moves down to
critical behavior – level three, learning – level two, and reaction – level one, respectively
(Kirkpatrick & Kirkpatrick, 2016). In the new model, the outcomes for level four is decided first,
and then the rest of the levels follow suit. Level four is the degree to which targeted outcomes
occur as a result of the training and the support and accountability package, then moves to the
behavior to which participants apply what they learned during training when back on the job.
Next, leading to learning, in which participants obtain the anticipated knowledge, skills, attitude,
and commitment based on their participation in the training, and finally ending with the degree to

85

which participants find the training favorable, engaging, and relevant to their jobs. The
Kirkpatrick Model is founded on the belief that training professionals can create and demonstrate
the organizational value of their training without hiring costly outside consultants (Kirkpatrick &
Kirkpatrick, 2016).
Organizational Purpose, Need and Expectations
The SEH mission is to strive to be the leader in quality healthcare that is compassionate,
innovative and stands for empowerment, integrity, collegiality, and vitality (St. Elsewhere,
2020). The purpose of practice being addressed in this study is the electronic patient-portal,
which offers the potential for increased patient-physician communication and improved clinical
care beyond the office visit. The patient-portal supports core functionality such as secure
messaging, appointment scheduling, bill management, access to select laboratory results, and
access to select electronic health records (EHR) data. Stakeholder goal is by June 2021, all PCPs
will send a secure message to 30% of their panel through the patient-portal during the reporting
period. The goal is to encourage PCPs to engage their patients to communicate via the patient-
portal. This goal achieves two wins for the organizations; first, it improves the frequency and
quality of communications the physicians have with their patients, and secondly, it fulfills
federal meaningful use metrics that are necessary for receiving financial incentives associated
with the federal government. The recommendation put forth in the project consists of a
comprehensive program that promotes the use of the patient-portal by the PCPs to close any gaps
and enable communication with their patients through the patient-portal.
Level 4: Results and Leading Indicators
Kirkpatrick and Kirkpatrick (2016) defined results as the degree to which targeted
outcomes occur as a result of the training and the support and accountability package. Table 11

86

shows the external and internal outcomes, metrics, and methods by which the PCPs will be
observed and measured. Short-term observations and measurements are called leading indicators
that suggest that critical behaviors are on track to create a positive impact on the desired results
(Kirkpatrick & Kirkpatrick, 2016). These indicators will advise the PCPs and leadership as to
whether the goal of sending secure messages through the patient-portal has been successful.
Table 11

Outcomes, Metrics, and Methods for External and Internal Outcomes
Outcome Metric(s) Method(s)
External Outcomes
Increase patient satisfaction score Yelp Reviews
CG-CAHPS Score Cards increase by
3% in “Access to Care” category

Monthly Yelp Reviews
Monthly CG-CAHPS Score
Reports

Improve communication to referring
physicians
Number of complaints from referring
physicians decrease by 2% of baseline
Feedback from community
practices

Increase the CG-CAHPS score on
providing lab results in a timely
manner
The time the physician received the
lab results to when physician contact
patient
Monthly CG-CAHPS Score
Reports

Monthly report from the IT
Department regarding lab
results

Internal Outcomes
Improve communication to patients Number of secure messages sent by
the PCPs to his/her patients by
increase by 3% of baseline
Aggregate monthly data from
Meaningful Use Reports

Monthly report from the IT
Department on the number of
secured messages sent out

Decrease the amount of time primary
care physicians spend in the patient-
portal outside of their workday
Time spent working in the patient-
portal outside of regular 10-hour day
Monthly report from IT
Department on physician-
patient-portal usage

Level 3: Behavior
Critical behaviors. Kirkpatrick and Kirkpatrick (2016) states that critical behaviors are the few
specific actions which, if performed consistently on the job, will have a most significant impact

87

on the desired results. Level Three is the most troublesome to traditional training evaluation
practices, and it is the most challenging part of an implementation and evaluation plan because of
the difficulty in supporting and holding the stakeholders accountable for applying their learning
(Kirkpatrick & Kirkpatrick, 2016). The stakeholders of focus are the PCPs. The first critical
behavior is that physicians need to know how the use of the patient-portal impacts their ability to
provide quality care to their patients. The second critical behavior is that physician understand
their role in increasing the use of the patient portal by actively promoting the patient portal to
their patients and discouraging the use of email or other electronic medium. Finally, the third
critical behavior is that the organization needs to develop resources and training to support the
patient-portal. Table 12 outlines the specific metrics, methods, and timing that demonstrate these
critical behaviors.
Table 12

Critical Behaviors, Metrics, Methods, and Timing for Evaluation

Critical Behavior Metric(s)
Method(s)
Timing
1. Physicians need to know how the use
of the patient-portal impacts their
ability to provide quality patient care.

Scores of the
patient-portal
Using the CG-CAHPS Scores plus
Patient Satisfaction Scores to measure
the quality improvement

Measured
monthly
2. Physicians understand their role in
increasing the use of the patient-portal
by actively promoting the patient-portal
to their patients and discouraging the
use of email or other electronic
medium.

The number of
behaviors
observed.

Providers will reinforce the use of the
patient-portal by letting the patient
know their commitment to the use of
the portal at each visit which can be
observed by the clinical staff.

Monthly
3. The organization needs to develop
resources and training to support the
patient-portal.
Number of
courses
completed

Compare courses available and a
budget for resources
Monthly
basis

Required drivers. The critical behaviors in Table 13 cannot occur on their own. For
mission-critical programs, there should be a solid plan for monitoring both compliance with
critical behaviors and progress toward desired results. The New World Kirkpatrick Model adds

88

required drivers, which are processes and systems that reinforce, monitor, encourage, and reward
the performance of critical behaviors on the job (Kirkpatrick & Kirkpatrick, 2016). Required
drivers fall under two headings: support and accountability. Support is vital post-training to
reinforce what was learned during the training and to reinforce it on the job. Reinforcing
behavior is a method that is designed and built in advance of the training, such as job aids,
reminders, refreshers, and on-the-job training modules. Encouraging behavior occurs on the job
by the supervisor and team members, encouraging each other by listening to challenges and
assisting in resolving them. Rewarding behavior ensues recognition and rewards for excellent
performance. Monitoring behavior is the method of accountability that must be in place for
essential initiatives and non-negotiable behaviors. The most common method is the post-program
90-day survey. Table 13 lists the required drivers to support the critical behaviors of the PCP
using the patient-portal to send secure messages and ongoing quality care to their patients.
Table 13
Required Drivers to Support Critical Behaviors
Method(s) Timing
Critical
Behaviors
Supported
1, 2, 3
Reinforcing

Upper management provides job aids outlining how to
communicate to patients through the patient-portal effectively.

Monthly

1, 2
Department meetings where upper management provides
training on the use of best practices when communicating with
patients through the patient-portal, encouraging physicians to
reflect on progress, generate ideas, and provide feedback.

Monthly

1, 2, 3
The administration is exploring additional training and funding
resources to promote the patient-portal.

Yearly 1, 2, 3
Provide ongoing training through HealthStream and the use of
physician champions.

Bi-
Annually

1, 2

Encouraging

89

Department Chief acknowledges individual physicians and the
collective work of his physicians that model the values and
enthusiasm of using the patient-portal when caring for their
patients.

Monthly

1, 2, 3

Rewarding

Department Chief’s newsletter that expresses appreciation and
recognition to the physicians that scored highest on patient
satisfaction scores and who sent the most secured messages to
their patients through the patient-portal.

Monthly

1, 2, 3
Monitoring

Tracking of the KPI’s and CG-CAHP Scores

Monthly 1, 2
Tracking the numbers of secured messages sent through the
patient portal

Monthly

1, 2

Organizational support. A successful implementation does not only depend on the
strategic level of management, but the operational and frontline levels also have their own roles
and tasks (Kujala et al., 2019). Implementations often happen in complex organizations, and
changes in these environments require a clear vision at all levels and especially from the top.
Effective leadership should be identified as necessary in the successful implementation of
ensuring all PCPs will send secure messages to their patients through the patient-portal. First,
leadership must commit to providing funding for on-going training, equipment, and resources on
effective ways the PCPs can communicate with their patients through the patient-portal. Second,
the leadership will commit to establishing regular meetings with the PCPs to provide evidence on
the benefits the patient receives now that the physicians are communicating through the patient-
portal. Also, develop policies and procedures to monitor the communications and reward
physicians that are consistent with their communication. Lastly, the organization must commit to
establishing a marketing campaign to promote a culture of open communication and
transparency to all medical professionals and staff.

90

Level 2: Learning
Kirkpatrick and Kirkpatrick (2016) defined Learning as the degree to which participants
develop the intended knowledge, skills, attitude, confidence, and commitment based on their
involvement in the training. The approach to evaluating at Level 2 is to combine a variety of
activities into the training that fundamentally tests participant knowledge.
Learning goals. Following the completion of the recommended solutions, the PCPs will be
able to:
1. Recognize the details of prior knowledge of sound clinical practices for physicians when
communicating with their patients through the patient-portal (Conceptual Knowledge)
2. Be confident in their ability to master the patient-portal (Self Efficacy)
3. Value the planning and monitoring the use of the patient-portal (Expectancy-Value)
Program. The name of the program is Patient-Portal Fundamentals (PPF). The PPF will
consist of two one-hour workshops that focus on the knowledge and skills the PCP has while
working in the portal, and examining what motivates their confidence to use the portal and the
values they get for using the patient-portal when communicating with their patients. Knowing
that it is challenging to get the PCPs in a group, each workshop will last one hour. Each
workshop will have 10 PCP participating; all workshops will be recorded for further training
purposes and will be championed by the Chief of Primary Care and the Director of Application
Architecture & IT Integration. The first PPF will provide a demonstration of the patient-portal,
which will include a user guide and job aid for effective and evidence-based strategies for
communication with patients through the patient-portal. After the 30-minute demonstration, the
group will move to break-out sessions where the PCPs will be offered hands-on experience with
the portal. They will be encouraged to discuss prior knowledge and experience they have had

91

working within the portal through self-reflection of their communications with their patients. At
the end of the workshop, the PCPs will be given a survey, which will capture the learning
understanding of the workshop. The PCPs will be asked to keep a journal of their
communications when working in the portal. A retrospective chart review will be conducted on
their correspondence through the portal of the 10 PCPs that attend the workshop.
The second workshop will be conducted one week later with the same PCPs. During the
one-hour session, the first 30-minutes will be a group discussion on how the week went after the
initial workshop. From the retrospective chart reviews, the Department Chief will roleplay
different real-life scenarios with the PCPs. After the 30-minute discussion, there will be a break-
out session where the PCPs will have an open discussion on the values and rationales of
investing time and resources into using the patient-portal. There will also be a member of the
legal team available to answer any legal concerns the physicians might have as to appropriate
documentation within the patient-portal and EHR. At the end of the workshop, the PCPs will be
given a survey, which will capture the learning understanding of the workshop. A member of the
management team will offer the PCPs mentorship and coaching, often referred to as a super-user.
Based on the individual retrospective chart review, physicians will be given an action plan with a
timeline of implementation. The effectiveness of the program will be measured by the Clinician
and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) scores,
plus patient satisfaction scores conducted by SEH, the Patient Feedback and Communication
committee, and by the number of secured messages each physician sends through the patient-
portal.
Evaluation of the components of learning. Learning is familiar to most training
professionals and the degree to which participants develop the intended knowledge, skills,

92

attitude, confidence, and commitment based on their participation in the training (Kirkpatrick &
Kirkpatrick, 2019). Table 14 lists the methods and activities of the five dimensions as part of the
PPF program detailed in the above section.
Table 14
Evaluation of the Components of Learning for the Program
Method(s) or Activity(ies) Timing
Declarative Knowledge “I know it.”

Group activity explaining and demonstrating key concepts During
Survey questions addressing understanding

During and after
Procedural Skills “I can do it right now.”

Roleplay real-life scenarios with peers

During
Survey questions addressing skills

After
Attitude “I believe this is worthwhile.”

Discussion about value and rationales of investing time and resources

During
Survey questions addressing attitudes and value

After
Confidence “I think I can do it on the job.”

Discussion in small groups of concerns and barriers

During
Survey questions addressing confidence

After
Mentorship and coaching with individuals

After an ongoing
Commitment “I will do it on the job.”

Create an action plan with a timeline to implementation

During
Survey questions addressing commitment

After

Level 1: Reaction
Level 1: Reaction has three components; engagement, relevance, and customer
satisfaction. Kirkpatrick and Kirkpatrick (2016) define reaction as the level to which participants
find the training favorable, engaging, and relevant to their jobs. The Patient-Portal Fundamentals
(PPF) program begins the process of implementation of effective and evidence-based strategies

93

into their two workshops. Table 15 lists the methods that will be used to determine whether the
PPF program is effective and relevant.
Table 15
Components to Measure Reactions to the Program
Method(s) or Tool(s) Timing
Engagement

Attendance During each session
Asking meaningful questions During each session
Course evaluation After each session and after the program
Relevance

Pulse check via survey and/or discussion During each session
Anonymous survey End of the program
Customer Satisfaction

Pulse-check via group discussion During each session
Course evaluation After each session and the end of the program

Evaluation Tools
Kirkpatrick and Kirkpatrick (2016) states all evaluation tools should be blended by
creating an evaluation form in which one can ask a question related to all dimensions of Level 1,
questions about confidence and commitment to apply what was learned on the job from Level 2
and questions related to anticipated application and outcomes. Using a blended evaluation tool,
SEH leadership can receive an evaluation of Level 1 reactions to the PPF program and Level 2
learning of knowledge, skills, attitude, confidence, what commitment is necessary to achieve the
goal, and whether the PPF program was engaging and relevant. Appendix C provides the
evaluation form used to assess the PPF program after each workshop.
Delayed for a period after the program implementation. It is useful to SEH and the
stakeholders to conduct a delayed evaluation after training physicians have applied what they
learned on the job, what support they have received (Level 3), and what kinds of results they

94

have accomplished (Level 4) due to the program (Kirkpatrick & Kirkpatrick, 2016). Appendix D
provides the delayed survey, which will be received by the PCPs six months after the completion
of the second workshop. Besides using the delayed evaluation survey tool, the Chief of Primary
Care will also review the results of the CG-CAHPS scores plus patient satisfaction scores and
review the number of secured messages each PCP sent through the patient-portal over the
previous six-months.
Data Analysis and Reporting
The data analysis process will be continuous and will incorporate the data gathered from
each workshop and all delayed surveys, which were administered six months after the program
completion. Kirkpatrick and Kirkpatrick (2016) state that there are three key questions the data
must answer: 1) Does … meet expectations? 2) If not, why? 3) If so, why?
The data analysis will also take into account the data from the patient satisfaction scores,
CG-CAHPS Score Cards, and the number of secured messages sent by the PCP. The Level 4
goal of the implementation plan is to provide the PCPs with the knowledge, motivation, and
organizational support to effectively reach the goal of sending secure messages through the
patient-portal successfully.
Summary
The New World Kirkpatrick Model is an outcome-focused model that evaluates the
outcomes of the organization, which provides a research-based method to effectively evaluate
professional development training (Kirkpatrick & Kirkpatrick, 2016). A well-designed and well-
received training program where training is relevant and the transfer of learning become
behavior is essential. The model categorizes four key areas where feedback is critical to ensure
the success of the training process: Level 4, which focuses on results of the training; Level 3,

95

which entails examining participant implementation and behavior back on the job; Level 2,
which explores whether participants acquired the knowledge needed for implementation; and
Level 1, which consists of reactions to the training as favorable, engaging and relevant
(Kirkpatrick & Kirkpatrick, 2016).
St. Elsewhere Hospital had several outcomes and metrics to pull from, both externally
and internally, to measure the success of the PPF program. The administration reviewed the
baseline of each metric and forecasted a 3% increase to be deemed successful. The take-away
from Level 4 results of the training was that the PCPs understand the impact they have on their
patients by providing quality care through their communications and maintaining their patient-
physician relationship in today’s digital age.
Level 3 focused on the performance of the PCPs after the two workshops and how they
implemented the information learned to their daily activities. To ensure the workshops were
successful, the PCPs had to demonstrate that they knew how the patient-portal worked and that
they saw value in the portal, and how it added to the complexity of care. Monitoring the number
of secure messages; each PCP sent to their patient is a method to ensure success in the
workshops.
The PCPs knowledge, skills, confidence, and commitment is what was assessed after
Level 2 learning. It was important for the SEH administration to be able to determine the
effectiveness of the training and whether the PCPs obtained the knowledge necessary to
communicate with their patients through the portal. The PCPs were evaluated on the components
of learning by reviewing completed surveys, roleplaying with their peers, and through
retrospective chart review of their patient-portal entries.

96

Healthcare today requires PCPs to conduct their services through an electronic portal.
Level 1 provided the leadership a glimpse into the engagement and relevance of the PPF
program on the PCPs. St. Elsewhere Hospital needs their PCPs to invest in the concepts of
communicating to their patient through the patient-portal to achieve the quality of care they
require.
Strengths and Weaknesses of the Approach
Clark and Estes (2008) conceptual framework was used in this study. Healthcare
institutions like SEH can examine performance gaps through a systematic, analytic framework
that identifies gaps between actual performance levels and performance goals, and the
organizational and stakeholder's performance goals. The approach used in the study was the
KMO model to examine the knowledge, motivation, and organizational influences that impact
key stakeholders, e.g., the PCPs, within the problem of practice. The purpose of the gap analysis
was to identify whether PCPs had adequate knowledge, motivation, and organizational support to
reach the goal of the PCPs sending a secure message to 30% of their panel through the patient-
portal by June 2021. The weakness of Clark and Estes (2008) conceptual framework is that the
research was conducted during a global pandemic, and the crisis at hand may have influenced the
PCP's knowledge and motivation to send secure messages through the patient-portal due to the
limitations of face-to-face interactions. The question to ask is whether the findings of the gap
analysis is relevant to the crisis and human behavior, or if there are real organizational barriers.
Limitations and Delimitations
One of the limitations of this study was the sample size of the PCPs at SEH. The study
population was limited at the academic institution, with only 32 PCPs that worked in ambulatory
clinics. Emails were sent out at the beginning of May 2020 and with a reminder email sent one

97

week later. The interviews took place in the middle of the COVID-19 worldwide pandemic when
all medical personnel were working tirelessly to provide for their patients. Ten physicians
volunteered their time for the interviews, which represented a 31% response rate. Due to
unforeseen circumstances, one of the 10 physicians interviewed (Amy), unfortunately, had to
lower her full-time status to 60% (part-time). The requirements for participation in the interviews
was that the physicians were to be practicing full-time within the ambulatory clinics. The
researcher was informed of the situation after all of the interviews were completed by the
interviewer. Due to the low sample size, the researcher chose to include Amy’s interview data in
the study. A larger sample size would have been preferred; however, to mitigate the small
sample size, a high threshold was set.
Another limitation was that the researcher holds the role of an operations director of
ambulatory services, whereby participants had a direct or indirect reporting relationship to the
researcher. Thus, considerations relating to the ethics of this study, the researcher had someone
else conduct the interviews. Even though the interviewer was very knowledgeable in the research
and in the use of the patient-portal, the limitation that the researcher did not have complete
control of how the research questions were asked or on the responses that were given made data
collection challenging.
The novel coronavirus COVID-19 pandemic was an unexpected limitation due to the fact
healthcare services had changed dramatically. Most of the ambulatory clinics closed or were
scaled back to urgent patients only. The pandemic may have persuaded different answers to the
research questions from the PCPs since the majority of the ambulatory clinic's appointments
were converted to telemedicine and telephone encounters.
Future Research

98

The focus of this study was on PCPs communicating with their patients through the
patient-portal. The patient-portal is such a broad topic, and there are many aspects of the medical
profession conducting medical care through electronic platforms. For instance, this study only
focused on PCPs corresponding to patients within ambulatory clinics. Patient-portals are being
used for in-patient care as well; the stakeholders for the use of the patient-portal could be
expanded to administration, nurses, and patients. From the outcome of the interviews, the
stakeholders for future studies could focus on the patients' motivation to use the patient-portal.
To better understand what their gaps are to receiving quality care through the patient-portal. Due
to the pandemic, telemedicine has become a staple for all medical institutions. Research needs to
be conducted regarding all aspects of this growing trend from types of user platforms, devices to
operate from, to billing for services.
Due to the small sample size of this study, it is recommended that the interviews be
conducted again and possibly expand the invitations to other Primary Care Departments at
academic institutions in the area. Since the interviews were conducted in the middle of a
pandemic, follow-up questions could be addressed of the 10 original interviewees to ensure their
answers were accurate to how they felt or whether the crisis swayed their responses.
Conclusion
Over the years, there has been an increased emphasis on understanding the needs of
primary care patients, so this study sought to explore the experiences of PCPs utilizing the
patient-portal for ongoing patient care. The stakeholders for the study were PCPs because
primary care is the initial introduction to a patient’s preventative, routine, and chronic health
needs. There is an ongoing government-mandate that all medical facilities adopt an EHR and
patient-portal. This study focused on the interactions and communications the PCP had with their

99

patients through the patient-portal. The use of Clark and Estes (2008) conceptual framework
focused on the knowledge, motivation, and organization needs that influence the PCPs to
communicate with their patients through the patient-portal. The evaluation revealed the PCPs
assumed knowledge and organizational influences were validated as gaps. The research
highlighted that the PCP's required attention, consistency, and commitment.
The proposed solution is for SEH to conduct a two-day program called Patient-Portal
Fundamentals. The two-day program will offer two one-hour workshops that will expand on the
knowledge and skills the PCP needs while communicating in the portal. The goal of the PPF
program is to give PCPs the tools that are useful and evidence-based strategies for
communication to their patients. The program strives to provide the physicians with mentorship
and coaching to allow the PCP will feel empowered to provide quality patient care through a
digital platform.
The COVID-19 pandemic has changed the face of healthcare forever. Within a span of
two-weeks back in April of 2020, SEH equipped and trained 480 physicians to conduct patient
care through telemedicine. St. Elsewhere was thankful that the patient-portal was established and
took on a more substantial role in how SEH communicated with their patients and community.
Before COVID, the primary care clinic saw 650 in-person appointments a day, with 12
telemedicine appointments. Currently, during COVID, the same clinic is now seeing 425
appointments daily, with 200 telemedicine appointments, 125 telephone appointments, and only
100 appointments in-person. Times have changed, and PCPs had to get on board and
communicate with their patients over several different digital platforms. Today, patients want to
book their appointment, conduct their medical care, and obtain psychiatry services on-line.

100

A significant change that stemmed from the pandemic is that the US national payer
network, CMS, State Medicaid, private payers, and state health agencies are ensuring patients get
the care by enabling physicians to be reimbursed for non-face-to-face services (Cohen et al.,
2020). These services are referred to as telemedicine or telehealth visits. Reimbursements apply
to telephone appointments as well. Gone are the days of trying to decipher a physician's
signature. We are at a point of realization that the digital age is the future of healthcare, and the
future is NOW.

101

References
Alkire, L., O'Connor, G. E., Myrden, S., & Köcher, S. (2020). Patient experience in the digital
age: An investigation into the effect of generational cohorts. Journal of Retailing and
Consumer Services, 57, 102221.
Alkureishi, M. A., Lee, W. W., Lyons, M., Press, V. G., Imam, S., Nkansah-Amankra, A., . . .
Arora, V. M. (2016). Impact of electronic medical record use on the patient-doctor
relationship and communication: A systematic review. Journal of General Internal
Medicine, 31(5), 548-560.
Ammentorp, J., Sabroe, S., Kofoed, P., & Mainz, J. (2007). The effect of training in
communication skills on medical doctors’ and nurses’ self-efficacy: A randomized
controlled trial. Patient Education and Counseling, 66(3), 270-277.
Arndt, B. G., Beasley, J. W., Watkinson, M. D., Temte, J. L., Tuan, W., Sinsky, C. A., &
Gilchrist, V. J. (2017). Tethered to the EHR: Primary care physician workload assessment
using EHR event log data and time-motion observations. The Annals of Family
Medicine, 15(5), 419-426.
Ayre, J., Bonner, C., Bramwell, S., McClelland, S., Jayaballa, R., Maberly, G., & McCaffery, K.
(2019). Factors for supporting primary care physician engagement with patient apps for type
2 diabetes self-management that link to primary care: Interview study. JMIR mHealth and
uHealth, 7(1), e11885.
Barrett, A. K. (2018). Electronic health record (EHR) organizational change: Explaining
resistance through profession, organizational experience, and EHR communication
quality. Health Communication, 33(4), 496-506.

102

Borg, H. W. (2017). Electronic health records: Agenda-based medicine. Journal of American
Physicians and Surgeons, (2), 48-55.
Bowman, F. M., Goldberg, D. P., Millar, T., Gask, L., & McGrath, G. (1992). Improving the
skills of established general practitioners: The long‐term benefits of group teaching. Medical
Education, 26(1), 63-68.
Bozan, K., & Mooney, D. (2018). Intervention study on the effect of training and support on
perceived level of patient portal ease of use among the elderly. Paper presented at
the Proceedings of the 51st Hawaii International Conference on System Sciences,
Bryan, M., Norton, D., Birstler, J., Chen, G., Cruz, L., & Hanrahan, L. (2020). Resource
utilization among portal users who send messages: A retrospective cohort
study. Wmj, 119(1), 26-32.
Bush, R. A., Connelly, C. D., Pérez, A., Chiang, N., Kuelbs, C., & Chiang, G. J. (2017).
Physician perception of the role of the patient portal in pediatric health. The Journal of
Ambulatory Care Management, 40(3), 238.
Chan, B., Lyles, C., Kaplan, C., Lam, R., & Karliner, L. (2018). A comparison of electronic
patient-portal use among patients with resident and attending primary care
providers. Journal of General Internal Medicine, 33(12), 2085-2091.
Cinite, I., & Duxbury, L. E. (2018). Measuring the behavioral properties of commitment and
resistance to organizational change. The Journal of Applied Behavioral Science, 54(2), 113-
139.
Clark, R. E., & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions. Charlotte, NC: Information Age Publishing, Inc.

103

Cohen, B. H., Busis, N. A., & Ciccarelli, L. (2020). Coding in the world of COVID-19: Non–
face-to-face evaluation and management care. CONTINUUM: Lifelong Learning in
Neurology, 26(3), 785-798.
Creswell, J.W., & Creswell, J.D., (2018). Research design: Qualitative, quantitative, and mixed
methods approaches. Thousand Oaks, CA: Sage Publications
Dendere, R., Slade, C., Burton-Jones, A., Sullivan, C., Staib, A., & Janda, M. (2019). Patient
portals facilitating engagement with inpatient electronic medical records: A systematic
review. Journal of Medical Internet Research, 21(4), e12779.
Dyrbye, L. N., Shanafelt, T. D., Sinsky, C. A., Cipriano, P. F., Bhatt, J., Ommaya, A., . . .
Meyers, D. (2017). Burnout among health care professionals: A call to explore and address
this underrecognized threat to safe, high-quality care. NAM (National Academy of Medicine)
Perspective,
Eccles, J. (2006). Expectancy-value motivational theory.
Eccles, J. (2009). Who am I and what am I going to do with my life? Personal and collective
identities as motivators of action. Educational psychologist, 44(2), 78-89.
Emont, S. (2011). Measuring the impact of patient portals. California Healthcare Foundation, 1-
20.
Ewelukwa, O., Perez, R., Carter, L. E., Fernandez, A., & Glover, S. (2018). Incorporation of
scribes into the inflammatory bowel disease clinic improves quality of care and physician
productivity. Inflammatory Bowel Diseases, 24(3), 552-557.
FACT SHEET. (2012). https://www.healthit.gov/sites/default/files/playbook/pdf/nlc-secure-
messaging-fact-sheet.pdf
Frankel, R. M. (2016). Computers in the examination room. JAMA Intern Med., 176(1), 128-129.

104

Fong, Z. V., Qadan, M., McKinney, R., Griggs, C. L., Shah, P. C., Buyske, J., . . . Altieri, M. S.
(2020). Practical implications of novel coronavirus COVID-19 on hospital operations, board
certification, and medical education in surgery in the USA. Journal of Gastrointestinal
Surgery, 1-5.
Giardina, T. D., Modi, V., Parrish, D. E., & Singh, H. (2015). The patient portal and abnormal
test results: An exploratory study of patient experiences. Patient Experience Journal, 2(1),
148.
Glesne, C. (2011). Becoming qualitative researchers an introduction. Pearson Education, Inc
Guo, U., Chen, L., & Mehta, P. H. (2017). Electronic health record innovations: Helping
physicians–One less click at a time. Health Information Management Journal, 46(3), 140-
144.
Hefner, J. L., MacEwan, S. R., Biltz, A., & Sieck, C. J. (2019). Patient portal messaging for care
coordination: A qualitative study of perspectives of experienced users with chronic
conditions. BMC Family Practice, 20(1), 57.
Hingle, S. (2016). Electronic health records: An unfulfilled promise and a call to action. Annals
of Internal Medicine, 165(11), 818-819.
Irizarry, T., Shoemake, J., Nilsen, M. L., Czaja, S., Beach, S., & Dabbs, A. D. (2017). Patient
portals as a tool for health care engagement: A mixed-method study of older adults with
varying levels of health literacy and prior patient portal use. Journal of Medical Internet
Research, 19(3), e99.
Johnson, B., & Christensen, L. (2014). Educational research fourth edition: Quantitative,
qualitative and mixed approaches.

105

Karakose, T. (2014). An evaluation of the relationship between general practitioners' job
satisfaction and burnout levels. Studies on Ethno-Medicine, 8(3), 239-244.
Kim, S., & Fadem, S. (2018). Communication matters: Exploring older adults’ current use of
patient portals. International Journal of Medical Informatics, 120, 126-136.
Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Kirkpatrick's four levels of training evaluation.
Association for Talent Development.Kornacker, D. (2019). A patient portal push toward
acceptance and utilization of the technology. Patient Experience Journal, 6(2), 20-27.
Kornacker, D., Fitzgerald, K., & Elder, E., (2019). A patient portal push toward acceptance and
utilization of the technology. Patient Experience Journal, 6(2), 20-27.
Krathwohl, D. R. (2002). A revision of bloom's taxonomy: An overview. Theory into
Practice, 41(4), 212-218.
Krousel-Wood, M., McCoy, A. B., Ahia, C., Holt, E. W., Trapani, D. N., Luo, Q., . . . Milani, R.
V. (2017). Implementing electronic health records (EHRs): Health care provider perceptions
before and after transition from a local basic EHR to a commercial comprehensive
EHR.Journal of the American Medical Informatics Association, 25(6), 618-626.
Kruse, C. S., Argueta, D. A., Lopez, L., & Nair, A. (2015). Patient and provider attitudes toward
the use of patient portals for the management of chronic disease: A systematic
review. Journal of Medical Internet Research, 17(2), e40.
Kujala, S., Hörhammer, I., Heponiemi, T., & Josefsson, K. (2019). The role of frontline leaders
in building health professional support for a new patient portal: Survey study. Journal of
Medical Internet Research, 21(3), e11413.

106

Lieu, T. A., Altschuler, A., Weiner, J. Z., East, J. A., Moeller, M. F., Prausnitz, S., . . . Awsare,
S. (2019). Primary care physicians’ experiences with and strategies for managing electronic
messages. JAMA Network Open, 2(12), e1918287.
Lyles, C. R., Allen, J. Y., Poole, D., Tieu, L., Kanter, M. H., & Garrido, T. (2016). “I want to
keep the personal relationship with my doctor”: Understanding barriers to portal use among
african americans and latinos. Journal of Medical Internet Research, 18(10), e263.
Maxwell, J.A. (2013). Qualitative Research Design. Thousand Oaks, CA: Sage Publications.
McCrudden, M. T., Schraw, G., & Hartley, K. (2006). The effect of general relevance
instructions on shallow and deeper learning and reading time. The Journal of Experimental
Education, 74(4), 291-310.
Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research: A guide to design and
implementation (fourth).
Michael, K., Dror, M. G., & Karnieli-Miller, O. (2019). Students’ patient-centered-care attitudes:
The contribution of self-efficacy, communication, and empathy. Patient Education and
Counseling, 102(11), 2031-2037.
Namada, J. M. (2018). Organizational learning and competitive advantage. Handbook of
research on knowledge management for contemporary business environments (pp. 86-104)
IGI Global.
Nouri, S., Khoong, E. C., Lyles, C. R., & Karliner, L. (2020). Addressing equity in telemedicine
for chronic disease management during the covid-19 pandemic. NEJM Catalyst Innovations
in Care Delivery, 1(3)
Oftedal, B., Karlsen, B., & Bru, E. (2010). Perceived support from healthcare practitioners
among adults with type 2 diabetes. Journal of Advanced Nursing, 66(7), 1500-1509.
Pajares, F. (2006). Self-Efficacy theory.

107

http://www.education.com/reference/article/self-efficacy-theory/
Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in
learning and teaching contexts. Journal of Educational Psychology, 95(4), 667.
Portz, J. D., Brungardt, A., Shanbhag, P., Staton, E. W., Bose-Brill, S., Lin, C., . . . Lum, H. D.
(2020). Advance care planning among users of a patient portal during the COVID-19
pandemic: Retrospective observational study. Journal of Medical Internet Research, 22(8),
e21385.
Rathert, C., Porter, T. H., Mittler, J. N., & Fleig-Palmer, M. (2019). Seven years after
meaningful use: Physicians’ and nurses’ experiences with electronic health records. Health
Care Management Review, 44(1), 30-40.
Reeves, J. J., Hollandsworth, H. M., Torriani, F. J., Taplitz, R., Abeles, S., Tai-Seale, M., . . .
Longhurst, C. A. (2020). Rapid response to COVID-19: Health informatics support for
outbreak management in an academic health system. Journal of the American Medical
Informatics Association, 27(6), 853-859.
Risling, T., Martinez, J., Young, J., & Thorp-Froslie, N. (2018). Defining empowerment and
supporting engagement using patient views from the citizen health information portal:
Qualitative study. JMIR Medical Informatics, 6(3), e43.
Robb, M. J. (2017). Fulfillment vs. burnout. Journal of American Physicians and
Surgeons, 22(4), 101-105.
Rubin, H. J., & Rubin, I. S. (2012). Qualitative interviewing: The art of hearing data. Sage.
Rueda, R. (2011). The 3 dimensions of improving student performance: Finding the right
solutions to the right problems. ERIC.
Schein, E.H., Schein, P. (2017). Organizational culture and leadership, 5th edition. San
Francisco: Jossey-Bass.

108

Schraw, G., & McCrudden, M. (2006). Information processing theory. Retrieved from
http://education.com/reference/article/information-proccessing-theory/
Shah, S. D., & Liebovitz, D. (2017). It takes two to tango: Engaging patients and providers with
portals. Pm&r, 9(5), S85-S97.
Shahmoradi, L., Safadari, R., & Jimma, W. (2017). Knowledge management implementation and
the tools utilized in healthcare for evidence-based decision making: A systematic
review. Ethiopian Journal of Health Sciences, 27(5), 541-558.
Shanafelt, T. D., Dyrbye, L. N., Sinsky, C., Hasan, O., Satele, D., Sloan, J., & West, C. P.
(2016). Relationship between clerical burden and characteristics of the electronic
environment with physician burnout and professional satisfaction. Paper presented at
the Mayo Clinic Proceedings, , 91(7) 836-848.
Shanafelt, T. D., & Noseworthy, J. H. (2017). Executive leadership and physician well-being:
Nine organizational strategies to promote engagement and reduce burnout. Paper presented
at the Mayo Clinic Proceedings, , 92(1) 129-146.
Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research
projects. Education for Information, 22(2), 63-75.
Sieck, C. J., Hefner, J. L., Schnierle, J., Florian, H., Agarwal, A., Rundell, K., & McAlearney, A.
S. (2017). The rules of engagement: Perspectives on secure messaging from experienced
ambulatory patient portal users. JMIR Medical Informatics, 5(3), e13.
Sinsky, C., Colligan, L., Li, L., Prgomet, M., Reynolds, S., Goeders, L., . . . Blike, G. (2016).
Allocation of physician time in ambulatory practice: A time and motion study in 4
specialties. Annals of Internal Medicine, 165(11), 753-760.

109

Sulieman, L., Robinson, J. R., & Jackson, G. P. (2020). Automating the classification of
complexity of medical decision-making in patient-provider messaging in a patient
portal. Journal of Surgical Research, 255, 224-232.
Tavares, J., & Oliveira, T. (2016). Electronic health record patient portal adoption by health care
consumers: An acceptance model and survey. Journal of Medical Internet Research, 18(3),
e49.
Usher, E. L., & Pajares, F. (2009). Sources of self-efficacy in mathematics: A validation
study. Contemporary Educational Psychology, 34(1), 89-101.
Van den Bulck, Steve Alfons, Hermens, R., Slegers, K., Vandenberghe, B., Goderis, G., &
Vankrunkelsven, P. (2018). Designing a patient portal for patient-centered care: Cross-
sectional survey. Journal of Medical Internet Research, 20(10), e269.
West, C. P., Dyrbye, L. N., & Shanafelt, T. D. (2018). Physician burnout: Contributors,
consequences and solutions. Journal of Internal Medicine, 283(6), 516-529.
Wigfield, A., & Eccles, J. S. (1992). The development of achievement task values: A theoretical
analysis. Developmental Review, 12(3), 265-310.
Wong, J. I. S., Steitz, B. D., & Rosenbloom, S. T. (2019). Characterizing the impact of health
literacy, computer ability, patient demographics, and portal usage on patient satisfaction
with a patient portal. JAMIA Open, 2(4), 456-464.
Zhong, X., Liang, M., Sanchez, R., Yu, M., Budd, P. R., Sprague, J. L., & Dewar, M. A. (2018).
On the effect of electronic patient portal on primary care utilization and appointment
adherence. BMC Medical Informatics and Decision Making, 18(1), 84.
Zhong, X., Park, J., Liang, M., Shi, F., Budd, P. R., Sprague, J. L., & Dewar, M. A. (2020).
Characteristics of patients using different patient portal functions and the impact on primary

110

care service utilization and appointment adherence: Retrospective observational
study. Journal of Medical Internet Research, 22(2), e14410.
Zickmund, S. L., Hess, R., Bryce, C. L., McTigue, K., Olshansky, E., Fitzgerald, K., & Fischer,
G. S. (2008). Interest in the use of computerized patient portals: Role of the provider–patient
relationship. Journal of General Internal Medicine, 23(1), 20-26.

111

Appendix A
Interview Protocols
Research Questions

3. What is St. Elsewhere Hospital physician’s knowledge and motivation related to the goal of
sending a secure message to 30% of their patient panel through the portal during the reporting
period, in 2021?

4. What is the interaction between St. Elsewhere Hospital’s culture and context, and physician
knowledge and motivation to the goal of sending a secure message to 30% of their patient panel
through the portal during the reporting period, in 2021?

Demographic

1. How many years have you been a primary care physician?

a. Sex
b. Age

2. How long have you been at the organization?

a. Where else have you practiced?

3. How many years have you been practicing in the ambulatory care setting?

Knowledge & Skills

1. What has been your experience using the patient portal?

a. In what way?

2. When working in the patient portal can you expand on what does or does not promote
better care for your patients

3. Has the patient portal impacted your face-to-face interactions with your patients?

a. Can you walk me through a previous portal exchange you had with a patient?

b. How often are you finding your patients prefer to communicate through the portal instead
of scheduling an appointment? Why?

4. What is the impact on the quality of your patient care as a result of the portal?

5. Give me an example of how the patient portal provides better or worse communication
between you and your patients?

112

6. How does the patient portal impact communication on productivity or workflow in your
practice?

7. Do you feel you are effective in your communications with your patients through the
portal?

a. If so, how?

8. What methods do you use to monitor correspondence with your patients in the patient
portal?

a. Does your staff answer?

9. How do you set expectations around communicating with your patients using the portal?

Motivation

10. Tell me the level of confidence you feel corresponding with your patients who use the
portal?

a. What are the specific factors that lead to your confidence level?

b. What would help you feel more confident in your ability to communicate with patients?

11. Tell me about a time you did not feel confident using the patient portal? Why

a. How consistent is it?

b. What are the particular factors within the EHR that impact your usage?

12. Has the patient portal improved the quality of care you are providing to patients?

a. What factors do you feel contribute to the quality of care?

13. Does the patient portal support chronic disease management to promote quality of care?
a. Or does it hinder the quality of care?

14. How is the patient portal a valuable tool you use with your patients?

a. Why?

Organization

15. Do you feel that the organization provides adequate training and ongoing support to
appreciate the patient portal?

a. If no, what type of support and training is lacking?

b. If yes, are there improvements to be made or additional training?

16. Tell me what types of tools and resources you feel you need to master the patient portal?

113

a. Can you be more specific on the tools you are referring too?

17. Give me an example of a new practice or structure the organization introduced in the past
year regarding the patient portal?

a. If there were no new practices or structures implemented, what would you like the
organization to introduce and support?

18. How has the organization adjusted to this new digital platform?

19. How has the organization shifted its focus to promote the patient portal to ensure active
participation?

20. How do you feel the organization communicates when making changes or upgrades to the
patient portal?

a. Give me examples of the communications the organization put out before and after any
changes or upgrades to the patient portal?

21. Is the organization communicating its position regarding the innovation of the digital
platform?

a. If yes, how?

b. If no, is the lack of communication from the organization, adding to your creating a
feeling of stress and frustration?

114

Appendix B
Knowledge Influences
Conceptual Knowledge

4: Management of Messages
(n = 10)

Positive for PCP 30%
Negative for PCP 70%

5: Effective Communicator
(n = 10)

Positive Communicator 100%
Negative Communicator 0%

6: Increase in Productivity
(n = 10)

Positive Increase 20%
Negative Increase 80%
Motivation Influences
Self-Efficacy Motivation

1: Physician’s Ability to Communicate
(n = 10)
Positive Communication 90%
Negative Communication 10%

2: Readjust Patient’s Expectations
(n = 10)
Positive Adjustment 100%
Negative Adjustment 0%

Expectancy - Value Motivation

3: Access to Care
(n = 10)
Positive Access 100%
Negative Access 0%

115

4: Improve Communication
(n = 10)
Positive Communication 90%
Negative Communication 10%
Organization Influences
Cultural Setting

1: Quality of Training.
(n = 10)
Yes, Provides Adequate Training 40%
Does Not, Provides Adequate Training 60%

2: Resources.
(n = 10)
Yes, Provides Resources 50%
Does Not, Provides Resources 50%

Cultural Setting

3: Patient Portal Enrollment.
(n = 10)
Yes, New Patient Enrollment 50%
No, New Practices 50%

4: Promoting the Portal.
(n = 10)
Yes, Promoted the Portal 60%
Does Not, Promote the Portal 40%

Cultural Model

5: Communication.
(n = 10)
Open Communication 30%
No Communication 70%

116

Appendix C
Immediate Evaluation Tool
Participant Evaluation
Instructions Date _________________

For questions 1 – 5, please use the following rating scale:

0 Strongly Disagree 10 Strongly agree

Please circle the appropriate rating to indicate the degree to which you agree with the statement,
and provide comments to explain your answer.

Rating
Strongly Disagree Strongly Agree

0 1 2 3 4 5 6 7 8 9 10

1. I took responsibility for being involved in
this program.

Comments

0 1 2 3 4 5 6 7 8 9 10

2. This program held my interest.

Comments

0 1 2 3 4 5 6 7 8 9 10

3. The presentation style of the facilitator
contributed to my learning experience.

Comments

0 1 2 3 4 5 6 7 8 9 10

4. The information in this program is
applicable to my work.

Comments

0 1 2 3 4 5 6 7 8 9 10

5. I would recommend this program to others.

Comments

117

Participant Evaluation Page 2

For questions 6 – 10, please use the following rating scale:

1 2 3 4 5
None or Very Very High Level
Low Level

Please circle the appropriate rating before the training and now (after the training) and
please provide comments to explain your ratings.

Before the Training After the Training

1 2 3 4 5

6. Knowledge of how I
should communicate with
my patients through the
portal.

1 2 3 4 5

Comments

1 2 3 4 5

7. How to set
communicational
boundaries.

1 2 3 4 5

Comments

1 2 3 4 5

8. My confidence level in
building communications
with my patients in the
portal.

1 2 3 4 5

9. How can this program be improved?

10. Please share any additional comments you may have.

118

Appendix D
Delayed Feedback Tool

Date of you completing the PPF training: __________________

Please answer the following questions regarding your experiences
communicating with your patients within the patient-portal.

1. How are you currently communicating with your patients within the patient-
portal?

2. What positive outcomes are you seeing by communicating through the patient-
portal? If you are not receiving positive outcomes? Can you comment why?

3. To what can you attribute your success in communicating with your patients?

4. If you are not corresponding with your patients through the patient-portal,
what are the reasons?

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

The attrition and lack of medical follow-up of patients in research in a primary care setting: a gap analysis

PDF

Improving the overall quality and star rating at the sub-acute care center

PDF

Physician burnout in the COVID-19 pandemic: Healthcare organization and leadership implications for patient care and clinical team leadership with nurses

PDF

Exploring the effectiveness of continuing medical education for physicians enrolled in an MBA program

PDF

Reducing gaps in healthcare delivery quality on American Indian reservations and Alaska Native communities: an improvement study

PDF

Assessing administrators’ perceptions of implementing trauma-informed care in K−12 schools

PDF

Leadership engagement to promote employee retention and career paths within the senior assisted living industry

PDF

Barriers to practice: primary care patient experience and quality health outcomes in the Veterans Health Administration

PDF

First-generation college students and persistence to a degree: an evaluation study

PDF

The integration of medicine and compassionate care: an evaluation study

PDF

Understanding the quality of geriatric healthcare and family caregiver advocacy for patients with dementia

PDF

Raising special needs: an evaluation study of respite care for medically fragile children living with autism and other illnesses

PDF

A qualitative examination of the methods church leaders use to increase young adult attendance in Christian churches: an evaluation study

PDF

Lack of culturally relevant teaching in international bilingual schools: a gap analysis

PDF

Health care disparities and the influence of nurse leader cultural competency: an evaluation study

PDF

Leadership in an age of technology disruption: an evaluation study

PDF

The moderating role of knowledge, motivation, and organizational influences on employee turnover: A gap analysis

PDF

Analyzing the implementation of a learning management system into a post-merger & acquisition organization and its effects on sales performance (improvement model)

PDF

Examining the adoption of electronic health records system in patient care and students’ education using the GAP analysis approach

PDF

Mitigating low employee engagement through improved performance management: an evaluation study

Asset Metadata

Creator Fleming, Eric Glenn (author)

Core Title Primary care physicians' experiences working within the patient-portal to improve the quality of patient care

School Rossier School of Education

Degree Doctor of Education

Degree Program Organizational Change and Leadership (On Line)

Publication Date 09/21/2020

Defense Date 08/25/2020

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag OAI-PMH Harvest,patient-portal,primary care physicians,quality of patient care,secured messages

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Muraszewski, Alison (committee chair), Adibe, Bryant (committee member), Hirabayashi, Kimberly (committee member)

Creator Email egfleming@gmail.com,eric.fleming@med.usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c89-376583

Unique identifier UC11666369

Identifier etd-FlemingEri-8999.pdf (filename),usctheses-c89-376583 (legacy record id)

Legacy Identifier etd-FlemingEri-8999.pdf

Dmrecord 376583

Document Type Dissertation

Rights Fleming, Eric Glenn

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA