Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Physician behavior and low-value care
(USC Thesis Other)
Physician behavior and low-value care
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
PHYSICIAN BEHAVIOR AND LOW-VALUE CARE
by
Marcella A. Kelley, MHS
A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(HEALTH ECONOMICS)
August 2022
ii
Epigraph
“There is no end to education. It is not that you read a book, pass an examination, and finish with
education. The whole of life, from the moment you are born to the moment you die, is a process of
learning.”
- Jiddu Krishnamurti
iii
Acknowledgements
I am grateful to the many people who have encouraged and indulged my curiosity in big and
small ways throughout this journey. As I conclude my 22
nd
year of formal education, I am
particularly appreciative of the teachers, classmates, and friends with whom I could explore new
ideas and perspectives.
Thank you to my teachers at Saint Joseph High School for introducing me to the wonderful
world of economics. In my final year at Georgetown University, I was fortunate enough to take my
first health economics course and have a professor who saw that my interests and abilities were not
best suited to a biology lab. Thank you, Kathleen Maguire-Zeiss, for encouraging me to pursue my
interests in clinical trials and pharmaceuticals.
I am particularly appreciative of the professors at the Johns Hopkins Bloomberg School of
Public Health who led me into a deep dive of public health, economics, and health policy to begin
my graduate school career. Thank you, John Bridges, for recognizing my passion for improving
patients’ lives with the help of numbers. Working in your stated-preference group alongside talented
researchers inspired me to pursue my PhD.
I am thankful for the funding, coursework, and fantastic research opportunities at the
University of Southern California. Thank you to my dissertation committee: Jason Doctor, Daniella
Meeker, and Bill Padula for your guidance and support. I learned so much from each of you. Thank
you to my classmates who worked together and encouraged each other to pursue all the
opportunities that we could. I am appreciative of my collaborators at the Keck School of Medicine,
Los Angeles County Department of the Medical Examiner-Coroner, Northwestern University
Feinberg School of Medicine, Sol Price School of Public Policy, and Teladoc® for your insight and
reminders of the important ways health economics can be applied to our health care system.
iv
Studying uncertainty was made easier by being certain of a few essential things. First and
foremost, the love and support of my family and friends for the past many years. I will always hold
your questions, phone calls, hugs, and pride close. And to my soon-to-be husband, Tyler, thank you
for moving across the country so I could pursue the subject I love with the person I love by my side.
You are the best teammate I could have imagined.
As I embark on this next chapter of my work in health economics, I am excited to dive into
new research with old and new colleagues. From what graduate school has shown me, there will
always be questions that need answers and people willing to explore them with me.
v
Table of Contents
EPIGRAPH ......................................................................................................................... ii
ACKNOWLEDGEMENTS ................................................................................................ iii
LIST OF TABLES .............................................................................................................. vii
LIST OF FIGURES ........................................................................................................... viii
ABBREVIATIONS ............................................................................................................. ix
ABSTRACT ......................................................................................................................... xi
CHAPTER 1 ......................................................................................................................... 1
1.1 LOW-VALUE CARE .............................................................................................................................. 1
1.2 OBJECTIVE .......................................................................................................................................... 4
1.3 SPECIFIC AIMS .................................................................................................................................... 4
1.4 OPIOID OVERUSE: HARMS & TREATMENT GUIDELINES ................................................................. 5
1.5 ANTIBIOTIC OVERUSE: HARMS & TREATMENT GUIDELINES DURING COVID-19 ....................... 6
1.6 BENZODIAZEPINE OVERUSE: HARMS & TREATMENT GUIDELINES ............................................... 9
REFERENCES........................................................................................................................................... 10
CHAPTER 2 ....................................................................................................................... 18
ABSTRACT ............................................................................................................................................... 19
2.1 BACKGROUND .................................................................................................................................. 20
2.2 METHODS ......................................................................................................................................... 21
2.2.1 Data Sources ................................................................................................................................ 21
2.2.2 Measures ...................................................................................................................................... 22
2.2.3 Outcomes...................................................................................................................................... 23
2.2.4 Statistical Analysis ....................................................................................................................... 23
2.3 RESULTS ............................................................................................................................................ 24
2.4 DISCUSSION ...................................................................................................................................... 28
2.5 CONCLUSIONS .................................................................................................................................. 31
REFERENCES........................................................................................................................................... 33
CHAPTER 3 ....................................................................................................................... 36
ABSTRACT ............................................................................................................................................... 37
3.1 BACKGROUND .................................................................................................................................. 39
3.2 METHODS ......................................................................................................................................... 41
3.2.1 Data Sources ................................................................................................................................ 41
3.2.2 Sample Selection ........................................................................................................................... 42
3.2.3 Measures ...................................................................................................................................... 43
3.2.4 Outcomes...................................................................................................................................... 43
3.2.5 Statistical Analysis ....................................................................................................................... 44
3.3 RESULTS ............................................................................................................................................ 46
3.4 DISCUSSION ...................................................................................................................................... 58
3.5 CONCLUSIONS .................................................................................................................................. 61
REFERENCES........................................................................................................................................... 63
APPENDIX ............................................................................................................................................... 68
vi
CHAPTER 4 ....................................................................................................................... 76
ABSTRACT ............................................................................................................................................... 77
4.1 BACKGROUND .................................................................................................................................. 79
4.2 METHODS ......................................................................................................................................... 81
4.2.1 Study Design and Data Sources ..................................................................................................... 81
4.2.2 Measures ...................................................................................................................................... 82
4.2.3 Outcomes...................................................................................................................................... 83
4.2.4 Statistical Analysis ....................................................................................................................... 83
4.2.5 Power Calculations ........................................................................................................................ 85
4.3 RESULTS ............................................................................................................................................ 85
4.4 DISCUSSION ...................................................................................................................................... 90
4.5 CONCLUSIONS .................................................................................................................................. 91
REFERENCES........................................................................................................................................... 93
APPENDIX ............................................................................................................................................... 96
CHAPTER 5 ...................................................................................................................... 100
5.1 AIMS AND HYPOTHESES REVISITED .............................................................................................100
5.2 FUTURE DIRECTIONS .....................................................................................................................102
REFERENCES.........................................................................................................................................105
vii
List of Tables
TABLE 2.1: SAMPLE CHARACTERISTICS ..................................................................................................... 25
TABLE 2.2: MODEL RESULTS ..................................................................................................................... 28
TABLE 3.1. SAMPLE CHARACTERISTICS BY SETTING................................................................................. 47
TABLE 3.2: ADJUSTED ODDS RATIO OF A SAME DAY ED ESCALATION, INAPPROPRIATE
PRESCRIPTION, OR TEST ORDER ........................................................................................................ 49
TABLE 3.3. INAPPROPRIATE PRESCRIPTION SAMPLE CHARACTERISTICS................................................. 54
TABLE 3.4: ADJUSTED ODDS RATIO OF A HOSPITALIZATION WITHIN 7 DAYS OR DEATH WITHIN 30
DAYS OF INDEX APPOINTMENT ........................................................................................................ 57
TABLE A3.1: INCLUSION CRITERIA ............................................................................................................ 68
TABLE A3.2: COVID-19 DIAGNOSIS CODES ............................................................................................ 69
TABLES A3.3 AND A3.4: VALUE SETS ....................................................................................................... 69
TABLE A3.5: FREQUENCY OF DRUGS PRESCRIBED WITHIN 3 DAYS OF INDEX APPOINTMENT ......... 70
TABLE A3.6: ADJUSTED ODDS RATIO OF A CLINICIAN ATTRIBUTED TEST AMONG PATIENTS WITH
SUSPECTED OR EXPOSED COVID-19............................................................................................... 71
TABLE A3.7: THE ASSOCIATION OF TEST ORDERS AND INAPPROPRIATE PRESCRIPTIONS (ODDS
RATIOS) ............................................................................................................................................... 74
TABLE 4.1: DECEDENT CHARACTERISTICS ............................................................................................... 86
TABLE 4.2: ENROLLED PRESCRIBER CHARACTERISTICS ........................................................................... 86
TABLE 4.3: MODEL RESULTS AS COEFFICIENTS ........................................................................................ 88
TABLE 4.4: ADJUSTED MEAN MONTHLY 2 MG DIAZEPAM PILL EQUIVALENTS DISPENSED PER
PRESCRIBER ........................................................................................................................................ 89
TABLE A4.1: LOG-ODDS OF A GREATER THAN 20% REDUCTION IN DME FROM PRE- TO POST-
PERIOD ................................................................................................................................................ 99
viii
List of Figures
FIGURE 1.1: TIMELINE OF COVID-19 TREATMENT MILESTONES. ......................................................... 7
FIGURE 2.1: UNADJUSTED NUMBER OF OPIOID-RELATED DEATHS IN 2019 AND 2020 ..................... 24
FIGURE 2.2: OPIOID-RELATED DEATHS BY TIME PERIOD AND RACE AND ETHNICITY ......................... 26
FIGURE 3.1: PATTERNS OF CARE FOR PATIENTS WITH INDEX COVID-19 APPOINTMENTS AT A
TELEHEALTH SYSTEM OR URGENT CARE CENTER IN 2020 ............................................................. 48
FIGURE 3.2: ADJUSTED PREDICTED PROBABILITY BY SETTING AND TIME WITH 95% CONFIDENCE
INTERVALS .......................................................................................................................................... 51
FIGURE 3.3: PERCENT OF COVID-19 PATIENTS WITH INAPPROPRIATE PRESCRIPTIONS ..................... 53
FIGURE 3.4: PERCENT OF COVID-19 PATIENTS WITH INAPPROPRIATE ANTIBIOTIC OR STEROID
PRESCRIPTIONS. .................................................................................................................................. 53
FIGURE A3.1: TESTING TRENDS BY SETTING. .......................................................................................... 70
FIGURE A3.2: PREDICTED PROBABILITY OF A TEST ORDER BY SETTING AND TIME WITH 95%
CONFIDENCE INTERVALS. ................................................................................................................. 73
FIGURE 4.1: CONSORT DIAGRAM............................................................................................................... 87
FIGURE 4.2: FREQUENCY OF BENZODIAZEPINE TYPE DISPENSED ........................................................ 87
FIGURE A4.1: QUANTILE-QUANTILE PLOT OF POSITIVE DAILY DMES ................................................. 98
FIGURE A4.2: QUANTILE-QUANTILE PLOT OF POSITIVE LOG(DAILY DME) ......................................... 98
ix
Abbreviations
Abbreviation Term
ACS American Community Survey
AIC Akaike Information Criterion
BIC Bayesian Information Criterion
CCI Charlson Comorbidity Index
CDC Centers for Disease Control and Prevention
CDM Optum's De-identified Clinformatics® Data Mart
CI Confidence Interval
CONSORT Consolidated Standard of Reporting Trials
COVID-19 Coronavirus Disease 2019
CURES California Controlled Substance Utilization Review and Evaluation System
DDS/DMD Dentistry
DME Diazepam Milligram Equivalents
DO Doctor of Osteopathy
ED Emergency Department
EUA Emergency Use Authorization
FDA United States Food and Drug Administration
HHS United States Department of Health and Human Services
HIPAA Health Insurance Portability and Accountability Act
HIV Human Immunodeficiency Virus
ICD-10-CM International Classification of Diseases, Tenth Revision, Clinical Modification
LA Los Angeles
MD Medical Doctor
ME Medical Examiner
MEDI MEDication Indication resource
MG Milligram
MME Milligram Morphine Equivalent
NIA The National Institute on Aging
NP Nurse Practitioner
OR Odds Ratio
OTC Over-the-counter
PA Physician Assistant
RR Relative Risk
Rx Prescription
x
Abbreviation Term
SE Standard Error
US United States
xi
Abstract
Inappropriate prescribing and overprescribing are examples of low- or no-value care that
result in high costs with little to no clinical benefit and patient harms. Suboptimal prescribing is the
result of suboptimal physician decision-making. Physician behavior is influenced by intrinsic and
extrinsic factors, such as reimbursement models, patient demand, diagnostic uncertainty, and poor
numeracy. When faced with numerous diagnostic and treatment decisions per day, physicians rely on
mental shortcut, or “heuristics”, that unconsciously alter their perception of the risks and benefits of
a treatment. While overtreatment or inappropriate prescribing occurs in many conditions, this
dissertation concentrates on opioids, COVID-19 treatments, and benzodiazepines.
The three aims of this dissertation include: (1) the downstream harms of opioid
overprescribing during COVID-19; (2) the impact of setting on physician behavior in treating
COVID-19 outpatients and associated outcomes; and (3) the effectiveness of a behavioral economic
intervention on inappropriate benzodiazepine prescribing. We primarily address these aims using LA
County Department of the Medical Examiner-Coroner autopsy reports, California Controlled
Substance Utilization Review and Evaluation System data, and claims data (Optum’s de-identified
Clinformatics® Data Mart Database (2007-2020)).
We identify community- and decedent-level characteristics associated with opioid-related
deaths following the implementation of stay-at-home orders in Los Angeles County. We estimate if
the likelihood of initial provider interventions for COVID-19, including inappropriate prescribing,
differs by appointment setting (i.e., urgent care center versus dedicated telehealth company) and if
inappropriate prescribing for COVID-19 is associated with adverse outcomes (i.e., hospitalizations
and mortality). Lastly, we measure the effect of a behavioral economic intervention in reducing
benzodiazepine prescribing in a secondary analysis of a randomized controlled trial.
1
Chapter 1
Introduction
Marcella A. Kelley
1.1 Low-value care
Low-value care, the use of wasteful health services that offer little to no net benefit to patients,
is widespread in the United States healthcare system (1–3). The value of health care is measured as
outcomes relative to costs, and the quality of care is assessed by its probability of improving health
outcomes and its concordance with treatment recommendations (4–6). Poor quality can mean too
much, too little, or wrong care (7). Quality can be assessed across three dimensions – structure,
process, and outcomes (8). The process quality of making diagnosis and treatment decisions is
generally assessed by their appropriateness and adherence to guidelines (7). Medical care is
considered appropriate if its expected health benefits (e.g., increased life expectancy, improved health
outcomes, increased quality of life) outweigh its expected health risks (e.g., mortality, morbidity,
anxiety anticipating the intervention) by a “wide enough” margin (9). Care is considered necessary if
there is a reasonably large chance that the patient will benefit and if it would be unethical not to
provide the care (7,10,11). Unnecessary care, overtreatment, overuse, and low-value care are some of
the largest contributors to U.S. health care expenditure waste, accounting for 10% to 17% (i.e., $82 to
$158 billion) of the $760 to $935 billion in excess annual spending in 2019 (12).
The underuse of medical care can leave patients untreated with no improvement in health
outcomes, while the overuse of treatments is costly and presents risks of treatment complications
unnecessarily (7). Measures of health outcomes are not always better in places with greater medical
care utilization in the United States (13). Even if a medical service proves beneficial for a selected
group, using this service in other groups can present harms (14). For example, antidepressants have
2
demonstrated effectiveness in the treatment of severe depression, but for patients with less severe
depression, the small benefits of treatment are outweighed by the adverse events (14,15). Some
medical services accrue similar benefits for a particular disease across all patients but the cost of their
adverse effects are higher when the risk of the disease is lower, such as with mammograms for younger
women (14,16).
Providers cite that 20.6% of medical care is unnecessary and that 22% of prescription
medications are unnecessary (17). There are numerous factors that contribute to a clinician providing
more care than is medically necessary. Two surveys of providers found that the most cited reasons for
overtreatment were malpractice concerns, patient pressure, difficulty accessing medical records,
clinical performance measures, and inadequate time to spend with patients (17,18). Patients may be
more satisfied with more care under the notion that “more care is better care,” and providers want to
achieve high patient satisfaction rates (17,19). Because of this, studies have shown that the use of
shared decision-making tools which educate the patient can lead to more conservative care (20,21).
Additional contributors to unnecessary care include reimbursement models that pay more for
procedures than visits (e.g., fee-for-service), patient demand for tests and treatments, moral hazard
among insured patients, the higher time cost of explaining no treatment compared to writing a
prescription, and defensive medicine (1,14,22). Additionally, unclear or outdated practice guidelines
can contribute to overdiagnosis and unnecessary services (17,23).
Throughout the day, a provider makes numerous treatment decisions. Within these decisions,
the provider may overestimate the benefits and underestimate the harms of a low-value treatment due
to poor numeracy, framing, and discomfort with uncertainty (1). Prescription medications offer great
benefits but present high risks when inappropriately prescribed to patients. The overuse of
prescription medications has high mortality, morbidity, and financial costs. Behavioral economics is
one approach to reducing overprescribing. Supply-side nudges, such as default options or electronic
3
health record alerts, can alter prescribing behavior. Default options have demonstrated effectiveness
in altering the quantity of opioids prescribed and increasing the use of generic alternatives (24–26).
Electronic health record alerts and nudges can reduce inappropriate antibiotic prescribing in primary
care and their effect on opioid prescribing for chronic noncancer pain is being evaluated in primary
care settings (27,28)
Overtreatment and inappropriate prescribing is particularly common with opioids for chronic
noncancer pain, benzodiazepines in combination with opioids, antibiotics in acute care settings, and
polypharmacy among elderly patients (7,29,30). Despite evidence on their lack of benefit and increased
harms, opioid prescribing for acute back pain increased from 2014 to 2018 among Medicare enrollees
(31). Nearly 20% of patients with opioid prescriptions had at least 1 day with an overlapping
benzodiazepine prescription in each month of 2017 despite CDC guidelines discouraging concurrent
benzodiazepine and opioid use (32,33). At least 30% of antibiotic prescriptions in outpatient settings
are inappropriate (34). Behavioral economic interventions and interventions combining provider,
patient, and public education have proven to be the most successful in encouraging the rational use
of antibiotics (27,35). Polypharmacy, characterized as having more than 5 regularly prescribed drugs,
occurs in 29% of elderly patients despite risks of frailty, falls, and mortality (30,36,37). One approach
to reducing polypharmacy in the elderly is to prioritize “deprescribing”, a process in which clinicians
oversee the patient’s tapering or stopping of drugs (30). Deprescribing is complicated by barriers such
as continuing treatment being perceived as higher value than discontinuing treatment, uncertainty
regarding changes to the patient-provider relationship, and clinical complexity (38,39). While
educational efforts and treatment guidelines have been established, continued efforts are needed to
reduce overtreatment and low-value care (31).
4
1.2 Objective
Provider decision-making is influenced by many factors, including the patient-provider
relationship, provider uncertainty, provider incentives, and clinical complexity. Discouraging low-
value care and encouraging the rational use of medications requires a multi-faceted approach. The
objective of this dissertation is to analyze downstream medication overuse harms, identify
inappropriate prescribing, and evaluate interventions to reduce overprescribing. To do this, we will
identify factors related to opioid overdose deaths during COVID-19, measure inappropriate
outpatient prescribing for COVID-19 and its associated outcomes, and estimate a behavioral
economic intervention’s effect on inappropriate benzodiazepine prescribing.
1.3 Specific Aims
AIM 1: To evaluate decedent- and community-level associations with an opioid-related death
following the implementation of stay-at-home orders in Los Angeles County.
Aim 1.1: We compare the monthly number of opioid-related deaths in January-April of 2019 and 2020
in Los Angeles County.
Aim 1.2: We estimate the change in opioid-related deaths following the implementation of stay-at-
home orders in Los Angeles County.
Aim 1.3: We estimate the relative risk of an opioid-related death following stay-at-home orders by
individual- and community-level factors.
AIM 2: To assess the variation in COVID-19 treatment patterns and associated outcomes by
appointment setting (i.e., urgent care center versus dedicated telehealth company).
Aim 2.1: We characterize the unadjusted variation in initial provider interventions (i.e., emergency
department referrals, inappropriate prescriptions, and COVID-19 test orders) and patient outcomes
of outpatients with COVID-like illness.
5
Aim 2.2: We estimate if the likelihood of each provider action differs in urgent care versus telehealth
over time, controlling for patient and appointment characteristics.
Aim 2.3: We assess provider group variation in the likelihood of provider actions.
Aim 2.4: We estimate if inappropriate prescribing for COVID-19 is associated with increased
hospitalizations or mortality, controlling for patient and appointment characteristics.
AIM 3: To measure the association of fatal overdose notification letters with prescription of
benzodiazepines through the secondary analysis of a randomized control trial
Aim 3.1: We estimate the change in daily benzodiazepines (average diazepam milligram equivalents
and 2 milligram diazepam equivalent pills) dispensed pre- and post-intervention between the control
and intervention groups.
Aim 3.2: We estimate if other changes in potentially high-risk prescribing, including the probability,
number, and average days supply of “new benzodiazepine starts”, differs pre- and post-intervention
between groups.
Aim 3.3: We assess if the notification was associated with potentially unsafe and considerable (> 20%)
reductions in daily diazepam milligrams from the pre- to post-intervention period, controlling for the
provider’s share of new patients and patients with co-prescribed opioids and benzodiazepines.
1.4 Opioid overuse: Harms & treatment guidelines
Opioids carry significant risks of overdose and addiction, and there is evidence that non-opioid
alternatives are safer and effective (40). A meta-analysis of opioid treatment for chronic noncancer
pain found no clinically significant benefit of opioid treatment (41). Approximately 19% of adults in
the United States filled an opioid prescription in 2018 (42). While opioid prescribing in the US has
decreased since its peak in 2010, the milligram morphine equivalent (MME) per capita remains
approximately two times higher than in 1999 (43,44). Despite this high level of prescribing, reports of
6
pain have not decreased (45,46). In 2016, the Centers for Disease Control and Prevention (CDC)
issued the "CDC Guideline for Prescribing Opioids for Chronic Pain" to address opioid initiation or
continuation for chronic pain, opioid selection, quantity, follow-up, tapering, concurrent prescribing,
and opioid harm (33). The CDC Guideline for Prescribing Opioids for Chronic Pain addresses the
high risks of prolonged opioid exposure (33). The guideline encourages the use of alternatives to
opioids and other practices that minimize harm to patients (33).
The downstream harms of opioid overprescribing are significant as 75% of heroin users in the
2000s first used prescription opioids (47). Aggregate costs for prescription opioid harms are estimated
to be over $78.5 billion (in 2013 US dollars) (48). There were 68,630 overdose deaths involving opioids
in the United States in 2020 (49). The emergence of Coronavirus disease 2019 (COVID-19) put the
United States in the midst of a pandemic and the ongoing opioid epidemic. Their simultaneous
occurrence posed a challenge for policymakers seeking to reduce infectious disease spread without
exacerbating opioid use and overdose. In the months following the introduction of state-mandated
stay-at-home orders, there were increases in non-fatal and fatal drug overdoses (50–55). Little is
known about whether COVID-19 and its related policies have resulted in rising opioid-related deaths
in some communities but not others.
1.5 Antibiotic overuse: Harms & treatment guidelines during COVID-19
During the COVID-19 pandemic, concerns of inappropriate antibiotics increased as providers
faced clinical uncertainty and prescribed repurposed drugs, often without robust evidence, to treat
COVID-19 (56,57). The overprescribing of antibiotics is associated with increased risk of antibiotic-
resistant bacterial infections, longer and more severe infections, adverse events, mortality, and
healthcare costs (35). In the United States, 2.8 million people get an antibiotic-resistant infection
annually, of whom more than 35,000 die (58). Treating 6 of the antibiotic resistant infections that are
7
of serious or urgent concern to human health costs $4.6 billion dollars annually in the United States
(59,60). Because of concerns of exacerbating antibiotic resistant infections and lack of proven efficacy,
the World Health Organization and the National Institutes of Health recommended against the use
of antibiotics in patients with mild to moderate COVID-19 (57,61). Despite these guidelines, the
antibiotic visit prescribing rate for Medicare beneficiaries with confirmed COVID-19 was 30% in the
first year of the pandemic (62).
The diagnosis of and treatment guidelines for COVID-19 changed rapidly throughout the
pandemic (Figure 1.1). As of April 2022, outpatient COVID-19 treatment guidelines recommend
medications for symptom management, Paxlovid and Veklury for mild to moderate COVID-19
patients with a high risk of clinical progression, and Bebtelovimab or Lagevrio when Paxlovid and
Veklury are not feasible or appropriate (63). Outpatient guidelines recommend against systemic
glucocorticoids, antibiotics, chloroquine or hydroxychloroquine with or without azithromycin,
lopinavir/ritonavir, and other HIV protease inhibitors (63).
Figure 1.1: Timeline of COVID-19 Treatment Milestones. A timeline of Food and Drug
Administration (FDA) and COVID-19 Treatment Guidelines Panel milestones from March 2020 to
April 2022. Emergency use authorizations are abbreviated as EUAs.
The use of inappropriate outpatient COVID-19 treatments, such as antibiotics,
corticosteroids, and hydroxychloroquine, presents risks of adverse events. One antibiotic,
8
azithromycin, has repeatedly demonstrated no benefit in treating COVID-19 but increases risks of
antimicrobial risk, and in combination with hydroxychloroquine, demonstrates no treatment effect
and risks of cardiac adverse events (64,65). Inappropriate corticosteroid presents risks of
hyperglycemia, neuropsychiatric symptoms, and secondary infections (63). In particular, the use of
one corticosteroid found to benefit hospitalized patients on respiratory support, dexamethasone, has
been associated with increased mortality rates when given to COVID-19 patients not requiring oxygen
(66). Despite guideline recommendations against systemic corticosteroid use in outpatients, 16% of
Medicare patients with COVID-19 were prescribed systemic corticosteroids in the first 16 months of
the pandemic (67).
COVID-19 and its related policies, such as lockdown measures, necessitated an increase in the
use of telehealth services. The impact of setting on antibiotic prescribing has been under scrutiny as
the utilization of retail health clinics and dedicated telehealth companies has increased (68–70). Quality
concerns regarding antibiotic prescribing in dedicated telehealth settings include the lack of a prior
physician-patient relationship, limited access to medical records, limited physical examination, and
barriers to testing (68). High rates of likely inappropriate antibiotic prescribing for respiratory tract
infections have been documented in dedicated telehealth companies (71). Telehealth appointments
with an antibiotic prescription have higher patient satisfaction and shorter visit times, which are both
incentives for the provider to continue prescribing antibiotics (72). However, patients selecting into
telehealth versus in-person settings may differ in unobserved ways, such as infection severity, patient-
provider relationship, or socioeconomic demographics, that are often not accounted for in
observational studies comparing in-person to telehealth settings. With increased numbers of patients
turning to telehealth for treatment in 2020, it is critical to understand if and how the value of care
delivered in telehealth differed from in-person urgent care, particularly regarding inappropriate
prescribing for COVID-19 and its associated outcomes.
9
1.6 Benzodiazepine overuse: Harms & treatment guidelines
In the last decade, the harms of antibiotic overuse and opioid overuse have appropriately
garnered national attention and public health interventions (73–75). However, much less attention has
been given to the alarming overprescribing of benzodiazepines (76). Benzodiazepines are psychotropic
drugs that are commonly prescribed but have a dose-dependent relationship associated with tolerance,
sedation, confusion, and increased mortality (77,78). (79). Treatment guidelines only recommend
short-term benzodiazepine use, typically as second- or third-line, for the treatment of anxiety or sleep
disorders (78,80–84). Long-term use is associated with memory impairment, vehicle accidents, falls,
overdoses, dependence, and severe withdrawal symptoms (85). Despite these guidelines and known
risks, benzodiazepine prescription fills, quantity per prescription, and mortality have increased in the
past 25 years in the United States (86,87). Increased utilization of this drug class is largely driven by
an increased number of patients with long-term use of benzodiazepines and increased outpatient
prescribing by primary care clinicians (86,88,89).
The costs of benzodiazepine use and misuse are high. Between 1991 and 2009, Medicaid
expenditures on benzodiazepines increased by approximately $40 million (90). From 1999 to 2020,
annual fatal overdoses involving benzodiazepines increased from 1,135 to 12,290 (49). Concurrent
opioid and benzodiazepine use increases the likelihood of respiratory depression, hospital admission,
emergency department visits, and fatal overdoses compared to opioid use alone (91–93). Despite CDC
guidelines discouraging concurrent benzodiazepine and opioid use, nearly 20% of patients with opioid
prescriptions had at least 1 day with an overlapping benzodiazepine prescription in each month of
2017 (32,33). In the first half of 2018, nearly one third of fatal opioid overdoses in the United States
involved benzodiazepines (79). Efforts to reduce inappropriate benzodiazepine prescribing are
needed.
10
References
1. Oakes AH, Radomski TR. Reducing Low-Value Care and Improving Health Care Value. JAMA.
2021 May 4;325(17):1715–6.
2. Powers BW, Jain SH, Shrank WH. De-adopting Low-Value Care: Evidence, Eminence, and
Economics. JAMA. 2020 Oct 27;324(16):1603–4.
3. Schwartz AL, Landon BE, Elshaug AG, Chernew ME, McWilliams JM. Measuring low-value
care in Medicare. JAMA Intern Med. 2014 Jul;174(7):1067–76.
4. Porter ME, Teisberg EO. Redefining Health Care: Creating Value-based Competition on Results.
Harvard Business Press; 2006. 506 p.
5. Porter ME. What is value in health care? N Engl J Med. 2010 Dec 23;363(26):2477–81.
6. Institute of Medicine, Committee to Design a Strategy for Quality Review and Assurance in
Medicare. Medicare: A Strategy for Quality Assurance, Volume I. National Academies Press;
1990. 468 p.
7. Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United
States? 1998. Milbank Q. 2005;83(4):843–95.
8. Donabedian A. Explorations in Quality Assessment and Monitoring: The definition of quality
and approaches to its assessment. Health Administration Press; 1980. 163 p.
9. Brook RH, Chassin MR, Fink A, Solomon DH, Kosecoff J, Park RE. A method for the detailed
assessment of the appropriateness of medical technologies. Int J Technol Assess Health Care.
1986;2(1):53–63.
10. Kahan JP, Bernstein SJ, Leape LL, Hilborne LH, Park RE, Parker L, et al. Measuring the necessity
of medical procedures. Med Care. 1994 Apr;32(4):357–65.
11. Laouri M, Kravitz RL, Bernstein SJ, French WJ, Leake B, Borowsky SJ, et al. Under use of
coronary angiography: application of a clinical method. Int J Qual Health Care. 1997 Feb;9(1):15–
22.
12. Shrank WH, Rogstad TL, Parekh N. Waste in the US Health Care System: Estimated Costs and
Potential for Savings. JAMA. 2019 Oct 15;322(15):1501–9.
13. Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ, Lucas FL, Pinder EL. The implications of
regional variations in Medicare spending. Part 2: health outcomes and satisfaction with care. Ann
Intern Med. 2003 Feb 18;138(4):288–98.
14. Grady D, Redberg RF. Less is more: how less health care can result in better health. Arch Intern
Med. 2010 May 10;170(9):749–50.
11
15. Fournier JC, DeRubeis RJ, Hollon SD, Dimidjian S, Amsterdam JD, Shelton RC, et al.
Antidepressant drug effects and depression severity: a patient-level meta-analysis. JAMA. 2010
Jan 6;303(1):47–53.
16. Woloshin S, Schwartz LM. The benefits and harms of mammography screening: understanding
the trade-offs. JAMA. 2010 Jan 13;303(2):164–5.
17. Lyu H, Xu T, Brotman D, Mayer-Blackwell B, Cooper M, Daniel M, et al. Overtreatment in the
United States. PLoS One. 2017 Sep 6;12(9):e0181970.
18. Sirovich BE, Woloshin S, Schwartz LM. Too Little? Too Much? Primary care physicians’ views
on US health care: a brief report. Arch Intern Med. 2011 Sep 26;171(17):1582–5.
19. Foster CB, Martinez KA, Sabella C, Weaver GP, Rothberg MB. Patient Satisfaction and
Antibiotic Prescribing for Respiratory Infections by Telemedicine. Pediatrics [Internet]. 2019
Sep;144(3). Available from: http://dx.doi.org/10.1542/peds.2019-0844
20. Behnke LM, Solis A, Shulman SA, Skoufalos A. A targeted approach to reducing overutilization:
use of percutaneous coronary intervention in stable coronary artery disease. Popul Health Manag.
2013 Jun;16(3):164–8.
21. McCullough JM, Zimmerman FJ, Rodriguez HP. Impact of clinical decision support on receipt
of antibiotic prescriptions for acute bronchitis and upper respiratory tract infection. J Am Med
Inform Assoc. 2014 Nov;21(6):1091–7.
22. Cutler DM, Zeckhauser RJ. Chapter 11 The anatomy of health insurance. Handbook of Health
Economics; Culyer, AJ, Newhouse, JP, Eds. :563–643.
23. Mangin D, Heath I, Jamoulle M. Beyond diagnosis: rising to the multimorbidity challenge. BMJ.
2012 Jun 13;344:e3526.
24. Delgado MK, Shofer FS, Patel MS, Halpern S, Edwards C, Meisel ZF, et al. Association between
Electronic Medical Record Implementation of Default Opioid Prescription Quantities and
Prescribing Behavior in Two Emergency Departments. J Gen Intern Med. 2018 Apr;33(4):409–
11.
25. Santistevan JR, Sharp BR, Hamedani AG, Fruhan S, Lee AW, Patterson BW. By Default: The
Effect of Prepopulated Prescription Quantities on Opioid Prescribing in the Emergency
Department. West J Emerg Med. 2018 Mar;19(2):392–7.
26. Patel MS, Day SC, Halpern SD, Hanson CW, Martinez JR, Honeywell S Jr, et al. Generic
medication prescription rates after health system-wide redesign of default options within the
electronic health record. JAMA Intern Med. 2016 Jun 1;176(6):847–8.
27. Meeker D, Linder JA, Fox CR, Friedberg MW, Persell SD, Goldstein NJ, et al. Effect of
Behavioral Interventions on Inappropriate Antibiotic Prescribing Among Primary Care
Practices: A Randomized Clinical Trial. JAMA. 2016 Feb 9;315(6):562–70.
12
28. Kelley MA, Persell SD, Linder JA, Friedberg MW, Meeker D, Fox CR, et al. The protocol of the
Application of Economics & Social psychology to improve Opioid Prescribing Safety Trial 1
(AESOPS-1): Electronic health record nudges. Contemp Clin Trials. 2021 Apr;103:106329.
29. Morgan DJ, Dhruva SS, Coon ER, Wright SM, Korenstein D. 2019 Update on Medical Overuse:
A Review. JAMA Intern Med. 2019 Nov 1;179(11):1568–74.
30. Scott IA, Hilmer SN, Reeve E, Potter K, Le Couteur D, Rigby D, et al. Reducing inappropriate
polypharmacy: the process of deprescribing. JAMA Intern Med. 2015 May;175(5):827–34.
31. Mafi JN, Reid RO, Baseman LH, Hickey S, Totten M, Agniel D, et al. Trends in Low-Value
Health Service Use and Spending in the US Medicare Fee-for-Service Program, 2014-2018.
JAMA Netw Open. 2021 Feb 1;4(2):e2037328.
32. Bohnert ASB, Guy GP Jr, Losby JL. Opioid Prescribing in the United States Before and After
the Centers for Disease Control and Prevention’s 2016 Opioid Guideline. Ann Intern Med. 2018
Sep 18;169(6):367–75.
33. Dowell D, Haegerich TM, Chou R. CDC Guideline for Prescribing Opioids for Chronic Pain -
United States, 2016. MMWR Recomm Rep. 2016 Mar 18;65(1):1–49.
34. Fleming-Dutra KE, Hersh AL, Shapiro DJ, Bartoces M, Enns EA, File TM Jr, et al. Prevalence
of Inappropriate Antibiotic Prescriptions Among US Ambulatory Care Visits, 2010-2011. JAMA.
2016 May 3;315(17):1864–73.
35. Llor C, Bjerrum L. Antimicrobial resistance: risk associated with antibiotic overuse and initiatives
to reduce the problem. Ther Adv Drug Saf. 2014 Dec;5(6):229–41.
36. Qato DM, Alexander GC, Conti RM, Johnson M, Schumm P, Lindau ST. Use of prescription
and over-the-counter medications and dietary supplements among older adults in the United
States. JAMA. 2008 Dec 24;300(24):2867–78.
37. Gnjidic D, Hilmer SN, Blyth FM, Naganathan V, Waite L, Seibel MJ, et al. Polypharmacy cutoff
and outcomes: five or more medicines were used to identify community-dwelling older men at
risk of different adverse outcomes. J Clin Epidemiol. 2012 Sep;65(9):989–95.
38. Anderson K, Stowasser D, Freeman C, Scott I. Prescriber barriers and enablers to minimising
potentially inappropriate medications in adults: a systematic review and thematic synthesis. BMJ
Open. 2014 Dec 8;4(12):e006544.
39. Reeve E, To J, Hendrix I, Shakib S, Roberts MS, Wiese MD. Patient barriers to and enablers of
deprescribing: a systematic review. Drugs Aging. 2013 Oct;30(10):793–807.
40. Krebs EE, Gravely A, Nugent S, Jensen AC, DeRonne B, Goldsmith ES, et al. Effect of Opioid
vs Nonopioid Medications on Pain-Related Function in Patients With Chronic Back Pain or Hip
or Knee Osteoarthritis Pain: The SPACE Randomized Clinical Trial. JAMA. 2018 Mar
6;319(9):872–82.
13
41. Busse JW, Wang L, Kamaleldin M, Craigie S, Riva JJ, Montoya L, et al. Opioids for Chronic
Noncancer Pain: A Systematic Review and Meta-analysis. JAMA. 2018 Dec 18;320(23):2448–60.
42. Schieber LZ, Guy GP Jr, Seth P, Losby JL. Variation in adult outpatient opioid prescription
dispensing by age and sex - United States, 2008-2018. MMWR Morb Mortal Wkly Rep. 2020 Mar
20;69(11):298–302.
43. Guy GP Jr, Zhang K, Bohm MK, Losby J, Lewis B, Young R, et al. Vital Signs: Changes in
Opioid Prescribing in the United States, 2006-2015. MMWR Morb Mortal Wkly Rep. 2017 Jul
7;66(26):697–704.
44. Centers for Disease Control and Prevention. 2019 Annual Surveillance Report of Drug-Related
Risks and Outcomes — United States Surveillance Special Report [Internet]. Centers for Disease
Control and Prevention, U.S. Department of Health and Human Services; 2019 Nov. Available
from: https://www.cdc.gov/drugoverdose/pdf/pubs/2019-cdc-drug-surveillancereport.pdf.
45. Chang H-Y, Daubresse M, Kruszewski SP, Alexander GC. Prevalence and treatment of pain in
EDs in the United States, 2000 to 2010. Am J Emerg Med. 2014 May;32(5):421–31.
46. Daubresse M, Chang H-Y, Yu Y, Viswanathan S, Shah ND, Stafford RS, et al. Ambulatory
diagnosis and treatment of nonmalignant pain in the United States, 2000-2010. Med Care. 2013
Oct;51(10):870–8.
47. Lankenau SE, Teti M, Silva K, Jackson Bloom J, Harocopos A, Treese M. Initiation into
prescription opioid misuse amongst young injection drug users. Int J Drug Policy. 2012
Jan;23(1):37–44.
48. Florence CS, Zhou C, Luo F, Xu L. The Economic Burden of Prescription Opioid Overdose,
Abuse, and Dependence in the United States, 2013. Med Care. 2016 Oct;54(10):901–6.
49. National Institute on Drug Abuse. Overdose death rates [Internet]. National Institute on Drug
Abuse. 2022 [cited 2022 Apr 28]. Available from: https://nida.nih.gov/drug-topics/trends-
statistics/overdose-death-rates
50. Alter A, Yeager C. COVID-19 impact on us national overdose crisis. Overdose detection
mapping application program. 2020.
51. Appa A, Rodda LN, Cawley C, Zevin B, Coffin PO, Gandhi M, et al. Drug Overdose Deaths
Before and After Shelter-in-Place Orders During the COVID-19 Pandemic in San Francisco.
JAMA Netw Open. 2021 May 3;4(5):e2110452.
52. Mason M, Welch SB, Arunkumar P, Post LA, Feinglass JM. Notes from the Field: Opioid
Overdose Deaths Before, During, and After an 11-Week COVID-19 Stay-at-Home Order —
Cook County, Illinois, January 1, 2018–October 6, 2020 [Internet]. Vol. 70, MMWR. Morbidity
and Mortality Weekly Report. 2021. p. 362–3. Available from:
http://dx.doi.org/10.15585/mmwr.mm7010a3
53. Friedman J, Akre S. COVID-19 and the Drug Overdose Crisis: Uncovering the Deadliest
Months in the United States, January ‒July 2020. Am J Public Health. 2021 Jul;111(7):1284–91.
14
54. Products - vital statistics rapid release - provisional drug overdose data [Internet]. 2021 [cited
2021 Oct 22]. Available from: https://www.cdc.gov/nchs/nvss/vsrr/drug-overdose-data.htm
55. U.S. Centers for Disease Control and Prevention. Suspected Nonfatal Drug Overdoses during
COVID-19 [Internet]. 2021 [cited 2021 Oct 22]. Available from:
https://www.cdc.gov/drugoverdose/nonfatal/states/covid-19.html
56. Cohen MS, Wohl DA, Fischer WA, Smith DM, Eron JJ. Outpatient Treatment of SARS-CoV-2
Infection to Prevent COVID-19 Progression. Clin Infect Dis [Internet]. 2021 May 28; Available
from: http://dx.doi.org/10.1093/cid/ciab494
57. Hsu J. How covid-19 is accelerating the threat of antimicrobial resistance. BMJ. 2020 May
18;369:m1983.
58. CDC. Antibiotic / antimicrobial Resistance [Internet]. 2021 [cited 2021 Oct 21]. Available from:
https://www.cdc.gov/drugresistance/index.html
59. CDC. Biggest Threats and Data [Internet]. 2021 [cited 2021 Oct 22]. Available from:
https://www.cdc.gov/drugresistance/biggest-threats.html
60. Centers for Disease Control and Prevention (U.S.). Antibiotic Resistance Threats in the United
States, 2019. U.S. Department of Health and Human Services, Centres for Disease Control and
Prevention; 2019. 115 p.
61. Nonhospitalized adults: Therapeutic management [Internet]. COVID-19 Treatment Guidelines.
[cited 2022 May 10]. Available from:
https://www.covid19treatmentguidelines.nih.gov/management/clinical-
management/nonhospitalized-adults--therapeutic-
management/?utm_source=site&utm_medium=home&utm_campaign=highlights
62. Tsay SV, Bartoces M, Gouin K, Kabbani S, Hicks LA. Antibiotic Prescriptions Associated With
COVID-19 Outpatient Visits Among Medicare Beneficiaries, April 2020 to April 2021. JAMA
[Internet]. 2022 Apr 8; Available from: http://dx.doi.org/10.1001/jama.2022.5471
63. Nonhospitalized Adults: Therapeutic Management [Internet]. [cited 2021 Oct 22]. Available
from: https://www.covid19treatmentguidelines.nih.gov/management/clinical-
management/nonhospitalized-adults--therapeutic-management/
64. Chloroquine or hydroxychloroquine [Internet]. COVID-19 Treatment Guidelines. [cited 2022
Apr 28]. Available from: https://www.covid19treatmentguidelines.nih.gov/therapies/antiviral-
therapy/chloroquine-or-hydroxychloroquine-and-or-azithromycin/
65. Kow CS, Hasan SS. Use of Azithromycin in COVID-19: A Cautionary Tale. Clin Drug Investig.
2020 Oct;40(10):989–90.
66. Crothers K, DeFaccio R, Tate J, Alba PR, Goetz MB, Jones B, et al. Dexamethasone in
hospitalised coronavirus-19 patients not on intensive respiratory support. Eur Respir J [Internet].
2021 Nov 25; Available from: http://dx.doi.org/10.1183/13993003.02532-2021
15
67. Bradley MC, Perez-Vilar S, Chillarige Y, Dong D, Martinez AI, Weckstein AR, et al. Systemic
Corticosteroid Use for COVID-19 in US Outpatient Settings From April 2020 to August 2021.
JAMA [Internet]. 2022 Apr 8; Available from: http://dx.doi.org/10.1001/jama.2022.4877
68. Uscher-Pines L, Mulcahy A, Cowling D, Hunter G, Burns R, Mehrotra A. Antibiotic Prescribing
for Acute Respiratory Infections in Direct-to-Consumer Telemedicine Visits. JAMA Intern Med.
2015 Jul;175(7):1234–5.
69. Palms DL, Hicks LA, Bartoces M, Hersh AL, Zetts R, Hyun DY, et al. Comparison of Antibiotic
Prescribing in Retail Clinics, Urgent Care Centers, Emergency Departments, and Traditional
Ambulatory Care Settings in the United States. JAMA Intern Med. 2018 Sep 1;178(9):1267–9.
70. Barnett ML, Ray KN, Souza J, Mehrotra A. Trends in Telemedicine Use in a Large Commercially
Insured Population, 2005-2017. JAMA. 2018 Nov 27;320(20):2147–9.
71. Martinez KA, Rood M, Jhangiani N, Kou L, Boissy A, Rothberg MB. Association Between
Antibiotic Prescribing for Respiratory Tract Infections and Patient Satisfaction in Direct-to-
Consumer Telemedicine. JAMA Intern Med. 2018 Nov 1;178(11):1558–60.
72. Martinez KA, Rood M, Jhangiani N, Boissy A, Rothberg MB. Antibiotic Prescribing for
Respiratory Tract Infections and Encounter Length: An Observational Study of Telemedicine.
Ann Intern Med. 2019 Feb 19;170(4):275–7.
73. Digital Communications Division (DCD). 5-Point Strategy to combat the opioid crisis [Internet].
2018 [cited 2021 Oct 22]. Available from: https://www.hhs.gov/opioids/about-the-
epidemic/hhs-response/index.html
74. Executive Order -- Combating Antibiotic-Resistant Bacteria [Internet]. 2014 [cited 2021 Oct 21].
Available from: https://obamawhitehouse.archives.gov/the-press-
office/2014/09/18/executive-order-combating-antibiotic-resistant-bacteria
75. Healthcare Professional Resources and Training [Internet]. 2021 [cited 2021 Oct 22]. Available
from: https://www.cdc.gov/antibiotic-use/training/index.html
76. Lembke A, Papac J. Our other prescription drug problem. The New England [Internet]. 2018;
Available from: https://pubmed.ncbi.nlm.nih.gov/29466163/
77. Weich S, Pearce HL, Croft P, Singh S, Crome I, Bashford J, et al. Effect of anxiolytic and
hypnotic drug prescriptions on mortality hazards: retrospective cohort study. BMJ. 2014 Mar
19;348:g1996.
78. Locke AB, Kirst N, Shultz CG. Diagnosis and management of generalized anxiety disorder and
panic disorder in adults. Am Fam Physician. 2015 May 1;91(9):617–24.
79. Gladden RM, O’Donnell J, Mattson CL, Seth P. Changes in Opioid-Involved Overdose Deaths
by Opioid Type and Presence of Benzodiazepines, Cocaine, and Methamphetamine - 25 States,
July-December 2017 to January-June 2018. MMWR Morb Mortal Wkly Rep. 2019 Aug
30;68(34):737–44.
16
80. Boggs JM, Lindrooth RC, Battaglia C, Beck A. Association between suicide death and
concordance with benzodiazepine treatment guidelines for anxiety and sleep disorders. Gen
Hosp Psychiatry [Internet]. 2020; Available from:
https://www.sciencedirect.com/science/article/pii/S0163834319303214
81. Katzman MA, Bleau P, Blier P, Chokka P, Kjernisted K, Van Ameringen M, et al. Canadian
clinical practice guidelines for the management of anxiety, posttraumatic stress and obsessive-
compulsive disorders. BMC Psychiatry. 2014 Jul 2;14 Suppl 1:S1.
82. Baldwin DS, Anderson IM, Nutt DJ, Allgulander C, Bandelow B, den Boer JA, et al. Evidence-
based pharmacological treatment of anxiety disorders, post-traumatic stress disorder and
obsessive-compulsive disorder: a revision of the 2005 guidelines from the British Association for
Psychopharmacology. J Psychopharmacol. 2014 May;28(5):403–39.
83. Matheson E, Hainer BL. Insomnia: Pharmacologic Therapy. Am Fam Physician. 2017 Jul
1;96(1):29–35.
84. Yan Li NF. Benzodiazepines for the treatment of adults with mental health conditions or sleep
disorders guidelines [Internet]. [cited 2021 Oct 22]. Available from:
https://www.cadth.ca/benzodiazepines-treatment-adults-mental-health-conditions-or-sleep-
disorders-guidelines
85. Fluyau D, Revadigar N, Manobianco BE. Challenges of the pharmacological management of
benzodiazepine withdrawal, dependence, and discontinuation. Ther Adv Psychopharmacol. 2018
May;8(5):147–68.
86. Agarwal SD, Landon BE. Patterns in Outpatient Benzodiazepine Prescribing in the United
States. JAMA Netw Open. 2019 Jan 4;2(1):e187399.
87. Bachhuber MA, Hennessy S, Cunningham CO, Starrels JL. Increasing Benzodiazepine
Prescriptions and Overdose Mortality in the United States, 1996–2013. Am J Public Health. 2016
Apr 1;106(4):686–8.
88. Kaufmann CN, Spira AP, Depp CA, Mojtabai R. Long-Term Use of Benzodiazepines and
Nonbenzodiazepine Hypnotics, 1999-2014. Psychiatr Serv. 2018 Feb 1;69(2):235–8.
89. Olfson M, King M, Schoenbaum M. Benzodiazepine use in the United States. JAMA Psychiatry.
2015 Feb;72(2):136–42.
90. Gorevski E, Bian B, Kelton CML, Martin Boone JE, Guo JJ. Utilization, spending, and price
trends for benzodiazepines in the US Medicaid program: 1991-2009. Ann Pharmacother. 2012
Apr;46(4):503–12.
91. Dasgupta N, Funk MJ, Proescholdbell S, Hirsch A, Ribisl KM, Marshall S. Cohort Study of the
Impact of High-Dose Opioid Analgesics on Overdose Mortality. Pain Med. 2016 Jan;17(1):85–
98.
92. Jones JD, Mogali S, Comer SD. Polydrug abuse: a review of opioid and benzodiazepine
combination use. Drug Alcohol Depend. 2012 Sep 1;125(1–2):8–18.
17
93. Sun EC, Dixit A, Humphreys K, Darnall BD, Baker LC, Mackey S. Association between
concurrent use of prescription opioids and benzodiazepines and overdose: retrospective analysis.
BMJ. 2017 Mar 14;356:j760.
18
Chapter 2
Opioid-related deaths before and after COVID-19 stay-at-home orders in
Los Angeles County
Marcella A. Kelley, Jonathan Lucas, Emily Stewart, Dana Goldman, Jason N. Doctor
Original version published in Drug and alcohol dependence. 2021 Nov 1;228:109028
19
Abstract
Background: Opioid-related morbidity and mortality has increased during the COVID-19
pandemic, yet specific information about the communities most affected remains unknown. Our
objective is to evaluate decedent-level associations with an opioid-related death following the
implementation of stay-at-home orders in Los Angeles County.
Methods: This retrospective cohort study used data from the L.A. County Medical Examiner-
Coroner to identify opioid-related deaths in 2019 and 2020. We used logistic regression to analyze
the change in opioid-related deaths following the start of stay-at-home orders (i.e., April 2020 versus
February 2020). Independent variables included decedent age, gender, race and ethnicity, heroin or
fentanyl present at the time of death, census tract-level education, and a scheduled drug prescription
in the year before death.
Results: Opioid-related deaths in L.A. County were most common in census tracts where a small
percentage of the population has a Bachelor’s degree. Following stay-at-home orders, non-Hispanic
White individuals had significantly more opioid-related deaths than Hispanic individuals (risk ratio
(RR): 1.82 [95% CI, 1.10 to 3.02]; P < 0.05) after adjusting for age, gender, and heroin or fentanyl
use. Racial and ethnic differences in mortality were not explained by census tract-level education or
recent scheduled drug prescriptions.
Conclusions: There has been an alarming rise in opioid-related deaths in L.A. County during 2020.
The increase in opioid-related overdose deaths following the onset of COVID-19 and related
policies occurred most among non-Hispanic White individuals. Further research on this trend's
underlying cause is needed to inform policy recommendations during these simultaneous public
health crises.
20
2.1 Background
The United States is amid both a coronavirus disease 2019 (COVID-19) pandemic and an
ongoing opioid epidemic. Their simultaneous occurrence poses a challenge for policymakers seeking
to reduce infectious disease spread without exacerbating opioid use and overdose. Restrictions to the
healthcare system, contraction of the economy, and general stress brought on by a deadly pandemic
may have contributed to increases in licit and illicit opioid use and subsequent overdose--both fatal
and non-fatal (1–5). In the months following the introduction of state-mandated stay-at-home
orders, there were increases in non-fatal and fatal drug overdoses across the United States (4,6–10).
These policies seeking to reduce the spread of COVID-19 may have also exacerbated mental health
issues that could trigger relapse and increased the risk of an isolated overdose without rescue (11).
As of March 2021, L.A. County has the largest number of confirmed COVID-19 cases and
deaths nationwide, despite statewide stay-at-home orders in effect since March 19, 2020 (12,13).
California has the 4th highest number of drug overdoses in the country, and L.A. County has
historically had a higher prevalence of prescription opioid and misuse than the national average (14).
From 2019 to 2020 the fentanyl-related mortality rate in L.A. County more than doubled (15).
Therefore, understanding how the COVID-19 pandemic and opioid epidemic have impacted L.A.
County provides useful information to prevent further exacerbation of the opioid crisis.
COVID-19 has disproportionately impacted historically marginalized groups and raised
concerns about health disparities (16). The COVID-19 and opioid epidemics are concentrated in
different racial and ethnic groups. There are higher rates of infection, hospitalization, and death
from COVID-19 among Black, Hispanic, Latino, American Indian, and Alaska Native individuals
compared to non-Hispanic White individuals (17). The opioid epidemic has been concentrated
among low-income non-Hispanic White individuals and people with lower educational attainment
(18–20). Little is known about whether COVID-19 and its related policies have resulted in rising
21
opioid-related deaths in some communities but not others. While aggregate level reports have shown
an increase in opioid overdoses during this time, no study has evaluated decedent- and community-
level socioeconomic characteristics. Our study's main objective is to identify factors associated with
opioid-related overdose deaths following the implementation of stay-at-home orders in L.A.
County.
2.2 Methods
2.2.1 Data Sources
This retrospective cohort study relied upon data from the Los Angeles County Department
of the Medical Examiner-Coroner autopsy reports, California Controlled Substance Utilization
Review and Evaluation System (CURES), and the American Community Survey. All drug overdose
deaths in L.A. County are investigated by the Department of the Medical Examiner-Coroner.
Completed autopsy reports for deaths with opioid-related causes occurring in L.A. County between
January 2019 and July 2020 were collected from an ongoing study with the L.A. County Medical
Examiner-Coroner. Demographic characteristics included in the death record were collected. Using
this decedent information, we queried CURES for the presence of a prescription for a scheduled
drug written by a prescriber in Los Angeles County in the year before death. The California
Controlled Substance Utilization Review and Evaluation System is the prescription drug monitoring
program for the state. It stores Schedule II, III, and IV controlled substance prescription dispensing
information, dates of a prescription fill, and quantities (21). The Medical Examiner-Coroner has the
authority to use CURES to educate practitioners and others, following the California State's Health
and Safety Code § 11165(c)(2). To capture census tract-level characteristics, we used the publicly
available American Community Survey 2018 data. The survey is conducted monthly among various
22
samples of the U.S. population. It collects current information on topics not reported in the U.S.
Census, such as education, employment, internet access, and transportation (22).
2.2.2 Measures
Medical Examiner-Coroner death reports provided complete information regarding the
decedent’s date of death, age, gender, race, ethnicity, and the presence of fentanyl or heroin at the
time of death. The controlled-substance utilization review evaluation system provided the decedent's
most recent address and scheduled drug prescriptions in the year before death. Addresses were
linked to census tract-level variables using the Census Geocoder (23). For errors where the geocoder
was unable to return results, street, zip code, and city received manual verification and updating. The
American Community Survey identified the average percentage of adult residents aged 25 or older in
the census tract with a Bachelor's degree or higher from 2013 to 2018. The Bachelor's degree was
considered the relevant measure for educational attainment based on previous research identifying a
large disparity in drug-related deaths between those with and without a Bachelor's degree (24).
For
decedents with missing or unmatched address information, no tract-level education information
could be collected from the American Community Survey. For those decedents, we used multiple
imputation, a Monte Carlo technique, to estimate missing education values that reflect the
uncertainty around the true value (25). We assumed education was missing at random and the
variables by which it can be predicted have a joint multivariate normal distribution (25,26). We also
evaluate the validity of this assumption (see Section 2.4). We selected the number of imputations
similar to the percentage of cases with incomplete education data based on the equation put forth in
Bodner’s conservative approach to estimating reliable point estimates, p-values, and standard errors
(27).
23
2.2.3 Outcomes
The outcome of interest was the change in the number of opioid-related deaths from
February 2020 to April 2020 following the implementation of stay-at-home orders. Our post-period
included deaths occurring on or before April 30
th
. The month of April was used as the post-period
because it was the first full calendar month of stay-at-home orders in L.A. County before
requirements for businesses to reopen were eased (28). Mobility patterns based on personal device
data suggest that adherence to stay-at-home orders in L.A. County was greatest during April 2020, as
demonstrated by the lowest average mobility since the onset of COVID-19 (29). We selected a
symmetric pre-period of one month. We did not include March 2020 data to account for differential
time to implement and adhere to stay-at-home orders within the county. For example, many persons
at risk of opioid overdose would not have a depleted supply of opioids or medication-assisted
treatments within the first few weeks of stay-at-home orders. Secondary outcomes include the
aggregate number of opioid-related deaths in January through April of 2019 compared to the same
months in 2020, and the change in the number of deaths by race and ethnicity from February 2019
to April 2019 to examine seasonality.
2.2.4 Statistical Analysis
Our primary model is the following logistic regression:
𝑌 = 𝛽 0
+ 𝛽 1
𝑋 1
+ 𝛽 2
𝑋 2
+ 𝛽 3
𝑋 3
+ 𝛽 4
𝑋 4
+ 𝘀
where Y is dichotomous for date of opioid-related death (April versus February), X
1
is continuous
for age, X
2
is sex, X
3
is the presence (or absence) of an illicit opioid in the death record, X
4
is
categorical for race and ethnicity, and 𝘀 is the model residual term. Expanded models include main
and interaction effects for the percentage of adults with a Bachelor’s degree in that census tract
(continuous) and the presence (or absence) of a prescription for a scheduled drug from a prescriber
in L.A. County in the year before the date of death. To validate the assumption that address data was
24
missing at random and educational status could be imputed without bias, we ran a logistic regression
with date of opioid-related death (April 2020 versus February 2020) as the outcome and a
dichotomous variable for missing address data as the independent variable. We conducted our
analyses and estimated risk ratios using Stata and its margins, mimrgns, and nlcom commands (30,31).
2.3 Results
More opioid-related deaths per month occurred in 2020 compared to the same period in
2019 (Figure 2.1). In total, there were 103 opioid-related deaths in February and April of 2020.
Figure 2.1: Unadjusted Number of Opioid-Related Deaths in 2019 and 2020. Blue bars
represent the number of opioid-related deaths in that month in 2019. Green bars represent the
number of opioid-related deaths in that month in 2020.
The decedents were mostly male (81.6%) and non-Hispanic White (48.5%). Illicit opioid use (i.e.,
heroin or fentanyl) was the cause of death for most of the sample (95.1%). There was no missing
information regarding the decedent’s date of death, age, gender, race, ethnicity, and the presence of
fentanyl or heroin at the time of death. Due to approximately 33% of decedents having missing or
29
23
36
29
31
49 49
54
0
10
20
30
40
50
60
January February March April
Opioid-Related Deaths
Month
2019 2020
25
unmatched address data, multiple imputation with 29 imputations estimated educational attainment
(27). Missing address data was not significantly associated with death in the pre- versus post-period,
consistent with data being missing at random (P > 0.1). Across included census tracts, the mean
prevalence of having a Bachelor's degree or higher among individuals 25 years or older was
approximately 11% lower in the sample than the average in L.A. County (20.9% and 31.8%,
respectively) (32). Sample characteristics compared to L.A. County population characteristics based
on ACS 2018 data are shown in Table 2.1. The sample of decedents with opioid-related deaths
consists of more males, non-Hispanic White individuals, and individuals residing in areas where a
smaller share of the adult population has a Bachelor's degree or higher compared to the L.A. County
population.
Table 2.1: Sample Characteristics
Sample
N=103
Los Angeles
County (2018)
Age, mean (SD) 36.0 (13.1) Median = 36.2
Male, No. (%) 84 (81.6%) 49.3%
Race and Ethnicity, No. (%)
American Indian 1 (1.0%) 1.6%
Asian 3 (2.9%) 16.2%
Black 14 (13.6%) 9.3%
Hispanic or Latin
American Ethnicity
33 (32.0%) 48.5%
Non-Hispanic White 50 (48.5%) 26.3%
Unknown 2 (1.9%)
Bachelor’s degree or higher,
mean (SD)
20.9% (11.7) 31.8%
Heroin or Fentanyl, No. (%) 98 (95.1%)
26
There was a statistically insignificant change in the probability of an opioid-related death in
the post-period compared to the pre-period (probability difference: 4.85 percentage points [95% CI,
-4.76 to 14.3]; P > 0.1). Our main model results demonstrate that decedents were 82% more likely to
be non-Hispanic White than Hispanic in the period following the implementation of stay-at-home
orders (risk ratio (RR): 1.82 [95% CI, 1.10 to 3.02]; P < 0.05), adjusting for age, gender, and
heroin/fentanyl. These two ethnic groups had the largest share of deaths in both periods and were
the only groups with significant differences between them. A histogram of opioid-related deaths by
time period and race and ethnicity shown in Figure 2.2.
Figure 2.2: Opioid-related deaths by time period and race and ethnicity. Blue bars represent
the number of opioid-related deaths in that period among individuals of American Indian, Asian,
Black, Hispanic, or Unknown race and ethnicity. Green bars represent the number of opioid-related
deaths in that period among non-Hispanic White individuals.
Additional models controlled for education and prescription drug access. Tract-level
education was not statistically significant (RR for a 10% increase in the mean percentage of adults
30
23
19
31
0
5
10
15
20
25
30
35
Pre-Stay-at-Home Orders Post-Stay-at-Home Orders
Opioid-Related Deaths
Time Period
American Indian, Asian, Black, Hispanic, or Unknown Non-Hispanic White
27
with at least a Bachelor’s degree: 1.00 [95% CI, 0.80 to 1.24]; P > 0.1) and did not modify the effect
of race and ethnicity on death in the post-period (RR of non-Hispanic White individuals compared
to Hispanic individuals: 1.82 [95% CI, 1.06 to 3.09]; P < 0.05), adjusting for age, gender, and
heroin/fentanyl. A prescription from a L.A. County prescriber for a scheduled drug in the year
before death, when adjusting for age, gender, heroin/fentanyl, and education, was not statistically
significant (RR: 0.71 [95% CI, 0.45 to 1.11]; P > 0.1) and increased the association of race and
ethnicity with death in the post-period (RR for non-Hispanic White individuals compared to
Hispanic individuals: 1.98 [95% CI, 1.15 to 3.41]; P < 0.05). Models controlling for the interactions
between education and race and ethnicity as well as between prescription drug access and race and
ethnicity resulted in statistically insignificant coefficients, indicating that the impact of race and
ethnicity on opioid-related deaths in the post-period is not dependent on tract-level education or
prescription scheduled drug access. Regression results are displayed in Table 2.2.
Opioid overdose deaths in April 2020 were 63% higher than in February 2020 among non-
Hispanic White individuals and 23% lower among all other racial and ethnic groups. In April 2019,
there was a 46% increase in opioid-related deaths among non-Hispanic White individuals and a 0%
change in opioid-related deaths among all other racial and ethnic groups compared to February
2019. However, we ran the main model on data from February and April of 2019 and did not find
significant differences in deaths by race and ethnicity (RR of non-Hispanic White individuals
compared to Hispanic individuals: 1.21 [95% CI, 0.64 to 2.26]; P > 0.1; RR of Black individuals
compared to Hispanic individuals: 1.33 [95% CI, 0.65 to 2.72]; P > 0.1).
28
Table 2.2: Model Results
VARIABLES Primary Model
Expanded Model with
Education
Expanded Model with
Education & Scheduled
Drug Prescription
Risk Ratio 95% CI Risk Ratio 95% CI Risk Ratio 95% CI
Age (5 year increase in
mean age)
1.00 (0.93, 1.08) 1.00 (0.93, 1.08) 1.01 (0.94, 1.08)
Male 1.09 (0.62, 1.92) 1.10 (0.62, 1.94) 1.11 (0.63, 1.95)
Race and Ethnicity
(Hispanic or Latin
American Ethnicity as
reference group)
American Indian - - - - - -
Asian - - - - - -
Black 1.17 (0.53, 2.60) 1.18 (0.53, 2.66) 1.25 (0.56, 2.81)
Non-Hispanic White 1.82** (1.10, 3.02) 1.82** (1.07, 3.12) 1.98** (1.15, 3.41)
Unknown 1.37 (0.31, 6.04) 1.37 (0.31, 6.00) 1.56 (0.39, 6.20)
Heroin or Fentanyl 2.75 (0.42, 17.84) 2.66 (0.40, 17.48) 2.58 (0.41, 16.34)
10% increase in mean %
of adults in census tract
with Bachelor’s degree
or higher
- - 1.00 (0.80, 1.24) 1.01 (0.81, 1.24)
Scheduled drug
prescription in the year
prior to death
- - - - 0.71 (0.45, 1.11)
Intercept
+
0.09* (0.01, 1.50) 0.09 (0.00, 1.96) 0.09 (0.00, 1.97)
+
intercepts are displayed as baseline odds, *** p<0.01, ** p<0.05, * p<0.1
2.4 Discussion
There has been a concerning rise in opioid-related deaths from 2019 to 2020. Our analysis
highlights that opioid-related deaths in L.A. County remain concentrated among individuals residing
in areas where a smaller percentage of the population has a Bachelor’s degree or higher. These
findings mirror previous findings that the most significant increase in mortality from suicide and
poisonings is concentrated among individuals with less than a Bachelor’s degree (20).
29
Our analysis identifies the most significant risk of opioid-related death following stay-at-
home orders among non-Hispanic White individuals compared to Hispanic individuals. Past studies
in the U.S. have identified increased morbidity and decreased mortality among non-Hispanic White
individuals aged 45-54 years old, driven predominantly by the increase in pain and opioid use (20).
The rise in mortality rate among middle-aged non-Hispanic White individuals from self-inflicted
deaths from guns, drug addiction, or alcohol are poignantly termed “deaths of despair”. Key drivers
to deaths of despair among white working-class individuals in the last 50 years are external forces,
such as declining wages, deterioration in job quality, and declining participation in organized religion,
community groups, and unions (24). Contrasting outcomes for other racial and ethnic groups may
have to do with stronger community ties and social supports that arose out of generations of
impoverished material conditions. Our findings suggest that COVID-19 has further exacerbated one
type of deaths of despair, fatal opioid overdoses, among non-Hispanic White individuals. The
underlying reasons for this were not tract-level education or prescription drug access. Black, Asian,
and Hispanic individuals experienced higher unemployment prior to the onset of COVID-19 (33).
Perhaps non-Hispanic White individuals faced a greater change in the unemployment rate or
responded more negatively to this change early in the pandemic. However, while the unemployment
rate among minority groups was higher than for non-Hispanic White groups in California before the
pandemic, the unemployment rate increased least among non-Hispanic White individuals from
quarter 1 to quarter 2 of 2020 compared to Black, Hispanic, and Asian individuals (10.1%, 12.3%,
12.7%, and 11.2% respectively) (33). Therefore, it could be the coping response to this change to
professional and social structures that is driving the difference in opioid-related mortality by race
and ethnicity.
Alternatively, the increased risk of overdose among non-Hispanic White individuals
following the implementation of stay-at-home orders may reflect a decrease in access to face-to-face
30
treatment, medications for addiction treatment, and formal support networks, such as Narcotics
Anonymous. Narcotics Anonymous membership is predominantly non-Hispanic White and
medications for addiction treatment are provided less frequently to racial minority groups (34–36).
Medication-assisted treatment for opioid use disorder may require daily clinic-based administration
of methadone or 30-day appointments or refills of buprenorphine which could be disrupted during
COVID-19 (34,37). Individual-level data regarding opioid use disorder treatment linked with
outcomes data would be necessary to evaluate this theorized decrease in medicine or support
networks and increased relapse. The decline in face-to-face encounters with medical providers may
have contributed to the decrease in opioid prescribing rates. A reduction in the supply of
prescription opioids may have led to diversion to non-prescription opioids. Since COVID-19
became a state of emergency, illicit fentanyl use has increased (3). Additionally, the lack of significant
opioid-related death increases among individuals of American Indian, Asian, Black, Hispanic, or
Unknown race and ethnicity could be explained by their higher rates of COVID-19-related mortality
(17). All-cause mortality data for L.A. County would be needed to explore this competing risk
hypothesis.
In our analysis of 2019 data, we do not find evidence of significantly more opioid-related
deaths among non-Hispanic White individuals compared to Hispanic individuals in April versus
February. The lack of seasonality in opioid-related deaths based on race and ethnicity in 2019
validates that our findings are specific to the time periods before and after stay-at-home orders in
2020.
A limitation of this study is its lack of decedent-level education data. While neighborhood
educational achievement predicts an individual’s educational status, previous research has identified
that an individual’s school, family socio-economic status, and parenting variables are also important
factors (38). Additionally, educational attainment at the tract-level is given in 5-year estimates, which
31
may not capture dynamic changes to education. Yet, one advantage of the 5-year estimates is its
increased stability and reliability for less populated census tracts (39). Nevertheless, our analysis of 5-
year estimates as point estimates does not take the distribution of each tract-level estimate into
account. Another limitation is that this study was limited to drug overdoses involving opioids, so any
shifts in all drug class utilization or overdose by race and ethnicity could not be measured. The
generalizability of these findings may be limited to Los Angeles County. However, L.A. County is a
diverse area of approximately 3% of the US population (39). Further analyses in other counties and
in areas where the peak of the COVID-19 outbreak coincided with policy implementation would
inform the generalizability of these results.
2.5 Conclusions
There has been a rise in opioid-related deaths in L.A. County during 2020 compared to 2019.
Stay-at-home orders were associated with increased opioid-related deaths among non-Hispanic
White individuals. One possibility is that changes to social and physical environments during
lockdown led to increased opioid overdoses among non-Hispanic White individuals. Further
research on this trend's underlying causes is needed to inform clinical and policy recommendations
during these simultaneous public health crises.
32
Acknowledgements
All study procedures were approved by the University of Southern California Institutional Review
Board (UP-19-00172). M.A. Kelley was a legal consultant for EconoMedRx. J.N. Doctor is an
independent consultant for Motley Rice LLC. D. Goldman reports research support from the
National Institutes of Health and Blue Cross Blue Shield of Arizona, Bristol Myers Squibb, Cedars-
Sinai Health System, Edwards Lifesciences, Gates Ventures, Genentech, Gilead Sciences, Johnson &
Johnson, Kaiser Family Foundation, Novartis, Pfizer, and Roche. He serves as a paid scientific
advisor to Biogen, GRAIL, and Precision Medicine Group.
Funding: The National Institute on Aging (NIA) at the National Institutes of Health
(R33AG057395) and NIA Roybal Center for Behavioral interventions (5P30AG024968-18) support
this work.
33
References
1. Slavova S, Rock P, Bush HM, Quesinberry D, Walsh SL. Signal of increased opioid overdose
during COVID-19 from emergency medical services data. Drug Alcohol Depend.
2020;214:108176.
2. American Medical Association. Issue brief: Reports of increases in opioid- and other drug-related
overdose and other concerns during COVID pandemic [Internet]. 2020. Available from:
https://www.ama-assn.org/system/files/2020-11/issue-brief-increases-in-opioid-related-
overdose.pdf
3. Millenium Health. COVID-19 Special Edition: Significant Changes in Drug Use During the
Pandemic [Internet]. Millennium Health; 2020. Available from:
https://resource.millenniumhealth.com/signalsreportCOVID
4. Alter A, Yeager C. COVID-19 Impact on US National Overdose Crisis [Internet]. Overdose
Detection Mapping Application Program; 2020. Available from:
http://www.odmap.org/Content/docs/news/2020/ODMAP-Report-June-2020.pdf
5. Rodda LN, West KL, LeSaint KT. Opioid Overdose-Related Emergency Department Visits and
Accidental Deaths during the COVID-19 Pandemic. J Urban Health. 2020;97(6):808–13.
6. Appa A, Rodda LN, Cawley C, Zevin B, Coffin PO, Gandhi M, et al. Drug Overdose Deaths
Before and After Shelter-in-Place Orders During the COVID-19 Pandemic in San Francisco.
JAMA Netw Open. 2021 May 3;4(5):e2110452.
7. Mason M, Welch SB, Arunkumar P, Post LA, Feinglass JM. Notes from the Field: Opioid
Overdose Deaths Before, During, and After an 11-Week COVID-19 Stay-at-Home Order - Cook
County, Illinois, January 1, 2018-October 6, 2020. MMWR Morb Mortal Wkly Rep.
2021;70(10):362–3.
8. U.S. Centers for Disease Control and Prevention. Suspected Nonfatal Drug Overdoses during
COVID-19 [Internet]. 2021 [cited 2021 Oct 22]. Available from:
https://www.cdc.gov/drugoverdose/nonfatal/states/covid-19.html
9. Friedman J, Akre S. COVID-19 and the Drug Overdose Crisis: Uncovering the Deadliest Months
in the United States, January ‒July 2020. Am J Public Health. 2021 Jul;111(7):1284–91.
10. Ahmad FB, Rossen LM, Sutton P. Provisional drug overdose death counts [Internet]. National
Center for Health Statistics; 2021 [cited 2021 Jul 19]. Available from:
https://www.cdc.gov/nchs/nvss/vsrr/drug-overdose-data.htm
11. Khatri UG, Perrone J. Opioid Use Disorder and COVID-19: Crashing of the Crises. J Addict
Med. 2020;14(4):e6–7.
12. Dong E, Du H, Gardner L. An interactive web-based dashboard to track COVID-19 in real time.
Lancet Infect Dis. 2020;20(5):533–4.
13. State of California. EXECUTIVE ORDER N-33-20 [Internet]. Department, Executive; 2020.
Available from: https://covid19.ca.gov/img/Executive-Order-N-33-20.pdf
34
14. County of Los Angeles. Opioid Abuse Prevention and Treatment in Los Angeles County
[Internet]. Health, Department of Public; 2018. Available from:
http://publichealth.lacounty.gov/sapc/MDU/DE/OpioidBriefFactSheet.pdf
15. Shover CL, Falasinnu TO, Dwyer CL, Santos NB, Cunningham NJ, Freedman RB, et al. Steep
increases in fentanyl-related mortality west of the Mississippi River: Recent evidence from county
and state surveillance. Drug Alcohol Depend. 2020 Nov;216:108314.
16. Artiga S, Garfield R, Orgera K. Communities of color at higher risk for health and economic
challenges due to COVID-19. San Francisco, CA [Internet]. 2020; Available from:
https://racism.org/images/pdf/Coronavirus/KffCommunitiesColorHigherRisk.pdf
17. U.S. Centers for Disease Control and Prevention. Risk for COVID-19 Infection, Hospitalization,
and Death By Race/Ethnicity [Internet]. 2021. Available from:
https://www.cdc.gov/coronavirus/2019-ncov/covid-data/investigations-
discovery/hospitalization-death-by-race-ethnicity.html
18. Song Z. Mortality Quadrupled Among Opioid-Driven Hospitalizations, Notably Within Lower-
Income And Disabled White Populations [Internet]. Vol. 36, Health Affairs. 2017. p. 2054–61.
Available from: http://dx.doi.org/10.1377/hlthaff.2017.0689
19. Friedman J, Kim D, Schneberk T, Bourgois P, Shin M, Celious A, et al. Assessment of
Racial/Ethnic and Income Disparities in the Prescription of Opioids and Other Controlled
Medications in California. JAMA Intern Med. 2019 Apr 1;179(4):469–76.
20. Case A, Deaton A. Rising morbidity and mortality in midlife among white non-Hispanic
Americans in the 21st century. Proc Natl Acad Sci U S A. 2015 Dec 8;112(49):15078–83.
21. California Department of Justice. Controlled Substance Utilization Review and Evaluation
System [Internet]. 2020 [cited 2020]. Available from: https://oag.ca.gov/cures
22. United States Census Bureau. The Importance of the American Community Survey and the 2020
Census [Internet]. 2020 [cited 2020]. Available from: https://www.census.gov/programs-
surveys/acs/about/acs-and-census.html
23. United States Census Bureau. Census Geocoder Documentation [Internet]. 2018 [cited 2020].
Available from: https://www.census.gov/programs-surveys/geography/technical-
documentation/complete-technical-documentation/census-geocoder.html
24. Case A, Deaton A, DeGruyter. Deaths of despair and the future of capitalism. Princeton, NJ:
Princeton University Press; 2020.
25. Demirtas H, Freels S, Yucel R. Plausibility of multivariate normality assumption when multiply
imputing non-Gaussian continuous outcomes: A simulation assessment. J Stat Comput Simul.
2008;78(1):69–84.
26. Bhaskaran K, Smeeth L. What is the difference between missing completely at random and
missing at random? Int J Epidemiol. 2014 Aug;43(4):1336–9.
35
27. Bodner TE. What Improves with Increased Missing Data Imputations? Structural Equation
Modeling: A Multidisciplinary Journal. 2008;15(4):651–75.
28. Kandel J. Timeline 2020: A Look Back at The Coronavirus Pandemic. 2020; Available from:
https://www.nbclosangeles.com/news/local/a-coronavirus-timeline/2334100/
29. Unacast. Unacast Social Distancing Dataset. 4 March 2021. https://www.unacast.com/data-for-
good; 2020.
30. StataCorp LLC. Stata statistical software: Release 16. College Station TX. 2019;
31. Klein D. MIMRGNS: Stata module to run margins after mi estimate. Statistical Software
Components, Boston College Department of Economics; 2014.
32. United States Census Bureau. Table DP02: Selected Social Characteristics in the United States
[Internet]. 2018. Available from:
https://data.census.gov/cedsci/table?q=ACSDP5Y2018.DP02%20Los%20Angeles%20Count
y,%20California&g=0500000US06037&tid=ACSDP5Y2018.DP02&hidePreview=true
33. Williams J. State unemployment by race and ethnicity [Internet]. 2020. (Institute, Economic
Policy, editor. Economic Indicators). Available from: https://www.epi.org/indicators/state-
unemployment-race-ethnicity/
34. Alexander GC, Stoller KB, Haffajee RL, Saloner B. An Epidemic in the Midst of a Pandemic:
Opioid Use Disorder and COVID-19. Ann Intern Med. 2020;173(1):57–8.
35. Wu LT, Zhu H, Swartz MS. Treatment utilization among persons with opioid use disorder in the
United States. Drug Alcohol Depend. 2016;169:117–27.
36. Narcotics Anonymous World Services Inc. 2018 Narcotics Anonymous Membership Survey
[Internet]. [cited 2021 Feb 16]. Available from:
https://www.na.org/admin/include/spaw2/uploads/pdf/conference/project/2301_MS_2018
_Nov19.pdf
37. Substance Abuse and Mental Health Services Administration. Federal Guidelines for Opioid
Treatment Programs. Vol. HHS Publication No. (SMA) PEP15-FEDGUIDEOTP. 2015.
38. Nieuwenhuis J, Hooimeijer P. The association between neighbourhoods and educational
achievement, a systematic review and meta-analysis. J Housing Built Environ. 2016;31(2):321–47.
39. United States Census Bureau. U.S. and World Population Clock [Internet]. 2020. Available from:
https://www.census.gov/popclock/
36
Chapter 3
Practice patterns for patients with COVID-19 symptoms receiving
telehealth versus in-person urgent care
Marcella A. Kelley, Jason N. Doctor, Wendy Mack, Daniella Meeker
37
Abstract
Background: Use of dedicated telehealth services for urgent care increased drastically in 2020.
During this time, guidelines for the outpatient management of COVID-19 were uncertain. In
comparison to facility-based services, virtual visits do not allow for comprehensive physical exams
or testing that may resolve uncertainty in diagnosis. Our objective is to explore whether telehealth
practice patterns differ from facility-based urgent care practices in the treatment of patients with
suspected, exposed, or confirmed COVID-19. We hypothesize that telehealth’s environment of
heightened uncertainty leads to a bias toward taking action over inaction, and we test this by
evaluating the frequency of inappropriate orders.
Methods: Using administrative health claims data provided by large commercial and Medicare
Advantage plans for 2019, we identified facility-based urgent care centers and dedicated telehealth
service providers. We identified encounters with suspected, exposed, or confirmed COVID-19
diagnoses at one of these settings in 2020. Descriptive statistics were used to characterize the most
common practice patterns. Multiple mixed effect logistic regression models were used to assess
variation in the use of initial interventions (i.e. emergency department (ED) referrals, inappropriate
prescriptions, and COVID-19 test orders) by telehealth versus urgent care settings and the
association of inappropriate prescriptions with hospitalizations and 30-day mortality. Inappropriate
prescriptions for COVID-19 included hydroxychloroquine, antibiotics, corticosteroids, ivermectin,
baricitinib, remdesivir, and steroids. Models controlled for quarter of 2020, patient characteristics,
state COVID-19 prevalence, and practice group random effects.
Results: 52,843 patients were identified with an index COVID-19 appointment at a dedicated
telehealth or urgent care setting. Telehealth patients were, on average, younger and healthier than
patients in urgent care. After adjusting for patient characteristics and state COVID-19 prevalence,
the probability of an ED referral was 0.03 higher in urgent care than telehealth at the onset of the
38
pandemic ([95% Confidence Interval (CI): 0.01 to 0.06]; P < 0.01) but did not differ in Quarters 2-4.
Of all of the patients who received a prescription, 37% received an inappropriate prescription for
COVID-19. The adjusted probability of an inappropriate prescription in urgent care was 0.20
higher than telehealth in quarter 1 ([95% CI: 0.14 to 0.27]; P < 0.001) and 0.07 lower than telehealth
in quarter 4 ([95% CI: -0.11 to -0.03]; P < 0.01). The adjusted probability of a test order was 0.09
lower in urgent care than telehealth in quarter 1 ([95% CI: -0.15 to -0.03]; P < 0.01) and 0.19 higher
in urgent care than telehealth in quarter 4 ([95% CI: 0.18 to 0.21]; P < 0.001). Patients who received
an inappropriate prescription at their index appointment had a 0.01 higher adjusted probability of
hospitalization within the week ([95% CI: 0.01 to 0.02]; P < 0.001) and a 0.004 higher adjusted
probability of death within 30 days ([95% CI: 0.001 to 0.006]; P < 0.01) compared to people without
an inappropriate prescription.
Conclusions: At the onset of the pandemic, inappropriate prescribing was high across both settings
and higher in facility-based urgent care settings. As the pandemic progressed, inappropriate
prescribing was higher in telehealth than urgent care, despite younger and healthier patients.
Inappropriate prescribing was associated with increased risk of hospitalization and death. Telehealth
and attendant uncertainties may lead to greater action bias to the detriment of healthcare quality.
39
3.1 Background
Clinicians often have a tendency toward action versus inaction, or "action bias", despite the
action possibly resulting in worse outcomes (1–4). The repeated bias toward action is influenced by
one’s perception of the value of the outcome as dependent on one’s involvement in causing it,
rather than the risks and benefits of the outcome (4). If the norm is action, then norm theory
suggests that an outcome caused by inaction prompts more intense regret than if the outcome is
caused by action (5,6). The tendency toward action over inaction when faced with diagnostic
uncertainty has been evidenced among primary care clinicians but has not been explored in the
context of COVID-19 (1).
During the COVID-19 pandemic, the delivery of outpatient medical care changed drastically
(7). COVID-19 and its related policies, such as social distancing and lockdown measures,
necessitated an increase in the use of telehealth services and a decrease in in-person encounters.
Telehealth visits accounted for two-thirds of the decrease in outpatient visits at brick-and-mortar
clinics among a privately insured population (7). Telehealth is offered in many forms, including
scheduled visits with the patient’s regular clinician or on-demand visits with a dedicated telehealth
clinician with whom the patient does not have a relationship (8). Visits with a dedicated telehealth
clinician are initiated by the patient, conducted via video or telephone on the patient’s devices, and
held around-the-clock (8,9). While telehealth may increase access to care and provide greater patient
and clinician flexibility, concerns have been raised regarding the quality of care, limited physical
examinations, and brief patient-clinician relationships (10,11).
In addition to setting changes, the diagnosis and treatment of respiratory infections also
changed greatly throughout the COVID-19 pandemic. In March 2020, hydroxychloroquine and
chloroquine received national attention and Food and Drug Administration (FDA) emergency use
authorization (EUA) for treating hospitalized COVID-19 patients (12,13). These announcements
40
were followed by a 7.2-fold increase in new outpatient prescriptions of hydroxychloroquine and
chloroquine in March 2020 compared to March 2019 (13). By mid-June, the FDA rescinded its
emergency use authorization and the federal COVID-19 Treatment Guidelines Panel issued two
recommendations against the use of hydroxychloroquine or chloroquine for COVID-19 based on
the risks outweighing the benefits (14,15). As of September 2020, there was no evidence-based drug
recommendation for outpatient COVID-19 treatment but numerous drugs were under exploration
(16). By November 2020, the FDA approved the first COVID-19 treatment, Veklury (remdesivir),
for hospitalized COVID-19 patients and granted emergency use authorization to a monoclonal
antibody product, Regen-Cov (casirivimab plus imdevimab), for early outpatient treatment (when
variant-appropriate) (17–19). In 2021 and thus far in 2022, two antiviral medications Paxlovid
(nirmatrelvir tablets; ritonavir tablets) and Lagevrio (molnupiravir) have received FDA emergency
use authorization for outpatient treatment, along with four additional monoclonal antibody products
(Bamlanivimab plus etesevimab when variant-appropriate, Sotrovimab, Tocilizumab, and
Bebtelovimab) (20–22). Additionally, the FDA approval of Veklury has since been expanded to
include outpatients (23). As of April 2022, outpatient COVID-19 treatment guidelines recommend
medications for symptom management, Paxlovid and Veklury for mild to moderate COVID-19
patients, and Bebtelovimab or Lagevrio when Paxlovid and Veklury are not feasible (24). Outpatient
guidelines recommend against systemic glucocorticoids, antibiotics, chloroquine or
hydroxychloroquine with or without azithromycin, lopinavir/ritonavir, and other HIV protease
inhibitors (24).
In 2020, there were great strides in understanding COVID-19, treating hospitalized patients,
and developing and administering vaccines, but the outpatient treatment of COVID-19 consisted of
drug repurposing, often without robust evidence (25). While studies have characterized the variation
in outpatient utilization during 2020 nationally and by region, few have evaluated the variation in
41
outpatient treatment of COVID-19 patients (7,26). Appointments at urgent care centers, like
dedicated telehealth companies, offer immediate care for acute, non-emergent conditions and are
often with a clinician whom the patient does not know (8,9,27,28). The lack of clinician-patient
relationship in these settings may increase action bias in the face of uncertainty since the harms of
the action may not be observable to the provider. Alternatively, urgent care and telehealth providers
may exhibit less action bias than primary care physicians because of the lack of a close relationship
and less perceived pressure to diagnose. With increased numbers of patients turning to telehealth for
treatment in 2020, it is critical to understand if and how the treatment delivered in telehealth differed
from in-person urgent care, particularly regarding inappropriate prescribing and its associated
outcomes. We sought to explore whether telehealth COVID-19 practice patterns differ from in-
person urgent care and if inappropriate prescriptions are associated with increased hospitalizations
and deaths.
3.2 Methods
3.2.1 Data Sources
We used Optum administrative medical and pharmacy claims data for years 2018-2020 from
the De-Identified Clinformatics® Data Mart (CDM) for this analysis (Optum’s de-identified
Clinformatics® Data Mart Database (2007-2020)). The Optum CDM was derived from a database of
administrative health claims for members of large commercial and Medicare Advantage health plans.
The Optum CDM contained de-identified patient-level information on inpatient and outpatient
medical claims including International Classification of Diseases, Tenth Revision, Clinical
Modification (ICD-10-CM) diagnosis and procedure codes, Current Procedural Terminology and
Healthcare Common Procedure Coding System procedure codes, pharmacy claims, member
enrollment, and date of death for commercially insured individuals in the United States. The database
42
included approximately 17-19 million annual covered lives. Data was longitudinal and records were
linked via unique anonymized patient identifiers that are Health Insurance Portability and
Accountability Act (HIPAA) compliant (Optum’s de-identified Clinformatics® Data Mart Database
(2007-2020)).
State-level COVID-19 case data came from the John Hopkins COVID-19 Unified Data Set
(29). Antibiotics were identified from the National Committee for Quality Assurance value sets for
appropriate treatment of upper respiratory infections and monoclonal antibody products included
those from CMS (30,31). The appropriateness of antibiotics and steroids was determined from the
MEDication Indication resource (MEDI) and clinical input (32).
3.2.2 Sample Selection
Our sample included patients with COVID-19-like illness diagnoses during a visit in a
dedicated telehealth setting or an in-person urgent care center in 2020. Telehealth settings and
urgent care centers were classified based on the clinician group’s proportion of all outpatient visits
that were in telehealth or urgent care settings in 2019 (Appendix Table A3.1). Appointment setting
(telehealth or urgent care) was defined using place of service, CPT codes or modifiers, HCPCS
codes, revenue codes, or type of service codes. Clinician groups in the 90th percentile of the
proportion of urgent care (telehealth) visits were included as urgent care centers (dedicated
telehealth groups). Outpatients with an exposed, suspected, or confirmed COVID-19 diagnosis in
one of these settings in 2020 were included in our sample. COVID-like illness diagnosis codes were
adapted from CDC guidance and adopted by a large dedicated telehealth company (Appendix
Table A3.2) (33,34). Patient inclusion criteria included an insurance enrollment window overlapping
the index date and one state of residence listed on the index date. Patients with unknown state or
sex were excluded from the sample.
43
3.2.3 Measures
Patient-level variables included age, sex, race and ethnicity (Asian, Black, Hispanic, White,
Unknown), Charlson Comorbidity Index (CCI), insurance type (Commercial or Medicare), index
COVID-19 diagnosis (suspected/exposed or confirmed), and COVID-19 prevalence in the state.
The patient’s Charlson Comorbidity Index was calculated based on diagnosis codes in the 2 years
prior to the index appointment. The COVID-19 prevalence was measured as a 7-day moving
average of the number of cases per 100,000 in that state. Appointment-level variables included
setting (i.e. telehealth or urgent care center) and quarter. Random effects for each practice group in
our analysis captured clinic-level variation.
3.2.4 Outcomes
Our outcomes were dichotomous variables for the clinician's initial intervention (i.e. ED
referral, inappropriate prescription, and COVID-19 test order) at the index COVID-19-related visit,
hospitalization within 1 week of the index visit, and 30-day mortality. Initial clinician interventions
were based on the most common evaluation and management actions at or following an index
COVID-19-related appointment. We did not include treatments administered after an ED referral
on the index date or a hospital admission. We did not include hospital-based management strategies.
Hospitalizations were identified based on place of service codes on claims within 0-7 days of the
index appointment.
Prescriptions written within 3 days of the index COVID-19-related appointment were
included. Inappropriate prescriptions based on the COVID-19 Treatment Guidelines Panel
recommendations included hydroxychloroquine, antibiotics, corticosteroids, ivermectin, baricitinib,
remdesivir, and steroids (35). All drugs were identified based on their National Drug Code, generic
name, brand name, American Hospital Formulary Service Classification, or First DataBank
classification. If the index COVID-19-related visit included diagnosis codes for which antibiotics are
44
sometimes or always appropriate, then an antibiotic prescription was not classified as inappropriate.
The same exclusion criterion was used for diagnosis codes that are appropriate for steroid treatment.
The appropriateness of antibiotics and steroids are listed in Appendix Tables A3.3 and A3.4 and
were determined from MEDI and clinical input (32).
COVID-19 tests were attributed to urgent care providers if they occurred within the same
claim as the index appointment. Since tests cannot be administered directly via telehealth, the
median time between the index appointment and test within the same claim in urgent care (0 days)
was used to attribute test orders to telehealth clinicians. Any COVID-19 tests on the same day as the
index appointment with the telehealth clinician were attributed to that clinician. COVID-19 tests
were considered positive if the patient had a U07.1 diagnosis 0-7 days after the test and presumed
negative if the patient did not.
3.2.5 Statistical Analysis
We characterized outpatient treatment patterns for patients with exposed, suspected, or
confirmed COVID-19 and their variation over time and across practitioners in the Optum dataset.
We used 2-sided chi-squared tests to compare the distribution of all follow-up actions and
demographic characteristics between patients seen in telehealth and urgent care. In three mixed
effect logistic regression models, we assessed the likelihood of different COVID-19 interventions.
We controlled for a vector of patient characteristics, the interaction of setting and time (quarter of
2020), and included a random effect for clinician group:
𝑙𝑜𝑔 (𝑌 𝑖𝑗𝑘
) = 𝛽 0
+ 𝛽 1
𝑋 1𝑖𝑗
+ 𝛽 2
𝑋 2𝑖𝑗𝑘
+ 𝛽 3
𝑋 3𝑖𝑗𝑘
+ 𝛽 𝑛 𝑋 𝑗𝑛
+ 𝛿 𝑖 + 𝘀 𝑖𝑗𝑘
[1]
where Y represented a dichotomous outcome for a COVID-19 evaluation and management action
by clinician i for patient j at the index appointment in quarter k. Dichotomous outcomes included a
same day ED referral, an inappropriate prescription, and a COVID-19 test order for a patient with
suspected or exposed COVID-19. X
1
was dichotomous for appointment setting (telehealth or
45
urgent care), X
2
was categorical for the quarter of 2020, X
3
is the interaction of setting and quarter,
and X
n
represented a vector of patient characteristics (including age, sex, race and ethnicity,
insurance type, Charlson comorbidity index, 7-day moving average of COVID-19 prevalence in the
patient’s state), 𝛿 represented the random effect for practice group i, and 𝘀
represented the model
residual term. The setting-quarter interaction term tested if the telehealth or urgent care association
with treatment actions changed over the 2020 follow-up period. In the model with test orders as the
outcome, the sample was limited to patients without confirmed COVID-19 diagnoses. To analyze if
changes in test availability contributed to changes in inappropriate prescribing, we conducted a
sensitivity analysis with inappropriate prescriptions as the outcome and test order as an additional
independent variable in Equation 1.
Using two mixed effect logistic regression models, we assessed the association of an
inappropriate prescription with an inpatient admission and 30-day mortality. We controlled for the
presence of an inappropriate prescription, a vector of patient characteristics, setting, quarter, and
included a random effect for the practice group:
𝑙𝑜𝑔 (𝜋 𝑖𝑗𝑘
) = 𝛽 0
+ 𝛽 1
𝑋 1𝑖𝑗
+ 𝛽 2
𝑋 2𝑖𝑗𝑘
+ 𝛽 3
𝑋 3𝑖𝑗
+ 𝛽 𝑛 𝑋 𝑗𝑛
+ 𝛿 𝑖 + 𝘀 𝑖𝑗𝑘
[2]
where 𝜋 was dichotomous for a hospitalization within 7 days of the index appointment and for a
death within 30 days of the index appointment for patient j at the index appointment with clinician i
in quarter k. As in Equation 1, X
1
was dichotomous for appointment setting (telehealth versus
urgent care), X
2
was categorical for the quarter of 2020, X
n
represented a vector of patient
characteristics (including age, sex, race, insurance type, Charlson comorbidity index, COVID-19 7-
day moving average prevalence in the patient’s state), 𝛿 represented the random effect for practice
group i, and 𝘀
represented the model residual term. X
3
was dichotomous for an inappropriate
prescription. Model results were presented as odds ratios and predicted probabilities based on
observed covariates. Analyses were performed using melogit in Stata version 16 (36).
46
3.3 Results
We identified 829 urgent care centers and 67 dedicated telehealth groups based on the 90th
percentile cutoff. 52,843 patients had an index COVID-19-related appointment at one of these
centers in 2020 and met our inclusion criteria. In our sample, the first case of COVID-19 occurred
on March 23, 2020. Prior to that date, 1,278 encounters with suspected or exposed COVID-19
appeared. The sample was predominantly female (56%), White (51%), residing in the South (48%),
diagnosed with suspected or exposed COVID-19 (93%), with few comorbidities (CCI=1.4), and
with a mean age of 55 (Table 3.1). In the overall sample, the most common initial clinician
interventions were ordering a COVID-19 test (23%), writing a prescription within 3 days (20%), and
referring the patient to the emergency department (1.6%). Across settings, patients significantly
differed in their race and ethnicity, sex, enrollment length, geographic region, Charlson Comorbidity
Index, and insurance type (P < 0.001). On average, patients seen in telehealth were younger,
predominantly male, less likely White, enrolled for a shorter period, had fewer comorbidities, and
were commercially insured.
In the overall sample, 2% of patients had an emergency department or inpatient referral on
their index appointment date, 20% received a prescription within 3 days, 60% had a COVID-19 test
within the following week (55% tested and presumed negative, 5% tested and presumed positive),
35% had a follow-up visit within the following week, 2% were hospitalized within the following
week, and less than 1% died within 30 days (Figure 3.1). More patients in urgent care than
telehealth received a COVID-19 test (P <0.001) or were hospitalized within 0-7 days of their index
appointment (P <0.001). More patients in telehealth than urgent care were sent to the emergency
department on the date of their index appointment (P <0.001), received a prescription within 3 days
of the index appointment (P <0.001), or had another visit in the following week (P <0.001).
47
Table 3.1. Sample characteristics by setting
Total Urgent Care Telehealth p-value
N=52,843 N=37,122 N=15,721
Age, mean (SD) 54.9 (20.8) 62.8 (18.6) 36.3 (12.2) <0.001
Sex, % (N) <0.001
Female 55.5% (29,327) 57.4% (21,314) 51.0% (8,013)
Male 44.5% (23,516) 42.6% (15,808) 49.0% (7,708)
Race Code, % (N) <0.001
Asian 3.2% (1,669) 3.0% (1,100) 3.6% (569)
Black 7.1% (3,754) 7.1% (2,633) 7.1% (1,121)
Hispanic 9.8% (5,177) 8.2% (3,039) 13.6% (2,138)
Unknown 28.9% (15,255) 28.7% (10,643) 29.3% (4,612)
White 51.1% (26,988) 53.1% (19,707) 46.3% (7,281)
Enrollment Length in Days,
mean (SD)
1316.5 (1125.8) 1426.7 (1131.3) 1056.5 (1068.5) <0.001
Geographic Region, % (N) <0.001
Midwest 15.0% (7,945) 13.4% (4,971) 18.9% (2,974)
Northeast 23.2% (12,235) 29.1% (10,803) 9.1% (1,432)
South 48.4% (25,580) 48.9% (18,148) 47.3% (7,432)
West 13.4% (7,083) 8.6% (3,200) 24.7% (3,883)
Charlson Comorbidity Index,
mean (SD)
1.4 (2.2) 1.8 (2.5) 0.3 (0.9) <0.001
ED Referral on Index Date, %
(N)
1.6% (871) 1.4% (508) 2.3% (363) <0.001
Inpatient Admission on Index
Date, % (N)
0.5% (285) 0.6% (230) 0.3% (55) <0.001
Index Date Diagnosis, % (N) <0.001
Suspected or Exposed COVID-19 92.5% (48,862) 94.5% (35,091) 87.6% (13,771)
Confirmed COVID-19 7.5% (3,981) 5.5% (2,031) 12.4% (1,950)
Insurance, % (N) <0.001
Commercial 46.4% (24,523) 24.1% (8,935) 99.2% (15,588)
Medicare with low-income
subsidy/Medicaid
7.3% (3,853) 10.3% (3,817) 0.2% (36)
Medicare 46.3% (24,467) 65.6% (24,370) 0.6% (97)
Received COVID-19 Test on
Index Date or in the following 7
Days, % (N)
60.6% (32,030) 72.0% (26,735) 33.7% (5,295) <0.001
Follow-up Appointment within 7
Days, % (N)
34.5% (18,231) 32.3% (11,976) 39.8% (6,255) <0.001
Hospitalization within 7 Days, %
(N)
2.3% (1,229) 2.8% (1,052) 1.1% (177) <0.001
30-day Mortality, % (N) 0.1% (61) 0.14% (53) 0.05% (8) <0.05
Received a Prescription within 3
Days of Index Date, % (N)
20.0% (10,590) 18.8% (6,962) 23.1% (3,628) <0.001
48
49
Based on Equation 1, the adjusted predicted probability of an ED referral decreased from
0.05 to 0.02 (-0.02 change [95% CI: -0.04 to -0.01]; P < 0.05) in urgent care and increased from .01
to .02 (0.01 change [95% CI: 0.001 to 0.02]; P < 0.05) in telehealth between quarters 1 and 4. The
probability of an ED referral was significantly higher in urgent care than telehealth (0.03 difference
[95% CI: 0.01 to 0.06]; P < 0.01) in quarter 1 but insignificantly different in the remaining quarters
as settings converged (Table 3.2; Figure 3.2A). Patients with more comorbidities and a confirmed
COVID-19 diagnosis were more likely to be referred to the ED (P < 0.001). Patients with index
appointments when cases were moderate in the area (3rd quintile of cases per 100k in state) were
less likely to be referred to the ED than patients with index appointments when cases were in the
lowest quintile (P < 0.001). There was significant practice group variation in the likelihood of
referring a patient to the ED (variance(𝛿 i
) = 2.0 [95% CI: 1.3 to 3.0]).
Table 3.2: Adjusted odds ratio of a same day ED escalation, inappropriate prescription, or
test order
ED Referral Inappropriate
Prescription
Test Order
VARIABLES Setting +
Time
Setting *
Time
Setting +
Time
Setting *
Time
Setting +
Time
Setting *
Time
OR (SE) OR (SE) OR (SE) OR (SE) OR (SE) OR (SE)
Telehealth 1.54 0.47 2.35*** 0.53*** 4.00*** 45.82***
(0.61) (0.23) (0.43) (0.12) (0.34) (26.90)
Quarter 2 1.22 0.49*** 0.38*** 0.13*** 7.01*** 31.95***
(0.21) (0.13) (0.03) (0.02) (1.20) (17.92)
Quarter 3 1.53** 0.58* 0.45*** 0.15*** 27.39*** 109.55***
(0.33) (0.17) (0.05) (0.02) (5.01) (61.81)
Quarter 4 1.29 0.54* 0.39*** 0.12*** 72.63*** 538.27***
(0.33) (0.17) (0.05) (0.02) (13.72) (305.09)
Telehealth#Quarter 2 3.82*** 4.44*** 0.20***
(1.33) (0.72) (0.12)
Telehealth#Quarter 3 4.02*** 4.48*** 0.15***
(1.40) (0.71) (0.09)
Telehealth#Quarter 4 3.12*** 5.54*** 0.02***
(1.07) (0.85) (0.01)
Reference Category: Age < 20
Age, 20 - 34 1.15 1.14 1.04 1.06 0.99 0.95
(0.30) (0.30) (0.12) (0.12) (0.09) (0.08)
Age, 35 - 49 1.40 1.41 1.81*** 1.86*** 0.90 0.89
50
(0.37) (0.37) (0.21) (0.21) (0.08) (0.08)
Age, 50 - 64 1.15 1.16 2.21*** 2.26*** 0.82** 0.82**
(0.31) (0.32) (0.26) (0.27) (0.08) (0.08)
Age, 65 - 79 1.00 1.00 1.34** 1.38** 0.91 0.89
(0.30) (0.30) (0.18) (0.19) (0.10) (0.10)
Age, 80 - 90 1.45 1.46 1.47*** 1.50*** 0.81* 0.82
(0.46) (0.46) (0.22) (0.22) (0.10) (0.10)
Male 0.93 0.93 0.70*** 0.70*** 0.99 0.98
(0.07) (0.07) (0.02) (0.02) (0.03) (0.03)
Reference Category: White
Asian 0.70 0.70 0.72*** 0.72*** 0.95 1.05
(0.17) (0.17) (0.09) (0.09) (0.08) (0.09)
Black 1.21 1.21 0.93 0.93 0.79*** 0.80***
(0.16) (0.16) (0.06) (0.07) (0.05) (0.05)
Hispanic 1.11 1.11 1.00 1.00 0.87*** 0.88**
(0.13) (0.13) (0.06) (0.06) (0.05) (0.05)
Unknown 1.01 1.00 1.09** 1.08* 0.90*** 0.91**
(0.09) (0.08) (0.04) (0.04) (0.03) (0.03)
Charlson Comorbidity
Index
1.18*** 1.18*** 1.09*** 1.09*** 1.01 1.01*
(0.02) (0.02) (0.01) (0.01) (0.01) (0.01)
Medicare 1.58** 1.57* 1.51*** 1.51*** 7.90*** 13.59***
(0.37) (0.36) (0.15) (0.15) (0.78) (1.43)
Reference Category: Lowest
quintile of cases per 100k in
state
Quintile 2 0.73** 0.76* 0.76*** 0.78*** 0.49*** 0.74***
(0.11) (0.11) (0.06) (0.06) (0.04) (0.06)
Quintile 3 0.48*** 0.53*** 0.84* 0.89 0.26*** 0.56***
(0.08) (0.09) (0.08) (0.08) (0.02) (0.05)
Quintile 4 0.71* 0.77 1.14 1.19* 0.48*** 0.92
(0.14) (0.16) (0.12) (0.12) (0.04) (0.09)
Highest quintile of cases
per 100k in state
0.83 0.89 1.24** 1.29** 0.50*** 0.92
(0.18) (0.20) (0.14) (0.14) (0.05) (0.09)
Confirmed COVID-19
Diagnosis on Index Date
2.34*** 2.31*** 1.91*** 1.83***
(0.24) (0.23) (0.11) (0.10)
Constant 0.004*** 0.009*** 0.093*** 0.266*** 0.004*** 0.001***
(0.00) (0.00) (0.01) (0.05) (0.00) (0.00)
Observations 52,843 52,843 45,240 45,240 48,862 48,862
Number of groups 896 896 801 801 856 856
*** p<0.01, ** p<0.05, * p<0.1
Note: The test order model is limited to patients with a suspected or exposed COVID-19 diagnosis at their index
appointment. Standard errors are in parentheses.
51
Figure 3.2: Adjusted predicted probability by setting and time with 95% confidence
intervals. Probabilities are estimated using observed covariates for patients with index appointments
at dedicated telehealth groups (red) or urgent care centers (blue) in that quarter of 2020. A)
Predicted probability of inappropriate prescribing; B) Predicted probability of ED referrals; C)
Predicted probability of a test order
52
Within 3 days of their index appointment, 10,590 patients received prescriptions, 10% of
which were azithromycin and 10% of which were albuterol sulfate (Appendix Table A3.5). 23% of
patients seen in telehealth and 19% of patients seen in urgent care received prescriptions (P < 0.05).
Of the prescriptions written, 37% were inappropriate for COVID-19. Unadjusted rates of
inappropriate antibiotic and corticosteroid prescribing were significantly higher in telehealth than
urgent care (antibiotic: 5.9% versus 4.5%, respectively; 𝜒 2
= 46.8; P < 0.05; P < 0.05; steroids:
4.5% versus 2.8%, respectively; 𝜒 2
= 102.9; P < 0.05). Unadjusted trends in inappropriate
prescribing for both urgent care and telehealth peaked in inappropriate prescribing in March,
declined until June, and increased from October through December (Figure 3.3). Telehealth
inappropriate prescribing overall, and among antibiotics and steroids specifically, was lower than
urgent care from January-April, but higher in all of the following months of 2020 (Figures 3.3 and
3.4). Of the 3,912 patients who received an inappropriate prescription for COVID-19, most were
female (62%), White (51%), residing in the South (57%), diagnosed with suspected or exposed
COVID-19 (87% ), and commercially insured (49%) (Table 3.3)
53
Figure 3.3: Percent of COVID-19 patients with inappropriate prescriptions. Percent of
possible or confirmed COVID-19 patients with inappropriate prescriptions in each month of 2020.
FDA and COVID-19 Treatment Guidelines Panel milestones are included. Points are scaled to the
volume of COVID-19 patients seen in dedicated telehealth groups (red) or urgent care centers (blue)
each month.
Figure 3.4: Percent of COVID-19 patients with inappropriate antibiotic or steroid
prescriptions. Percent of possible or confirmed COVID-19 patients with inappropriate antibiotic
(solid) or corticosteroid (dashed) prescriptions at dedicated telehealth groups (red) or urgent care
centers (blue) in each month of 2020.
54
Table 3.3. Inappropriate prescription sample characteristics
Patients with
Inappropriate
Prescriptions
N=3,912
Age, mean (SD) 55.3 (19.3)
Sex, % (N)
Female 62.0% (2,424)
Male 38.0% (1,488)
Race Code, % (N)
Asian 2.1% (82)
Black 7.1% (277)
Hispanic 10.3% (403)
Unknown 29.7% (1,161)
White 50.8% (1,989)
Enrollment Length in Days,
mean (SD)
1346.8 (1173.9)
Geographic Region, % (N)
Midwest 14.3% (560)
Northeast 17.2% (671)
South 56.9% (2,225)
West 11.7% (456)
Charlson Comorbidity
Index, mean (SD)
1.6 (2.3)
Index Date Diagnosis, %
(N)
Suspected or Exposed
COVID-19
86.8% (3,397)
Confirmed COVID-19 13.2% (515)
Insurance, % (N)
Commercial 49.1% (1,920)
Medicare with low-income
subsidy/Medicaid
11.0% (429)
Medicare 40.0% (1,563)
Based on Equation 1, the adjusted predicted probability of an inappropriate prescription for
COVID-19 decreased from 0.38 to 0.09 (-0.29 change [95% CI: -0.34 to -0.24]; P < 0.001) in urgent
care and decreased from 0.17 to 0.16 (-0.01 change [95% CI: -0.04 to 0.01]; P > 0.1) in telehealth
between quarters 1 and 4. The probability of an inappropriate prescription was significantly higher in
urgent care than telehealth (0.20 difference [95% CI: 0.14 to 0.27]; P < 0.001) in quarter 1 and
significantly lower than telehealth in quarter 4 (-0.07 difference [95% CI: -0.11 to -0.03]; P < 0.01)
55
(Table 3.2; Figure 3.2B). Compared to patients less than 20 years old, patients 35 years and older
were significantly more likely to receive an inappropriate prescription (P < 0.05). Patients with more
comorbidities, with Medicare insurance coverage, in areas with the highest case prevalence
compared to the lowest, and with a COVID-19 diagnosis code were more likely to receive an
inappropriate prescription (P < 0.05). Asian patients compared to White patients, males compared
to females, and patients in areas with the second lowest quintile of cases compared to the lowest
case prevalence were less likely to receive an inappropriate prescription (P < 0.01) (Table 3.2).
There was significant practice group variability in the likelihood of writing an inappropriate
prescription (variance(𝛿 i
) = 0.4 [95% CI: 0.3 to 0.6]).
Testing rates in March were low with less than 5% of patients being tested but increased to
20% in both settings by August (Appendix Figure A3.1). From August to December, the urgent
care testing rate more than doubled to 42% and the telehealth testing rate declined to 18%. The
adjusted predicted probability of a test order among patients without a confirmed COVID-19
diagnosis increased from 0.15 to 0.55 (0.40 change [95% CI: 0.33 to 0.46]; P < 0.001) in urgent care
and increased from 0.24 to 0.36 (0.11 change [95% CI: 0.09 to 0.14]; P < 0.001) in telehealth
between quarters 1 and 4. The probability of a test order was significantly lower in urgent care than
telehealth (-0.09 difference [95% CI: -0.15 to -0.03]; P < 0.01) in quarter 1 and significantly higher
than telehealth in quarter 3 (0.01 difference [95% CI: 0.00003 to 0.03]; P < 0.05) and quarter 4 (0.19
difference [95% CI: 0.18 to 0.21]; P < 0.001) (Table 3.2; Figure 3.2C). Compared to White
patients, Black, Hispanic, and patients of unknown race were significantly less likely to have a
provider ordered test (P < 0.05). Patients aged 50-64 compared to patients under the age of 20 and
patients in areas with low to moderate case prevalence compared to patients in areas with the lowest
case prevalence were less likely to have a provider ordered test (P < 0.05). Patients with Medicare
and more comorbidities were significantly more likely to have a provider ordered test (P < 0.05).
56
Practice group variation was substantial in the likelihood of ordering a COVID-19 test (variance(𝛿 i
)
= 22.4 [95% CI: 17.6 to 28.5]). Subgroup analyses of the likelihood of a test order among patients
less than 40 years old and those 40 and older demonstrated that the increased probability of ordering
a test in telehealth in quarter 1 is concentrated among younger patients and the increased probability
of ordering a test in urgent care in quarter 4 is concentrated among patients 40 and older (Appendix
Table A3.6; Figure A3.2).
Sensitivity analyses of the association between test orders and inappropriate prescriptions
among all patients demonstrated that the likelihood of an inappropriate prescription increased with a
test order in Quarter 1 and decreased in the following quarters (P < 0.05). This pattern held after
controlling for patient and appointment characteristics. However, when we limited the analyses to
urgent care where tests were on the same claim as the visit, there was no significant association
between test orders and inappropriate prescriptions (P > 0.1) (Appendix Table A3.7).
Hospitalizations within the week and deaths within 30 days of the index appointment were
rare in the overall sample (< 3%). Based on Equation 2, patients who received an inappropriate
prescription at their index appointment had a significantly higher adjusted predicted probability of
hospitalization compared to patients without an inappropriate prescription (0.01 difference [95% CI:
0.01 to 0.02]; P < 0.001). The adjusted predicted probability of a hospitalization for patients with an
inappropriate prescription was 0.03 ([95% CI: 0.02 to 0.04]; P < 0.001) and was 0.02 for those
without ([95% CI: 0.01 to 0.02]; P < 0.001). Hospitalizations were significantly more likely among
males, patients 50 years and older compared to those less than 20 years old, those with higher
comorbidities, those with Medicare, and those with confirmed COVID-19 (P < 0.01). Individuals in
areas with moderate COVID-19 case prevalence were less likely to be hospitalized than individuals
in areas with the lowest case prevalence (P < 0.05). The adjusted predicted probability of death
within 30 days of the index appointment was significantly higher among patients who received an
57
inappropriate prescription compared to patients who did not (0.004 difference [95% CI: 0.001 to
0.006]; P < 0.01). Deaths were rare among patients with an inappropriate prescription (0.005 [95%
CI: 0.002 to 0.007]; P < 0.001) and those without (0.001 [95% CI: 0.001 to 0.001]; P < 0.001)
(Table 3.4). Deaths were significantly more likely among individuals with more comorbidities and
confirmed COVID-19 (P < 0.001). There was no interaction effect between setting and
inappropriate prescription for hospitalizations or deaths (P > 0.1).
Table 3.4: Adjusted odds ratio of a hospitalization within 7 days or death within 30 days of
index appointment
Hospitalization
30-Day Mortality
VARIABLES
Inappropriate
Rx + Setting
Inappropriate
Rx * Setting
Inappropriate
Rx + Setting
Inappropriate
Rx * Setting
OR (SE) OR (SE) OR (SE) OR (SE)
Inappropriate Prescription 1.44*** 1.35** 4.05*** 3.85***
(0.17) (0.17) (1.44) (1.41)
Telehealth 1.98* 1.82 1.78 1.35
(0.73) (0.68) (2.03) (1.79)
Inappropriate
Prescription#Telehealth
1.53 2.51
(0.46) (3.82)
Quarter 2 0.87 0.87 0.52 0.52
(0.25) (0.25) (0.39) (0.40)
Quarter 3 1.11 1.11 0.89 0.89
(0.36) (0.36) (0.51) (0.51)
Quarter 4 1.04 1.04 - -
(0.36) (0.36)
Reference Category: Age 20 - 34
Age, < 20 - - - -
Age, 35 - 49 1.39 1.36 - -
(0.33) (0.32)
Age, 50 - 64 2.44*** 2.36*** 1.83 1.64
(0.58) (0.57) (2.52) (2.25)
Age, 65 - 79 2.31*** 2.24*** 0.92 0.83
(0.61) (0.60) (1.38) (1.23)
Age, 80 - 90 3.26*** 3.16*** 5.43 4.85
(0.91) (0.88) (8.09) (7.25)
Male 1.32*** 1.32*** 1.82* 1.83*
(0.10) (0.10) (0.61) (0.61)
Reference Category: White
Asian 0.84 0.84 0.80 0.80
58
(0.26) (0.26) (0.84) (0.84)
Black 1.10 1.10 0.24 0.24
(0.17) (0.17) (0.25) (0.25)
Hispanic 1.22 1.22 0.46 0.46
(0.17) (0.17) (0.34) (0.34)
Unknown 0.93 0.93 0.60 0.60
(0.09) (0.09) (0.26) (0.26)
Charlson Comorbidity Index 1.18*** 1.18*** 1.25*** 1.26***
(0.01) (0.01) (0.05) (0.05)
Medicare 2.09*** 2.13*** 7.16 8.59
(0.54) (0.56) (9.79) (12.54)
Reference Category: Lowest
quintile of cases per 100k in state
Quintile 2 0.69* 0.69* 0.70 0.70
(0.13) (0.13) (0.46) (0.46)
Quintile 3 0.63** 0.63** 0.36 0.36
(0.13) (0.13) (0.25) (0.26)
Quintile 4 0.72 0.72 0.43 0.43
(0.16) (0.16) (0.35) (0.35)
Highest quintile of cases per
100k in state
0.60** 0.60** 0.42 0.42
(0.14) (0.14) (0.36) (0.36)
Confirmed COVID-19
Diagnosis on Index Date
4.00*** 4.01*** 7.11*** 7.21***
(0.50) (0.50) (2.92) (2.97)
Constant 0.002*** 0.002*** 0.0001*** 0.0001***
(0.00) (0.00) (0.00) (0.00)
Observations 43,347 43,347 35,297 35,297
Number of groups 773 773 713 713
*** p<0.01, ** p<0.05, * p<0.1
3.4 Discussion
Our analyses demonstrate a bias toward action (i.e. prescribing, referring, or testing) at the
onset of the COVID-19 pandemic when treating patients with possible or confirmed COVID-19 in
urgent care or telehealth. In the first quarter when uncertainty was greatest, urgent care providers
had higher unadjusted rates of referring patients to the emergency department, prescribing an
inappropriate treatment for COVID-19, and ordering tests. In adjusted models, inappropriate
prescribing and ED referrals were highest in the first quarter and decreased in the following
59
quarters, while testing increased over time in both settings. The speed and magnitude of changes in
treating COVID-19 varied between urgent care and telehealth.
During the first month of the pandemic, inappropriate prescribing peaked for both settings,
but at a much higher rate in urgent care. The outcomes associated with inappropriate prescribing in
our sample are concerning, namely increased hospitalizations and mortality. Some common drivers
of inappropriate antibiotic prescribing, such as diagnostic uncertainty and patient demand, may
explain the inappropriate prescribing trends observed (37). Early in the COVID-19 pandemic,
providers faced great clinical uncertainty and prescribed more inappropriately. The varied
presentation and severity of COVID-19, as well as the lack of widespread testing in the early months
of 2020 made it challenging to determine one’s diagnosis (38–40). Diagnostic uncertainty has been
identified as a driver of antibiotic prescribing, with clinicians more likely to exhibit a mismatch
between their diagnosis and treatment when uncertainty is high (41,42).
Although patients using telehealth service providers were significantly younger and healthier
than patients at urgent care centers, they were equally or more likely to be sent to the emergency
department or receive an inappropriate prescription in Quarters 2-4 of 2020. The lack of physical
examinations or COVID-19 tests during telehealth appointments may have increased diagnostic
uncertainty and inappropriate prescribing in this setting. Inappropriate prescribing in telehealth
surpassed the rate in urgent care in May and remained higher for the rest of 2020. During this time,
testing rates in urgent care more than doubled as testing became more accessible. Increased testing
rates in urgent care, and therefore increased diagnostic certainty, coincided with decreased
inappropriate prescribing rates but the effect was not significant with and without adjusting for
patient characteristics. In the overall sample, providers who were more likely to inappropriately
prescribe were also more likely to order tests in the first quarter and to substitute test orders for
inappropriate prescriptions thereafter. Subgroup analyses of test orders by age demonstrate that
60
telehealth is erring more on the side of action than urgent care when patients are higher risk (older).
Our analysis of the association between testing and inappropriate prescribing and the likelihood of a
test order is limited by the lack of data on tests ordered in telehealth appointments.
Patient demand and patient satisfaction may also have contributed to the differences
observed by setting. Patients may be more satisfied with more care under the notion that “more care
is better care”, and providers want to achieve high patient satisfaction rates (43,44). Past research has
shown that patient satisfaction scores are higher for telehealth appointments when they receive an
antibiotic prescription (45). Patient demand for a treatment may have been satiated by test orders in
place of prescriptions to some extent, explaining the observed substitution between tests and
inappropriate treatments in Quarters 2-4 in our sensitivity analyses.
The lack of tests substituting prescriptions in models limited to urgent care indicates that
other factors specific to these setting contexts influenced the observed differences in inappropriate
prescribing. Telehealth requires patient and provider knowledge about how to conduct the
appointment. Patients must have the language to describe their symptoms and clinicians need
training in how to conduct a physical examination via telehealth (46). It is plausible that many new
providers entered these telehealth groups in 2020 without telehealth expertise and drove the
increases in inappropriate prescribing, but we cannot test this with our data due to a high level of
missingness for individual provider variables.
With the rapid expansion of telehealth in 2020, it is critical to understand if and how the
value of care delivered in these settings differs from in-person outpatient care. We find variation in
the care delivered across telehealth and urgent care groups using real world evidence. Past research
has shown variation in the quality of care delivered across telehealth companies using standardized
patients (47). Our results are consistent with recent findings of higher rates of inappropriate
antibiotic prescribing for COVID-19 Medicare patients in telehealth visits compared to urgent care
61
visits (48). High rates of inappropriate corticosteroid use for COVID-19 treatment has also been
found in outpatient settings (49). We find increased risk of hospitalization and mortality among
patients who received inappropriate prescriptions. In addition, the overprescribing of antibiotics
increases the risks of antimicrobial resistance and high rates of inappropriate outpatient
corticosteroid use present risks of hyperglycemia, neuropsychiatric symptoms, venous
thromboembolism, and sepsis (24,50,51).
One limitation of our study is the inability to identify the clinical severity, including oxygen
use, of patients, which is likely associated with hospitalization and death. Higher referral rates in
telehealth may be due to more severely ill patients using telehealth than urgent care but we cannot
directly test this hypothesis. However, differences in clinical severity would not make for appropriate
reasons for prescribing COVID-19 inappropriate treatments more in telehealth than urgent care.
Tests administered and not billed to insurance are not included in our data. The generalizability of
these findings is limited to commercially insured patients and telehealth companies or urgent care
centers established prior to 2020.
3.5 Conclusions
COVID-19 treatment patterns differed between in-person urgent care and telehealth. At the
onset of the pandemic, providers in both settings demonstrated a bias toward action, as evidenced
by high rates of inappropriate prescribing, when uncertainty was greatest. Inappropriate prescribing
was associated with increased risk of hospitalization and death. Providers in urgent care had higher
rates of inappropriate prescribing and referring patients to the emergency department at the start of
the pandemic. Over time, providers in telehealth exhibited higher rates of inappropriate prescribing
and emergency department referrals. Telehealth and attendant uncertainties may lead to greater
action bias to the detriment of healthcare quality.
62
Acknowledgements
We appreciate Steve Haenchen, Jeffrey A. Linder, Bridget McCabe, Stephen D. Persell, and Jason
Tibbels for their contributions to study conceptualization.
Funding: The Agency for Healthcare Research and Quality (5R01HS028127) support this work.
63
References
1. Kiderman A, Gur I, Bdolah-Abram T, Brezis M. in primary care: Evidence of action bias. J Fam
Pract [Internet]. 2013;62(8). Available from: https://cdn.mdedge.com/files/s3fs-
public/Document/September-2017/6208_JFP_Article1.pdf
2. Ayanian JZ, Berwick DM. Do Physicans Have a Bias toward Action?: A Classic Study Revisited.
Med Decis Making. 1991 Aug 1;11(3):154–8.
3. Thorpe A, Sirota M, Juanchich M, Orbell S. Action bias in the public’s clinically inappropriate
expectations for antibiotics. J Exp Psychol Appl. 2020 Sep;26(3):422–31.
4. Patt A, Zeckhauser R. Action Bias and Environmental Decisions. J Risk Uncertain. 2000 Jul
1;21(1):45–72.
5. Bar-Eli M, Azar OH, Ritov I, Keidar-Levin Y, Schein G. Action bias among elite soccer
goalkeepers: The case of penalty kicks. J Econ Psychol. 2007 Oct 1;28(5):606–21.
6. Kahneman D, Miller DT. Norm theory: Comparing reality to its alternatives. Psychol Rev. 1986
Apr;93(2):136–53.
7. Patel SY, Mehrotra A, Huskamp HA, Uscher-Pines L, Ganguli I, Barnett ML. Trends in
Outpatient Care Delivery and Telemedicine During the COVID-19 Pandemic in the US. JAMA
Intern Med. 2021 Mar 1;181(3):388–91.
8. Li KY, Zhu Z, Ng S, Ellimoottil C. Direct-to-consumer telemedicine visits for acute respiratory
infections linked to more downstream visits. Health Aff . 2021 Apr;40(4):596–602.
9. Ashwood JS, Mehrotra A, Cowling D, Uscher-Pines L. Direct-To-Consumer Telehealth May
Increase Access To Care But Does Not Decrease Spending. Health Aff . 2017 Mar 1;36(3):485–
91.
10. Dorsey ER, Topol EJ. State of Telehealth. N Engl J Med. 2016 Jul 14;375(2):154–61.
11. Saliba-Gustafsson EA, Miller-Kuhlmann R, Kling SMR, Garvert DW, Brown-Johnson CG,
Lestoquoy AS, et al. Rapid Implementation of Video Visits in Neurology During COVID-19:
Mixed Methods Evaluation. J Med Internet Res. 2020 Dec 9;22(12):e24328.
12. Gautret P, Lagier J-C, Parola P, Hoang VT, Meddeb L, Mailhe M, et al. Hydroxychloroquine and
azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial.
Int J Antimicrob Agents. 2020 Jul;56(1):105949.
13. Bull-Otterson L, Gray EB, Budnitz DS, Strosnider HM, Schieber LZ, Courtney J, et al.
Hydroxychloroquine and Chloroquine Prescribing Patterns by Provider Specialty Following
Initial Reports of Potential Benefit for COVID-19 Treatment - United States, January-June 2020.
MMWR Morb Mortal Wkly Rep. 2020 Sep 4;69(35):1210–5.
14. Information on COVID-19 treatment, prevention and research [Internet]. [cited 2021 Oct 18].
Available from: https://www.covid19treatmentguidelines.nih.gov/
64
15. Office of the Commissioner. Coronavirus (COVID-19) update: FDA Revokes Emergency Use
Authorization for Chloroquine and hydroxychloroquine [Internet]. 2020 [cited 2021 Oct 18].
Available from: https://www.fda.gov/news-events/press-announcements/coronavirus-covid-
19-update-fda-revokes-emergency-use-authorization-chloroquine-and
16. Cheng A, Caruso D, McDougall C. Outpatient Management of COVID-19: Rapid Evidence
Review. Am Fam Physician. 2020 Oct 15;102(8):478–86.
17. Office of the Commissioner. Coronavirus (COVID-19) update: FDA authorizes monoclonal
antibodies for treatment of COVID-19 [Internet]. 2020 [cited 2021 Oct 18]. Available from:
https://www.fda.gov/news-events/press-announcements/coronavirus-covid-19-update-fda-
authorizes-monoclonal-antibodies-treatment-covid-19
18. Office of the Commissioner. FDA approves first treatment for COVID-19 [Internet]. 2020 [cited
2021 Oct 18]. Available from: https://www.fda.gov/news-events/press-announcements/fda-
approves-first-treatment-covid-19
19. Office of the Commissioner. Coronavirus (COVID-19) update: FDA limits use of certain
monoclonal antibodies to treat COVID-19 due to the omicron variant [Internet]. U.S. Food and
Drug Administration. 2022 [cited 2022 Apr 27]. Available from: https://www.fda.gov/news-
events/press-announcements/coronavirus-covid-19-update-fda-limits-use-certain-monoclonal-
antibodies-treat-covid-19-due-omicron
20. COVID-19 monoclonal antibodies [Internet]. [cited 2022 Apr 21]. Available from:
https://www.cms.gov/monoclonal
21. Office of the Commissioner. Coronavirus (COVID-19) update: FDA authorizes first oral
antiviral for treatment of COVID-19 [Internet]. U.S. Food and Drug Administration. 2021 [cited
2022 Apr 21]. Available from: https://www.fda.gov/news-events/press-
announcements/coronavirus-covid-19-update-fda-authorizes-first-oral-antiviral-treatment-
covid-19
22. Office of the Commissioner. Coronavirus (COVID-19) update: FDA authorizes additional oral
antiviral for treatment of COVID-19 in certain adults [Internet]. U.S. Food and Drug
Administration. 2021 [cited 2022 Apr 21]. Available from: https://www.fda.gov/news-
events/press-announcements/coronavirus-covid-19-update-fda-authorizes-additional-oral-
antiviral-treatment-covid-19-certain
23. Office of the Commissioner. FDA takes actions to expand use of treatment for outpatients with
mild-to-moderate COVID-19 [Internet]. U.S. Food and Drug Administration. 2022 [cited 2022
Apr 21]. Available from: https://www.fda.gov/news-events/press-announcements/fda-takes-
actions-expand-use-treatment-outpatients-mild-moderate-covid-19
24. Nonhospitalized Adults: Therapeutic Management [Internet]. [cited 2021 Oct 22]. Available
from: https://www.covid19treatmentguidelines.nih.gov/management/clinical-
management/nonhospitalized-adults--therapeutic-management/
65
25. Cohen MS, Wohl DA, Fischer WA, Smith DM, Eron JJ. Outpatient Treatment of SARS-CoV-2
Infection to Prevent COVID-19 Progression. Clin Infect Dis [Internet]. 2021 May 28; Available
from: http://dx.doi.org/10.1093/cid/ciab494
26. Weiner JP, Bandeian S, Hatef E, Lans D, Liu A, Lemke KW. In-Person and Telehealth
Ambulatory Contacts and Costs in a Large US Insured Cohort Before and During the COVID-
19 Pandemic. JAMA Netw Open. 2021 Mar 1;4(3):e212618.
27. Allen L, Cummings JR, Hockenberry JM. The impact of urgent care centers on nonemergent
emergency department visits. Health Serv Res. 2021 Aug;56(4):721–30.
28. Chipidza FE, Wallwork RS, Stern TA. Impact of the Doctor-Patient Relationship. Prim Care
Companion CNS Disord [Internet]. 2015 Oct 22;17(5). Available from:
http://dx.doi.org/10.4088/PCC.15f01840
29. Badr HS, Zaitchik BF, Kerr GH, Nguyen N-LH, Chen Y-T, Hinson P, et al. Unified real-time
environmental-epidemiological data for multiscale modeling of the COVID-19 pandemic
[Internet]. bioRxiv. medRxiv; 2021. Available from:
http://dx.doi.org/10.1101/2021.05.05.21256712
30. COVID-19 Vaccines and Monoclonal Antibodies [Internet]. [cited 2021 Oct 18]. Available from:
https://www.cms.gov/medicare/medicare-part-b-drug-average-sales-price/covid-19-vaccines-
and-monoclonal-antibodies
31. Centers for Medicare & Medicaid Services. Appropriate Treatment for Upper Respiratory
Infection (URI) [Internet]. Electronic Clinical Quality Improvement (eCQI) Resource Center.
[cited 2021 May 9]. Available from: https://ecqi.healthit.gov/
32. Wei W-Q, Cronin RM, Xu H, Lasko TA, Bastarache L, Denny JC. Development and evaluation
of an ensemble resource linking medications to their indications. J Am Med Inform Assoc. 2013
Sep;20(5):954–61.
33. Centers for Disease Control and Prevention. ICD-10-CM Official Coding Guidelines -
Supplement Coding encounters related to COVID-19 Coronavirus Outbreak [Internet]. National
Center for Health Statistics. 2020 [cited 2021 Mar 16]. Available from:
https://www.cdc.gov/nchs/data/icd/ICD-10-CM-Official-Coding-Gudance-Interim-Advice-
coronavirus-feb-20-2020.pdf
34. Centers for Disease Control and Prevention. New ICD-10-CM code for the 2019 Novel
Coronavirus (COVID-19) [Internet]. National Center for Health Statistics. 2020 [cited 2021 Mar
16]. Available from: https://www.cdc.gov/nchs/data/icd/Announcement-New-ICD-code-for-
coronavirus-3-18-2020.pdf
35. Nonhospitalized adults: Therapeutic management [Internet]. COVID-19 Treatment Guidelines.
[cited 2022 May 10]. Available from:
https://www.covid19treatmentguidelines.nih.gov/management/clinical-
management/nonhospitalized-adults--therapeutic-
management/?utm_source=site&utm_medium=home&utm_campaign=highlights
36. StataCorp LLC. Stata statistical software: Release 16. College Station TX. 2019;
66
37. Richards AR, Linder JA. Behavioral Economics and Ambulatory Antibiotic Stewardship: A
Narrative Review. Clin Ther. 2021 Oct;43(10):1654–67.
38. CDC. COVID data tracker [Internet]. Centers for Disease Control and Prevention. 2020 [cited
2022 Apr 27]. Available from: https://covid.cdc.gov/covid-data-tracker/
39. Coleman JJ, Manavi K, Marson EJ, Botkai AH, Sapey E. COVID-19: to be or not to be; that is
the diagnostic question. Postgrad Med J. 2020 Jul;96(1137):392–8.
40. Gandhi TK, Singh H. Reducing the Risk of Diagnostic Error in the COVID-19 Era. J Hosp Med.
2020 Jun;15(6):363–6.
41. Whaley LE, Businger AC, Dempsey PP, Linder JA. Visit complexity, diagnostic uncertainty, and
antibiotic prescribing for acute cough in primary care: a retrospective study. BMC Fam Pract.
2013 Aug 19;14:120.
42. Lum EPM, Page K, Whitty JA, Doust J, Graves N. Antibiotic prescribing in primary healthcare:
Dominant factors and trade-offs in decision-making. Infection, Disease & Health. 2018 Jun
1;23(2):74–86.
43. Lyu H, Xu T, Brotman D, Mayer-Blackwell B, Cooper M, Daniel M, et al. Overtreatment in the
United States. PLoS One. 2017 Sep 6;12(9):e0181970.
44. Foster CB, Martinez KA, Sabella C, Weaver GP, Rothberg MB. Patient Satisfaction and
Antibiotic Prescribing for Respiratory Infections by Telemedicine. Pediatrics [Internet]. 2019
Sep;144(3). Available from: http://dx.doi.org/10.1542/peds.2019-0844
45. Martinez KA, Rood M, Jhangiani N, Boissy A, Rothberg MB. Antibiotic Prescribing for
Respiratory Tract Infections and Encounter Length: An Observational Study of Telemedicine.
Ann Intern Med. 2019 Feb 19;170(4):275–7.
46. Telediagnosis for acute care: Implications for the quality and safety of diagnosis [Internet]. [cited
2022 May 2]. Available from: https://www.ahrq.gov/patient-safety/reports/issue-briefs/teledx-
3.html
47. Schoenfeld AJ, Davies JM, Marafino BJ, Dean M, DeJong C, Bardach NS, et al. Variation in
Quality of Urgent Health Care Provided During Commercial Virtual Visits. JAMA Intern Med.
2016 May 1;176(5):635–42.
48. Tsay SV, Bartoces M, Gouin K, Kabbani S, Hicks LA. Antibiotic Prescriptions Associated With
COVID-19 Outpatient Visits Among Medicare Beneficiaries, April 2020 to April 2021. JAMA
[Internet]. 2022 Apr 8; Available from: http://dx.doi.org/10.1001/jama.2022.5471
49. Bradley MC, Perez-Vilar S, Chillarige Y, Dong D, Martinez AI, Weckstein AR, et al. Systemic
Corticosteroid Use for COVID-19 in US Outpatient Settings From April 2020 to August 2021.
JAMA [Internet]. 2022 Apr 8; Available from: http://dx.doi.org/10.1001/jama.2022.4877
50. Llor C, Bjerrum L. Antimicrobial resistance: risk associated with antibiotic overuse and initiatives
to reduce the problem. Ther Adv Drug Saf. 2014 Dec;5(6):229–41.
67
51. Waljee AK, Rogers MAM, Lin P, Singal AG, Stein JD, Marks RM, et al. Short term use of oral
corticosteroids and related harms among adults in the United States: population based cohort
study. BMJ. 2017 Apr 12;357:j1415.
68
Appendix
Table A3.1: Inclusion criteria
Inclusion Exclusion
Telehealth Clinician Group Proportion of 2019 outpatient
visits that were telehealth is in the
90th percentile of clinician groups
Urgent Care Face to Face
Clinician Group
Proportion of 2019 outpatient
visits that were urgent care is in
the 90th percentile of clinician
groups
Sample Index suspected, exposed, or
confirmed COVID-19 diagnosis at
TH or UC; insurance enrollment
window overlapping index date
Unknown state or sex; 2 states
listed on index date
Drugs Prescribed 0-3 days after index
appointment
Prescribed on or after the date of
inpatient admission; prescribed on
or after the ED escalation
Clinician-attributed test
order
Urgent Care: same claim;
Telehealth: same day
69
Table A3.2: COVID-19 diagnosis codes
ICD-10 Code Code Description COVID-19 classification
B34.2 Coronavirus infection, unspecified Suspected or exposed
B97.21
SARS-associated coronavirus as the cause of
diseases classified elsewhere
Suspected or exposed
B97.29
Other coronavirus as the cause of diseases
classified elsewhere
Suspected or exposed
B97.89
Other viral agents as the cause of diseases
classified elsewhere
Suspected or exposed
M35.81 Multisystem inflammatory syndrome Suspected or exposed
M35.89
Other specified systemic involvement of
connective tissue
Suspected or exposed
U07.2
COVID-19, virus not identified (clinically
diagnosed)
Suspected or exposed
Z03.818
Encounter for observation for suspected exposure
to other biological agents ruled out
Suspected or exposed
Z20.822
Contact with and (suspected) exposure to
COVID-19
Suspected or exposed
Z20.828
Contact with and (suspected) exposure to other
viral communicable diseases
Suspected or exposed
J12.81 Pneumonia due to SARS-associated coronavirus Confirmed
J12.82 Pneumonia due to coronavirus disease 2019 Confirmed
U07.1 COVID-19, virus identified (lab confirmed) Confirmed
Tables A3.3 and A3.4: Value Sets
ICD-10 Codes
A3.3 -Antibiotic appropriate (potentially or always) Link to Table
A3.4 - Steroid appropriate (potentially or always) Link to Table
Note: If the index claim does not contain an antibiotic (steroid) appropriate diagnosis code and an antibiotic (steroid)
was prescribed within 3 days, then it is treated as an inappropriate antibiotic (steroid) prescription.
70
Table A3.5: Frequency of Drugs Prescribed within 3 Days of Index Appointment
Generic Name Drug Classification Percent
AZITHROMYCIN ANTIBIOTICS 10.11
AMOXICILLIN/POTASSIUM CLAV ANTIBIOTICS 3.54
AMOXICILLIN ANTIBIOTICS 3.36
DOXYCYCLINE HYCLATE ANTIBIOTICS 2.38
CEFDINIR ANTIBIOTICS 1.10
DOXYCYCLINE MONOHYDRATE ANTIBIOTICS 1.01
ALBUTEROL SULFATE
BRONCHODILATOR/BETA-
AGONIST/ANTICHOLINERGIC 9.58
IPRATROPIUM BROMIDE
BRONCHODILATOR/BETA-
AGONIST/ANTICHOLINERGIC 1.12
BENZONATATE ANTITUSSIVES 6.91
PREDNISONE CORTICOSTEROIDS 5.55
FLUTICASONE PROPIONATE CORTICOSTEROIDS 3.06
METHYLPREDNISOLONE CORTICOSTEROIDS 2.33
ONDANSETRON ANTIEMETICS 1.89
ONDANSETRON HCL ANTIEMETICS 1.02
OSELTAMIVIR PHOSPHATE ANTIVIRALS 2.45
Note: Only listing prescriptions with > 1% frequency and excluding monoclonal antibodies (n=2).
Figure A3.1: Testing trends by setting. Percent of patients with possible or confirmed COVID-
19 who receive a test at dedicated telehealth groups (red) or urgent care centers (blue) in each month
of 2020. Urgent care tests are defined as COVID-19 tests on the same claim as the index visit.
Telehealth tests are defined as COVID-19 tests on the same day as the index visit.
71
Table A3.6: Adjusted odds ratio of a clinician attributed test among patients with suspected
or exposed COVID-19
Test Order Test Order
Sample < 40
years old
OR (SE)
Sample > 40
years old
OR (SE)
VARIABLES
Telehealth 13.48* 37.55***
(19.30) (25.99)
Quarter 2 14.84** 39.59***
(19.35) (24.47)
Quarter 3 42.74*** 158.08***
(55.62) (98.55)
Quarter 4 257.19*** 743.09***
(336.70) (466.11)
Telehealth#Quarter 2 0.39 0.19**
(0.52) (0.13)
Telehealth#Quarter 3 0.24 0.14***
(0.32) (0.10)
Telehealth#Quarter 4 0.03*** 0.02***
(0.04) (0.02)
Reference Category: Age < 20
Age, 20 - 34 0.91
(0.08)
Age, 35 - 49 0.99
(0.10)
Age, 50 - 64 0.99
(0.07)
Age, 65 - 79 1.03
(0.09)
Age, 80 - 90 0.95
(0.10)
Male 0.91* 1.01
(0.05) (0.04)
Reference Category: White
Asian 0.97 1.08
(0.14) (0.11)
Black 0.78** 0.82***
(0.09) (0.06)
Hispanic 0.85* 0.93
(0.08) (0.07)
Unknown 0.96 0.90**
(0.06) (0.04)
Charlson Comorbidity Index 1.09** 1.01
(0.05) (0.01)
Medicare 1.71 14.83***
(0.67) (1.81)
Reference Category: Lowest quintile of cases per 100k in
state
Quintile 2 1.14 0.57***
(0.14) (0.06)
Quintile 3 1.21 0.38***
72
(0.17) (0.04)
Quintile 4 1.16 0.73***
(0.19) (0.09)
Highest quintile of cases per 100k in state 1.37* 0.71***
(0.24) (0.09)
Constant 0.001*** 0.0005***
(0.00) (0.00)
Observations 14,218 34,644
Number of groups 357 759
*** p<0.01, ** p<0.05, * p<0.1
73
Figure A3.2: Predicted probability of a test order by setting and time with 95% confidence
intervals. Probabilities are estimated using observed covariates for patients with index appointments
at dedicated telehealth groups (red) or urgent care centers (blue) in that quarter of 2020. A) Sample
limited to patients under the age of 40; B) Sample limited to patients 40 and older
74
Table A3.7: The association of test orders and inappropriate prescriptions (Odds Ratios)
VARIABLES Test +
Quarter
Test*
Quarter
Test* Quarter,
Urgent Care
Adjusted
Test* Quarter
Adjusted Test*
Quarter,
Urgent Care
Test Order 1.04 2.95*** 5.63 2.86*** 3.67
(0.05) (1.09) (6.82) (1.08) (4.42)
Quarter 2 0.37*** 0.38*** 0.12*** 0.39*** 0.13***
(0.03) (0.03) (0.02) (0.03) (0.02)
Quarter 3 0.41*** 0.41*** 0.13*** 0.46*** 0.14***
(0.03) (0.03) (0.02) (0.05) (0.02)
Quarter 4 0.49*** 0.52*** 0.14*** 0.40*** 0.12***
(0.04) (0.04) (0.02) (0.05) (0.02)
Test Order#Quarter 2 0.40** 0.19 0.38** 0.24
(0.16) (0.23) (0.15) (0.30)
Test Order#Quarter 3 0.36*** 0.20 0.34*** 0.26
(0.14) (0.24) (0.13) (0.31)
Test Order#Quarter 4 0.33*** 0.21 0.33*** 0.28
(0.12) (0.25) (0.12) (0.33)
Telehealth 1.56** 1.56** 2.34***
(0.31) (0.30) (0.43)
Reference Category: Age < 20
Age, 20 - 34 1.04 1.72***
(0.12) (0.33)
Age, 35 - 49 1.81*** 2.95***
(0.21) (0.55)
Age, 50 - 64 2.21*** 2.88***
(0.26) (0.54)
Age, 65 - 79 1.34** 1.83***
(0.18) (0.36)
Age, 80 - 90 1.47*** 2.07***
(0.22) (0.42)
Male 0.70*** 0.75***
(0.02) (0.03)
Reference Category: White
Asian 0.72*** 0.81
(0.09) (0.13)
Black 0.93 0.95
(0.07) (0.09)
Hispanic 1.00 0.97
(0.06) (0.09)
Unknown 1.09** 1.19***
(0.04) (0.06)
Charlson Comorbidity
Index
1.09*** 1.07***
(0.01) (0.01)
Medicare 1.51*** 1.76***
(0.15) (0.20)
Reference Category: Lowest
quintile of cases per 100k in
75
state
Quintile 2 0.77*** 0.89
(0.06) (0.09)
Quintile 3 0.84* 0.89
(0.08) (0.10)
Quintile 4 1.14 1.22
(0.12) (0.16)
Highest quintile of cases
per 100k in state
1.24** 1.23
(0.14) (0.17)
Confirmed COVID-19
Diagnosis on Index Date
1.91*** 2.15***
(0.11) (0.17)
Constant 0.18*** 0.18*** 0.56*** 0.09*** 0.17***
(0.02) (0.02) (0.07) (0.01) (0.04)
Observations 45,240 45,240 32,064 45,240 32,064
Number of groups 801 801 752 801 752
*** p<0.01, ** p<0.05, * p<0.1
76
Chapter 4
Association of fatal overdose notification letters with prescription of
benzodiazepines: A secondary analysis of a randomized control trial
Marcella A. Kelley, Andy Nguyen, Roneet Lev, Jonathan Lucas, Tara Knight, Emily Stewart, Michael
Menchine, Jason N. Doctor
Condensed version will be published in a forthcoming issue of JAMA Internal Medicine 2022
77
Abstract
Background: There is a need for interventions that reduce inappropriate benzodiazepine prescribing
to mitigate increased benzodiazepine-related fatal overdoses in the last 25 years. Our objective was to
determine whether a behavioral economic intervention that reduced opioid prescribing was also
effective for benzodiazepines.
Methods: In this cluster randomized controlled trial, 861 clinicians from 170 decedent clusters were
randomized to the intervention (n=404 prescribers) or control group (n=457 prescribers). Participants
were scheduled drug prescribers to individuals who died of a scheduled drug-related overdose between
July 2015 and June 2016 in San Diego County. Clinicians in the intervention group received a letter
signed by the Chief Deputy Medical Examiner notifying them of their patient’s fatal overdose and
clinicians in the control group received no notification. We assessed the change in benzodiazepine
prescriptions dispensed by clinicians’ patients 3 months prior (pre-period) to 4 months after (washout
and post-period) intervention implementation. Our primary outcomes were the change in daily
diazepam milligram equivalents (DMEs) and the change in the daily quantity of 2 milligram diazepam
equivalent pills dispensed pre- to post-intervention between study groups. Secondary outcomes
included substantial (>20%) reductions in DME by group and the change in the probability, number,
and average days supply of new starts from pre-to post-intervention between groups. We conducted
an intent-to-treat analysis with a difference-in-difference estimator and nested random effects.
Results: 743 prescribers (intervention n=353; control n=390) had a patient who dispensed a
benzodiazepine prescription during the study period. Daily DMEs filled by patients of study
prescribers decreased 5.6% ([95% CI: -9.1% to -2.0%]; P < 0.01) more in the intervention than control
group pre-to-post intervention. The number of 2 mg pills dispensed decreased 3.7% ([95% CI: -6.9%
to -0.5%]; P < 0.05) more in the intervention group compared to the control group pre-to-post
intervention. From pre-to post-intervention between groups, there were no significant differences in
78
the probability, daily number, or daily average days supply of “new benzodiazepine starts” and > 20%
reductions in daily DMEs, controlling for the share of new or co-prescribed patients.
Conclusions: Provider awareness of a recent overdose death decreases benzodiazepine prescribing.
Fatal overdose notification letters are a cost-effective solution to reduce benzodiazepine and opioid
overprescribing.
Trial Registration: This is a secondary analysis of the original trial registered at
https://clinicaltrials.gov (NCT02790476).
79
4.1 Background
Many public health interventions and policies are appropriately aimed at reducing opioid
misuse and abuse, but few address the alarming overprescribing of benzodiazepines (1–4).
Benzodiazepines are psychotropic drugs that are commonly prescribed but have a dose-dependent
relationship associated with tolerance, sedation, confusion, and increased mortality (5,6). The
effectiveness of benzodiazepines is limited to short-term improvements in the treatment of anxiety
disorders (7,8). However, long-term use is associated with memory impairment, vehicle accidents,
falls, overdoses, dependence, and severe withdrawal symptoms (9). Treatment guidelines only
recommend short-term benzodiazepine use, typically as second- or third-line, for the treatment of
anxiety or sleep disorders (6–8,10–12). Despite these guidelines and known risks, benzodiazepine
prescription fills, quantity per prescription, and mortality have increased in the past 25 years in the
United States (13,14). Increased utilization of this drug class is largely driven by an increased number
of patients with long-term benzodiazepine use and increased outpatient prescribing by primary care
clinicians (13,15,16). In fact, the rate of benzodiazepine prescribing at ambulatory care visits increased
by 95% from 2003 to 2015 (13). Discontinuing long-term use is difficult for many prescribers, who
view tapering as low-value, uncompassionate, and likely to fail (17,18). Additional efforts to reduce
inappropriate benzodiazepine prescribing are needed.
Fatal overdose on benzodiazepines is a serious problem. From 1999 to 2020, annual fatal
overdoses involving benzodiazepines increased from 1,135 to 12,290 (19). Fatal overdose is more
likely when benzodiazepines are used in combination with opioids, licit or illicit (19). In the first half
of 2018, nearly one third of fatal opioid overdoses in the United States involved benzodiazepines (20).
Based on the harms of concurrent benzodiazepine and opioid use, the 2016 CDC Guideline for
Prescribing Opioids for Chronic Pain advises clinicians to avoid co-prescribing opioids and
benzodiazepines whenever possible (21). Despite this guideline, nearly 20% of patients with opioid
80
prescriptions had at least 1 day with an overlapping benzodiazepine prescription in each month of
2017 (22). The overprescribing of benzodiazepines may also be fueling the entry of new forms of
high-potency benzodiazepines into the illicit market, analogous to the increase of heroin and fentanyl
following opioid overprescribing (1). Once dependence on the drug has formed, transition to illicit
use is more probable. This places the individual at risk of exposure to pills laced with fentanyl. Across
23 states, 67% of benzodiazepine overdose deaths involved illicitly manufactured fentanyls in the first
half of 2020 (23).
One effective intervention in reducing opioid prescribing is notifying clinicians of their
patient’s scheduled drug-related death (24). In 2017, clinicians in San Diego County were randomized
to a control or intervention group following their patient’s accidental overdose involving a schedule
II, III, or IV drug. After receiving the letter, clinicians in the intervention arm prescribed significantly
fewer milligram morphine equivalents (-9.7% [95% CI, -13.2 to -6.2%]; P < 0.001), started fewer
patients on opioid therapy, and wrote fewer high-dose opioid prescriptions than the control group
(24). Since this study, the county of Los Angeles adopted this intervention to reduce opioid
overprescribing (25).
Although the intervention targeted prescribers with any scheduled drug death in their practice,
the effect of this notification letter on drugs outside of opioids is currently unknown. Given the
problem of inappropriate benzodiazepine prescribing, we sought to study the intervention's effect on
prescribing for this drug class. Of the 170 deaths included in the trial, 132 involved a benzodiazepine
alone or in combination with an opioid. Since learning of a patient’s fatal overdose is psychologically
impactful, our hypothesis is that the notification letters lead to reductions not only in opioid
prescribing, but also benzodiazepine prescribing. Using intervention assignment and prescription data
from the San Diego trial and California’s prescription drug monitoring program, we estimate whether
learning of a patient’s fatal overdose impacts dispensed quantities of benzodiazepines.
81
4.2 Methods
4.2.1 Study Design and Data Sources
Trial participants were prescribers to individuals who died of a scheduled drug overdose
between July 2015 and June 2016 in San Diego County. Eight-hundred sixty-one prescribers wrote a
scheduled drug prescription to a decedent in the year prior to their death. These clinicians were
grouped into 170 decedent clusters and randomized to the intervention or control group. In January
2017, clinicians in the intervention group received a letter signed by the Chief Deputy Medical
Examiner notifying them of their patient’s death and clinicians in the control group received no
notification. We tabulated if the decedent received a prescription from a prescriber with a single or
multiple fatal overdose(s), and if the cause of death was opioids only, benzodiazepines only, or opioids
in combination with benzodiazepines. Six lists of decedents were created based on these crossed strata
levels. Study personnel used the random sequence generator from random.org to determine the
decedent order in each list (26). Prescribers to the decedents in the first half of the list were assigned
to the intervention group and the remaining prescribers were assigned to the control group. The
University of Southern California Institutional Review Board approved this study and waived
informed consent. Study procedures are explained in greater detail in Doctor et al. (24).
The treatment letter notified prescribers of their patient’s fatal overdose involving a schedule
II, III, or IV drug (Appendix Supplementary Text). The behavioral economic underpinning of this
notification letter is the availability heuristic. When people can easily recall an event, meaning the event
is more available to them, their judgment of the probability of that event increases (27). Therefore,
learning of a patient's overdose may cause clinicians to be more cautious of a future overdose when
prescribing scheduled drugs.
In the present analysis, we used prescription drug monitoring data from the California
Controlled Substance Utilization Review and Evaluation System (CURES). CURES contains Schedule
82
II, III, and IV controlled substance prescription dispensing information, dates of a prescription fill,
and quantities (28). Our analysis was limited to prescriptions in the benzodiazepine drug class from
the 170 clusters of prescribers randomized to the intervention or control group. We followed the
Consolidated Standard of Reporting Trials (CONSORT) reporting guideline.
4.2.2 Measures
Benzodiazepine prescriptions written by clinicians in our study were converted to daily
diazepam milligram equivalents (DMEs/day) (29). The formula for converting benzodiazepine
prescriptions to DMEs is a product of strength, quantity, the inverse of days supply, and a conversion
factor:
For prescriptions with a quantity less than days supply, quantity was set equal to days supply. If the
patient had multiple benzodiazepine prescriptions, DMEs/day was an aggregate of these
prescriptions. Daily DMEs were converted to daily diazepam pill equivalents assuming 1 diazepam
pill equals to 2 DMEs, rounded to the nearest integer. Each prescription was labeled as pre- or post-
intervention based on if the prescription fill date fell before or after the intervention start date.
Prescriptions written between October 27, 2016 and January 27, 2017 were included in the pre-period
and prescriptions written between February 27, 2017 and May 27, 2017 were included in the post-
period. Intervention assignment data from the original study was included to delineate the intervention
and control groups. It is important to note that the DMEs/day calculation reflects the daily amount
dispensed under the physician’s name and not the daily amount consumed by each patient.
The first benzodiazepine prescription for a patient in the study period was considered a “new
benzodiazepine start”. Patients with co-prescribed opioids and benzodiazepines were identified if the
patient dispensed a benzodiazepine on the same day as or after an opioid prescription written by any
83
prescriber in the sample. We calculated the share of a provider’s patients with new starts and co-
prescribed opioids and benzodiazepines among their patients who dispensed a benzodiazepine or
opioid.
4.2.3 Outcomes
Building upon the trial’s prespecified and previously reported outcomes related to opioid
prescribing, we are exploring changes in benzodiazepine prescribing. Our primary outcomes were the
change in daily benzodiazepines (DMEs) dispensed and the change in the daily quantity of 2 mg
diazepam pill equivalents dispensed pre-to post-intervention between study groups. We log-
transformed daily DMEs and retained zero values to approximate a normal distribution (Appendix
Figures A4.1 and A4.2). Secondary outcomes included changes in potentially high-risk prescribing,
including the difference in the probability of “new benzodiazepine starts”, the difference in the daily
number of new starts, and the difference in the daily average days supply of those new benzodiazepine
starts pre-to post-intervention between groups. Additionally, we analyzed the likelihood of a greater
than 20% reduction in DMEs from the pre- to the post-intervention period by group.
4.2.4 Statistical Analysis
To estimate the effect of the notification letter on benzodiazepine prescribing in the post-
intervention period, we conducted an intent-to-treat analysis with a difference-in-difference estimator
and nested random effects. The analysis was intent-to-treat because adherence to the intervention (i.e.
receiving and reading the letter) was unobserved. The simplified version of our five models is as
follows:
𝑌 𝑖𝑗𝑘
= 𝛽 0
+ 𝛽 1
𝑋 1𝑖𝑗𝑘
+ 𝛽 2
𝑋 2𝑖𝑗
+ 𝛽 3
𝑋 3𝑖𝑗𝑘
+ 𝛿 𝑖𝑗
+ 𝘀 𝑖𝑗𝑘
where Y
ijk
is one of our primary or secondary outcomes, X
1
denotes the post-intervention period
(dichotomous), X
2
denotes the intervention group (dichotomous), and X
3
is the interaction of X
1
and
X
2
(dichotomous) for every j
th
prescriber from the i
th
decedent cluster at time k. 𝛿 𝑖𝑗
is the random
84
effect for the j
th
prescriber nested in the i
th
decedent cluster and 𝘀 is the model residual term. 𝛽 3
is the
difference-in-difference estimator of interest for each model.
In the analysis of the change in clinician’s daily DMEs, Y
ijk
was left-truncated at zero for days
when the clinician did not prescribe a benzodiazepine and was continuously distributed over positive
values. Because of this, we used a censored regression (i.e. tobit) model with log-transformed censored
diazepam milligram equivalent as the outcome (30). 𝛽 3
is our difference-in-difference estimator and
provides the partial effect of X
3
on DMEs if data are uncensored (DMEs*), but the effect of greater
interest is the partial effect of X
3
on the observed outcome (DMEs). We used a scale factor to adjust
𝛽 3
to approximate the partial effect of X
3
on DMEs as a percentage change (30). Additionally, we
estimated a count model where Y
ijk
was the daily quantity of 2 milligram (mg) diazepam pill equivalents
dispensed. We used a mixed effects zero-inflated negative binomial model to account for excess zeroes
and overdispersion of the dependent variable.
To estimate the effect of the notification letter on the likelihood of new benzodiazepine starts,
we estimated a mixed effects logistic model where Y
ijk
was dichotomous for new starts per prescriber
per day. Additionally, we estimated the effect of the notification letter on the daily number of new
starts per prescriber using a mixed effects zero-inflated negative binomial model where Y
ijk
was the
count of new starts per day. For the analysis of the difference in daily average days supply of new
benzodiazepine starts, we limited the sample to new starts and used a negative binomial mixed model
where Y
ijk
was the daily average days supply rounded to the nearest integer. Logistic regression was
used to analyze problematic sudden diazepam milligram equivalent (DME) reductions by a prescriber
(> 20%), controlling for the proportion of patients with new starts and with opioid co-prescriptions,
which the letter advises against. Analyses were performed using STATA version 16, SAS version 9.4,
and R version 3.6.0 (31–33).
85
4.2.5 Power Calculations
We calculated the sample size needed to detect a 5% change in daily DMEs. We assumed a
two-tailed t-test with a 0.05 probability of Type I error and 80% statistical power. We assumed a mean
daily dose of 11 DMEs based on the average daily benzodiazepine dose among primary care patients
in a 2016 study in Massachusetts (34). We conducted our power analysis with 5.5 prescribers per
decedent cluster, the mean cluster size from the San Diego study (24). Most clinician clusters in
multicenter trials for process measures have an intracluster correlation coefficient between 0.05 and
0.15 (35). We assumed the intracluster correlation coefficient for clinicians in our study was the median
of this range, 0.10. Using a cluster-adjusted formula for estimating sample size and assuming a standard
deviation of 2 DMEs, we found that with 602 clinicians, we have a greater than 80% chance to detect
a 5% change in daily benzodiazepine dosage (36).
4.3 Results
861 prescribers from 170 decedent clusters were randomized to the intervention or control
group (Tables 4.1 and 4.2). The analysis included any prescribers from the original study whose
patient filled a benzodiazepine prescription in the pre- or post-intervention period. There were 743
prescribers (intervention n=353; control n=390) from 161 decedent clusters (Figure 4.1). Alprazolam
and lorazepam were the most frequently dispensed benzodiazepines (Figure 4.2).
86
Table 4.1: Decedent characteristics
Characteristic Letter
(n=82)
Control
(n=85)
Statistic p-value
Age (SD) 49.75 (11.15) 48.21 (14.85) 𝑡 =0.059 0.954
Male 53 (65%) 44 (52%) 𝜒 2
= 0.591 0.442
Race 𝜒 2
= 4.752 0.447
Black 6 (7%) 8 (9%)
Hispanic 8 (10%) 4 (5%)
Native American 0 1 (1%)
Asian/Pacific Islander 0 1 (1%)
Non-Hispanic White 65 (79%) 70 (82%)
Other 3 (4%) 1 (1%)
Cause of death
𝜒 2
= 5.300
0.380
Prescription 41 (50%) 53 (62%)
Prescription and illicit 27 (33%) 15 (18%)
Prescription and alcohol 10 (12%) 13 (15%)
Prescription, illicit, and alcohol 2 (2%) 2 (2%)
Prescription and OTC 1 (<1%) 1 (<1%)
Prescription, alcohol, and OTC 1 (<1%) 1 (<1%)
From Doctor et al. 2018 Table 1
Table 4.2: Enrolled prescriber characteristics
Professional Practice Letter Control Statistic p-value
Medical doctor (MD) 277 315
Doctor of osteopathy (DO) 30 29
Nurse Practitioner (NP) 24 26
Physician assistant (PA) 44 48
𝜒 2
= 1.173*
0.883*
Dentistry (DDS/DMD) 13 20
Total 388 438
*For all professional practices; From Doctor et al. 2018 Table 2
87
Figure 4.1: Consort diagram. Consort diagram depicting analytic sample selection. ME, medical
examiner
Figure 4.2: Frequency of benzodiazepine type dispensed. Frequency of benzodiazepine type
dispensed in the pre- and post-intervention periods.
88
In the mixed effects censored regression model, convergence could not be achieved with
weekend observations so only weekdays were included. The results of this model demonstrated
weekday daily DMEs filled among patients of prescribers in the intervention group decreased by 5.6%
more than in the control group between the pre- and post-intervention period (-5.6% [95% CI: -9.1%
to -2.0%]; P < 0.01) (Table 4.3). We ran additional models using all available data, including weekend
dispenses, to confirm the observed significant reduction in prescribing. In the mixed effects zero-
inflated negative binomial model of the daily quantity of 2 mg diazepam pill equivalents dispensed,
there was a 3.7% greater decrease from the pre- to the post-intervention period among patients of
prescribers in the intervention group compared to the control group (-3.7% [95% CI: -6.9% to -0.5%];
P < 0.05) (Table 3). On average, there were 2.9 fewer 2 mg diazepam pills dispensed per prescriber
per month in the intervention group compared to the control group (Table 4.4).
Table 4.3: Model results as coefficients
Variables
Daily DMEs
(adjusted)
Daily Number
of Diazepam
Pills
Daily Number
of New Starts
Daily Average
Days Supply
of New Starts
Daily
Log(New
Starts)
Received Letter -0.03 -0.06 0.04 -0.03 0.04
Post-Period -0.00 -0.01 -0.11*** -0.04*** -0.11***
Received Letter*Post-
Period
-0.06*** -0.04* 0.01 -0.03 0.01
Weekend -1.13*** -0.99*** -0.05*** -1.10***
Constant -1.16*** 0.57*** -2.43*** 2.54*** -2.54***
Observations 95,847 133,740 133,740 12,876 133,740
Number of decedents 161 161 161 161 161
Log-likelihood -90969 -188319 -43526 -45239 -35744
AIC 181952 376656 87071 90493 71502
BIC 182018 376744 87159 90553 71570
*** p<0.01, ** p<0.05, * p<0.1
89
Table 4.4: Adjusted mean monthly 2 mg diazepam pill equivalents dispensed per prescriber
Letter Control
Prescribers followed, n 353 390
Pre-intervention,
mean (95% CI)
76.0
(74.1 to 77.9)
82.9
(81.0 to 84.8)
Post-Intervention,
mean (95% CI)
72.3
(70.6 to 74.0)
82.0
(80.2 to 83.8)
Difference in Difference,
mean (95% CI)
-2.9
(-5.43 to -0.01)
Difference in Difference
p-value
p < 0.05
Note: Pre-intervention values are trimmed at 95% with bootstrapped means and confidence intervals.
In the analysis of the change in the likelihood of “new benzodiazepine starts” per day, there
was no significant difference by group from the pre- to post-intervention period (Odds Ratio (OR):
1.0 [95% CI: 0.9 to 1.1]; P > 0.1). There was a significant decrease in the daily likelihood of a “new
benzodiazepine start” in the post-intervention period compared to the pre-intervention period (OR:
0.9 [95% CI: 0.8 to 0.9]; P < 0.01) (Table 4.3). In the zero-inflated negative binomial model of the
number of new patients started per prescriber per day, there was a 10.2% reduction in the number of
new patients prescribed benzodiazepines in the post-intervention period compared to the pre-
intervention period (95% CI: -14.2% to -6.0%; P < 0.01) but no significant difference between groups
from the pre-intervention period to the post-intervention period (0.8% [95% CI: -5.7% to 7.7%]; P >
0.1) (Table 4.3).
The difference in the average days supply of new starts from the pre- to the post-intervention
period was significant (-4.4% [95% CI: -7.4% to -1.3%]; P < 0.01) but did not significantly differ
between groups (-2.6% [95% CI: -7.0% to 2.0%]; P > 0.1) (Table 4.3). In all models including
weekend observations, the estimate for weekends was negative and statistically significantly different
from zero. Prescribers in the intervention group were significantly more likely to reduce daily DMEs
by > 20% from the pre- to the post-intervention (OR: 1.3 [95% CI: 1.0 to 1.8]; P < 0.05), but the
90
significant association dissipated when the proportion of their patients with new prescriptions and
with an opioid co-prescription was included in the model as main and interaction effects with
intervention assignment (OR: 0.9 [95% CI: 0.6 to 1.6]; P > 0.1) (Appendix Table A4.1).
4.4 Discussion
In addition to reducing opioid prescribing, notifying clinicians of their patient’s scheduled
drug-related fatal overdose also significantly decreased benzodiazepine prescribing. After receiving the
letter, there was a greater decrease in benzodiazepine dispenses (i.e. diazepam milligram and pill
equivalents) per day among patients of clinicians who received the letter compared to patients of
clinicians who did not. The effect of the notification was not significant in reducing the probability,
number, or average days’ supply of new benzodiazepine starts. The unadjusted effect of the
notification on the likelihood of a prescriber substantially reducing their daily DMEs (>20%) was
confounded by the association of the treatment group and changes in prescribing with the share of
new and co-prescribed patients. Prescribers in the intervention group had a smaller share of patients
with co-prescriptions and new starts, suggesting safer prescribing that contributed to their reduction
in DMEs. The remaining effect of the notification letter seems to be concentrated among continuing
prescriptions, which is precisely what drove the national increase in benzodiazepine prescribing from
2005-2012 (37). Concerns of substituting opioids for benzodiazepines have been raised as
benzodiazepine prescribing for chronic pain increases (13). With recent increases in benzodiazepine
overdose visits and deaths with and without opioid co-involvement, interventions to reduce
inappropriate benzodiazepine prescribing are needed (23). Our study finds that this behavioral
economic intervention effectively reduces both opioid and benzodiazepine prescribing across the
sample of patients.
91
Of the five recommendations in this letter, all pertain to opioid prescribing, and one contains
information on the dangers of co-prescribing opioids and benzodiazepines. Although
benzodiazepines are less emphasized in the letter, there was a significant reduction in benzodiazepine
prescribing. Physician inertia in tapering benzodiazepines is anchored in their belief of the treatment’s
high efficacy and low risk of adverse events (18). High-rate benzodiazepine prescribers perceive a
lower risk of related harms than low-rate benzodiazepine prescribers (38). The notification letter may
increase the salience of harms for opioid and benzodiazepine prescribing among clinicians in the
intervention group, thereby increasing the clinicians’ perceived probability of harms related to their
use (27).
Behavioral economic interventions, such as this notification letter campaign, have not been
widely explored to reduce benzodiazepine prescribing among a broad panel of patients. One
behavioral economics study using peer benchmarking identified a decrease in the benzodiazepine
prescribing rate among elderly inpatients (39). Much of the research on reducing benzodiazepine
prescribing uses education, medication reviews, and auditing to target an elderly patient population
(40). A strength of our study is the inclusion of all benzodiazepine prescriptions written to patients of
any age in San Diego County. An important limitation is that we did not assess the clinical well-being
of the patients or clinicians in the intervention group. The generalizability of these findings may be
limited to San Diego County.
4.5 Conclusions
Increasing a clinician’s awareness of harms related to benzodiazepine use decreased their
prescribing. Fatal overdose notification letters are a cost-effective intervention to reduce both opioid
and benzodiazepine prescribing.
92
Acknowledgments
We appreciate M. Small and T. Farrales at the U.S. Department of Justice for their assistance with
CURES. The University of Southern California Institutional Review Board approved all study
procedures and waived informed consent for participants under HHS regulations at 45 CFR
46.116(c). M.A. Kelley was a legal consultant for EconoMedRx. J.N. Doctor is a legal consultant for
Motley Rice Law Group.
Funding: The California Health Care Foundation (19413), the National Institute on Aging (NIA) at
the National Institutes of Health (R21-AG057395-01 and R33-AG057395), National Institute on
Drug Abuse (R01 DA046226) and the NIA Roybal Center for Behavioral Interventions
(P30AG024968) support this work.
93
References
1. Lembke A, Papac J. Our other prescription drug problem. The New England [Internet]. 2018;
Available from: https://pubmed.ncbi.nlm.nih.gov/29466163/
2. Hedberg K, Bui LT, Livingston C, Shields LM, Van Otterloo J. Integrating Public Health and
Health Care Strategies to Address the Opioid Epidemic: The Oregon Health Authority’s Opioid
Initiative. J Public Health Manag Pract. 2019;25(3):214–20.
3. U.S. Centers for Medicare & Medicaid Services. CMS Opioid Roadmap: Strategy to Fight the
Opioid Crisis [Internet]. 2020. Available from: https://www.cms.gov/About-CMS/Agency-
Information/Emergency/Downloads/Opioid-epidemic-roadmap.pdf
4. CDC’s response to the opioid overdose epidemic [Internet]. 2021 [cited 2022 Mar 2]. Available
from: https://www.cdc.gov/opioids/strategy.html
5. Weich S, Pearce HL, Croft P, Singh S, Crome I, Bashford J, et al. Effect of anxiolytic and hypnotic
drug prescriptions on mortality hazards: retrospective cohort study. BMJ. 2014 Mar 19;348:g1996.
6. Locke AB, Kirst N, Shultz CG. Diagnosis and management of generalized anxiety disorder and
panic disorder in adults. Am Fam Physician. 2015 May 1;91(9):617–24.
7. Boggs JM, Lindrooth RC, Battaglia C, Beck A, Ritzwoller DP, Ahmedani BK, et al. Association
between suicide death and concordance with benzodiazepine treatment guidelines for anxiety and
sleep disorders. Gen Hosp Psychiatry. 2020 Jan;62:21–7.
8. Katzman MA, Bleau P, Blier P, Chokka P, Kjernisted K, Van Ameringen M, et al. Canadian
clinical practice guidelines for the management of anxiety, posttraumatic stress and obsessive-
compulsive disorders. BMC Psychiatry. 2014 Jul 2;14 Suppl 1:S1.
9. Fluyau D, Revadigar N, Manobianco BE. Challenges of the pharmacological management of
benzodiazepine withdrawal, dependence, and discontinuation. Ther Adv Psychopharmacol. 2018
May;8(5):147–68.
10. Baldwin DS, Anderson IM, Nutt DJ, Allgulander C, Bandelow B, den Boer JA, et al. Evidence-
based pharmacological treatment of anxiety disorders, post-traumatic stress disorder and
obsessive-compulsive disorder: a revision of the 2005 guidelines from the British Association for
Psychopharmacology. J Psychopharmacol. 2014 May;28(5):403–39.
11. Matheson E, Hainer BL. Insomnia: Pharmacologic Therapy. Am Fam Physician. 2017 Jul
1;96(1):29–35.
12. Yan Li NF. Benzodiazepines for the treatment of adults with mental health conditions or sleep
disorders guidelines [Internet]. [cited 2021 Oct 22]. Available from:
https://www.cadth.ca/benzodiazepines-treatment-adults-mental-health-conditions-or-sleep-
disorders-guidelines
13. Agarwal SD, Landon BE. Patterns in Outpatient Benzodiazepine Prescribing in the United States.
JAMA Netw Open. 2019 Jan 4;2(1):e187399.
94
14. Bachhuber MA, Hennessy S, Cunningham CO, Starrels JL. Increasing Benzodiazepine
Prescriptions and Overdose Mortality in the United States, 1996-2013. Am J Public Health. 2016
Apr;106(4):686–8.
15. Kaufmann CN, Spira AP, Depp CA, Mojtabai R. Long-Term Use of Benzodiazepines and
Nonbenzodiazepine Hypnotics, 1999-2014. Psychiatr Serv. 2018 Feb 1;69(2):235–8.
16. Olfson M, King M, Schoenbaum M. Benzodiazepine use in the United States. JAMA Psychiatry.
2015 Feb;72(2):136–42.
17. Cook JM, Marshall R, Masci C, Coyne JC. Physicians’ perspectives on prescribing
benzodiazepines for older adults: a qualitative study. J Gen Intern Med. 2007 Mar;22(3):303–7.
18. Anderson K, Stowasser D, Freeman C, Scott I. Prescriber barriers and enablers to minimising
potentially inappropriate medications in adults: a systematic review and thematic synthesis. BMJ
Open. 2014 Dec 8;4(12):e006544.
19. National Institute on Drug Abuse. Overdose Death Rates [Internet]. 2022 [cited 2022 Mar 29].
Available from: https://nida.nih.gov/drug-topics/trends-statistics/overdose-death-rates
20. Gladden RM, O’Donnell J, Mattson CL, Seth P. Changes in Opioid-Involved Overdose Deaths
by Opioid Type and Presence of Benzodiazepines, Cocaine, and Methamphetamine - 25 States,
July-December 2017 to January-June 2018. MMWR Morb Mortal Wkly Rep. 2019 Aug
30;68(34):737–44.
21. Dowell D, Haegerich TM, Chou R. CDC Guideline for Prescribing Opioids for Chronic Pain -
United States, 2016. MMWR Recomm Rep. 2016 Mar 18;65(1):1–49.
22. Bohnert ASB, Guy GP Jr, Losby JL. Opioid Prescribing in the United States Before and After
the Centers for Disease Control and Prevention’s 2016 Opioid Guideline. Ann Intern Med. 2018
Sep 18;169(6):367–75.
23. Liu S, O’Donnell J, Gladden RM, McGlone L, Chowdhury F. Trends in Nonfatal and Fatal
Overdoses Involving Benzodiazepines - 38 States and the District of Columbia, 2019-2020.
MMWR Morb Mortal Wkly Rep. 2021 Aug 27;70(34):1136–41.
24. Doctor JN, Nguyen A, Lev R, Lucas J, Knight T, Zhao H, et al. Opioid prescribing decreases
after learning of a patient’s fatal overdose. Science. 2018 Aug 10;361(6402):588–90.
25. LA county will notify doctors of patient opioid overdoses in effort to reduce opioid deaths
[Internet]. [cited 2021 Oct 25]. Available from:
https://hahn.lacounty.gov/la_county_will_notify_doctors_of_patient_opioid_overdoses_in_ef
fort_to_reduce_opioid_deaths
26. Haahr M. RANDOM.ORG - sequence generator [Internet]. [cited 2016 Oct 1]. Available from:
https://www.random.org/sequences/
27. Tversky A, Kahneman D. Availability: A heuristic for judging frequency and probability. Cogn
Psychol. 1973 Sep 1;5(2):207–32.
95
28. Controlled substance utilization review and evaluation system [Internet]. 2016 [cited 2021 Oct
25]. Available from: https://oag.ca.gov/cures
29. Borrelli EP, Bratberg J, Hallowell BD, Greaney ML, Kogut SJ. Application of a diazepam
milligram equivalency algorithm to assess benzodiazepine dose intensity in Rhode Island in 2018.
J Manag Care Spec Pharm. 2022 Jan;28(1):58–68.
30. Wooldridge JM. Introductory econometrics: A modern approach: Cengage Learning. A Figures.
2012;18(2).
31. StataCorp LLC. Stata statistical software: Release 16. College Station TX. 2019;
32. R Core Team. R: A Language and Environment for Statistical Computing [Internet]. Vienna,
Austria: R Foundation for Statistical Computing; 2021. Available from: https://www.R-
project.org/
33. SAS Institute Inc. SAS Software, Version 9.4. Cary, NC. 2016;
34. Kroll DS, Nieva HR, Barsky AJ, Linder JA. Benzodiazepines are Prescribed More Frequently to
Patients Already at Risk for Benzodiazepine-Related Adverse Events in Primary Care. J Gen
Intern Med. 2016 Sep;31(9):1027–34.
35. Cook JA, Bruckner T, MacLennan GS, Seiler CM. Clustering in surgical trials--database of
intracluster correlations. Trials. 2012 Jan 4;13:2.
36. Friedman LM, Furberg CD, DeMets DL, Reboussin DM, Granger CB. Sample Size. In: Friedman
LM, Furberg CD, DeMets DL, Reboussin DM, Granger CB, editors. Fundamentals of Clinical
Trials. Cham: Springer International Publishing; 2015. p. 165–200.
37. Kaufmann CN, Spira AP, Depp CA, Mojtabai R. Continuing Versus New Prescriptions for
Sedative-Hypnotic Medications: United States, 2005–2012. Am J Public Health. 2016 Nov
1;106(11):2019–25.
38. Subelj M, Vidmar G, Svab V. Prescription of benzodiazepines in Slovenian family medicine: a
qualitative study. Wien Klin Wochenschr. 2010 Aug;122(15–16):474–8.
39. Giorno RD, Del Giorno R, Greco A, Zasa A, Clivio L, Pironi M, et al. Combining prescription
monitoring, benchmarking, and educational interventions to reduce benzodiazepine prescriptions
among internal medicine inpatients; a multicenter before and after study in a network of Swiss
Public Hospitals [Internet]. Vol. 130, Postgraduate Medicine. 2018. p. 627–36. Available from:
http://dx.doi.org/10.1080/00325481.2018.1504594
40. Gould RL, Coulson MC, Patel N, Highton-Williamson E, Howard RJ. Interventions for reducing
benzodiazepine use in older people: meta-analysis of randomised controlled trials. Br J Psychiatry.
2014 Feb;204(2):98–107.
96
Appendix
Supplementary Text
San Diego County Medical Examiner Office
{Address}
{Phone}
{Email}
Date
Dear ______________(prescriber name),
This is a courtesy communication to inform you that your patient (Name, Date of Birth) died on
(date). Prescription drug overdose was either the primary cause of death or contributed to the death.
The San Diego Medical Examiner’s office sees between 250 and 270 prescription medication-related
deaths each year. A significant proportion of deaths are due to the combination of multiple
prescription medications. Patients may obtain legitimate prescriptions for opioids, benzodiazepines,
muscle relaxants, and sleep aids from more than one prescriber. When taken in any combination,
these medications put patients at greater risk of death. We also see many deaths that are a result of
long-term therapeutic prescribing.
Controlled Substance Utilization Review and Evaluation System (CURES) helps prescribers who are
dedicated to avoiding prescribing controlled substances when they are likely to do more harm than good. CURES
contains information about whether other clinicians had prescribed controlled substances to your
patient. This type of information can help prescribers make informed decisions and avoid duplicate
or additive types of medications from being provided to patients. We ask that you commit to
prescribe safely by registering for and regularly logging in to CURES before prescribing controlled
substances. On the CURES website you may run a report on any patient you are considering
prescribing controlled substances to in order to find their detailed prescription history. CURES data
is available for only the last 12 months for patients.
You can register for CURES at
https://cures.doj.ca.gov/registration/confirmEmailPnDRegistration.xhtml.
You can access CURES at https://cures.doj.ca.gov
The following evidence-based interventions also lower overdose death rates:
1. Avoid co-prescribing an opioid and a benzodiazepine. We found this combination in over 50% of
CURES reports and in over 20% of toxicology results of patients who died of an overdose.
2. Minimize opioid prescribing for acute pain. According to the Centers for Disease Control and
Prevention (CDC), clinicians should avoid opioids, and when necessary, start with the lowest
effective dose of immediate-release opioids. Three days or less will often be sufficient. Opioids
should not be considered first-line or routine therapy for chronic pain.
1
1
Recommendations #1, 6, CDC Guideline for Prescribing Opioids for Chronic Pain, 2016
97
3. Taper opioids to safer doses. The CDC recommends that for patients already on long-term opioid
high dose opioid therapy, taper to a dose that is lower than 50 milligrams of morphine equivalent and
that slow opioid tapers as well as pauses in the taper may be needed for long-term users.
2
4. Avoid “the 90-day cliff.” We found that nearly 70% of patients who died were prescribed the same
medication for 3 consecutive months. The CDC recommends opioids should be discontinued if
benefits do not outweigh risks (if realistic goals for pain and function have not been met).
3
5. The CDC recommends prescribing naloxone to patients on higher than 50 milligrams of
morphine equivalents daily.
4
We are aware of the challenges in balancing the potential harm and benefit of controlled medication
prescribing for your patients. Therefore, please visit:
http://sandiegosafeprescribing.org/
and click the link named “Did you get a letter from the Medical Examiner?” Here you will find links
to the CDC guidelines, local addiction referral resources, including medication-assisted treatment, a
clinical advice hotline, regimens for successful tapering and other information.
Learning of your patient’s death can be difficult. We hope that you will take this as an opportunity
to join us in preventing future deaths from drug overdose.
Sincerely,
Jonathan Lucas, MD
Chief Deputy Medical Examiner
2
Recommendation #5, 7, CDC Guideline for Prescribing Opioids for Chronic Pain, 2016
3
Recommendation # 2, 7, CDC Guideline for Prescribing Opioids for Chronic Pain, 2016
4
Recommendation #8, CDC Guideline for Prescribing Opioids for Chronic Pain, 2016
98
Figure A4.1: Quantile-quantile plot of positive daily DMEs. A plot of quantiles of weekday
daily DME per clinician against quantiles of normal distribution demonstrates that daily DME does
not follow a normal distribution.
Figure A4.2: Quantile-quantile plot of positive log(daily DME). A plot of quantiles of weekday
log(daily DME) per clinician against quantiles of normal distribution demonstrates that log(daily
DME) is similar to a normal distribution.
99
Table A4.1: Log-odds of a greater than 20% reduction in DME from pre- to post-period
(1) (2) (3)
Variables Unadjusted Adjusted Interacted
Received letter 0.29** 0.31* -0.05
(0.15) (0.16) (0.28)
% of patients that are new starts - -0.37 -1.29
- (0.74) (1.01)
Letter * % of patients that are new
starts
- -
-
2.07
- (1.52)
% of patients with opioid co-
prescription
- 0.39 0.32
- (0.89) (1.14)
Letter * % of patients with opioid co-
prescription
- - 0.40
- - (1.87)
Constant -0.48*** -0.39** -0.24
(0.10) (0.16) (0.19)
Observations 743 642 642
Log-likelihood -502.5 -437.8 -436.4
AIC 1009 884 885
BIC 1018 901 912
*** p<0.01, ** p<0.05, * p<0.1
100
Chapter 5
Conclusion
Marcella A. Kelley
5.1 Aims and Hypotheses Revisited
In this dissertation, we analyzed downstream harms of opioid overprescribing, quantified
inappropriate outpatient prescribing for COVID-19, and evaluated the effectiveness of a behavioral
economic intervention in reducing benzodiazepine overprescribing. Overprescribing and
inappropriate prescribing cost financial, staff, and time resources and presents unnecessary health
risks (1). The inappropriate prescribing of opioids and benzodiazepines contributes to increased
rates of opioid and benzodiazepine involved overdoses. Once dependence on the drug has formed,
transition to illicit use and exposure to fentanyl is more probable. Fatal overdoses involving
benzodiazepines or opioids increased in 2020, with fentanyl as a large contributor to both (2,3). The
use of inappropriate outpatient COVID-19 treatments, such as antibiotics, corticosteroids, and
hydroxychloroquine, increases risks of antibiotic resistance, infections, hospitalizations, and
mortality (4–6).
In Aim 1, we found opioid-related deaths increased in 2020 compared to 2019 in Los
Angeles County. Following the onset of COVID-19 related stay-at-home orders, opioid-related
overdoses did not increase overall but increased among Non-Hispanic Caucasians compared to
Hispanic individuals (risk ratio: 1.82 [95% CI, 1.10 to 3.02]; P < 0.05). Possible explanations for the
exacerbation of fatal opioid overdoses among Non-Hispanic Caucasians are the decrease in access to
face-to-face treatment, medications for addiction treatment, and formal support networks, which are
predominantly provided to and utilized more by Non-Hispanic Caucasians (7–9). Additionally, the
lack of significant opioid-related death increases among individuals of American Indian, Asian,
101
Black, Hispanic, or Unknown race or ethnicity could be explained by their higher rates of COVID-
19-related mortality (10). The trends we observed were not explained by recent scheduled drug
prescriptions or tract-level educational attainment.
In Aim 2, we used Optum’s de-identified Clinformatics® Data Mart Database (2007-2020)
to measure the effect of in-person versus telehealth urgent care setting on physician behavior during
the first year of the COVID-19 pandemic. During this time, physicians faced great clinical
uncertainty and rapidly changing treatment guidelines. In response, they exhibited a bias toward
action (i.e. emergency department referrals, inappropriate prescriptions, and COVID-19 test orders)
over inaction (11). At the onset of COVID-19 in the United States, the probability of emergency
department referrals was 3 percentage points higher ([95% CI: 0.01 to 0.06]; P < 0.01), the
probability of an inappropriate prescription was 20 percentage points higher ([95% CI: 0.14 to 0.27];
P < 0.001), and the probability of a COVID-19 test order was 9 percentage points lower ([95% CI:-
0.15 to-0.03]; P < 0.01) in urgent care than telehealth after adjusting for patient and appointment
characteristics. By the fourth quarter of 2020, the adjusted probability of an ED referral did not
differ significantly by setting (P > 0.05), the probability of an inappropriate prescription was 7
percentage points lower ([95% CI: -0.11 to -0.03]; P < 0.01), and the adjusted probability of a test
order was 19 percentage points higher in urgent care than telehealth ([95% CI: 0.18 to 0.21]; P <
0.001). Greater utilization of testing in urgent care settings over time did not explain their decrease
in inappropriate prescribing (P > 0.05), indicating that other factors contributed to the patterns
observed. Patients who received an inappropriate prescription at their index appointment had a 1
percentage point higher adjusted probability of hospitalization within the week ([95% CI: 0.01 to
0.02]; P < 0.001) and a 0.4 percentage point higher adjusted probability of death within 30 days
([95% CI: 0.001to 0.006]; P < 0.01) compared to people without an inappropriate prescription. We
102
identified provider group-level variation in the likelihood of all initial interventions (i.e. emergency
department escalations, inappropriate prescriptions, and test orders).
In Aim 3, we quantified the effect of a letter notifying a clinician of their patient’s fatal
overdose death on the clinician’s benzodiazepine prescribing in a cluster randomized controlled trial
in San Diego County in 2017. Weekday diazepam milligram equivalents (DMEs) filled by patients of
study prescribers decreased 5.6% ([95% CI: -9.1% to -2.0%]; P < 0.05) more in the intervention
than control group pre-to-post intervention. The number of 2 mg pills dispensed decreased3.7%
([95% CI: -6.9% to -0.5%]; P <0.05) more in the intervention group compared to the control group
pre-to-post intervention. There were no significant differences in the probability, number, and
average days supply of “new benzodiazepine starts” from pre-to post-intervention between groups.
The considerable (> 20%) reductions in daily DMEs observed in the treatment group were related
to smaller shares of patients with new benzodiazepine prescriptions or opioid co-prescriptions.
Although benzodiazepines are less emphasized than opioids in the letter’s recommendations, there
was a significant reduction in benzodiazepine prescribing. The notification letter seemed to increase
the salience of harms for opioid and benzodiazepine prescribing among clinicians in the intervention
group, thereby increasing the clinicians’ perceived probability of harms related to their use (12).
5.2 Future Directions
We find evidence of downstream harms of opioid prescribing, low-value outpatient care
during COVID-19, and an effective intervention to reduce inappropriate benzodiazepine
prescribing. Discouraging low-value care and encouraging the rational use of medications requires a
multi-faceted approach. Behavioral economics recognizes that physician decision-making isn’t as
perfectly rational and utility maximizing as neoclassical economics would predict. Instead, when
making complex decisions with uncertainty and time pressure, physician decision-making may be
103
influenced by framing and heuristics (13). As Eddy wrote in 1984, “Uncertainty creeps into medical
practice through every pore. Whether a physician is defining a disease, making a diagnosis, selecting
a procedure, observing outcomes, assessing probabilities, assigning preferences, or putting it all
together, he (or she) is walking on very slippery terrain” (14). Although uncertainty cannot be
eliminated from medical decision-making, it can be acknowledged and reduced.
Uncertainty is not widely acknowledged or accepted in medical care. With a physician's
difficulty to accept uncertainty comes premature decision-making that leaves rooms for biases,
errors, and overtreatment (15). One approach to reduce the harms associated with physicians
struggling to acknowledge uncertainty is to encourage reasoning, exploration, and being more
comfortable with uncertainty in medical education (15). The rapid onset of COVID-19 led to
increased physician uncertainty in making a diagnosis and selecting a treatment. In response, urgent
care and telehealth providers prescribed many inappropriate medications at the onset of the
pandemic. As the pandemic progressed, inappropriate prescribing remained higher in the setting
with less diagnostic certainty, telehealth.
One way to reduce uncertainty in telehealth is through more research regarding what
conditions or symptoms benefit most from an in-person examination. Telehealth experts purport
that most elements of the physical examination can be done via telehealth with proper clinician
training and patient education (16). The accuracy of diagnoses in primary care via telehealth is
comparable to in-person primary care for nonurgent conditions, but more research is needed to
understand the accuracy of diagnoses and the appropriateness of triage advice administered for
urgent conditions (17). Our research contributes to this area by identifying inappropriate prescribing
during COVID-19 in 2020.
In opioid and benzodiazepine prescribing, physician uncertainty regarding the probability of
adverse outcomes influences inappropriate prescribing. As the availability heuristic suggests,
104
increasing a physician’s awareness about opioid- and benzodiazepine-related harms increases their
perceived probability of these adverse outcomes (12,18). This information increases the cost of the
treatment from the physician’s perspective, causing some prescribers to reduce their prescribing.
Our findings are further proof of physician’s perceived probability altering their decision-making
since the true probability of these adverse outcomes was unchanged by the intervention. Since the
original trial, this letter has been adopted by Los Angeles County and is sent to every scheduled drug
prescriber of a person who suffers a fatal overdose (19). With our findings of increased benefits
associated with this low-cost intervention, other jurisdictions may adopt this policy. Future
directions include evaluating the effect of this letter on prescribers to patients who experience a
nonfatal overdose, a trial which is currently underway (20). Further research on identifying and
reducing inappropriate prescribing and its downstream consequences is warranted and welcomed.
105
References
1. Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United
States? 1998. Milbank Q. 2005;83(4):843–95.
2. National Institute on Drug Abuse. Overdose death rates [Internet]. National Institute on Drug
Abuse. 2022 [cited 2022 Apr 28]. Available from: https://nida.nih.gov/drug-topics/trends-
statistics/overdose-death-rates
3. Liu S, O’Donnell J, Gladden RM, McGlone L, Chowdhury F. Trends in Nonfatal and Fatal
Overdoses Involving Benzodiazepines - 38 States and the District of Columbia, 2019-2020.
MMWR Morb Mortal Wkly Rep. 2021 Aug 27;70(34):1136–41.
4. Nonhospitalized Adults: Therapeutic Management [Internet]. [cited 2021 Oct 22]. Available
from: https://www.covid19treatmentguidelines.nih.gov/management/clinical-
management/nonhospitalized-adults--therapeutic-management/
5. Hsu J. How covid-19 is accelerating the threat of antimicrobial resistance. BMJ. 2020 May
18;369:m1983.
6. Crothers K, DeFaccio R, Tate J, Alba PR, Goetz MB, Jones B, et al. Dexamethasone in
hospitalised coronavirus-19 patients not on intensive respiratory support. Eur Respir J [Internet].
2021 Nov 25; Available from: http://dx.doi.org/10.1183/13993003.02532-2021
7. Alexander GC, Stoller KB, Haffajee RL, Saloner B. An Epidemic in the Midst of a Pandemic:
Opioid Use Disorder and COVID-19. Ann Intern Med. 2020;173(1):57–8.
8. Wu LT, Zhu H, Swartz MS. Treatment utilization among persons with opioid use disorder in the
United States. Drug Alcohol Depend. 2016;169:117–27.
9. Narcotics Anonymous World Services Inc. 2018 Narcotics Anonymous Membership Survey
[Internet]. [cited 2021 Feb 16]. Available from:
https://www.na.org/admin/include/spaw2/uploads/pdf/conference/project/2301_MS_2018
_Nov19.pdf
10. CDC. Risk for COVID-19 infection, hospitalization, and death by race/ethnicity [Internet]. 2021
[cited 2021 Oct 22]. Available from: https://www.cdc.gov/coronavirus/2019-ncov/covid-
data/investigations-discovery/hospitalization-death-by-race-ethnicity.html
11. Patt A, Zeckhauser R. Action Bias and Environmental Decisions. J Risk Uncertain. 2000 Jul
1;21(1):45–72.
12. Tversky A, Kahneman D. Availability: A heuristic for judging frequency and probability. Cogn
Psychol. 1973 Sep 1;5(2):207–32.
13. Oakes AH, Radomski TR. Reducing Low-Value Care and Improving Health Care Value. JAMA.
2021 May 4;325(17):1715–6.
14. Eddy DM. Variations in physician practice: the role of uncertainty. Health Aff . 1984
Summer;3(2):74–89.
106
15. Simpkin AL, Schwartzstein RM, Others. Tolerating uncertainty—the next medical revolution. N
Engl J Med. 2016;375(18):1713–5.
16. Telediagnosis for acute care: Implications for the quality and safety of diagnosis [Internet]. [cited
2022 Apr 28]. Available from: https://www.ahrq.gov/patient-safety/reports/issue-
briefs/teledx-3.html
17. Telediagnosis for acute care: Implications for the quality and safety of diagnosis [Internet]. [cited
2022 Apr 28]. Available from: https://www.ahrq.gov/patient-safety/reports/issue-
briefs/teledx-2.html
18. Doctor JN, Nguyen A, Lev R, Lucas J, Knight T, Zhao H, et al. Opioid prescribing decreases
after learning of a patient’s fatal overdose. Science. 2018 Aug 10;361(6402):588–90.
19. LA county will notify doctors of patient opioid overdoses in effort to reduce opioid deaths
[Internet]. [cited 2021 Oct 25]. Available from:
https://hahn.lacounty.gov/la_county_will_notify_doctors_of_patient_opioid_overdoses_in_ef
fort_to_reduce_opioid_deaths
20. Kelley MA, Persell SD, Linder JA, Friedberg MW, Meeker D, Fox CR, et al. The protocol of the
Application of Economics & Social psychology to improve Opioid Prescribing Safety Trial 1
(AESOPS-1): Electronic health record nudges. Contemp Clin Trials. 2021 Apr;103:106329.
Abstract (if available)
Abstract
Inappropriate prescribing and overprescribing are examples of low- or no-value care that result in high costs with little to no clinical benefit and patient harms. Suboptimal prescribing is the result of suboptimal physician decision-making. Physician behavior is influenced by intrinsic and extrinsic factors, such as reimbursement models, patient demand, diagnostic uncertainty, and poor numeracy. When faced with numerous diagnostic and treatment decisions per day, physicians rely on mental shortcut, or “heuristics”, that unconsciously alter their perception of the risks and benefits of a treatment. While overtreatment or inappropriate prescribing occurs in many conditions, this dissertation concentrates on opioids, COVID-19 treatments, and benzodiazepines.
The three aims of this dissertation include: (1) the downstream harms of opioid overprescribing during COVID-19; (2) the impact of setting on physician behavior in treating COVID-19 outpatients and associated outcomes; and (3) the effectiveness of a behavioral economic intervention on inappropriate benzodiazepine prescribing. We primarily address these aims using LA County Department of the Medical Examiner-Coroner autopsy reports, California Controlled Substance Utilization Review and Evaluation System data, and claims data (Optum’s de-identified Clinformatics® Data Mart Database (2007-2020)).
We identify community- and decedent-level characteristics associated with opioid-related deaths following the implementation of stay-at-home orders in Los Angeles County. We estimate if the likelihood of initial provider interventions for COVID-19, including inappropriate prescribing, differs by appointment setting (i.e., urgent care center versus dedicated telehealth company) and if inappropriate prescribing for COVID-19 is associated with adverse outcomes (i.e., hospitalizations and mortality). Lastly, we measure the effect of a behavioral economic intervention in reducing benzodiazepine prescribing in a secondary analysis of a randomized controlled trial.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Assessing value defects in limb preservation care
PDF
Essays on opioid prescribing and abatement
PDF
Evaluating approaches to reduce inappropriate antibiotic use in the United States
PDF
Impact of medication persistence on clinical and health care cost outcomes in patients with major depressive disorder using retrospective claims data
PDF
Evaluating treatment options for metastatic, castration-resistant prostate cancer: a comprehensive value assessment
PDF
Outcomes of antibiotic use among children with acute respiratory tract infections
PDF
Essays in health economics
PDF
Opioid withdrawal symptoms, opioid use, and injection risk behaviors among people who inject drugs (PWID)
PDF
Value in oncology care and opportunities for improvement
PDF
Essays in opioid use and abuse
PDF
The impact of healthcare interventions using electronic health records: an evaluation within an integrated healthcare system
PDF
A qualitative study of street fentanyl in Dayton, Ohio: drug markets, trajectories, and overdose risk reduction
PDF
A multi-system evaluation of medication for opioid use disorder for people who use opioids
PDF
Crohn’s disease: health outcomes and resource utilization in the biologic era
PDF
Physician burnout in the COVID-19 pandemic: Healthcare organization and leadership implications for patient care and clinical team leadership with nurses
PDF
Three essays on emerging issues in hemophilia care
PDF
Essays in health economics and provider behavior
PDF
Neurodivergent-affirming care within applied behavior analysis
PDF
Designing infectious disease models for local level policymakers
PDF
Economic, clinical, and behavioral outcomes from medical and pharmaceutical treatments
Asset Metadata
Creator
Kelley, Marcella Austin
(author)
Core Title
Physician behavior and low-value care
School
School of Pharmacy
Degree
Doctor of Philosophy
Degree Program
Health Economics
Degree Conferral Date
2022-08
Publication Date
07/25/2022
Defense Date
06/01/2022
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
antibiotic overuse,behavioral economics,benzodiazepine overuse,Health Economics,low-value care,OAI-PMH Harvest,opioid overuse
Format
application/pdf
(imt)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Doctor, Jason (
committee chair
), Meeker, Daniella (
committee member
), Padula, William (
committee member
)
Creator Email
makelley@usc.edu,marcellaakelley@gmail.com
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-oUC111375328
Unique identifier
UC111375328
Legacy Identifier
etd-KelleyMarc-10975
Document Type
Dissertation
Format
application/pdf (imt)
Rights
Kelley, Marcella Austin
Type
texts
Source
20220728-usctheses-batch-962
(batch),
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given.
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Repository Email
cisadmin@lib.usc.edu
Tags
antibiotic overuse
behavioral economics
benzodiazepine overuse
low-value care
opioid overuse