University of Southern California Dissertations and Theses

Challenges in the implementation of Risk Evaluation Mitigation Strategies (REMS): a survey of industry views

(USC Thesis Other)

Challenges in the implementation of Risk Evaluation Mitigation Strategies (REMS): a survey of industry views

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

CHALLENGES IN THE IMPLEMENTATION OF RISK EVALUATION
MITIGATION STRATEGIES (REMS): A SURVEY OF INDUSTRY VIEWS

by

Alicia M. McDowell

A Dissertation Presented to the
FACULTY OF THE USC SCHOOL OF PHARMACY
UNIVERSITY OF SOUTHERN CALIFORNIA
In Fulfillment of the
Requirements for the Degree
DOCTOR OF REGULATORY SCIENCE

December 2018

Copyright 2018 Alicia M. McDowell

2
DEDICATION
I dedicate this dissertation to my family. To my parents Ken and Jeannie who always
supported me and stressed the importance of education. To my husband Jim, for his
patience, understanding, love and support during this process. To my step children,
Nicholas, Isabella and Alexandra, for your encouragement to “keep going”, I hope I can
provide you all the support you need to achieve your goals. To my Ava James and new
baby to be, by far, my greatest accomplishments, you inspire me every day and teach me
more than I could ever learn in a formal setting. You’re never too old to learn!

3
ACKNOWLEDGEMENTS
I would like to thank my thesis supervisor, Dr. Eunjoo Pacifici, Pharm D, PhD, for all of
her guidance, encouragement, friendship and hospitality. Also for her patience in
reviewing so many iterations of this dissertation while I navigated life’s journey. Her
encouragement helped me complete this chapter of my life. I would also like to thank Dr.
Frances Richmond, PhD for all of her guidance and support and additional time spent
with me to get me moving in the right direction. This program and what I have learned is
invaluable. I would like to thank Dr. Nancy Smerkanich, DRSc for all of the help she
provided with reviewing my documents, providing me relevant publications and
assistance with Qualtrics. To my other thesis committee members, Dr. Benson Kuo,
PhD, and Dr. Ellen Whalen, RN, DRSc for the invaluable feedback that they provided
throughout the process. I would also like to acknowledge the help that Randa Issa, PhD
and Erin Chow have provided me to navigate all of my administrative hurdles. I would
also like to thank Beatriz Rocha, MD, PhD – my boss, my mentor, my friend – for all of
your support in my career, educational and personal goals. Your guidance and
mentorship have helped me through this process. In closing, I would like to thank the
other students in the 2012 Doctoral Cohort and the staff of the Regulatory Science
program for their support professionally and personally and for making this such an
enjoyable endeavor.

4
TABLE OF CONTENTS
DEDICATION .................................................................................................................... 2
ACKNOWLEDGEMENTS ................................................................................................ 3
TABLE OF CONTENTS .................................................................................................... 4
LIST OF TABLES .............................................................................................................. 8
ABSTRACT ...................................................................................................................... 11
CHAPTER 1. OVERVIEW ...................................................................................... 13
1.1 Introduction .................................................................................................. 13
1.2 Risk Management History ........................................................................... 14
1.3 Establishment of REMS ............................................................................... 16
1.4 Statement of the Problem ............................................................................. 18
1.5 Purpose of the Research ............................................................................... 19
1.6 Delimitations, Limitations, Assumptions ..................................................... 19
1.7 Organization of Thesis ................................................................................. 20
CHAPTER 2. LITERATURE REVIEW .................................................................. 22
2.1 Introduction .................................................................................................. 22
2.2 Evolution of Pharmacovigilance .................................................................. 22
2.2.1 Elixir of Sulfanilamide ................................................................... 23
2.2.2 Abuse of Amphetamines ................................................................ 24
2.2.3 Thalidomide Tragedy ..................................................................... 25
2.3 WHO Development of a Pilot Reporting System for Adverse Events ........ 27

5
2.4 CIOMS Program and ICH Regulations Approach to Pharmacovigilance ... 28
2.5 Development of European Union Regulations and the Risk
Management Plan ......................................................................................... 30
2.5.1 EU Risk Management Approaches ................................................. 32
2.6 Development of US Regulations .................................................................. 35
2.6.1 Fen-Phen and PDUFA III ............................................................... 35
2.6.2 Vioxx .............................................................................................. 37
2.6.3 FDAAA of 2007 and the Initiation of REMS ................................. 40
2.6.4 REMS Elements ............................................................................. 42
2.6.5 Assessment of REMS ..................................................................... 47
2.6.6 Modifying existing REMS .............................................................. 48
2.7 REMS Challenges ........................................................................................ 49
2.7.1 OIG Report Findings and Recommendations ................................. 51
2.8 Addressing REMS Challenges ..................................................................... 53
2.8.1 REMS Integration Initiative ........................................................... 54
2.8.2 FDA Priority Projects ..................................................................... 55
2.8.3 Measuring Effectiveness of REMS ................................................ 58
2.9 Where does industry stand? ......................................................................... 60
2.9.1 Studying industry views on REMS Implementation ...................... 60
2.9.2 Research Framework ...................................................................... 63

6
2.10 Additional Efforts and Issues with REMS Since the Launch of the
Survey .......................................................................................................... 65
CHAPTER 3. METHODOLOGY ............................................................................ 69
3.1 Introduction .................................................................................................. 69
3.2 Development of the Initial Survey ............................................................... 69
3.3 Survey Modification by Focus Group Input ................................................ 70
3.4 Survey Deployment and Analysis ................................................................ 71
CHAPTER 4. RESULTS .......................................................................................... 73
4.1 Analysis of Survey Results .......................................................................... 73
4.2 Profiles of Respondents ............................................................................... 73
4.3 How companies modify their REMS Programs – Innovation and
Sustainability ................................................................................................ 84
4.4 Views on Transparency ................................................................................ 86
4.5 Collaboration on innovation and sustainability............................................ 97
4.6 Harmonization/Degree of Burden on Innovation and Sustainability ......... 101
4.7 Modifying REMS ....................................................................................... 117
CHAPTER 5. DISCUSSION .................................................................................. 122
5.1 Industry Views on Implementation of REMS ............................................ 122
5.2 Methodological Considerations ................................................................. 122
5.3 Consideration of Results ............................................................................ 128
5.3.1 Transparency ................................................................................ 128

7
5.3.2 Collaboration Across the Key Stakeholders ................................. 132
5.3.3 Harmonization/ Consistency ......................................................... 133
5.3.4 Other Areas of Concern ................................................................ 136
5.3.5 Modifying REMS ......................................................................... 137
5.4 Conclusions and Future Directions ............................................................ 138
REFERENCES ............................................................................................................... 139
APPENDIX A. FINAL VERSION OF SURVEY ......................................................... 148

8
LIST OF TABLES
Table 1. Modules and Parts of RMP .............................................................................33
Table 2. REMS Elements and Descriptions ..................................................................43
Table 3. FDA REMS Priority Projects .........................................................................57
Table 4. Research Framework ......................................................................................64
Table 5. Focus Group Participants ................................................................................70
Table 6. Comments on Which Department has Responsibility for Developing
REMS .........................................................................................................78
Table 7. Comments for Other Activities in regard to REMS .......................................81
Table 8. Other Therapeutic Areas .................................................................................82
Table 9. “Other” Documents Consulted When Modifying REMS ...............................86
Table 10. Statements on Transparency ...........................................................................87
Table 11. Comments from Respondents that Answered “Yes” ......................................89
Table 12. Comments from Respondents that Responded “No” ......................................90
Table 13. Comments from Respondents that Responded “Yes” ....................................91
Table 14. Comments from Respondents that Responded “No” ......................................92
Table 15. Comments from Respondents that Responded “Yes” ....................................94
Table 16. Comments from Respondents that Responded “No” ......................................94
Table 17. Comments from Those that Responded “Yes” ...............................................96
Table 18. Comments from Respondents That Responded “Yes” ...................................98
Table 19. Comments from Respondents That Responded “No” ....................................98
Table 20. Statements on Communication with Stakeholders when REMS are
Modified or Rescinded ...............................................................................99

9
Table 21. Statements on Collaboration with Stakeholders ...........................................100
Table 22. Statements on Regular Communication with Stakeholders ..........................101
Table 23. Biggest Overall Challenge With REMS Programs .......................................102
Table 24. Statements on Consistency Across REMS ...................................................106
Table 25. Weighted Calculation of Rank Order of Most Burden on Healthcare
Delivery System .......................................................................................108
Table 26. Results of Rank Order for Individual Statements .........................................108
Table 27. “Other” Comments on Most Burden on Healthcare Delivery System .........109
Table 28. Comments on Greatest Barriers to Developing Harmonized REMS ...........110
Table 29. Feasibility of Harmonization ........................................................................113
Table 30. Comments on feasibility to harmonize REMS programs across
therapeutic areas.......................................................................................114
Table 31. Comments on feasibility to harmonize REMS within a therapeutic
area ...........................................................................................................114
Table 32. Comments on feasibility to harmonize REMS within a product class .........115
Table 33. Additional Initiatives that May Help Harmonization ...................................116
Table 34. Other initiatives that would help harmonize REMS .....................................116
Table 35. First Priority to Change the Way REMS are Structured ...............................119
Table 36. Additional Comments on REMS Experience ...............................................121

LIST OF FIGURES
Figure 1. Size of Company .............................................................................................74
Figure 2. Number of REMS Programs Handled by Respondents ..................................75

10
Figure 3. Roles of Respondents .....................................................................................76
Figure 4. Department Responsible for REMS ...............................................................77
Figure 5. Number of People Working on a REMS at the Company ..............................79
Figure 6. Activities with REMS .....................................................................................80
Figure 7. REMS Therapeutic Areas ...............................................................................82
Figure 8. Types of Molecules .........................................................................................83
Figure 9. Elements of REMS Worked on by Respondents ............................................84
Figure 10. Documents Consulted When Modifying REMS ............................................85
Figure 11. FDA Provides Adequate Direction on How to Sustain or Modify
REMS .........................................................................................................88
Figure 12. Regulators are Transparent with Regard to REMS ........................................91
Figure 13. FDA Provides Clear Guidance on how a Program Can Be Rescinded ..........93
Figure 14. Have You Been Directly Involved with a REMS Program That Has
Been Rescinded? ........................................................................................95
Figure 15. Was Clear Rationale for Rescission Provided? ..............................................96
Figure 16. Communicates Regularly with Stakeholders ..................................................97
Figure 17. Weighted Calculation of Rank Order of Most Burden on Healthcare
Delivery System .......................................................................................107
Figure 18. Should REMS be Kept As Is, Modified, or Eliminated ...............................118

11
ABSTRACT

It has taken over a hundred years and a number of high profile tragic events for the US to
establish the current risk management framework for drugs and biologics. In 2007, the
Food and Drug Administration Amendments Act (FDAAA) was passed by Congress
requiring sponsors of New Drug Applications (NDAs), Abbreviated New Drug
Applications (ANDAs), or Biologics License Applications (BLAs) to submit and
implement a formalized Risk Evaluation Mitigation Strategy (REMS) if FDA determines
that special controls are necessary to ensure safe use of a product. After almost a decade,
REMS programs have been criticized for imposing heavy burden on the healthcare
system without demonstrating that it is in fact mitigating risks as intended. Hence, FDA
has committed to developing strategies to assess, standardize and improve REMS and has
been gathering input from various stakeholders, including patients, healthcare providers,
and pharmacies on the burden of REMS on healthcare delivery systems and patient
access. Up to now, however, input from another principal stakeholder, the industry that
is tasked to design, implement and pay for the REMS activities, seems to be lacking. The
purpose of this study was to explore the views of the biopharmaceutical industry utilizing
a framework based on implementation science. The research presented here extends our
understanding of the views of industry with regard to the innovation and sustainability
stages of REMS implementation. Survey methods were used to examine the views of
respondents who are experienced in complex REMS programs. The primary data
analysis was conducted on responses from 41 participants, 3 of whom indicated they had

12
not worked on a REMS and were taken to the end of the survey. Results showed that
REMS are recognized as serving a purpose in the risk management of high risk products,
but the program needs to be modified. Factors that appear to contribute to the thought
that REMS need to be modified are gaps in program transparency, inconsistencies among
REMS programs, and the lack of a framework to assess the effectiveness of a REMS
program.

13
CHAPTER 1. OVERVIEW
1.1 Introduction
Medications are widely used in our society for a myriad of conditions. Through their
pharmacologic actions, medications have improved people’s lives by helping them feel or
function better or live longer. However, along with these beneficial effects can be risks.
Many drugs produce undesirable effects that require medical intervention or, even cause
death. Therefore, identifying and mitigating risks associated with medication use are
central to the role of the United States Food and Drug Administration (FDA), whose
mandate is to protect public health. The FDA evaluates safety of a drug throughout its life
cycle, from the moment it enters clinical trials to the point where it is no longer on the
market. This continued scrutiny is important because the safety profile of a drug can only
be understood with time and experience. During clinical trials, adverse events are
tracked and quantified as a key part of the dataset that will be used in a New Drug
Application (NDA) or Biologics License Application (BLA) to seek marketing
authorization. The FDA must then make a decision about the acceptability of the drug by
assessing its therapeutic benefit relative to its safety risk. However, this assessment is
based on limited information obtained from a relatively small population of trial
participants studied under tightly controlled conditions. It is not until the product has
been on the market and used more widely that the understanding of the risk-benefit
profile of a product becomes better defined. Thus, FDA has a strong post-marketing
surveillance system for collecting and compiling the most accurate and up-to-date
information that can be used to monitor and assess a product’s safety profile.

14
Some medications inherently pose higher safety risks than others. For example,
chemotherapeutic products typically have cytotoxic actions that can kill healthy as well
as fast-growing cancer cells, leading to severe side effects that require close management.
Generally, medications declare their risks through approved product labeling. This
information, referred to in the United States (US) as the Prescribing Information, is based
on the totality of data submitted in support of an NDA/BLA and can be modified later
based on post-marketing experience. The content of the product labeling is the subject of
negotiations between the sponsor and the FDA during the marketing application review
process to assure that it provides the information necessary to assure the safest possible
use of the drug. However, some products may pose risks that cannot be managed by the
Prescribing Information alone. These products would benefit from additional measures,
such as programs for patients and health care providers to educate them on the specific
risks associated with the product and controls to ensure that the products are distributed
and dispensed under restricted conditions. Such measures make it possible, by adding
additional ways to minimize risks, to justify the commercialization of a harmful product
that has important therapeutic use.
1.2 Risk Management History
It took over a hundred years and a number of high profile tragic events for the US to
establish the current risk management framework for drugs and biologics. The Bureau of
Chemistry in the Department of Agriculture first exerted authority over drug safety with
the passage of the 1906 Pure Food and Drug Act. This law required manufacturers to
label products in a way that would identify its contents truthfully. It further introduced

15
the concepts of misbranding and adulteration of drugs, which are still in use today. The
law, however, did not require a company to demonstrate the product’s safety or efficacy,
allowing harmful products to remain on the market and forcing the public to fend for
themselves under a “buyer beware” culture (Food and Drug Administration, 2015b).
Hence, the public had no assurance that the drugs on the market were safe or effective.
Three more decades and a high profile tragic event finally convinced the US Congress to
pass a landmark legislation that initiated FDA’s transformation into the premier
regulatory body that it is today. The Food, Drug, and Cosmetic (FD&C) Act of 1938
repealed and replaced the original 1906 law and is the basis of the modern food and drug
regulatory framework. It introduced the concept of premarket review by requiring
manufactures to submit safety data to the FDA for its review prior to commercialization.
This was an important first step toward establishing FDA’s science based decision-
making process and its role in mitigating risks associated with new drugs.
Between 1940 and 1970, additional laws were passed to amend the FD&C Act and
strengthen FDA’s ability to protect public health. One such amendment, the Durham-
Humphrey Amendment, required that drugs with high safety risks be acquired only
through a physician’s prescription (Food and Drug Administration, 2015b). This new law
aimed to curb uncontrolled distribution and administration of medications that are habit-
forming or potentially dangerous by placing physicians and pharmacists as professional
intermediaries to provide additional oversight and mitigate safety risks. On the heels of
another tragedy, the Kefauver-Harris Amendments of 1962 further strengthened the
Food, Drug and Cosmetic Act by requiring pharmaceutical companies to demonstrate

16
safety and efficacy of a new drug through clinical trials; and to report serious adverse
events so that safety signals could be detected quickly, and risks managed effectively.
Nonetheless, it became evident as experience was gained with the new laws and
regulations that risks posed by new and old drugs could still be better managed.
Therefore, when discussions took place in 2002 for the third reauthorization of the
Prescription Drug User Fee Act (PDUFA), there was strong support to include provisions
for greater surveillance of drug safety. Accordingly, FDA was mandated to allocate user-
fee funds to support post-marketing safety-related activities, including collecting and
analyzing data from real world experience; and developing a risk management plan for
high-risk medications. This new initiative introduced new tools including education and
outreach. However, with the new risk management program, called RiskMAPs, FDA
could only make recommendations regarding post marketing studies, label changes, or
risk communications (Nicholson, 2012). FDA still lacked the authority to require
sponsors to develop a more systematic risk management strategy for their products.
1.3 Establishment of REMS
As high-profile safety concerns continued to emerge, it became evident that a deeper
understanding of the safety of a product is obtained only after it has been on the market.
Hence, politicians and regulators were pressured to develop a better system to monitor,
update, and manage safety risks of a product after market approval. This more systematic
approach was seen to not only identify risks associated with drugs with dangerous side-
effects but also communicate them more effectively to patients and healthcare providers
who could then help to prevent harm from these products. To effect the changes, the

17
fourth reauthorization of PDUFA, the Food and Drug Administration Amendments Act
(FDAAA), passed by Congress in 2007, required applicants submitting New Drug
Applications (NDAs), Abbreviated New Drug Applications (ANDAs), or Biologics
License Applications (BLAs) to include a formalized Risk Evaluation Mitigation Strategy
(REMS) if FDA determines that special controls are necessary to ensure that the benefits
of a drug continue to outweigh its risks. REMS could also be required for a drug that was
already marketed, if FDA were to become aware of new safety concerns that are
associated with its use.
The elements of a REMS program escalate with the extent to which control was judged to
be necessary to minimize a particular drug risk. Most simply, it could involve providing
patients with a medication guide whose information might help them to avoid serious
adverse events. At the other extreme, it could include a range of interventions, described
collectively as Elements to Assure Safe Use (ETASU) that might include
training/certification programs for prescribers, pharmacies, practitioners, or healthcare
settings as well as necessary testing for patients. The REMS programs also included
implementation and monitoring requirements that could be quite extensive. For
example, it could require the sponsor to ensure that all wholesalers and distributors are
also enrolled in the REMS program and that they maintain a validated database including
every certified healthcare facility. The sponsor might also be required to monitor and
review data relating to patient enrollments and product distribution to assess ongoing
compliance.

18
Companies have some latitude to design their REMS programs, so considerable
variability exists across products. Thus, as a result of experience with more than 260
products between 2007-2016, it became apparent that the differing and often duplicative
approaches of companies to their individual products led to some level of confusion
among physicians, pharmacists and patients. Thus, the 2012 PDUFA V reauthorization
of the Prescription Drug User Fee Act (PDUFA) directed the FDA to explore strategies to
assess, standardize and improve the REMS programs.
1.4 Statement of the Problem
It has been a decade since the inception of the REMS program. During this time, over
260 products have been subjected to a REMS program and the experience has provided
insights into its impact on the healthcare system. We know that REMS requirements
represent significant time and effort. In particular, the most rigorous of REMS, the
ETASU program is considered to be extremely burdensome for sponsors, healthcare
providers and pharmacies. Without clear evidence that REMS programs are effective,
concerns were raised that they may actually discourage physicians from prescribing
products with extensive REMS requirements and deny patients access to medications that
would benefit them. According to one physician, a complicated REMS program for a
product is viewed as a “kiss of death” that can lead to a dramatic drop in its use (A.
Jennings, personal communication, December 6, 2016). Thus, FDA has been tasked with
examining the effectiveness of REMS programs by convening public meetings and
workshops where stakeholders can provide their experience and comments. The
outcomes from these activities confirm that REMS programs impose a heavy burden on

19
the healthcare system, without clear evidence that they are in fact mitigating risks as
initially intended. At the same time, however, it is unclear what challenges are present
for sponsors who are responsible for developing, implementing and assessing REMS
programs. Their input has been particularly lacking. For example, we do not understand
if the sponsors are asked for their input as the FDA is considering modifying the REMS
program; and also, if and how the large amounts of data being collected by the sponsors
in the REMS process are being used. According to the FDA, REMS should not represent
undue burden to the patients, healthcare professionals, or the healthcare system but there
is little reference to the sponsor’s views (Food and Drug Administration, 2009a).
1.5 Purpose of the Research
The purpose of this study was to explore the views of the biopharmaceutical industry on
the stages implementation across elements of transparency, collaboration and
harmonization/degree of burden of FDA’s REMS. The study used a novel survey
instrument to assess how the industry’s perspectives affect the development of a REMS
program and identify potential gaps in the program development process. In this context,
the study further explored whether industry believes that FDA’s programs are meeting
their objectives of providing patients with controlled but adequate access to medications
with heightened safety issues.
1.6 Delimitations, Limitations, Assumptions
This study is delimited to industry professionals who have had experience with REMS
programs, which may narrow the potential respondent pool. Another delimitation is the

20
point in time in which the survey is taken such that its relevance must be viewed in the
context of the current regulatory and clinical environment.
Several limitations must also be considered as part of this research. Only a limited
number of products are subject to REMS and this survey was further limited to
respondents with experience with mature REMS programs. Because this study was
intended to survey perspectives from industry professionals, identifying respondents with
direct experience with a REMS and their implementation was challenging. Potential
responders are likely to be busy individuals and, hence, it was challenging to obtain
sufficient participation from a relatively narrow population of survey candidates. The
survey tool may have been limited in its ability to solicit feedback from different
professional perspectives in a comparable manner. Another limitation is the length of the
survey. Limiting the number of questions that the participant needs to respond to helps to
ensure completion of the survey but also limits the scope of data that could be collected
for analysis. Using a focus group to critique and help construct the survey provided
valuable input from various perspectives.
It is assumed that the responders are honest in their responses to questions about current
or previous experience with REMS.
1.7 Organization of Thesis
Chapter 1 introduces the problem to be explored in this thesis and explains the approach
to be taken in the research. Chapter 2 provides background to the problem through a
literature search. The search though literature traces the history of risk management and
pharmacovigilance as shaped by relevant events and shows the evolution of risk

21
management to the REMS system in place today. Furthermore, it shows the issues with
the current REMS system and frames the basis for the research for this thesis. Chapter 3
describes the methodology by which the survey was developed and conducted. The
manner in which the survey was be deployed is also discussed. Chapter 4 describes the
results of the study in the form of a narrative, tables, and figures. Chapter 5 discusses the
implications of the research. It interprets the results in the context of new literature and
includes conclusions and recommendations for further research.

22
CHAPTER 2. LITERATURE REVIEW
2.1 Introduction
“Where no counsel is, the people fall; but in the multitude of counsellors there is safety”
(Proverbs 11:14, English Standard Version).
This two-thousand-year-old quote retains its relevance today. The safety of modern
medicine requires the coordinated efforts of many individuals and groups - regulators,
sponsors of drug development, health care providers, pharmacists and patients. New
drugs have dramatically improved the health of our society, with new treatments that
have increased the quality of life and have extended the lifespan of most the world’s
population. That said, drugs also have risks to users. A key set of tools for
understanding those risks contribute to a branch of regulatory science called
pharmacovigilance, a science directed at detecting, assessing, intervening and, where
possible, preventing end-user harm from the use of medications (Fischhoff, 2011)
2.2 Evolution of Pharmacovigilance
The practice of pharmacovigilance has emerged as a reaction to a series of tragedies that
starkly illustrated the need for continued safety oversight across the lifecycle of a
product. One of the earliest documented needs to systematize the handling of adverse
events occurred in 1848 when a woman died while undergoing a surgery for an ingrown
toenail. Her death was suspected to be caused by ventricular fibrillation induced by the
chloroform used for anesthesia (Routledge, 1998). The concern over the safety of agents
used for anesthesia led to the establishment of a special commission, which then invited
physicians in Britain and its colonies to report anesthesia-related deaths, as a way to

23
gather information needed to understand the risks associated with anesthesia. The
findings were subsequently published in The Lancet in 1893 and disseminated to alert
other physicians.
2.2.1 Elixir of Sulfanilamide
The adverse event reporting process established for incidents associated with chloroform
was relatively informal and reactionary. It was clearly insufficient to deal with the
tragedy that occurred in 1937, when an elixir of sulfanilamide formulated in diethylene
glycol led to the death of over a hundred people, including many children. Sulfanilamide
is the active metabolite of sulfonamidochrysodine, which was the first sulfa antibiotic
introduced in the 1930s. Not readily soluble in water, sulfanilamide provided a special
challenge for those attempting to develop a liquid dosage formulation for children
(Jarrell, 2012). At the time when sulfanilamide entered the market, FDA did not require
the company to provide any assurances that the new product would be safe. Chemists at
the manufacturer, Massingill, therefore decided to use diethylene glycol as the solvent
because of its ability to dissolve the active substance and its slightly sweet taste.
However, they were unaware that diethylene glycol is deadly to humans. As a result, the
new product was distributed extensively throughout the United States before reports
began to identify alarming incidents in which many children who were treated for sore
throats experienced convulsions, pain, nausea, and kidney failure (Jarrell, 2012).
Because the FDA lacked a mechanism to gather adverse event information in a way that
would allow the agency to establish a correlation between the adverse event and the
causative product, it was impossible to respond quickly. By the time FDA and the
manufacturers took action, over 100 people, many of them children, had already died.

24
This tragedy prompted public outcry, pressuring the Congress and the President to take
action. Shortly thereafter, a comprehensive new law, the Food, Drug and Cosmetic Act
(FD&C Act), was passed in 1938. Under this Act, the manufacturer was required to label
a new drug with adequate directions for safe use and to submit evidence of safety as part
of a preview process. A new drug application (NDA) required data on the product’s
ingredients, methods of assay and quality standards, formulation and manufacturing
process, pharmacology and toxicology studies in animals, tissues, or cell cultures, and
clinical trials on human subjects.
2.2.2 Abuse of Amphetamines
Even as the FD&C Act was being operationalized and enforced, it was almost
immediately challenged by risks associated with a new class of compounds called
amphetamines. Amphetamines were first patented for nasal congestion and then further
developed and marketed for various other conditions including narcolepsy, Parkinsonism,
and minor depression. During World War II, the popularity of amphetamines grew to
such an extent that even the US military provided amphetamines to its soldiers.
Amphetamines were also becoming popular among women for weight loss. By 1945, the
consumption rate of amphetamines in the Unites States was at an all-time high. The
national (civilian) consumption rate for the United States in 1945 was sufficient to supply
half a million Americans with two tablets daily. However, the addictive qualities of
amphetamines quickly gave rise to abuse and dependency (Rasmussen, 2008). At the
same time, barbiturates, another class of addictive drugs was gaining popularity as
hypnotics, anxiolytics, anticonvulsants, anesthetic inducers, and treatment for headaches,
migraines, and other types of pain, despite early evidence of its dependence and abuse.

25
It rapidly became clear that amphetamines and barbiturates required more rigorous
controls. Some states attempted to manage the distribution of specific products, but their
efforts were typically inadequate and inconsistent. Eventually, Congress enacted the
Durham-Humphrey Amendments in 1951 to separate drugs that required oversight by
health professionals from those that could be safely marketed over the counter (OTC)
based on their levels of risk. Those drugs that did require oversight, because of their
propensity for dependence or narrow therapeutic index, for example, were restricted to
sale only by the prescription of a licensed practitioner (Food and Drug Administration,
2015c).
2.2.3 Thalidomide Tragedy
Although safety information was required by the FD&C Act, the standards for that data
were not well defined. The quality of evidence considered sufficient to demonstrate
safety of a new drug came under scrutiny in the 1960s when the existing US and
European systems for evaluation, largely done through pre-market testing in animals and
humans, were challenged by another tragedy. In October 1957, a new synthetic drug
thalidomide, was introduced as a sedative and tranquilizer in West Germany by the
company Grünenthal. Soon it was on the pharmacy shelves in over 50 countries (Tuffs,
2007). During the aggressive marketing campaign for thalidomide, patients were told
that it could be “given with complete safety to pregnant women and nursing mothers
without any adverse effect on mother and child”. During the four years that it was on the
market, many doctors prescribed it to pregnant women as a nontoxic treatment for
morning sickness and sleeplessness (Williams, 2012). By early 1959, evidence started to
surface that thalidomide had associated toxicities such as peripheral neuritis, evidence

26
that was suppressed for many years. It was not until 1961 that an Australian doctor and a
German doctor separately made the association between pregnant women taking
thalidomide and giving birth to deformed babies. It took another four months to
withdraw the drug from the market even after the correlation between the drug and
adverse event was made. By then, over 100,000 pregnant women were affected by
thalidomide; the drug appears to have caused at least 90,000 miscarriages and the birth of
over 10,000 deformed babies (Williams, 2012).
Meanwhile in the US, thalidomide was never approved for sale. Dr. Frances Oldham
Kelsey, an FDA medical officer, refused to approve thalidomide despite its widespread
use and strong pressure for marketing authorization from the US manufacturer. Dr.
Kelsey challenged the efficacy of the sleeping pill for two reasons. First, she was
concerned that the drug did not appear to cause sleepiness in animals. More significantly,
she recognized that the drug might have safety issues, because of an English study that
documented peripheral neuritis as a side effect on the nervous system, (Kim, 2011). She
withheld approval in the US and was persistent in requesting further information. Her
persistence was vindicated when her actions averted a potential epidemic of birth defects
in the US like that being experienced in Europe. As a consequence of this incident,
however, regulators and politicians were sensitized to the importance of more rigorous
testing and better safety detection. Thus, drug regulations were further strengthened in
1962 by the Kefauver Harris Amendment. This amendment tightened controls over
prescription drugs, new drugs, and investigational drugs in several ways. Notably it
acknowledged that no drug could be considered sufficiently risk-free if it provided no

27
efficacy benefits. Substantial evidence of efficacy, therefore, became required prior to
marketing. Drug manufacturers were also now required to send adverse reaction reports
to the FDA so that safety signals could be detected and investigated quickly. The new
amendment also required that drug advertising in medical journals contain balanced
information in which risks as well as benefits were disclosed (Janssen, 1981). This
amendment laid the groundwork for the current regulatory framework for the US
biopharmaceutical industry that focuses on benefit-risk balance.
2.3 WHO Development of a Pilot Reporting System for Adverse Events
The consequences of thalidomide misuse served as a huge wake-up call for the
international community. Systematic international efforts were initiated to address drug
safety issues that might appear after a drug is commercialized. By 1968, the World
Health Organization (WHO) developed the Pilot Research Project for International
Monitoring of Adverse Reactions to Drugs for collecting and rapidly disseminating
information regarding Adverse Drug Reactions (ADR). Regulatory authorities from ten
countries with national drug monitoring centers - Australia, Canada, Czechoslovakia,
The Federal Republic of Germany, the Netherlands, Ireland, New Zealand, Sweden, the
UK, and the US - participated in this early pilot phase which was initially supported
financially by the US government. Their activities focused on developing a drug reaction
reporting form, an adverse reaction dictionary, and a drug dictionary to facilitate the
systematic collection and documentation of adverse events. They also developed a
suitable computer system for processing data, reports to capture those data, an alert
system that would be triggered by an increase in event reporting, and an information

28
retrieval system. During the 2 years of the pilot phase, over 24,000 adverse events case
reports were received.
After the pilot phase was completed successfully, the project transitioned to a primary
operational phase in 1971. The organization now allowed more countries to join and
published papers on methodologies for drug monitoring, epidemiology of drug use, and
the economics of ADRs (Lindquist, 2010). This program created a wide network of
people through which regulatory agencies worldwide began to know each other, and to
discuss and advise each other on regulatory matters.
2.4 CIOMS Program and ICH Regulations Approach to Pharmacovigilance
The Council for International Organizations of Medical Sciences (CIOMS) is another
organization that further helped to develop a systematic approach to pharmacovigilance.
As an international, non-governmental, non-profit organization established jointly by
WHO and UNESCO in 1949, CIOMS facilitated and promoted international activities by
forming working groups with the objective of fostering communication between
regulatory agencies and the industry regarding product safety. These working groups,
composed of senior scientists from regulatory authorities, pharmaceutical industry, and
academia, convened to discuss a broad range of safety topics and developed consensus
guidelines on pharmacovigilance. They produced final documents and numbered them
sequentially. CIOMS I developed a standardized form for the international reporting of
individual cases of serious, unexpected adverse drug reactions. CIOMS II, III, and IV
established harmonized approaches to Periodic Drug Safety Update Summaries and Core
Clinical Safety Information on Drugs. CIOMS V identified the most important elements

29
that should be considered in addressing drug safety of a marketed drugs (Lindquist,
2010). From these international activities, pharmacovigilance quickly evolved into a
more sophisticated regulatory activity that moved from collecting and disseminating
safety information to minimizing drug-induced injury and ensuring that the benefits of a
drug outweigh its risks. In this regard, CIOMS VI examined ways to minimize risks
before a product is marketed by requiring Development Safety Update Reports (DSUR)
to capture drug safety data from clinical trials. Later, the CIOMS VII working group
discussed the rationale, format and content for DSURs and published a report in 2006.
Most recently, in 2014, CIOMS IX published its report on “Practical Approaches to Risk
Minimisation for Medicinal Products”, which provides guidance on identifying risks that
need additional risk minimization approaches, as well as tools for implementation and
assessment of risk minimization strategies.
At the same time as the 1980s and 1990s saw an explosion of international drug research
and development, it also experienced dramatic increase in the global trade of
pharmaceutical products. This new culture created a need to harmonize drug regulations
among the nations involved in the trade. Their efforts to illuminate country-to-country
dissonance were pioneered by the European Community (EC) in the 1980s, which was
successful in developing a harmonized regulatory path and a single market for
pharmaceuticals from a number of European partners. Discussions also took place
between Europe, Japan and the US on the possibilities for cross-regional harmonization.
These led to harmonization of a joint initiative between the regulators and the industry in
1990 through the formation of the International Council for Harmonization of Technical

30
Requirements for Pharmaceuticals for Human Use (ICH). Many of the recommendations
from the CIOMS working groups were adopted by ICH and incorporated into a
harmonized tripartite guideline E2E titled “Pharmacovigilance Planning”, with
approaches to manage pharmacovigilance activities during the early post-marketing
period of a new drug (Arora, 2012).
The failure to detect certain types of risks was not the only concern as regulations and
risk management systems were being developed. Once a serious risk was identified,
decisions had to be made about whether to withdraw the product from commercial
distribution. Such a drastic step was problematic because it would deprive patients of a
drug with an important therapeutic benefit. Thus, measures were needed to mitigate
specific risks associated with a drug so that it can remain on the market. Drug
management began to enter a new era, in which proactive risk management became key
to approval.
2.5 Development of European Union Regulations and the Risk Management Plan
As the efforts of international bodies like WHO, CIOMS, and ICH produced guidance
documents and standards on pharmacovigilance, regional and national regulatory
authorities looked for ways to incorporate pharmacovigilance into their own regulatory
frameworks. These activities in Europe were to some degree similar to those in the USA.
Comparison of these two systems is germane to this dissertation that centers on the
management of risk for drugs with challenging safety issues.
Specific legislation regarding pharmacovigilance did not exist in Europe until January
1995, when, the European Medicines Agency (EMA) was established with funding from

31
the European Union and the pharmaceutical industry to harmonize (but not replace) the
national medicine regulatory bodies. An important part of its mandate was to provide
guidance about how the pharmacovigilance activities of more than 20 countries could be
harmonized. Their efforts produced European Directive 2001/83/EC and Regulation (EC)
No 726/2004 in 2001, with requirements that imposed safety measures for medicines
after they were put on the market. The new requirements required companies to
designate a person be responsible for pharmacovigilance who would keep detailed reports
of all presumed ADRs. They were also required to submit periodic safety update reports
(PSUR) at specified intervals.
Despite these new measures, ADRs were still causing approximately 197,000 deaths per
year in the EU (Bouvy, 2015). In response, the European Commission initiated a
comprehensive review of the safety monitoring system and, in 2005, the European
Medicines Agency (EMA) issued a detailed guideline requiring the submission of risk
management plans as part of the marketing authorization applications of new medicinal
products. This led to the adoption of a new directive (Directive 2010/84/EU) by the
European Parliament and Council of Ministers in December 2010 that imposed further
safety monitoring measures across the EU. They included mechanisms for direct patient
reporting, more effective collection of safety information, improvements to package
leaflets and labelling and the provision of better information on pharmacovigilance to the
public.
History soon provided evidence that these measures were insufficient; the withdrawal of
the drug Mediator (benfluorex) in October 2012 pointed to unresolved challenges in

32
pharmacovigilance. Mediator, used to treat diabetes and prescribed as an appetite
suppressant, was associated with reports of cardiac valvulopathy and pulmonary arterial
hypertension. Although safety concerns had been raised as early as 1999, the product
remained on the market until 2009, when it was finally forced off the market by the
French regulatory agency. Shortly thereafter, the product was removed from the entire
EU market. These delays suggested that a more robust pharmacovigilance system was
needed to collect better safety data and assessments, and then deliver effective regulatory
actions. The 2012 amendments, brought in to modify Directive 2012/26/EU and
Regulation (EU) No 1027/2012, required prompt notification and assessment of safety
issues (European Parliament, 2012a, 2012b). They further introduced an automatic
emergency procedure, including an EU safety evaluation and possible EU-wide
withdrawal if, for example, a member state was to withdraw a drug from the market. The
changes also forced companies to be more transparent. If a company were to withdraw a
drug, that company was required to state explicitly whether the withdrawal was instigated
by problems of safety. The EMA also published the Guideline on Good
Pharmacovigilance Practice (GVP) to provide sponsors with practical measures to
implement the pharmacovigilance legislation (European Medicines Agency, 2015).
2.5.1 EU Risk Management Approaches
Different drug products pose different risks. For example, thalidomide is a teratogenic
chemotherapeutic agent that can cause neutropenia. The active ingredient in Epogen,
epoetin alpha, filgrastim, can increase the risk of death or tumor growth in patients who
have cancer. The elements of a risk management plan must therefore be customized
according to product, indication, and considerations of the specifics of the risk that the

33
particular product presents. The approach to the risk management plan requires an
accurate product safety profile to identify specific risks, a strategy for preventing or
minimizing the risks, plans for studies and other activities to gain further knowledge
about the safety and efficacy of the medicine, the risk factors for developing side effects,
and an assessment plan for measuring the effectiveness of risk-minimization strategy.
This information contained in the risk management plan serves as the basis for an action
plan for pharmacovigilance and risk minimization activities and provides the safety
profile of a medicine at the beginning of its commercial life cycle.
The requirements that form the EU risk management plan (RMP) include seven parts:
product overview, safety specifications, pharmacovigilance plan, plans for post-
authorization efficacy studies, risk minimization measures, summary of the risk
management plan, and annexes (Table 1).
Table 1. Modules and Parts of RMP
Part I Product(s) overview
Part II Safety specification
 Module SI Epidemiology of the indication(s) and target population(s)
 Module SII Non-clinical part of the safety specification
 Module SIII Clinical trial exposure
 Module SIV Populations not studied in clinical trials
 Module SV Post-authorization experience
 Module SVI Additional EU requirements for the safety specification
 Module SVII Identified and potential risks
 Module SVIII Summary of the safety concerns
Part III Pharmacovigilance plan
Part IV Plans for post-authorization efficacy studies
Part V Risk minimization measures (including evaluation of the effectiveness of risk
minimization measures)
Part VI Summary of the risk management plan
Part VII Annexes

34
Each product that is subject to RMP requirements must have its own Risk Management
Plan based on the EMA template, available on its website. The RMP is viewed as a
“living document” that should be modified and updated as new information becomes
available. Currently, a RMP must be submitted in full every time the plan is updated,
even in part, but in the future, only the modules that have been updated will need to be
submitted (European Medicines Agency, 2015). An RMP may include routine activities,
such as ensuring that suitable warnings are included, and that the product is properly
labeled and packaged, or additional pharmacovigilance activities appropriate to the
product profile. For example, the product may require active surveillance by examining
medical records or patient registries, or by conducting retrospective or prospective
epidemiology studies. Further clinical or marketing studies may be needed to explore the
safety effects of additional doses or longer durations of treatment, to understand how the
drug affects particular populations or to evaluate how the drug is marketed, prescribed
and used. An RMP may include additional educational materials, dear healthcare
provider letters, training programs and restrictions on the use of the medicine.
A European network of centers for pharmacovigilance and pharmacoepidemiology
(ENCePP) was established in order to facilitate various pharmacovigilance activities
across Europe. ENCePP is led by the EMA and brings together expertise and research in
the fields of pharmacoepidemiology and pharmacovigilance. The network includes
research and medical-care centers, healthcare databases, electronic patient registries and
other research networks.

35
2.6 Development of US Regulations
A formalized system for risk management in the US lagged a few years behind that of
Europe. However, as in Europe, its development was driven by serious safety issues
affecting public health. Particularly influential were the problems associated with the
popular weight loss drug known as “fen-phen”, that became recognized in the 1990s.
2.6.1 Fen-Phen and PDUFA III
The name “fen-phen” was a short-form for a drug combination of
fenfluramine/dexfenfluramine and phentermine, both of which had effects on weight loss.
The combination was actively promoted by Robins Pharmaceuticals and Wyeth-Ayerst,
two divisions of American Home Products. Physicians then exercised their prerogative
to prescribe the two drugs together and made fen-phen one of the best-selling
pharmaceutical products at the time. In 1996 alone, as many as 18 million prescriptions
were written for fen-phen (Pitts, 1998). However, that combination of drugs had never
been studied in combination. It came as a great surprise, then when an echocardiography
technician in North Dakota discovered what appeared to be a relationship between fen-
phen usage and valvular heart disease, early in 1997. This association, subsequently
confirmed by the Mayo Clinic, was reported in the New England Journal of Medicine in
August 1997 along with a public warning about the dangers of fen-phen use. The
mounting evidence convinced the FDA to take action. In September of 1997, the FDA
requested that one of the two components, fenfluramine/dexfenfluramine, be removed
from the market (Barrett, 2004). This event was significant because it drew attention to
the challenges for FDA to anticipate the risk of a product. It pointed to the need for a
more formalized and effective risk management framework.

36
After the fen-phen experience, the US Congress looked for ways to insert risk
management into FDA’s regulatory framework. The opportunity came in 2002, when the
Prescription Drug User Fee Act (PDUFA) was due for reauthorization. PDUFA was
designed to link user fees for FDA with specific goals for the five-year period for which
the payments are authorized. According to PDUFA legislation of 2002, called PDUFA
III, FDA was instructed to produce guidance documents for the pharmaceutical industry
on risk management activities for drug and biological products. To meet these goals,
three guidance documents were published over the next 3 years aimed at four main goals:
1) assessing the benefit-risk balance of a medication, 2) developing and implementing
tools to minimize risks while preserving benefits, 3) evaluating effectiveness of the tools
and reassessing the product’s benefit-risk balance, and 4) making adjustments, as
appropriate, to the risk management tools to further improve the benefits-risk balance.
From the goals of PDUFA III came the RiskMAP program defined by the FDA as “the
strategic safety program designed to meet specific goals and objectives in minimizing
known risks of a medication while preserving its benefits” (Craig, 2010). This program
required industry to develop risk management programs associated with FDA approved
products. For most of the products, specific labeling and routine post marketing reporting
requirements of adverse events were considered sufficient to manage risks reasonably.
However, for some products for which additional risk management measures were
deemed necessary to ensure that the benefits of a drug outweigh the risks, a more
elaborate Risk Minimization Action Plan (RiskMAP) would have to be submitted when
filing an NDA. The RiskMAP plan would be designed to minimize known safety risks
through such processes as targeted education and outreach to healthcare practitioners or

37
patients to communicate risks; reminder systems to assist in reducing risks during
prescribing; and performance-linked access systems to help guide prescribing,
dispensing, and use of the product to maximize its benefits while minimizing certain
risks. The RiskMAP for Accutane (isotretinoin) illustrates one such example. Accutane
is contraindicated for female patients who are or may become pregnant because of a high
risk of birth defects that are associated with taking the medication. The RiskMAP for
Accutane, therefore, included several elements to prevent this use. The product could
only be distributed to registered wholesalers, pharmacists and healthcare providers;
pregnancy testing was linked to isotretinoin dispensing, and a pregnancy registry was
established, so that root cause analysis pregnancy exposure could be recognized and
investigated to assess root causes of the exposure.
2.6.2 Vioxx
Despite the implementation of the RiskMAP program, safety issues and product
withdrawals continued. The most publicized and politically charged market withdrawal
in the 2000s was that of Vioxx, a Cox-2 selective inhibitor used for the treatment of pain.
Cox-2 selective inhibitors were promoted for their ability to treat pain while avoiding the
gastrointestinal (GI) side effects that were often associated with existing nonsteroidal
anti-inflammatory drugs (NSAIDs) that interacted with both Cox-1 as well as Cox-2
receptors. The first Cox-2 selective inhibitor, Pfizer’s Celebrex, was approved by the
FDA in 1998, and a second, Vioxx, by Merck in 1999. At the time of their launch,
Celebrex and Vioxx both included warnings in their labels about the possibility of GI side
effects because the FDA was not convinced that the clinical data were sufficiently robust
to prove that such disturbances were eliminated.

38
Vioxx was commercialized with an aggressive marketing campaign and achieved
blockbuster status within a few years. By 2003, it served more than 20 million patients
and accounted for US$2.5 billion in sales (Fiedler, 2005). However, the continued need
to list the GI side effects on the label of Vioxx was felt to be a continuing concern. Thus,
Merck initiated a large clinical trial titled Vioxx Gastrointestinal Research (VIGOR) that
was designed to show that Vioxx was superior to an older drug, naproxen, for treating
pain from rheumatoid arthritis because it caused fewer GI side effects. Results from over
8,000 rheumatoid arthritis patients in that trial indeed demonstrated that patients in the
Vioxx arm had a reduced risk of internal bleeding. At the same time, however, data
suggested that Vioxx increased the risk of heart attacks, strokes and thrombus formation
in the legs (Fiedler, 2005). The results of the VIGOR study were published in the New
England Journal of Medicine. However, the estimated incidence of cardiovascular
adverse events was inaccurate as cardiovascular risk of those kinds were not anticipated
when the study was designed, and thus had no pre-specified adjudication process. When
FDA subsequently reviewed the raw data, it found a much higher incidence of major
cardiovascular events than originally reported (Karha, 2004).
Skepticism regarding the clinical trial results for Vioxx led other researchers to conduct
additional investigations. In one study, a meta-analysis of cardiovascular risks across the
clinical trials of Celebrex and Vioxx showed that Vioxx was associated with higher rates
of cardiovascular events than Celebrex. Although the merits of such analyses were
debated, it did fuel public concern about the potential cardiac risks associated with
Vioxx. Both Merck and Pfizer were pushed by the research community to conduct

39
further clinical investigations into this cardiovascular safety concern, but both refused.
Other researchers, however, continued to conduct epidemiological studies, and study
after study showed a higher risk of cardiovascular events with Cox-2 selective inhibitors
when compared to traditional, nonspecific Cox inhibitors. However, the merit and
reliability of these studies were questioned and neither company pursued any action in
response.
The confusion about adverse events related to Cox-2 selective inhibitors clearly required
better clarity. Thus, in February 2001, the FDA convened an advisory committee
meeting during which the committee did not recommend withdrawal but proposed label
changes to include increased cardiovascular risks (Fiedler, 2005). Merck responded by
altering its label to list the potential benefit of fewer GI side effects followed by the
increased risk of cardiovascular events. Merck also modified the protocol of an ongoing
clinical trial for the prevention of colon polyps to monitor cardiovascular events more
closely. It was this trial that caused a major reevaluation of Merck’s approach, when
interim results analyzed by the study’s data safety monitoring board (DSMB) showed that
the patients in the Vioxx group were twice as likely to have a heart attack or stroke as
those in the placebo group. Merck halted the trial and shortly thereafter voluntarily
withdrew Vioxx from the worldwide market. By that time, an estimated 20 million
Americans had taken the drug. Of these individuals, 88,000 had suffered heart attacks
and 38,000 had died (Juni, 2004).
This incident sparked much public and political debate. It called into question the ethics
of Merck, a long respected company, as well as the ability of the FDA to properly track

40
and manage safety issues of medications once they are approved and on the market. The
Vioxx event, like other past tragic events, would lead to another landmark FDA
legislative act to prevent such events from recurring.
2.6.3 FDAAA of 2007 and the Initiation of REMS
The Vioxx case highlighted the shortcomings of a regulatory system that was still
primarily reactive in nature. Safety measures such as a black box warning, limited
distribution, or product withdrawal were almost exclusively retrospective in nature- they
were usually imposed after an adverse effect was found to be associated with an already
marketed and widely used drug. A small number of drug and biological products were
initially approved with a RiskMAP but, typically, a safety event would need to occur
before the FDA could intervene. The highly publicized Vioxx safety issues led
legislators and regulators to recognize that the system in place was not adequate to
protect the public’s safety. Therefore, when the Food and Drug Administration
Amendments Act (FDAAA) was signed into law as PDUFA IV, it incorporated
provisions to further strengthen FDA’s authority over a product after it is placed on the
market.
The 2007 FDAAA gave FDA the authority to require sponsors to make drug labeling
changes and conduct additional post market studies. The FDA could also require an
applicant to submit a formalized Risk Evaluation and Mitigation Strategy (REMS) for a
product with risks that could be reduced if special controls were put in place. FDA now
had the option to approve effective but potentially harmful medications by requiring
specific risk management actions and/or surveillance. It could further require a REMS for

41
any NDA, ANDA, or BLA after the product is on the market. Under these new rules, a
call by FDA for a REMS for a marketed product would require the affected sponsor or
sponsors to submit that plan in 120 days. Otherwise the drug would be considered
misbranded under section 502(y) of FDAAA, with civil monetary penalties of $250,000
per violation that could escalate to $10 million per proceeding for delays in compliance
(Food and Drug Administration, 2009a).
The REMS program, now in place for a decade, calls for driving behavioral changes that
would minimize risks associated with a drug while still maintaining its benefits. Under
REMS, each drug has its own custom tailored program with special controls that are
complementary to the product labeling. For example, a product that may cause liver
damage will require liver function monitoring whereas one that may cause birth defects
will require pregnancy testing prior to dispensing. If the FDA determines that REMS is
necessary, the sponsor designs, develops, and submits the proposed REMS for FDA’s
review and approval. To assist this REMS process, the FDA published a draft guidance
for industry in 2009 titled “Format and Content of Proposed Risk Evaluation and
Mitigation Strategies (REMS), REMS Assessments, and Proposed REMS Modifications”
(Food and Drug Administration, 2009a). The guidance document provides format and
content of a proposed REMS, including REMS supporting documentation, the content of
assessments and proposed modifications of approved REMS, what identifiers to use on
REMS documents, and how to communicate with FDA about a REMS. To date, the
guidance has not been finalized apart from specific information related to REMS
modification and revision, which will be described later.

42
2.6.4 REMS Elements
The FDA considers several factors to determine if a REMS is needed: 1) size of the
population that would likely use the drug, 2) seriousness of the disease, 3) expected
benefit of the drug, 4) duration of treatment, 5) seriousness of known or potential adverse
events, and 6) if the drug is a new molecule. The required REMS may contain any or all
of the following four elements: medication guide or patient package insert,
communication plan, elements to assure safe use (ETASU), and an implementation
system (Table 2). Whereas the first two items are passive measures to provide
information to patients and providers, the second two items are active measures that aim
to modify provider behavior and assess REMS implementation.

43
Table 2. REMS Elements and Descriptions

Element Description
Medication Guide Provides FDA approved patient-friendly
labeling
Communication Plan FDA approved materials used to aid
sponsor’s implementation of the REMS
and/or informs healthcare providers about
serious risks. May include:
 Dear healthcare professional letters
 Dissemination of information to
healthcare professionals through
professional societies
 Information about the REMS to
encourage implementation
Elements to Assure Safe Use May include:
 Certification or training for
prescribers
 Certification for dispensing
 Dispensing in certain healthcare
settings (i.e. hospitals)
 Dispensing only to patients with
evidence of safe-use (i.e. lab tests)
 Patient monitoring
 Patient registry

Implementation System Applicant to take reasonable steps to
monitor and evaluate implementation of
such ETASU elements by those
responsible and work to improve
implementation

The most straightforward REMS element is the medication guide. The medication guide
must be dispensed with the drug and is written in non-technical language at the 6
th
-8
th

grade reading level so that it can be understood by the “lay person”. The medication
guide is provided to the patient in addition to the patient leaflet that is part of the routine

44
product labeling. This added information is provided for outpatient prescription products
that pose “serious and significant” public health concern (Food and Drug Administration,
2009a, 2009b). The medication guide is required if it could prevent serious adverse
events, help patients make informed decisions regarding product use, or promote patient
compliance. An example of a medication guide is that for Januvia
®
(sitagliptin) for
diabetes, which informs the patient of serious risk of acute pancreatitis associated with
the use of drug. This element of REMS was approved on February 2010 and rescinded
on March 2011 as part of a supplemental New Drug Application approval (Food and
Drug Administration, 2011b). Although the medication guide may no longer part of
REMS, it must remain a part of the approved labeling. Over the past several years, the
FDA has considered any new Medication Guide to be a part of a REMS. However, the
Agency has now determined that every Medication Guide will not necessarily be part of a
REMS. Depending on the risks involved, the Agency will now approve some Medication
Guides under only labeling regulations, and occasionally, also as part of a REMS.
The next REMS element, the communication plan, is more demanding. It supports the
implementation of the REMS by educating the health care providers about one or more
specific risks associated with the drug. A communication plan educates, informs, and
raises awareness of risk. Examples of communication plan tools include Dear Healthcare
Professional letters, relevant information to be disseminated to healthcare professionals
through professional societies, and information about the REMS program to encourage
implementation. An example of a communication plan is that for Actemra
®

(tocilizumab) that was approved for rheumatoid arthritis. Following FDA’s

45
determination that REMS was necessary for this drug due to its serious risks including
infections and gastrointestinal perforations, the sponsor, Genentech, Inc. submitted its
proposed communication plan and timetable for submission of assessments (Food and
Drug Administration, 2013a). In this case, the goal was to inform healthcare providers
about the serious risks associated with the drug. The plan specified a long list of
healthcare providers who will be targeted, including rheumatologists, infectious disease
specialists, and general practitioners; elements of the communication plan, which
consisted of a Dear Healthcare Provider Letter, prescriber education slide deck,
information to be presented through scientific meetings and journals; and a timetable for
submission assessment, which was specified as 18 months, 3 years, 5 years, and 7 years
after REMS approval. The proposed REMS also contained examples of the elements of
communication plan as attachments. meeting the requirements of the communication
plan is more challenging in part because it can be difficult to identify a distribution
method capable of reaching and educating a majority of healthcare providers in an
effective way. An additional challenge relates to the ongoing need to identify and notify
new prescribers. In the case of Actemra
®
, the REMS program was eliminated after the 3-
year assessment milestone was reached, in order to minimize burden on the healthcare
delivery system (Food and Drug Administration, 2015a). According to the FDA letter to
the manufacturer, the Agency determined that the communication met its goal and is no
longer necessary.
The most demanding form of REMS is the “Elements to Assure Safe Use”, typically
referred to by the acronym, ETASU. Its requirements are intended to reduce a single

46
specific risk that is identified in the product label, by addressing that risk aggressively in
multiple ways. For example, an ETASU program may require prescribers to receive
specific training and/or certification before they can prescribe the product or require
pharmacies and/or pharmacists and wholesalers to be certified in order to be able to
dispense it. In addition, ETASU requirements may mandate that the drug be dispensed
only in certain healthcare settings (e.g. hospitals), or after evidence of safe-use conditions
are provided through laboratory test results. In some cases, patients may be subject to
monitoring and/or be enrolled in a registry. An example of an ETASU is that for
Zyprexa
®
Relprevv™ (olanzapine) that was approved for schizophrenia. The single most
serious risk identified with the use of this product relates to complications associated with
post-injection delirium/sedation syndrome (PDSS) (Food and Drug Administration,
2009c). As part of its NDA approval process, FDA required the submission of a REMS
program that consisted of a Medication Guide, a communication plan, ETASU, an
implementation system, and a timetable for submission of assessments of the REMS. As
seen here, ETASU is typically accompanied by other elements of REMS. The goal of
this REMS program is to mitigate the risk by ensuring that the healthcare providers are
aware of the risk and that the patients are observed for at least 3 hours after drug
administration in a certified healthcare facility with ready access to emergency response
services. The ETASU for this drug requires that the product can be prescribed only by
certified physicians and dispensed only by certified pharmacies. Furthermore, each
patient using the drug is to be enrolled in the Zyprexa Relprevv Patient Care Registry.

47
REMS programs require their implementation be assessed, but the implementation
programs needed for ETASU elements are particularly challenging. To satisfy the
requirement, the sponsor must collect and analyze large amounts of data in order to
evaluate and improve ETASU implementation. This may involve administering surveys,
monitoring prescriber compliance with certification and REMS requirements, and
reviewing data to identify which patients are getting the drug and under what conditions.
For the Zyprexa
®
Relprevv™ example, the sponsor’s proposed implementation system
included maintaining a database of all certified dispensers to be able to link each risk to
the enrolled patient and the associated dispenser (Food and Drug Administration, 2014d).
To ensure compliance with the REMS program, the sponsor, Lilly, pledged to review the
drug distribution data to determine if only certified dispensers are dispensing the drug,
evaluate healthcare settings to determine if they can assure safe use of the product, and
take reasonable steps to improve implementation based on its findings.
2.6.5 Assessment of REMS
Following the passage of FDAAA in 2007, the REMS program expanded quickly.
Before long, REMS were required for over 260 drugs, each customized according to the
product’s unique risk profile (Food and Drug Administration, 2016b) . Most required
only a medication guide or a communication plan but a number of them required an
ETASU element, often in combination with other. As of May 2016, there have been or
are currently 74 REMS programs that include an ETASU element (Hyman, 2016). Thus,
it was important to ensure that each REMS is effective in achieving its goals while
minimizing burden to the healthcare system and other stakeholders.

48
REMS assessments are reviewed by the FDA according to a specified timetable,
proposed by the sponsor and approved by the FDA, and can be modified or discontinued
altogether based on the data submitted (Food and Drug Administration, 2016a). FDA
may determine that the risk of a product can be adequately mitigated with the product
label alone or with less stringent elements of a REMS program. REMS assessments may
include: 1) survey data to gauge understanding by healthcare professionals on the safe
use of the drug, 2) summary of adverse events associated with the drug, 3) record of
prescriber compliance with certification and REMS requirements, and 4) use data to
understand how the product is used.
2.6.6 Modifying existing REMS
The FDA provides specific instructions for modifying REMS. A 2009 draft guidance on
format and content was finalized in October 2017. Parts related to REMS modification
and revisions have also been finalized in a separate guidance for industry in 2015 as was
stipulated in FDASIA (One Hundred Twelfth Congress of the United States of America,
2012). The information in the new guidance supersedes corresponding parts in the draft
guidance mentioned above (Food and Drug Administration, 2015f). Previously, a
proposed change to approved REMS required a Prior Approval Supplement (PAS)
submission, which had to be reviewed and approved by FDA within 180 days. The 2015
guidance on modification and revision of REMS categorizes REMS revisions and
modifications into different submission types, from revisions and minor modifications
that have little or no impact on risk to major modifications that require closer scrutiny.
Revisions that are editorial in nature and do not change the risk information can be
implemented without any prior approval by the FDA and are documented in the NDA

49
annual report. Minor modifications can be submitted using the Changes Being Effected
in 30 days (CBE-30) supplement route. For these, FDA has 60 days to review but the
change can be implemented in 30 days. Major REMS modifications must be submitted
as a PAS and may not be implemented until approved by FDA within 180 days of receipt.
2.7 REMS Challenges
From its beginnings the REMS programs have been controversial because of the way that
it changes the requirements for many of the affected stakeholders. For example, the
required REMS may greatly impact a physician’s prescribing behavior and, as a
consequence, deny patients access to medications that could be potentially beneficial.
The REMS for GlaxoSmithKline’s (GSK) Avandia (rosiglitazone maleate) is a case in
point. The ETASU for this product placed a heavy burden on the healthcare system and
contributed to a precipitous drop in its use.
Initially approved in May 1999, Avandia was once one of the most prescribed diabetes
medications in the US, becoming a blockbuster and reaching nearly 3 billion dollars in
sales (Orelli, 2013). During initial review of Avandia’s New Drug Application, FDA
reviewers expressed concerns about the cardiac risks potentially posed by the product and
FDA consequently requested a post-marketing study to evaluate the hepatic, metabolic,
cardiovascular and hematologic effects of the drug. In 2007, while GSK was still
conducting its prospective, controlled study, results of a high-profile meta-analysis of
data from 42 different clinical trials were published, indicating that Avandia was
associated with an increased risk of myocardial infarction and death. FDA responded
swiftly, exercising its new authority under FDAAA, by imposing the most severe form of

50
REMS. The program greatly restricted drug access to include only patients who were
already treated successfully with Avandia, whose blood glucose levels could not be
controlled with other drugs, or who have previously refused pioglitazone-containing
medicines (Food and Drug Administration, 2012b). These draconian restrictions made
Avandia essentially the last drug of choice for patients with diabetes. There were other
challenges. The program required healthcare providers and patients to be enrolled in the
“Avandia-Rosiglitazone Medicines Access Program” and the drug to be obtained only
through certified mail order pharmacies participating in the REMS program. By the end
of 2012, annual sales from Avandia had plummeted from $3 billion to $9 million.
In 2013, six years after results of a meta-analysis raised safety concerns and imposed a
strict REMS program for Avandia, GSK completed its prospective, multicenter, open-
label cardiovascular outcomes study, RECORD. Contrary to the earlier meta-analysis,
the results of this study did not demonstrate an elevated risk of heart attacks or death in
patients treated with Avandia when compared to standard-of-care. In light of the new
information, FDA’s advisory committee recommended that many of Avandia’s REMS
components be removed. Subsequently, all of Avandia’s REMS restrictions were
rescinded (Food and Drug Administration, 2015g).
The Avandia case raised concerns about both the questionable merit of meta-analyses and
the impact of REMS on the health care system and patient access. Over the last few
years, the total number of REMS programs has greatly decreased after FDA’s
assessments rescinded more than 170 (Food and Drug Administration, 2016b). This
number, however, may be misleading because most of the rescinded programs were in

51
the least rigorous category. They involved the dissemination of medication guides that
were, in fact, not eliminated but, as allowed by FDA’s new guidance in 2011, integrated
into the product labeling (Food and Drug Administration, 2011a). In contrast, the
number of ETASUs, which are the most burdensome components of REMS, has
remained generally the same; only five have been rescinded as of May 2016 (Hyman,
2016). Thus, the REMS program continues to receive criticism for straining the
healthcare system without clear evidence demonstrating its effectiveness in improving
drug safety. Some of this criticism was based on a study conducted by the Office of
Inspector General (OIG) of the Department of Health and Human Services that examined
data from the first 3 years of the REMS program (Office of the Inspector General, 2014).
2.7.1 OIG Report Findings and Recommendations
OIG reviewed 199 REMS that were approved between 2008 and 2011 to determine if the
REMS assessments were being conducted appropriately. They examined assessment
submissions by the sponsors and review memoranda by the FDA to determine if REMS
evaluations were as complete and timely as legally required. Of the 199 REMS that were
reviewed, 119 required medication guides, 48 required communication plans, and 32
required ETASUs. Seventy-four of these programs were implemented as part of the drug
approval process and the remaining 125 were put into place after the drug was on the
market in response to new safety information. OIG found that the sponsors were usually
late in submitting their assessments and did not provide all information requested in FDA
assessment plans (Office of the Inspector General, 2014). Furthermore, they were not
using the data to make program improvements that would help them meet their stated
goals. Deficiencies identified by the FDA reviewers remained unchanged in subsequent

52
assessments and, hence, FDA was unable to determine whether REMS programs were
meeting their goals. FDA was also consistently late in their review of the assessments
submitted by the sponsors. In its report, OIG criticized the FDA for not having
developed a plan to “identify, develop, validate, and assess the effectiveness of REMS
components” and not meeting its annual requirement to evaluate the ETASU of at least
one drug. Of the 32 REMS with ETASUs approved between 2008 and 2011, FDA was
able to complete only one by the end of 2011. With limited data and the lack of a reliable
methodology, evaluating the effectiveness of REMS has proven to be difficult and
challenging.
Based on their findings, OIG made a number of recommendations including the need for
the FDA to work closely with the sponsors to obtain data missing in their assessment
submissions and to ensure that the assessment reviews are completed in a timely manner.
Above all, OIG identified an urgent need for the FDA to develop methods to assess the
effectiveness of REMS rather than merely relying on the assessments provided by the
sponsors. To do this, OIG encouraged the FDA to continue to hold meetings with
stakeholders to discuss how best to assess the effectiveness of REMS components. The
findings and recommendations of OIG are reflected in the REMS section of the fifth
reauthorization of the Prescription Drug User Fee Act (PDUFA V), Food and Drug
Administration Safety and Innovation Act, described below (One Hundred Twelfth
Congress of the United States of America, 2012).

53
2.8 Addressing REMS Challenges
Many of the assessment metrics currently in place to evaluate REMS programs focus on
processes and not outcomes. It is easier to obtain information such as the number of
certified healthcare providers, number of prescriptions filled, and number of “Dear
Healthcare Provider” letters mailed, rather than information related to how well REMS is
working to control risks associated with a particular product. Further, without outcomes-
related data from a pre-REMS period against which to compare, it is difficult to assess
the effectiveness of REMS requirements. Nevertheless, FDA is obligated to address this
issue, under FDAAA, by convening the Drug Safety and Risk Management (DSaRM)
Advisory Committee at least annually to discuss one or more REMS with ETASUs.
These meetings aim to determine if a REMS program is effective in assuring safe use of
the drug without obstructing access or placing an excessive burden on the healthcare
delivery system. Generally, the DSaRM discussions have focused either on a drug or a
class of marketed drugs such as isotretinoin (2011) and alosteron (2013) or on a new drug
under review, such as sugammadex (2015). The meetings often take place jointly with
another advisory committee that has expertise in the specific therapeutic area that is
under discussion. The results of these deliberations are mixed. Opinions have varied
even among the advisory committee members as to the effectiveness of existing REMS
and many have called for further studies to understand their impact on patient safety and
healthcare (Food and Drug Administration, 2012c, 2014a). Inconsistencies across
programs continue to be troublesome. As noted by one committee member,
“…the formats vary so widely that we have to remember, well, for
Aranesp, they have to sign this consent form, and for Soliris we have to

54
make sure to distribute the materials and document the appropriate
education. And for lenalinomide, the patient has to take a telephone
survey. They’re all different…”.

Accordingly, initiatives have been underway to examine how best to evaluate and
modify REMS (Food and Drug Administration, 2014b). These include a “REMS
Integration Initiative” and a set of “REMS Priority Projects”, described below.
2.8.1 REMS Integration Initiative
While FDAAA provided FDA with the authority to require REMS, it did not require the
development of a standardized framework or a scientific methodology to measure the
effectiveness of REMS. Hence, the FDA created the REMS integration initiative in 2011
as part of the discussions in anticipation of the fifth reauthorization of the Prescription
Drug User Fee Act (PDUFA V). The goal of this initiative was to evaluate and improve
the implementation of REMS by obtaining input from key stakeholders including
industry representatives, patient and consumer advocates, and healthcare professionals,
with respect to the issues and challenges associated with the development,
implementation, and assessment of REMS. To support the work of the Initiative, a
REMS Integration Steering Committee was put in place to oversee the activities of three
subordinate work groups, each charged with a specific objective: 1) developing guidance
on the interpretation of statutory criteria for requiring a REMS (Policy Work Group), 2)
improving standardization and assessment of REMS (Design and Standardization Work
Group), and 3) improving integration of REMS into the healthcare system (Evaluation
Work Group) (Food and Drug Administration, 2016c). These groups were engaged in
producing specific outcomes and recommendations. The Policy Work Group has
developed a draft guidance document, issued in September 2016, to clarify how FDA

55
intends to determine when a REMS is needed to ensure that the benefits of a drug
outweigh the risks. This guidance document explains that the determination about
whether a REMS is necessary is a complex, drug-specific inquiry that includes the
integrated consideration of each of the six statutory factors. The Evaluation Work Group
lead FDA efforts to develop an evidence-based approach to assess the effectiveness and
burden of REMS, and to develop a draft guidance on appropriate methodologies for
assessing the effectiveness of REMS, but, to date, a guidance document on this topic has
not been issued. The Design and Standardization Work Group lead efforts to identify
best practices for the future design of REMS programs so that tools can be standardized,
and programs can be integrated into the health care delivery system more seamlessly.
This group was instrumental in identifying and developing FDA priority projects that are
described below.
2.8.2 FDA Priority Projects
As part of its efforts to seek stakeholder feedback, FDA held public meetings to examine
issues associated with the development and implementation of REMS. In July 2010,
sponsors and health care providers gathered to share their views including challenges
associated with collecting data on the compliance of third parties (e.g., patients,
pharmacies, drug distributors), concerns about patient confidentiality, and the lack of a
standardized format for sponsor assessments (Office of the Inspector General, 2014).
These discussions continued as the Food and Drug Administration Safety and Innovation
Act (FDASIA) was signed into law in July 2012 as part of the reauthorization of PDUFA
(PDUFA V). The new law required the FDA to measure the effectiveness of REMS

56
program, standardize it across products, and integrate it into existing and evolving
healthcare systems.
In June 2012, the FDA held a public workshop to discuss ways to assess REMS
effectiveness, including survey methodologies and instruments that can be used (Food
and Drug Administration, 2012a). In general, the panelists agreed that surveys should be
tested before use and should measure the level of knowledge by patients and health care
providers regarding risks of drugs marketed with REMS. Throughout 2013, FDA reached
out to stakeholders in a variety of ways including face-to-face meetings, roundtables,
teleconferences, webinars, symposia and seminars, as well as a meeting with the FDA
Drug Safety Oversight Board (Food and Drug Administration, 2014b). The series of
stakeholder meetings culminated in identifying four priority projects for FDA. In
September 2014, FDA published a report, “Standardizing and Evaluating Risk Evaluation
and Mitigation Strategies (REMS)”, in which the agency outlined its initiatives for
REMS administration and work plans of projects in priority areas as described in Table 3
(Food and Drug Administration, 2014b).

57
Table 3. FDA REMS Priority Projects
Project FDA proposed actions
Patient benefit/risk information under
REMS: Providing patient benefit/risk
information by improving tools for
prescriber-to-patient counseling
 conduct research into existing
REMS patient counseling tools
 seek stakeholder feedback to
identify opportunities to improve
upon the content, format,
processes, techniques, tools, and
delivery of effective counseling
within REMS programs
 develop a report for stakeholders
that could serve as the basis for
designing new tools and validating
them
Prescriber education under REMS:
Prescriber education – REMS and
continuing education for health care
providers
 assess if it is feasible to accredit
continuing education (CE) with
certain REMS
 for new drugs, determine at what
stage the development of CE
would best fit in the new drug
approval process (pre or post
approval)
 provide an analysis of time and
resources required related to
developing and using CE to
conduct REMS-related training
Pharmacy systems under REMS:
standardizing REMS information for
inclusion into pharmacy systems
using structured product labeling
(SPL)
 investigate developing SPL for
REMS content
 develop a structured method to
share clear and consistent
information about REMS content
 make structured information
available to HCPs, patients and
FDA
 provide a single source of
comprehensive information about
REMS programs
 facilitate the integration of REMS
into pharmacy systems
 improve the efficiency of review
by allowing FDA to receive REMS

58
information from sponsors in a
consistent format
 support FDA’s ongoing
standardization efforts by
cataloging the similarities and
differences between REMS
programs
Practice settings under REMS:
providing a central source of REMS
information for practice settings
 investigate a centralized,
standardized, user-friendly source
of information about what
stakeholders are required to do in
each REMS program
 help stakeholders quickly learn
about REMS programs
 help stakeholders understand and
comply with REMS requirements
 allow stakeholders to compare
requirements across REMS and
minimize confusion associated
with complying with multiple
REMS programs

2.8.3 Measuring Effectiveness of REMS
In October 2015, FDA hosted another public meeting, this time to discuss REMS in the
context of “understanding and evaluating their impact on the health care delivery systems
and patient access” (Food and Drug Administration, 2015e). Of particular interest at that
meeting was a presentation given by CVS Health, the largest U.S. provider of
prescriptions and specialty pharmacy services. They identified a number of challenges
with REMS programs that have an ETASU, including disruption to the pharmacy
workflow, difficulties with program adherence, a lack of standardization in training and
certification, poor access to relevant REMS information, and delays that could impact
patient safety. The American Society of Hematology (ASH) also shared their concerns
over having to manage multiple REMS that are often duplicative of existing safety

59
protocols in a hematology practice. The added workload for the healthcare providers
may compromise patient care by delaying access to important therapeutic agents without
enhancing patient safety. Biogen shared their experience with REMS for Tysabri and
voiced concerns over the unbalanced nature of REMS materials that focus mostly on the
potential risks and not enough on the potential benefits of a product. The concern here is
that patients who may benefit from the treatment may refuse that treatment as a result of
unbalanced and inadequate information.
The two-day meeting brought together many of the stakeholders including regulators, the
scientific community, the pharmaceutical industry, patients and their advocates, health
care system administrators, prescribers, dispensers, and hospitals. Of the 26 panel
members, however, only two were from industry (Food and Drug Administration,
2015d). This lack of input from industry is consistent with that observed in previous
forums. For example, out of approximately 54 panel members in the July 2010 meeting
to discuss FDA priority projects, only six panel members were potentially representing
views of industry. At the June 2012 workshop convened to discuss survey methodologies
and instruments, only 7 out of 30 panel members were representatives from industry
(Food and Drug Administration, 2010). Hence, although the sponsors are legally
mandated to develop and implement REMS programs, there is relatively little
involvement from industry to inform the regulators about the challenges created by the
requirements and their views on how the system could be improved.

60
2.9 Where does industry stand?
The REMS program is at a crossroads, where policy could be revised in different ways.
The usefulness of that policy revision likely depends on the collective inputs of many
stakeholders. The working groups and meetings that are described above have probed in
depth the views of regulators, pharmacists, physicians, and academicians, but have
provided surprisingly little information about the views of industry on the current state
and future improvements for the REMS system. While FDA is considering how to
redesign REMS, it would be informative to investigate current views of the industry.
According to Cronin, the long-term sustainability of risk management efforts requires
that REMS processes be embedded into and across the health care system with a clear
articulation of the roles of key stakeholders of which industry is clearly one (Cronin,
2014).
2.9.1 Studying industry views on REMS Implementation
Policy theories suggest that initiatives often fail to reach their goals when policies are
developed to meet specific objectives without taking into account the logistics and
challenges of the real world setting (Fixsen, 2005). This sometimes occurs because policy
makers are initially most concerned with the goals and regulatory restrictions associated
with the policy development and have less experience or interest in defining the details of
a policy’s execution. However, policies can only be successful if they can be
implemented effectively. Problems with implementation might be avoided if policy
makers were sensitive to the need to develop policies that are unambiguous and helpful in
their instructions for implementation (Eaton Baier, 1986), a state that is best achieved by
understanding the implementation environment. Some might argue that a simplified

61
policy is very difficult to develop around a complex problem such as that posed by the
REMS initiatives. Certainly, the medications that are subject to REMS differ greatly
from one another in the nature of risk, target patient population, and the clinical setting.
Nevertheless, the recurring theme in all of the analyses and meetings held in the last 5
years suggest that stakeholders would appreciate more assistance with the challenges of
implementation. It seems therefore appropriate to use some of the lessons from
implementation science to better understand the major challenges of REMS programs
through the eyes of the principal stakeholders responsible for its implementation (Smith
& Morrato, 2014).
As a discipline, implementation science emphasizes three interacting factors that
collectively influence program effectiveness: the attributes of the proposed intervention
such as its requirements, clarity, transparency and consistency; the characteristics of the
intended adopters with respect to their capabilities to collaborate, cooperate and devote
time to the endeavor; and aspects of the intervention delivery context such as degree of
burden of the program (Rabin, 2006). The three factors can be used as a framework to
guide an evaluation of REMS implementation through the eyes of a stakeholder such as
the industry. Each may have an effect in facilitating or limiting the implementation of the
policy.
Implementation science also identifies clearly delineated stages of program
implementation. The work of Fixsen divides implementation into certain phases:
exploration and adoption; program installation; initial implementation; full operation;
innovation; and sustainability (Fixsen, 2005). The exploration and adoption phase is part

62
of program development. It involves activities to match the structure of the program to
the needs of key stakeholders including the industry, FDA and healthcare providers,
pharmacists and patients. During this phase, the REMS provider must determine the
resources needed to implement the REMS, such as funding, space and staffing; and must
assess conditions that might pose barriers to the implementation. Based on the findings
from the exploration step, buy-in from stakeholders is obtained in order to move to the
next stage. In the installation stage, activities such as designing and setting up training
modules and establishing SOPs needed for the healthcare practice and/or pharmacy
would be carried out in preparation for REMS implementation. In the subsequent initial
implementation phase, the complex work of changing the existing procedures and
practices begins with the launch of the program to outside entities and individuals such as
healthcare providers and patients. Implementation activities may include, for example,
the dissemination of training materials, the initial teaching of educational and
certification programs, and the population of databases to track enrollment and safe-use
conditions. The maintenance phase is characterized by the full operation of the REMS
program after it has been completely integrated in the health care community. During
this phase inefficiencies and problems may be discovered that need remediation. The
innovation phase is that in which the developed REMS program is modified to improve
its performance. The last stage of implementation is described as sustainability, after the
program has become mature and forms part of a standardized way of doing business.
These stages can be used to provide a vocabulary to describe the state of implementation
of a REMS program, so that it can be dissected systematically for study.

63
2.9.2 Research Framework
To explore the industry’s views on effectiveness of REMS, it is helpful to have an
approach based on a scientific platform. The work of Fixsen provides a way of looking at
the implementation of REMS longitudinally. That of Rabin frames it according to certain
essential features known to be key determinants of success. Their two frameworks can
work in a complementary fashion to explore more systematically the vulnerabilities of
REMS implementation as shown in Table 4. At each stage of implementation,
participants in the survey may have views on the way in which certain attributes of the
policy, identified by Rabin, have helped or hindered implementation.
At the same time, REMS programs are not new; much work that has been done
previously by advisory committees and FDA to define and drive the earlier stages of
program development and implementation. Thus, REMS programs are now at later
stages of implementation. It is on these stages that little research has been conducted to
understand the perspective of the companies responsible for the ongoing innovation to
improve suboptimal programs as they strive for sustainability. Because so many REMS
programs are now mature, it seems timely to investigate industry views specifically
related to these later stages. Thus, the research here focused on later stages of
implementation; bolded sections in Table 4 identify the areas of concentration for my
research.

64

Table 4. Research Framework
Stage of
Implementation
Transparency Collaboration Harmonization/Degree
of Burden
Exploration and
adoption
Needs of a REMS
program and the
steps for its
development are
clearly articulated
and disclosed
Stakeholders are
engaged during
development of the
program
New REMS programs
are developed to be
similar to other
programs as appropriate
to be consistent
Program installation REMS requirements
including registries,
training materials
and learning
modules are
announced and
available for review
Stakeholders
provide input into
establishing the
program
New REMS programs
can utilize existing
procedures and SOPs
Initial
implementation
Information on the
launch of REMS
programs including
target audiences and
facilities is
announced and
available for review
Stakeholders
participate in
program launch,
including training
and education
New REMS programs
are integrated into
existing operations or
workflow
Full operation Information
regarding what is
working and what is
not working in a
REMS program is
publicly available
Stakeholders
provide input as to
what is working and
what is not working
in a REMS
program; and
modify program to
facilitate the
purpose of the
REMS
REMS programs are
continuously monitored
to assess “how it is
working” without
significant effort
Innovation Information and
process used in
identifying
programs that
need adjustments
are publicly
available
Stakeholders work
together to
improve the
program so that it
is more efficient at
achieving its goal
Stakeholders
consistently provide
input on REMS
effectiveness to help
improve the program
Sustainability Information and
process used to
determine if a
REMS program
should be sustained
Stakeholders work
together to monitor
and improve the
program; or
determine that the
Consistent approach is
used to assess the
program at regular
intervals after
implementation.

65
or rescinded are
publicly available
program is no
longer needed

2.10 Additional Efforts and Issues with REMS Since the Launch of the Survey
As previously discussed, FDA launched the REMS Integration Initiative in 2011 which
ended in 2017. In totality, FDA held 3 public meetings with its stakeholders, 5
workshops, 15 listening sessions, and a REMS structured product labeling pilot program
with 9 REMS application holders to test and refine the REMS data model/technology.
As such, specific stakeholder concerns were addressed with FDA publishing 5 guidance
documents, 3 of which were published after the close of this survey. The guidance
documents published include: 1) Use of a Drug Master File for Shared System REMS
Submissions, 2) Format and Content of a REMS Document, 3) Providing Regulatory
Submissions in Electronic Format, 4) FDA’s Application of Statutory Factors in
Determining When a REMS Is Necessary, and 5) Risk Evaluation and Mitigation
Strategies: Modifications and Revisions (Food and Drug Administration, 2017b).
Guidance documents 1 through 3 listed above were all published after the survey was
closed and 4 and 5 were published in 2016 and 2015, respectively. It will take some
time to determine if these new guidance documents are helpful.
One of the priority projects previously discussed focused on the standardization of REMS
by integrating REMS in structured product labeling (SPL) format. SPL is a data
standard for capturing information about drug products and covers product information
beyond product labeling. REMS in SPL makes REMS information easier to understand,

66
more accessible and helps integrate REMS into the care process. This pilot wrapped up
as a result of the end of the REMS Integration Initiative and FDA is currently working on
drafting a guidance document such that REMS submissions would be required in SPL
format (Food and Drug Administration, 2017b).
With the ending of the REMS Integration Initiative, other activities are underway to
improve REMS implementation. For example, the REMS Platform Standards Initiative,
created in 2015, is working to standardize REMS and integrate the program into the
health care system. The initiative’s goal is to leverage electronic health data standards to
standardize certain activities in REMS with ETASU and integrate them into health IT
systems.
Other interesting events that have taken place since the close of the survey relate to how
the industry is abusing REMS systems for commercial benefit. Criminal and civil actions
were filed against Aegerion Pharmaceuticals for failing to comply with REMS. On
October 22, 2017, Aegerion Pharmaceuticals Inc. agreed to plead guilty in the United
States District Court for the District of Massachusetts to two misdemeanor counts of
violating the FD&C Act involving the introduction of misbranded Juxtapid into interstate
commerce. Juxtapid was deemed misbranded because Aegerion failed to comply with
the requirements of the REMS program and because the drug’s labeling lacked adequate
directions for all of Juxtapid’s intended uses. Aegerion did not follow REMS
requirements to distribute to a narrow indication such that they sought to render the rare
disease diagnosis as vague and indefinite as possible to extend the product use to
additional patient populations. This resulted in both Aegerion and one of its vice

67
presidents entering in to a consent decree. The consent decree includes a
comprehensive compliance program and legal tools for FDA to enforce that the
defendants comply with the law, which is subject to judicial oversight (Food and Drug
Administration, 2017a). FDA is committed to enforcing legal actions for noncompliance
and will continue to pursue those not following their approved REMS. In regard to this
incident, FDA Commissioner Scott Gottlieb, M.D. stated:
… By failing to follow the safety requirements that Aegerion had
agreed to, the company put patients’ lives at risk and didn’t honor the
safety commitments they made as a condition of gaining approval for
their drug. This is unacceptable. We will continue to pursue those who
skirt the law, and flout patient safety and other post-market
commitments, using all of the enforcement tools available to us. Post-
market safety requirements are a key element of the FDA’s public
health protections and we will ensure that they are fulfilled.

Also during this time, REMS have come under fire because the closed distribution
systems imposed as part of a REMS are being used by companies to manipulate product
access for commercial advantages. For example, generic companies need access to
samples for analytical and clinical testing to generate data needed to bring a generic
product to market. However, innovator companies are utilizing the closed distribution
system as part of a REMS to restrict access to the samples. Also, FDA is on the lookout
for brand drug makers that deliberately slow collaboration on REMS to delay generics
from coming to market. FDA wants to make it easier for innovator companies and
generic drug makers to develop a common file for the implementation of REMS for the
same chemical entity. When the innovator companies drag out these negotiations, it’s a

68
stall tactic for generic entry to the market. In light of these REMS abuses, the FDA has
released two new draft Guidance documents on developing a shared system REMS, and
waivers associated with the single, shared system REMS requirement in June 2018
(Brennan, 2018).
Congress has taken notice of the REMS abuse and created the Creating and Restoring
Equal Access to Equivalent Samples (CREATES) Act which takes steps to curb abuses
of REMS and non-REMS restricted distribution programs and enables participation in a
shared REMS. The CREATES Act allows a generic drug manufacturer facing a delay
tactic from the denial of samples to bring an action to federal court for injunctive relief to
obtain the samples needed. Furthermore, the new law, if enacted, would allow the FDA
to approve alternate REMS programs rather than require the innovator company and
generic company to collaborate on a shared safety protocol (Dabrowska, 2018). There
are many supporters for obvious reasons as this promotes the creation of affordable
generic drugs. Opponents of the Act indicate that it would encourage unnecessary
litigation and upend a carefully balanced REMS by allowing multiple REMS programs
for the same drug. Though the CREATES Act is a bipartisan bill, it has yet to receive the
support needed to be passed.

69
CHAPTER 3. METHODOLOGY
3.1 Introduction
The purpose of this study was to assess the views of industry regarding the challenges
that might limit the implementation and improvement of REMS systems. A novel survey
instrument was developed by utilizing a framework based on concepts from
implementation science in order to examine the elements of transparency, collaboration
and harmonization/degree of burden across the certain identified stages of program
implementation. A preliminary draft of the survey was presented to a focus group whose
observations, comments and recommendations were used to revise the survey.
Respondents of the survey were solicited from a population of industry professionals
involved in REMS program development.
3.2 Development of the Initial Survey
The survey was developed using the well-regarded electronic survey platform, Qualtrics
(www.qualtrics.com). Qualtrics software enables online data by providing a sophisticated
tool for designing, distributing and evaluating the survey. It allows survey questions to
be compiled in a number of different question formats, including multiple choice, Likert
scale and text box options. An initial survey of about 30 questions was drafted. The first
few questions established the backgrounds and affiliations of the respondents and the
remaining questions focused on areas of interest identified by the previously discussed
framework.

70
3.3 Survey Modification by Focus Group Input
Nine people from industry and academia who have a strong knowledge of REMS and/or
experience with survey development were invited to participate in the focus group (Table
5).
Table 5. Focus Group Participants
Participant Title
Frances Richmond, PhD Director and Professor, International Center For
Regulatory Science
Eunjoo Pacifici, PharmD, PhD President-Elect, PFC
Director, International Center for Regulatory
Science
Assistant Professor, Department of Regulatory
and Quality Sciences
Nancy Pire-Smerkanich, DRSc, MS Assistant Professor, Department of Regulatory
and Quality Sciences
Michael Jamieson, DRSc Adjunct Professor, Titus Family Department of
Clinical Pharmacy and Pharmaceutical
Economics & Policy
C. Benson Kuo, PhD Associate Director, Consulting Center,
International Center of Regulatory Science
Wendi Lau, MS Executive Director, Business Operations &
Portfolio Analytics, Astellas Pharma
Penny Ng, MS Senior Director, Regulatory Affairs, Celgene
Donatus Ako-Arrey, MS Senior Director, Regulatory Affairs,
Mallinckrodt Pharmaceuticals
Meredith Smith, PhD, MPA Global Risk Management Officer, Global
Patient Safety, Amgen

The objective of the focus group was to critique and approve the draft survey by
examining its content, formatting and clarity. The focus group participants were provided
with an electronic version of the survey prior to the meeting and also with a paper copy at
the focus group meeting. A 90-minute face-to-face meeting at the University of Southern

71
California was conducted with five participants attending in person and four via
teleconference or videoconference. The meeting began with a short presentation of the
intended research and purpose of the study, followed by a discussion of the survey
questions and intended survey recipients. The survey questions were subsequently
modified to incorporate comments from the focus group participants.
The final survey was developed to have 34 questions and was validated prior to
dissemination to research participants by first sending it to several individuals who are
either past or present students or faculty in the doctoral program. This step was important
to ensure that the email notification arrived properly, and the answers were returned and
analyzed appropriately. A few issues were identified and were resolved prior to survey
launch.
3.4 Survey Deployment and Analysis
Initially, the target population was intended to obtain responses from 75 industry
professionals involved in the development of REMS programs. Potential respondents
were identified by examining product approval letters on the FDA website for products
that have been required to have REMS and supplemented by identifying attendees at
FDA workshops and symposia related to REMS. The survey was also distributed to DIA
Pharmacovigilance workshop attendees. In addition, fellow classmates and contacts
through colleagues known to be active in developing REMS were solicited. All potential
participants were contacted by email to identify their willingness to receive the survey,
and to clarify that they met the criteria for inclusion into the surveyed populations. The
survey link was distributed through Qualtrics initially to all who agreed to participate.

72
Survey recipients also had the option to forward the survey to other individuals who they
felt may be appropriate participants in the survey based on their knowledge and
participation in REMS programs. Some individuals had some issues opening the survey
which was due to the way I had distributed the link. This was corrected and emails with
links to the survey were sent out to some willing participants again. Due to slow uptake
of the survey, a link to the survey was also posted on LinkedIn to a closed REMS group
which consisted of 700 members.
Potential respondents were assured that their responses will be anonymous, and that they
will be able to receive a copy of the results after the survey is analyzed. The respondents
who have requested a summary of the results will be provided a high-level overview of
the survey results after analysis.
The survey results were collected and saved electronically. Survey responses were also
printed, and a second copy was saved electronically in a separate location. Data collected
was analyzed and displayed using descriptive text, tables, and figures. Many comments
were received and were analyzed for trends or common topics.

73
CHAPTER 4. RESULTS
4.1 Analysis of Survey Results
The survey was open from March until September 2017. The survey was initially
disseminated to individuals identified to have worked on REMS programs through
literature, FDA public hearings, and professional connections. This group of
approximately 200 potential survey participants was contacted individually by email in
March 2017 and invited to participate. Emails with the survey link were sent out to 193
contacts; 30 surveys were started and 26 responses were received to provide an 80%
completion rate. After additional solicitation, a total of 41 participants initiated the
survey with three that indicated that they had not worked on a REMS and were taken
directly to the end the survey. Therefore, the total number of respondents who completed
the survey was 38. Not all respondents, however, completed every question in the
survey. Hence, the number of respondents corresponding to each question has been
reported along with the results.
4.2 Profiles of Respondents
Respondents were mostly from large companies with over 2000 employees (68%, 26/38),
followed by mid-size companies with 250 – 2000 employees (16%, 6/38) and small
companies with less than 250 employees (16%, 6/38), as shown in Figure 1.

74
Figure 1. Size of Company
What is the size of the company for which you worked on a REMS? (can be current
company or previous experience)

Most respondents have experience working with more than one REMS. For example,
73% (26/36) have worked on at least two REMS programs and 42% (15/36) on more than
three (Figure 2).

75

Figure 2. Number of REMS Programs Handled by Respondents
How many REMS have you worked on?

Figure 3 shows that the job positions of respondents varied but most were at the director
level at their respective companies (39%, 14/36). Three respondents who chose “other”
commented that they were either a senior director or executive director and, hence,
should also be included in the director category. Other categories of roles included
Associate/Specialist (14%, 5/36), Manager (11%, 4/36), and Consultant/CRO (22%,
8/36). None were at the VP/CEO level.

76

Figure 3. Roles of Respondents
What job level best describes your role when you worked on a REMS?

When asked which department at their companies was responsible for developing REMS,
30% (10/36) respondents selected Regulatory Affairs and, similarly, 31% (11/36)
selected Safety (Figure 4). Twelve respondents who chose “Other” further clarified their
choices with comments (Table 6). Only 3 respondents chose Medical Affairs, and none
chose Marketing/Market Access.
5
8
0
14
4
5
0 2 4 6 8 10 12 14 16
Other
Consultant/CRO
VP/CEO
Director
Manager
Associate/Specialist

77

Figure 4. Department Responsible for REMS
Which department at your company has primary responsibility for developing REMS?

12
0
11
3
10
0 2 4 6 8 10 12 14
Other
Marketing/Market Access
Safety
Medical Affairs
Regulatory Affairs

78
Table 6. Comments on Which Department has Responsibility for Developing
REMS
Which department at your company has primary responsibility for developing REMS?
Epidemiology
Risk and Program Management
Not applicable since I work for PMO.
REMS Customer Management
It was shared by RA/commercial and safety
Our Company has a separate REMS Operations Group
My company is a vendor who implements REMS and then continues to manage the operations
after implementation. It is the responsibility of the implementation team to develop the REMS
with the FDA, the sponsor(s), and all internal cross functional work streams.

The number of people working on REMS at a company varied, with 43% (12/28)
reporting 1 to 10 people, 36% (10/28) reporting 11 to 50 people, 21% (6/28) reporting
more than 50 people (Figure 5).

79

Figure 5. Number of People Working on a REMS at the Company
How many people at your company currently work to maintain your REMS program(s)?

Internal company communications (30) and FDA interactions (27) were the most
frequently selected activities engaged by the survey respondents (Figure 6). But many
also participated in external interactions with patients/patient advocacy groups (11),
healthcare providers (16), and pharmacies (13). Thirteen respondents selected “other”
activities, which were further characterized in Table 7 to include CRO activities,
interactions with other regulatory authorities, and REMS implementation studies.
6
10
12
0 2 4 6 8 10 12 14
More than 50
11-50
1-10

80

Figure 6. Activities with REMS
What best describes your activities in regard to the REMS? (check all that apply)

13
30
13
16
11
27
0 5 10 15 20 25 30 35
Other
Internal company communication
Contact with pharmacies
Contact with health care providers
Contact with patients/patient advocacy
groups
FDA interaction

81
Table 7. Comments for Other Activities in regard to REMS
What best describes your activities in regard to the REMS? (check all that apply)
Interaction with other global regulatory authorities interested in conditions of US
REMS programs
Program management, contact with sponsors and vendors
CRO Interaction
Develop and implement REMS evaluation studies.
IRB
Manage vendors for sponsors with REMS requirements
program management
Program Management Office
Support the Patient Services group that facilitates the program.
REMS Administrator, REMS Consulting and Strategy
Work directly with vendors who implement the HCP surveys as well as build the
implementation systems, also pregnancy registry vendors, advertising companies,
printers
Shared REMS program program/vendor/financial management

Although REMS programs handled by the respondents’ organizations spanned across
many different therapeutic areas (Figure 7 and Table 8), oncology (16) and pain (13)
were most frequently selected.

82
Figure 7. REMS Therapeutic Areas
Check all therapeutic area(s) that are covered by your REMS program(s)

Table 8. Other Therapeutic Areas
Check all therapeutic area(s) that are covered by your REMS
program(s)Infectious disease
Neurology
Dermatology
Rheumatology
Gastroenterology
Cardiology
Smoking cessation
HIV prophylaxis
18
7
4
6
10
9
13
16
0 5 10 15 20
Other
Metabolic/Endocrinology
Reproductive/Urology
Psychiatry
Immunology
Rare Disease
Pain
Oncology

83
Inflammation
Retinoid
Osteoporosis

Small molecules (25) were most frequently selected by the respondents as the type of
molecule covered by their REMS, followed by biologics (15), generics (12), and
biosimilars (2) (Figure 8).
Figure 8. Types of Molecules
Check all the type(s) of molecules that are covered by your REMS program(s)

25
12
15
2
Small molecule Generic drug
Biologic Biosimilar drug

84
Most respondents reported that they were experienced across all three main elements of
REMS; Medication Guide (30), Communication Plan (28) and Elements to Assure Safe
Use (32) (Figure 9).
Figure 9. Elements of REMS Worked on by Respondents
Check all types of REMS programs in which you were involved to date

4.3 How companies modify their REMS Programs – Innovation and Sustainability
Respondents were polled on what resources they consult when modifying a REMS
program. As can be seen in Figure 10, documents and resources available from FDA are
the most widely used resources. Key opinion leaders and peer reviewed literature are
also frequently consulted. On the other hand, most respondents did not access CIOMS
Working Group IX, EMA GVP Module V, or EMA GVP Module XVI when modifying
32
28
30
0 5 10 15 20 25 30 35
Elements to Assure Safe Use
Communication Plan
Medication Guide

85
their REMS. Items in the “other” category included resources from manufacturers and
companies as well as experiences gained from previous REMS programs (Table 9).
Figure 10. Documents Consulted When Modifying REMS
What resources do you consult when modifying a REMS program? (check all that apply)

2
3
5
7
14
15
21
22
29
29
0 5 10 15 20 25 30 35
EMA GVP Module XVI
EMA GVP Module V
CIOMS Working Group IX
Other
Key opinion leaders
Peer reviewed literature
Product REMS websites
REMS@FDA
FDA guidance documents
FDA regulatory documents

86
Table 9. “Other” Documents Consulted When Modifying REMS
What resources do you consult when modifying a REMS program? (check all that apply)
Results from REMS evaluation studies
Manufacturers
Internal stakeholders to review feedback from REMS stakeholders, audit results etc.
Companies that specialize in this area
Manufacturers of products in the REMS Program
looking to see what precedence is set for other products is useful. Having worked on so
many and having so many contacts in the industry, it’s easier to follow trends in FDA
thinking

4.4 Views on Transparency
Most respondents strongly agreed or somewhat agreed that FDA provided easy access to
information needed to manage REMS, but many had mixed views regarding the
transparency of its actions (Table 10). For example, while 71% (22/31) strongly agreed
or somewhat agreed that information needed to manage REMS program was easily
accessible, 45% (14/31) somewhat disagreed or strongly disagreed that FDA was
transparent in its rationale for modifying or rescinding a REMS program and 52%
(16/31) somewhat disagreed or strongly disagreed that the FDA communicated its actions
in a timely manner.

87
Table 10. Statements on Transparency
Thinking about REMS programs on which you have worked, please provide your level of
agreement with the following statements.

Similarly, most respondents (61%, 17/28) felt that the FDA provides adequate direction
on how to sustain or modify REMS programs (Figure 11) but some of the comments
(highlighted in bold) provided by this group point to areas of inadequacy (Table 11).
However, the strongest comments were received from respondents who answered “no”
(Table 12). Their comments point to deficiencies by the FDA in communications and
timeliness.

Question
Strongly
agree
Somewhat
agree
Combined
agree
Neither agree
nor disagree
Somewhat
disagree
Strongly
disagree
Combined
disagree
Cannot
say Total
Information needed to manage my
REMS program is easily
accessible
4 18 22 2 6 1 7 0 31
FDA's anticipated actions on a
REMS program are transparent.
6 6 12 5 7 5 12 2 31
The REMS program has provided
market access to my product that
would otherwise not have been
approved.
5 7 12 4 4 5 9 6 31
REMS programs make it difficult
for patients to have access to these
medications.
1 10 11 5 6 7 13 2 31
FDA's rationale for modifying or
rescinding a REMS program is
easily accessible to key
stakeholders.
4 7 11 5 11 3 14 1 31
FDA's anticipated actions on a
REMS program are
communicated in a timely manner
7 4 11 3 9 7 16 1 31

88
Figure 11. FDA Provides Adequate Direction on How to Sustain or Modify
REMS
Do you feel that the FDA provides adequate direction on how to sustain or modify your
REMS program? (please provide any relevant examples)

17
11
Yes No

89
Table 11. Comments from Respondents that Answered “Yes”
Do you feel that the FDA provides adequate direction on how to sustain or modify your
REMS program? (please provide any relevant examples)
collaborative communication
All communication is done through pharma.
Reviews periodic evaluation reports and provides feedback.
FDA will send a letter with information on what they are looking for - generally
speaking that is - but as for specific instructions, FDA seems to leave that to the
organization managing the REMS to interpret
FDA provides specific modification requests
Although we can usually discern the agencies direction for modifications to REMS
program many times it requires communication with the FDA to clarify the scope
and requirement
Most of the time, modifications are based on the annual report/assessment data.
If no communication from FDA, direction is to continue as normal.
Removal/modification of existing REMS to make more easily accessible by
patients or other stakeholders is rarely/never initiated by FDA.

90
Table 12. Comments from Respondents that Responded “No”
Do you feel that the FDA provides adequate direction on how to sustain or modify your
REMS program? (please provide any relevant examples)
The FDA does not have a model as to how REMS should operate.
FDA could do a better job explaining why items are needed.
The iterative review process for a REMS modification does not typically include
rationale for changes made
They have simply stated that "the program is not meeting its goals" without elaborating
on why, which goals, or how we can improve
FDA provides comments in a piecemeal fashion which is not strategically helpful when
modifying the educational pieces which also affects the website and other stakeholder
materials. There are examples of when the Agency requests a certain closed distribution
for drug delivery for which only one vendor is available which leads to higher costs and
the quality of their work is substandard. The Agency does not have great insights or
understanding to what it takes to make any changes and some edits they make are
fundamental system changes which affect the implementation system and potentially
patient safety or access
Responses during submission work could be provided in a timelier fashion,
teleconferences help to understand the meaning behind some of the FDAs comments.

Although a majority (59%, 17/29) thought that the activities of FDA regulators are
transparent with regard to REMS (Figure 12), the comments by this group indicated, as
highlighted in bold, that the FDA’s responses were slower than ideal (Table 13).
Respondents in the “no” group provided extensive comments on how FDA’s work is
often not transparent, and its explanations are not always complete (Table 14).

91
Figure 12. Regulators are Transparent with Regard to REMS
Do you think that the activities of FDA regulators with regard to REMS are transparent?
(please provide any relevant context)

Table 13. Comments from Respondents that Responded “Yes”
Do you think that the activities of FDA regulators with regard to REMS are transparent?
(please provide any relevant context)
but, slower than would be optimal
this is a very broad question, so I cannot answer having worked on 2 REMS projects
Somewhat transparent, often delayed
Typically, the regulatory fashion of REMS programs is focused on the elements to
assure safe use.

17
12
Yes No

92
Table 14. Comments from Respondents that Responded “No”
Do you think that the activities of FDA regulators with regard to REMS are transparent?
(please provide any relevant context)
their work off-site is not transparent
Explanations are not always complete.
While FDA clearly communicates about its goals for standardization, the reality of
implementation of REMS approvals and modification is far from consistent.
Very broad question that is difficult to answer yes / no.
The discussion of standardization is one that has been made public but you cannot
standardize language and requirements because they should be tailored to the patient
population as well as the specialty who treats the patients. Drugs dispensed only in the
hospital in an inpatient setting should not have implementation REMS as the drug is
being used in a hospital with trained medical professionals It increases cost of drug,
burden to prescribers and pharmacists not to mention the nursing staff
FDA decision making if/when a REMS is required is often difficult to obtain - instead
"risks outweigh benefits" or "benefits outweigh risks" are often thrown around without
specific reference to either benefit or risk. No objective approach to risk management
by the FDA, leading to judgement calls on all sides to require a REMS, and further,
what to include in a REMS.

Communication problems continued to be highlighted with a majority of respondents
(59%, 13/22) indicating that the FDA does not provide clear guidance on how a program
can be rescinded (Figure 13). One respondent commented that there were situations
where a company was “unexpectedly released from REMS” or denied requests to remove
specific requirements. Both groups provided comments to further elaborate their
responses (Tables 15 and 16). As can be seen in Table 15, even among those respondents

93
who thought that the FDA provided clear guidance, there were comments indicating that
FDA can be vague.

Figure 13. FDA Provides Clear Guidance on how a Program Can Be Rescinded
Thinking about the REMS program(s) on which you have worked, does FDA provide
clear guidance on how a REMS program can be rescinded? (please elaborate)

9
13
Yes No

94
Table 15. Comments from Respondents that Responded “Yes”
Thinking about the REMS program(s) on which you have worked, does FDA provide
clear guidance on how a REMS program can be rescinded? (please elaborate)
written documentation
It was clear due to interaction with FDA and their desire to be collaborative
It is clear that if you have a communication plan REMS after you have demonstrated
that the HCPs understand the risks and that your Communication plan worked you can
ask to remove the requirement. Also if you have a REMS for a potential risk and you
can show data to support that your product does not have the risk that piece of the
REMS can be eliminated.
Vague, but still clear.
I have seen both sides, FDA typically provides clear guidance, however, I have seen
cases where they do not.

Table 16. Comments from Respondents that Responded “No”
Thinking about the REMS program(s) on which you have worked, does FDA provide
clear guidance on how a REMS program can be rescinded? (please elaborate)
we have had examples where we were unexpectedly released from REMS, and another
that despite good effectiveness results for the 18-mo and 36-mo assessments, our
request to remove the survey assessment requirement (on a medguide only rems) was
denied.
Typically, FDA does not provide this guidance and it is on the sponsor to prove the
REMS is not needed.

It is important to note that only 32% (10/31) of respondents have been involved with a
REMS program that has been rescinded (Figure 14). Of them, 71% reported that a clear
rationale was given by the FDA for its decision to rescind (Figure 15, Table 17).

95
Figure 14. Have You Been Directly Involved with a REMS Program That Has
Been Rescinded?
Have you been directly involved with a REMS program that has been rescinded?

10
21
Yes No

96
Figure 15. Was Clear Rationale for Rescission Provided?
To your knowledge, did FDA provide a clear rationale for their decision to rescind the
REMS program? (please elaborate if yes)

Table 17. Comments from Those that Responded “Yes”
To your knowledge, did FDA provide a clear rationale for their decision to rescind the
REMS program? (please elaborate if yes)
After the client proved the REMS was not needed, the FDA provided clear agreement,
which was the rationale.
The REMS program clearly stated the FDA would review the program to determine if
the goals were being met and if continuation of the REMS would be necessary. The
newer REMS program do not include this language.

5
2
Yes No reason given

97
4.5 Collaboration on innovation and sustainability
Most of the respondents (85%, 23/27) thought that their companies collaborate
sufficiently with stakeholders (FDA, healthcare providers, pharmacies and patients) to
modify a REMS program (Figure 16) but some of the comments like “somewhat” and
“At the time it appeared so” indicate a level of uncertainty (Table 18 Table 19).
Figure 16. Communicates Regularly with Stakeholders
Do you feel you that your company collaborates sufficiently with stakeholders (FDA,
Health care Providers, Pharmacies, Patients) to modify REMS programs?

23
4
Yes No

98
Table 18. Comments from Respondents That Responded “Yes”
Do you feel you that your company collaborates sufficiently with stakeholders (FDA,
Health care Providers, Pharmacies, Patients) to modify REMS programs?
somewhat
At the time it appeared so
the company can propose modification based on feedback from HCPS but the FDA has
to agree.
We try to gain insights from the stakeholders before proposing changes or
modifications. It is important to obtain stakeholder feedback as they are impacted by
the changes. We also engage the stakeholders in our discussions with the Agency when
Agency comments/requests are not aligned with current medical practice or the
marketplace.
Most REMS use Scientific Advisory Boards or conduct user testing around major
modifications
Communication to all stakeholders is adequate.

Table 19. Comments from Respondents That Responded “No”
Do you feel you that your company collaborates sufficiently with stakeholders (FDA,
Health care Providers, Pharmacies, Patients) to modify REMS programs?
As a REMS Administrator we deal mainly with manufacturers when it comes to REMS
modification discussions

More respondents strongly agreed or somewhat agreed that healthcare providers (54%,
15/28), pharmacies (59%, 16/27), and patients (52%, 14/27) are provided with clear
communication regarding when and why REMS are modified or rescinded (Table 20). In
looking at stakeholder involvement, more respondents somewhat disagreed or strongly
disagreed that healthcare providers (45%, 13/29), pharmacies (41%, 12/29), and patients
(46%, 13/28) are involved in modifying REMS (Table 21). Overall, more respondents

99
strongly agreed or somewhat agreed that they regularly communicate with health care
providers (54%, 15/28), pharmacies (59%, 16/27), and patients (51%, 14/27) (Table 22).

Table 20. Statements on Communication with Stakeholders when REMS are
Modified or Rescinded
Regarding existing REMS programs, please provide your level of agreement with the
following statements.

Question
Strongly
agree
Somewhat
agree
Combined
agree
Neither
agree nor
disagree
Somewhat
disagree
Strongly
disagree
Combined
disagree Total
Health care providers are
provided with clear
communication regarding
when and why REMS are
modified or rescinded.
5 10 15 6 6 1 7 28
Pharmacies/Pharmacy
Groups are provided with
clear communication
regarding when and why
REMS are modified or
rescinded.
5 11 16 4 4 3 7 27
Patients/Patient Advocacy
Groups are provided with
clear communication
regarding when and why
REMS are modified or
rescinded.
2 12 14 3 6 4 10 27

100
Table 21. Statements on Collaboration with Stakeholders
Regarding modifying REMS programs, please provide your level of agreement with the
following statements.

Question
Strongly
agree
Somewhat
agree
Combined
agree
Neither agree
nor disagree
Somewhat
disagree
Strongly
disagree
Combine
d
disagree
Total
Health care providers are
involved in developing
modifications to REMS
programs.
0 12 12 4 7 6 13 29
Pharmacists/Pharmacy
groups are involved in
developing modifications
to REMS programs.
0 11 11 6 7 5 12 29
Patients/Patient advocacy
groups are involved in
developing modifications
to REMS programs.
0 7 7 8 7 6 13 28

101
Table 22. Statements on Regular Communication with Stakeholders
Regarding existing REMS programs, please provide your level of agreement with the
following statements regarding the way your company handles REMS.

4.6 Harmonization/Degree of Burden on Innovation and Sustainability
The respondents had many different comments to share when asked about their thoughts
on the biggest overall challenge with REMS programs (Table 23). There are, however,
some common themes that seem to emerge. One theme is the lack of consistency across
programs and reviewers. Another common theme is that of timing. According to
comments, FDA has been slow to communicate, slow to acknowledge difficulties in
programs and expect unrealistic timing for programs. Finally, another recurring
comment is the lack of choice of vendors for closed distribution systems as part of
Question
Strongly
agree
Somewhat
agree
Combined
agree
Neither agree
nor disagree
Somewhat
disagree
Combined
disagree
Don't
know
Total
Your company
communicates
regularly with
health care
providers.
9 10 19 6 2 8 2 29
Your company
communicates
regularly with
pharmacies/phar
macy groups.
8 11 19 5 1 6 2 27
Your company
communicates
regularly with
patients/patient
advocacy groups.
4 7 11 10 2 12 3 26

102
ETASUs. FDA is making sponsors work with one vendor which was seen to diminish
quality and deter competitive bids.

Table 23. Biggest Overall Challenge With REMS Programs
Please describe your biggest overall challenge with REMS program(s).
Dealing with the FDA is like dealing with the Wizard of Oz behind the green curtain.
You don't know what they are thinking or how they want programs to run. They have
been trying to standardize some aspects of REMS programs, but that is not apparent
unless you study changes to other REMS programs over time.
Best interest welfare and safety of patients.
Opioid REMS is the first to include accredited CME, which has been more difficult for
the FDA, the ACCME community and clinicians to understand
Pharma clients sometimes do not pre-test educational materials with patients or
physicians.
The REMS that I worked on was right after the passage of FDAAA in 2007. That
particular product was under a previous Risk Management Program or (RMP) and it
was challenging to convert the RMP's specific requirements for that product to the new
REMS framework. I was involved during the conversion phase of the RMP to REMS
for the product around 2009.
Evaluation of the performance of individual components of a given REMS program is
challenging.
Our challenge was that our REMS program was in 2008, early years when FDA didn't
provide much information or guidance so this was new to all of us in the pharma
industry.
The biggest challenge is not having a framework for evaluating the effectiveness of the
REMS. Knowledge, Attitude, Behavior surveys are the most commonly used
evaluation tool; however, the analyses are descriptive rather than predictive or
explanatory. So, if understanding is low for example, we can never truly point to a
reason why, then address it. It just doesn't make sense why FDA wouldn't demand or
come to expect reasons why. If we know why then we can at least attempt address

103
deficiencies in knowledge and behavior gaps, thus ultimately helping to improve a
drug's benefit risk profile.
lack of consistency and predictability; official FDA determination of whether a REMS
is needed comes too late in the review cycle.
FDA sets unattainable timelines. More guidance materials are needed. The vendor
pool needs to be expanded, especially for closed system REMS with ETASUs.
The FDA seems to think a REMS program is like a clinical study. The companies have
essentially no control over pharmacies and physicians.
Single product REMS are not written with the future in mind, as all REMS eventually
become shared REMS when the product loses exclusivity.
Communication between FDA and sponsors while working out changes to a REMS
program can be limited, slow, and thus less effective than it needs to be in order to
work out the details of REMS modifications at an appropriate level of detail in a timely
manner.
I was involved with REMS programs when they first were instituted so there has been
learnings over the years by FDA and industry. REMS are far from perfect but there
does seem to be some improvement in understanding of stakeholders concerns by the
FDA. Note: I don't currently work on any REMS programs.
The biggest challenge is lack of standardization across REMS Program. This causes
frustration across stakeholders, mainly prescribers and pharmacies, because each
program can be different causing stakeholders to implement differing processes for
different programs. In addition, the FDA's process for reviewing REMS submissions
and modifications is not well understood by the industry. Understanding their review
process would allow manufacturers to align their submission processes with the FDA
in a more constructive manner in order to improve the overall time to REMS approval.

104
FDA has been slow to acknowledge the hurdles with implementing the REMS
program, especially for products that have several risks and primarily affect an elderly
population that may not fully understand the multiple risks. This particular product is
given every 6 months so expecting an elderly patient to remember what their prescriber
told them several months prior is just not feasible. The REMS has been very costly
with minimal results.
Issues associated with REMS: The REMS I was involved in was required early in the
days of REMS implementation. There were issues with inconsistent use of REMS
between FDA divisions and across therapeutic areas. the amount of resource required
to design and implement a program was huge. I don't believe FDA had any sense of
what went into setting up a program that could actually meet the program "goals".
Modifying the REMS to remove the ETASU.
The Agency makes recommendations for restricted distribution, and currently there is
one vendor who has market exclusivity to this technology. They will not allow other
platform vendors to work with their pharmacy vendor. The Agency is making all
companies work with one vendor. There are quality issues, there are cost challenges as
you cannot get a competitive bid. Additionally, it appears the Agency does not
understand the process to implement any of the changes they request for educational
materials. It is not efficient to get feedback piecemeal with no apparent overall
strategy. There is no 2-way communication. It sometimes seem they want to hit a
timeline regardless of what it takes to build a platform and test it etc.
There is still some inconsistency between REMS programs that are similar in nature.
FDA has taken steps to standardize language and REMS elements, but more can be
done. Additionally, many REMS are moving toward using the insurance adjudication
transaction (i.e., the Switch) at the pharmacies. While this greatly reduces the burden
on Pharmacy (the REMS authorization can happen within the pharmacy management
system instead of outside of the pharmacist normal workflow), there is only 1 vendor
with this capability. They don't allow for REMS program sponsors to only purchase
this transaction service but bundle it with other REMS services like the call center and
IT services (i.e., website, database).
-Transition of single REMS to shared REMS program -Vendor pool considerations for
REMS services -Consistency of FDA communications and communications
expectations across REMS programs
Lack of operational/implementation experience within the FDA, leading to either too
much or too little interaction on how/should be the REMS is implemented, as well as
reviewer-to-reviewer variation with communication styles, preferences, evaluation, etc.

105
The rationale from FDA is not always clear as to their recommendations for
implementation
Implementation of a shared REMS program is very time consuming, mainly due to the
FDAs extended time to respond to a submission. The FDAs comments must be
assessed and input into implementation deliverables and then resubmitted for
additional feedback. I believe that easier scheduled teleconferences would assist in
streamlining this process. An additional challenge about shared REMS programs is
having multiple sponsors agree on requirements for a program.

The respondents had mixed views when thinking about consistency across REMS (Table
24). Amongst individual statements, 45% somewhat disagree or strongly disagree that
ETASUs are consistent across all products, and 61% somewhat disagree or strongly
disagree that REMS are consistent across therapeutic areas. More respondents strongly
disagree or somewhat disagree (39%, 12/31) that current FDA documents allow for
consistent modifications of REMS, while almost as many neither agree nor disagree
(32%, 10/31). More respondents strongly agree or somewhat agree that FDA provides
clarity on why modifications are needed (61%, 19/31) and that FDA engages industry to
discuss modifications (42%, 13/31).

106
Table 24. Statements on Consistency Across REMS
Thinking about your knowledge of REMS programs in general, please provide your level
of agreement with the following statements.

Respondents were asked to rank REMS attributes according to associated burden on the
health care delivery system (Table 26). To further analyze the ranking to clearly identify
the order, a weighted calculation was performed (Table 25) by giving items ranked first a
higher value or “weight”. In this rank order, there were four choices so the number of
Question
Strongly
agree
Somewhat
agree
Combined
agree
Neither agree
nor disagree
Somewhat
disagree
Strongly
disagree
Combined
disagree
Total
The elements of
ETASUs are
designed in a
consistent way
across all products.
1 10 11 6 9 5 14 31
REMS programs are
designed
consistently across
therapeutic areas.
1 2 3 9 11 8 19 31
Current FDA
guidance documents
allow for consistent
modifications to
REMS.
1 8 9 10 10 2 12 31
FDA provides
clarity to the
company on why
modifications to
existing programs
are needed.
5 14 19 7 3 2 5 31
FDA actively
engages industry to
discuss the
modifications to
their REMS
program.
5 8 13 8 7 3 10 31

107
statements ranked 1 were multiplied by 4, statements ranked 2 were multiplied by 3,
statements ranked 3 were multiplied by 2 and statements ranked 4 were multiplied by 1
and then the amounts were added to obtain a score for each statement. The weighted
calculation of the rank order results showed that most ranked “the need for health care
providers to take training and certifications” as the greatest burden, followed by a “lack
of harmonization across REMS programs” and “the need to monitor the outcome of the
implemented program” (Figure 17). Respondents who selected “other” commented that a
lack of vendor options, inconsistencies in FDA feedback, and the inability for patients to
opt out presented burdens to the healthcare system (Table 27).
Figure 17. Weighted Calculation of Rank Order of Most Burden on Healthcare
Delivery System
In your opinion, please rank which following puts the most burden on the health care
delivery system, 1 being the highest burden? (please click and drag in order)

32
71
78
87
0 20 40 60 80 100
Other
The need to monitor the outcome of the
implemented program
Lack of harmonization across REMS
programs.
The need for health care providers to
take training and certification programs.

108
Table 25. Weighted Calculation of Rank Order of Most Burden on Healthcare
Delivery System
In your opinion, please rank which following puts the most burden on the health care
delivery system, 1 being the highest burden? (please click and drag in order)
Question Rank 1
x 4
Rank 2
x 3
Rank 3
x 2
Rank 4
x 1
Total
Lack of harmonization across
REMS programs
36 15 26 0 78
The need to monitor the
outcome of the implemented
program
24 21 24 2 71
The need for health care
providers to take training and
certification programs
48 36 2 1 87
Other 0 6 2 24 32

Table 26. Results of Rank Order for Individual Statements
In your opinion, please rank which following puts the most burden on the health care
delivery system, 1 being the highest burden? (please click and drag in order)

Question Rank 1 Rank 2 Rank 3 Rank 4 Total
Lack of harmonization
across REMS programs.
9 5 13 0 27
The need to monitor
the outcome of the
implemented program.
6 7 12 2 27
The need for health
care providers to take
training and
certification programs.
12 13 1 1 27
Other 0 2 1 24 27

109
Table 27. “Other” Comments on Most Burden on Healthcare Delivery System
In your opinion, please rank which following puts the most burden on the health care
delivery system, 1 being the highest burden? (please click and drag in order)
Lack of vendor options for ETASUs, which results in higher prices and lower quality.
lack of consistency in FDA feedback/rationales
Challenges for incorporating requirements of REMS into the HDS so that
stakeholder can 'easily' comply
If by monitoring the outcome of the program- do you mean the surveys that have to be
implemented for HCPS and sometimes patients? That is expensive and really adds no
value. The PV aspects of the program are already being implemented registries, drug
safety processes, so if that is what is meant by monitor outcomes its part of the pharma
business. Offering training materials is not a problem- its tracking who did the
training and having them recertify at predefined times. Depending on the REMs, they
may not be in a platform or system for the REMs so the follow up to get them to
recertify is also burdensome and costly.
Hard stops for patients with life-threatening conditions. No option for patients to opt
out of a risk management program/REMS, based on personal
accountability/recognizance.

When asked about the greatest barriers to developing a harmonized REMS program, a
recurring theme in comments was the lack of information-sharing across companies for
competitive reasons (Table 28).

110
Table 28. Comments on Greatest Barriers to Developing Harmonized REMS
Thinking about the REMS program(s) on which you have worked, what are the greatest
barriers to developing a harmonized REMS program across the products in the same
class or therapeutic area?
The commercial teams and companies are competitive with each other, and the FDA
has not done anything to discuss common aspects that should be in REMS programs in
the same therapeutic area.
executives need to serve as the governing decision makers, in order to remediate
competing opinions
Unsure
federal and state legislation that determines educational needs, lack of communication
between FDA and DEA, as well as medical communities
We had communication only with one pharma client who marketed only 1 product
relevant to the REMS programs
I know that the particular REMS I worked on now is applicable to both the original
drug and a generic and it appears that at least in that respect there is harmonization
across same class products.
lack of shared (public) information on the successes and failures of previous REMS.
Doesn't allow for a learning system. Many companies are reluctant to share this
information publicly (not mine!) and the FDA does not make the assessment reports
public on its website.
Different REMS goals.
Communication. The FDA does not communicate well and companies don't talk to
each other as they are competing for the same prescription.

111
Harmonizing for products in the same class is fairly straight forward (although even
this can be a challenge depending on the NDA and ANDA holders involved); it
becomes more challenging across therapeutic areas and nearly impossible across all
programs. A one-size-fits-all approach does not take into account the nuance of
different products. For example, there are programs where the risk of not getting drug
because of REMS barriers is as serious if not more serious a risk than the risk the
REMS is mitigating. Also, use of technology, like integration with the pharmacy
network, is not practical for all products.
The delivery system is complex and as more stakeholders move to electronic medical
records or paperless systems, how to design REMS Program and the support in a
method that all stakeholders can access. And access outside of the sponsor call center
business hours. The elements of each program vary and for collection purposes, a
registry cannot be 'off the shelf' as the requirements need to be built to support the IT
infrastructure and the cost and time associated are burdensome. Even within the same
therapeutic area, if requirements are not in synergy, it is difficult to design systems that
are generic enough to support both REMS program. For harmonization, if the
organization wants to have 1 platform or registry, how does the organization set up an
online presence or portal presence that can support all stakeholders within 1 therapeutic
class?
You can't develop a REMS program across therapeutic area since it is product specific.
Also, FDA determines if the same class of products have the same REMS program
even that is difficult. One example of REMS for same class was the ESA REMS
developed by Amgen with agreement with FDA.
I don't think harmonization by TA would always work as harmonizing completely
different drugs just because they are both in the same area won't always work. The
same class of drug can be harmonized but there are potential legal concerns among
others if companies had to work together. This isn't a simplistic issue.
The biggest barrier is obtaining alignment and consensus among the manufacturers to
REMS program requirements.
The burden on physicians and prescribers makes it more likely that they would choose
a medication without or with a simpler REMS over a more complicated REMS in the
same product class. Even within a class, the differences in risks make harmonization
slightly more cumbersome.

112
The REMS I worked on was for a 1st in class product. The issues being addressed
were unique to that product so there was no harmonization required across the class
and therapeutic area. The biggest difficulty for the REMS we were required to put
together was that it was inconsistent with precedent across other therapeutic areas (i.e.
it required laboratory monitoring while other products with similar issues were not
subject to REMS).
The REMS program was harmonized across products in the same class, but actually
should have been differentiated slightly from the start due to slightly different safety
profile.
The barrier issues are when there is a class REMS and your product is in the class but
is treated in a hospital setting for example. Should you have all the same ETASU as
another product that is in retail setting? Harmony is appropriate when the drugs are all
in retail or specialty pharmacies. It does not make sense when the drug is used in a
controlled environment like a hospital acute care setting. The Agency needs to truly
understand the healthcare setting that product is dispensed or used vs blanketing a drug
with ETASU due to class- especially if the risks are managed at the hospital and the
patient is not discharged with that product. It’s a barrier to use in the hospital when you
have a REMS with ETASU.
The burden on the prescriber and pharmacist are the biggest challenges, especially in
program that are not mandatory (i.e., if you can still prescriber even if you are not
enrolled).
-variations in ETASU -timing of approval and difficultly of modifying REMS program
retroactively -inconsistency in FDA expectations across programs
Varied patient populations, resistance to additional work by stakeholders, lack of FDA
understanding of the health care system, different product risks and etiologies.
With multiple vendors and multiple pharmaceutical companies involved in this process
it is difficult to develop a standard that would harmonize REMS programs.

To gain insight on feasibility on harmonizing REMS, participants were provided a series
of statements and asked their importance on a scale between strongly agree and strongly
disagree. In response to feasibility of harmonizing REMS programs across therapeutic
areas, the number of respondents who agree (41%, 11/27) is equal to those who disagree
that it is feasible to harmonize REMS programs across therapeutic areas. However, a

113
majority of respondents strongly agree or somewhat that it’s feasible to harmonize a
REMS within a therapeutic area (70%, 19/27) and within a product class (86%, 24/28)
(Table 29). Respondents were given space to further comments on their choices (Table
30, Table 31, Table 32).
Table 29. Feasibility of Harmonization
Regarding REMS programs, please provide your level of agreement with the following
statements. (please elaborate in boxes provided)

Question
Strongly
agree
Somewhat
agree
Combined
agree
Neither agree
nor disagree
Somewhat
disagree
Strongly
disagree
Combined
disagree
Total
It is feasible to
harmonize REMS
programs across
therapeutic areas.
4 7 11 5 9 2 11 27
It is feasible to
harmonize REMS
within a therapeutic
area.
7 12 19 2 5 1 6 27
It is feasible to
harmonize REMS
within a product
class.
11 13 24 1 2 1 3 28

114
Table 30. Comments on feasibility to harmonize REMS programs across
therapeutic areas
Regarding REMS programs, please provide your level of agreement with the following
statements. (please elaborate in boxes provided)
not always
Different products have different needs, so harmonization is possible on a high level
but needs to be limited at a detailed level
To the extent that the underlying issue is similar. exceptions could be trying to
harmonize
The decision must be based on patient population, how is the drug available, what is
the risk and who are the prescribers that treat the patient. Is one product easier to
manage then another and should that product be burdened because it’s in the
therapeutic area. I don't think generalizing is feasible.
High-level harmonization with variation, as needed. Modular approach to risks.
Yes, with the appropriate oversight by the FDA I believe that it is feasible to
harmonize REMS programs.

Table 31. Comments on feasibility to harmonize REMS within a therapeutic
area
Regarding REMS programs, please provide your level of agreement with the following
statements. (please elaborate in boxes provided)
should be utilized
Depends on the mechanism of action of the given drugs/biologics being compared
Easier than across therapeutic areas if the issues are similar
The decision must be based on patient population, how is the drug available, what is
the risk and who are the prescribers that treat the patient. Is one product easier to
manage then another and should that product be burdened because it’s in the
therapeutic area. I don't think generalizing is feasible.
High-level harmonization with variation, as needed. Modular approach to risks.

115
Table 32. Comments on feasibility to harmonize REMS within a product class
Regarding REMS programs, please provide your level of agreement with the following
statements. (please elaborate in boxes provided)
absolute need
Absolutely (assuming issues are the same)
The decision must be based on patient population, how is the drug available, what is
the risk and who are the prescribers that treat the patient. Is one product easier to
manage then another and should that product be burdened because it’s in the
therapeutic area. It’s also makes it difficult because all the companies are forced to
work together and that is burdensome no some companies who care about delivering a
good REMS and who use their internal resources to manage the programs. I don't
think generalizing is feasible.
High-level harmonization with variation, as needed. Modular approach to risks.

Most respondents strongly agree or somewhat agree that additional initiatives would be
helpful to harmonize implementation of REMS. Specifically, 82% of the respondents
(23/28) would like to see additional FDA initiatives and some would like to also see
initiatives by ICH, WHO, and CIOMS (Table 33). Comments associated with these
responses pointed to a need to share more information across companies and within FDA
(Table 34).

116
Table 33. Additional Initiatives that May Help Harmonization
Would additional initiatives be helpful to harmonize implementation of REMS programs
by providing consistent approaches to risk management?

Table 34. Other initiatives that would help harmonize REMS
Would additional initiatives be helpful to harmonize implementation of REMS programs
by providing consistent approaches to risk management?
Need to find a way for companies to discuss competitive issues. Maybe there should
be educational sessions at medical meetings.
sharing results data consistently, as would more consistency WITHIN the FDA on its
approaches to REMS. Too much inter-divisional variation; old products held to
different (looser) standards than newer products; more research on the regulatory
science to drive REMS decision-making for ALL stakeholders (regulators, industry,
prescribers, patients, others touched by the REMS systems).
More detailed guidance materials.
There needs to be more collaboration between manufacturers and FDA to
understand the business challenges of implementing a REMS, as what seems practical
on paper is often difficult to implement in the real world
REMS implementation strategies will vary by region depending on the health care
system so I don't see international efforts helping to harmonize REMS implementation.
The underlying issues is all of these efforts are ways to ensure physicians are doing
what they're supposed to be doing. Since FDA can't regulate medical practice they
Question
Strongly
agree
Somewhat
agree
Combined
agree
Neither
agree nor
disagree
Somewhat
disagree
Strongly
disagree
Combined
disagree
Total
Additional FDA
initiatives (e.g.
PDUFA VI)
9 14 23 5 0 0 0 28
ICH initiatives 2 11 13 9 3 1 4 26
WHO initiatives 3 8 11 11 3 2 5 27
CIOMS initiatives 2 10 12 11 2 1 3 26

117
make regulated industry monitor/control physician behavior. Equal efforts should be
put on medical schools/HCP training programs on the importance of using products
safety, reading labels, monitoring patients, etc. There can be checks/balances built
into the health care system (pharmacies, insurers, etc.) that help ensure proper usage.
Harmonization is too broad. The HCPs do not like to be told how to practice medicine
and do not like to have to be trained on materials. SPL is good as that will help
everyone get the REMS information in a unified manner. Restrictive REMS programs
cost the manufacturers allot of money and in most cases the question is has the risk
been mitigated. I don't think we can answer that question. Outcomes studies should
be done to compare patients managed by the REMS vs those who were not mandated
to REMS and see if the risk we are trying to reduce was reduced.
Standardized labeling

4.7 Modifying REMS
Respondents were asked to consider the totality of their experiences with REMS and
recommend if the concept of REMS programs as laid out by the FDA should be kept “as
is”, modified, or eliminated. None of the respondents wanted to see REMS programs
eliminated but almost everyone (89%, 26/30) thought that they should be modified. A
few (13%, 4/30) thought that the programs should be kept “as is” (Figure 18).

118
Figure 18. Should REMS be Kept As Is, Modified, or Eliminated
Thinking about your totality of experience with REMS to date, in your opinion, the
concept of REMS programs as laid out by FDA should be:

When respondents were asked to comment on what would be their first priority if they
were to change in the way REMS are structured, common themes that emerged include
standardization, goals and assessments, integration into the healthcare system, and
education (Table 35).

26
4
0
5
10
15
20
25
30
Modified Kept as is Eliminated

119
Table 35. First Priority to Change the Way REMS are Structured
If you were to change the way REMS are structured, what would be your first priority?
Uniformity
The elements to assure safe use portion of the REMS are the ones with the most need of
modification. These are the most complicated, cumbersome and costly portions of the
program. Can some of the checks/balances to use of products be built into the
health care system (especially with advent of EHRs).
The assessment of goals is not on target. FDA needs to be clearer on what their
rationale for the REMS is and ensure a program is designed that can truly assess whether
the reason for the REMS (i.e. goals) have been met. For example, conducting surveys
have never been optimal as one of the measures.
That is a very broad question. I would state that the key aspect that is important is if an
ETASU is required, the goal should be measurable and the FDA should provide
guidance on what outcome is acceptable to remove a REMS.
Standardizing REMS programs (language, stakeholder requirements)
Redefine restricted distribution. Try to build into the HC system a way to make sure lab
results are available to prescribers before they are able to write a prescription.
Many REMs are reliant on the pharmacy management systems and one vendor which is
limiting.
More accurate ways to assess success of program - patient and prescriber surveys seem
to not be the best method
It is hard to say as I have not kept up with the current environment
Introduce true shared decision making with patients, with opportunity for patient to
assume risk with demonstrated knowledge/acceptance of the product and condition.
industry should have a seat at the table to contribute to FDA initiatives regarding
standardization and a common REMs platform

120
I think REMS are structured OK. The ETASUs are the most complicated, but that said,
the drugs that require that type of REMS have the worst benefit risk profiles. Requiring a
predictor/explanatory variable or variables to help answer the "why" question in REMS
evaluations.
I support the idea, with modification
focus groups, patient’s interviews, and physician interviews should be conducted to
pretest all educational materials.

The educational materials for patients are often way too complex and potentially “scary"
to a patient.
Enhance educational opportunities.
difficult to answer: there are so many REMS elements that do little to actually protect
patient safety but add significant burden. I.e., Most do not educate nor effect the
behavior change which they are intended to do.
Developing a standard process across all REMS programs, this would allow for a more
streamlined approval process.
Consistent guidance (published and not published)
Consistency by class of drug
In the same therapeutic area, may be a summary of the diff of REMS/risk for easy view
to health care providers on the diff. REM requirement/details of the diff. drugs treating
the same disease
A more structured iterative review process with FDA such that new concepts are not
introduced late in the REMS development process.

As the last questions of the survey, respondents were asked to provide any additional
comments regarding experience with REMS (Table 36). Comments point to lack of
harmonization across the FDA and also a lack of industry perspective within the agency.

121
Table 36. Additional Comments on REMS Experience
Please provide any additional comments you would like to share regarding your
experience with REMS.
We work directly with pharma and thus we would not do any of the above.
REMS do seem to be changing from where we started when first implemented. There is
a lot of work that still needs to be done but having supported some policy work there has
been the ability to provide industry thinking to the FDA. Ironically, this has not been
much different than what we put in our RTQs while negotiating an ETASU REMS.
FDA does seem to be working towards more change. It should be noted that there were
huge differences between review divisions and how they thought about REMS programs
so there is inconsistency within the FDA. Hard to harmonize when the FDA isn't
even harmonized.
I work on a wide range of REMS over the past 14 years and was part of the approval
process for some of the early REMS transitioning from risk management programs to
REMS. I have and currently interact with the Agency on REMS programs and the
education between Sponsor knowledge of the product, the patient and the HCP including
how the product is used in the marketplace is important. Some divisions work more
collaboratively with sponsors then other divisions.
FDA has hired allot of new people and the learning curve is steep, as many don't have
industry experience which is needing to develop and implement REMS successfully.

One other topic is working in single shared REMS is challenging too and is another topic
to add to the complexity of harmonization. NDA/BLA owners are challenged to lead
the REMs strategically and competitors work together to mitigate risk. Many of the
companies do not have the resources to add any value in the process. PMOs are hired
to manage the processes and budgets etc. but the regulatory and safety burdens usually
fall on a few of the organizations.

Many NDA/BLA holders who managed their REMS internally are forced to outsource
and work in the single share system. It is easier for the Agency but some of the nice
services are no longer available as we create one REMS for the class.

122
CHAPTER 5. DISCUSSION
5.1 Industry Views on Implementation of REMS
Since its launch in 2007, REMS implementation has faced challenges and criticism
primarily because the program presents a heavy burden to the healthcare system.
However, the challenges faced by the sponsors who are responsible for developing,
implementing, and assessing REMS programs have not been studied formally. This
study thus was designed to obtain insights into how industry professionals view FDA’s
policies and REMS implementation by using a customized survey. According to the
survey findings, industry’s biggest challenges were related to a lack of transparency
behind FDA’s actions and inconsistencies across different programs. Nonetheless,
challenges were present across all domains and the overall level of satisfaction was not
high. The particular details of these views will be examined below. Overall, these
findings are consistent with what is observed in literature and in public forums that have
criticized REMS for being complex, burdensome, and inconsistent.
5.2 Methodological Considerations
Delimitations
In the present study, a survey was developed to obtain views of industry experts on their
experiences with REMS programs that had been implemented fully by their companies.
It was further delimited by examining those programs through the lens of
implementation, by reference to two well-established frameworks in implementation
science, that of Fixsen, which defines implementation stages (Fixsen, 2005), and that of

123
Rabin, which attempts to identify key determinants of implementation success (Rabin,
2006). However, neither framework was adopted in its entirety in order to narrow the
focus of the results. I took only part of the proposed model of Fixsen (Fixsen, 2005), so
that the survey could be focused on mature REMS programs. Such an approach seemed
reasonable given the relative maturity of REMS policies and programs in general.
Launched over ten years ago, the REMS portfolio now includes 74 fully implemented
programs of different types (Food and Drug Administration, 2018a). Those REMS
programs should now be in the final stage of implementation, called “full
implementation” by Fixsen. However, by having this focus, the survey was delimited to a
particular group of respondents who have direct experience with a mature REMS
program and are thus familiar with elements associated with maintaining, modifying, or
terminating REMS.
Because the numbers of programs are relatively small, it is perhaps not surprising that the
number of survey participants who matched the requirements for direct experience were
relatively hard to find, and several methods had to be used to secure a sample sufficient
to have confidence that the results were saturated. The first subjects to be recruited were
identified using a method of nominated expert sampling to identify a group of experts
who might provide the views of the wider group (Trotter, 2012). By collecting names
from FDA public meetings, workshops and product specific letters, I was able to initially
identify a small number of potential respondents who would be considered subject
experts in REMS. Nonetheless, the importance of assuring a wider set of views from
respondents required that I cast a wider net to individuals who were in a closed group on

124
LinkedIn called “Risk Evaluation & Mitigation Strategy REMS”. It is likely that at least
some of the participants in this group might not typically go to the meetings whose
participants were approached initially or serve as the lead manager of a REMS program
named in an FDA communication. Thus, despite the fact that the number of complete
responses in this study was small (41), most belonged to a relatively rare population with
experience in multiple types of REMS programs across multiple areas of drug
development. Further, the responses did suggest that the same types of comments were
repeated frequently, and I felt that adding more respondents would not enrich the results
with major new insights. As identified by Trotter, it is generally not unusual to reach
saturation with as few as 15-25 subjects if the subjects have similar backgrounds and
experiences (Trotter, 2012).
Using the framework of Rabin (Rabin, 2006) the three key domains of transparency,
collaboration, and harmonization/ degree of burden were selected to examine the nature
of interactions with FDA. It was seen to be a particular advantage that the respondents
were so experienced and could provide unique insights into these aspects of REMS to
dissect policies and procedures that are working well, posing challenges or needing
improvements. The fact that respondents had considerable experience in REMS across
various therapeutic areas and types of molecules helps to allay a potential concern that
the survey might only select respondents in a particular subsector and would not be able
to represent various product types. Further, the concern that the results would come from
individuals with insufficient experience seems addressed by the fact that most
respondents were at a director level with responsibilities for interacting with both internal

125
and external stakeholders, including healthcare providers, pharmacies, patient groups,
and the FDA.
As a whole, the respondent group in this study included individuals with a range of
experiences in small molecules, biologics, generics, and biosimilars and across all
elements of REMS, including ETASU, the most challenging form of the REMS program.
The profiles of respondents clearly show that the survey targeted the intended population
of experts in REMS and provide assurance that the study results are likely to be
representative of the broader cohort of individuals engaged in REMS management. Of
particular value were the numerous additional comments provided by these experts. The
sheer number of comments reinforced my view that the three domains on which the
survey focused were appropriate to give a more systematic and deep accounting of views
and challenges related to the later-stage management of REMS. The results provide a rich
picture of current REMS practices and interactions. However, such an approach will
restrict the conclusions that can be made about other aspects of REMS. It will not tell us,
for example, whether the same experiences are typical in other constituencies, where
similar risk management programs are in place.
Responses are further delimited to a particular period of time; whether those same views
will be typical in a few years is open to question. Since the launch of this survey, the
FDA has made a concerted effort to improve REMS by publishing guidance documents
and holding public forums. At the same time, new issues and concerns have surfaced
pointing to unintended consequences of REMS. With the sixth renewal of PDUFA in
2017 (Food and Drug Administration Reauthorization Act or FDARA), the REMS

126
Integration Initiative, established in 2011 to evaluate and improve the implementation of
REMS, came to an end. This initiative produced five new guidance documents for
industry with four still in their draft stages. A continuing project is the REMS Platform
Standards Initiative through which the FDA is seeking to encourage the development of
platform standards. All of these initiatives will undoubtedly change the experience and
views of the regulatory teams in those companies in which REMS must be developed in
the future. The level of satisfaction with the REMS approaches may then improve but
new challenges may also come to light. For example, some sponsors are using features
of ETASU, the most rigorous form of REMS, to restrict access to their products and gain
unfair commercial advantage (Florko, 2017). These practices have been brought to the
attention of the FDA and are prompting new actions in response related to the ability of
generic drugs to divorce themselves from the participation in a single shared REMS
system to which they previously had to subscribe. The details of this change are not
directly relevant to the goals of the study discussed here but do illustrate the fact that the
REMS system is a program that has been used strategically to manage relationships with
competitors by some companies (Food and Drug Administration, 2018b).
Limitations
One of the most challenging aspects of the research that had the potential to limit this
survey was the difficulty to assure that a sufficiently large and representative grouping of
respondents respond to the survey. It is of course important to delimit the sample to
appropriately experienced individuals as discussed above, but even with names secured,
it is not possible to force them to fill out the survey. A problem related to surveys in

127
general is the low response rate, especially for surveys that are distributed electronically
(Sheehan, 2001) and can be easily deleted and forgotten. It is not unusual for an
electronic survey to have response rates in the single digits (Sheehan, 2001). In this
particular case, however, we anticipate that many of the non-responders may have
usefully recognized that they would be inappropriate respondents, either because their
experience was not well matched to the target participant profile or because the REMS
programs on which they were working were still at early stages.
For the respondents who did agree to participate, I was concerned that a long and difficult
survey would cause what has been called “survey fatigue” in the literature (Pecoraro,
2012). Regulatory professionals in challenging jobs often cannot easily find the time to
complete a survey. The decision to keep the survey to 34 questions was based on
feedback from the focus group indicating that busy professionals would be reluctant to
complete a longer survey. Even with only 34 questions, some respondents did not answer
every question. This may have been due to time limitations because many questions
requested a text response in addition to a multiple-choice answer. It could also reflect a
respondent’s lack of knowledge related to a particular question. Because of this
limitation on the number of questions, it was difficult to explore certain topics fully.
Further, the ability of each question to probe the views and experience of the audience
might have been limited by the way in which the questions were worded. For example, a
few of the questions asked about “your REMS”, which, in hindsight, could have been
interpreted as either a question regarding their personal experiences or that of their
companies.

128
The level of engagement of individuals once they began the survey was high. As
identified above, surveys are plagued by the tendency for some respondents to leave the
survey as soon as they become tired of participating. However, in this study, most of
those who started the survey continued to the end, and additionally put much effort into
giving detailed answers in text boxes. From these behaviors, it would seem safe to infer
that the respondents were interested in having their views known and potentially having a
role in affecting the future of the REMS program by providing input.

5.3 Consideration of Results
A number of aspects related to the implementation and interactions associated with
REMS programs were explored in this dissertation, but the majority of questions were
concerned with three aspects of the experience of respondents- transparency,
collaboration and harmonization- during their interactions with regulators and
stakeholders. These areas were highlighted in chapter 2 as areas of particular concern to
those implementing REMS programs.
5.3.1 Transparency
Perhaps one of the most discussed areas of REMS programs is the transparency with
which guidance is given and decisions are made by regulatory agencies with whom the
company must interact. Transparency is defined as something that is apparent or
understood (Ball, 2009). Thus, it can encompass both documents and actions taken by
the agency. It has different facets- clarity of communications, accessibility of needed

129
information, and accountability for the outcomes of decisions (Solberg & Richmond,
2012).
The questions asked in this survey addressed two main areas where transparency is often
viewed to be important. First are the policies and written materials that are available to
those who must maintain and modify a mature REMS program. It might not be
surprising to find that documentation related to the ongoing sustainability or termination
of a REMS program would be weaker than that associated with initial stage of program
implementation. When a new program is introduced, regulators often pay considerable
effort to provide clear rules for initial submissions and interactions but may have less
experience to anticipate the types of written materials that might be needed later. After
programs have begun, however, the resources to develop materials can diminish and it
may be difficult to assure the same level of attention to materials relevant to program
modifications or sustainability. Nevertheless, such materials to guide later stage
activities remain important to the stakeholders, as reflected in the views expressed by
respondents that the documents and resources available from FDA were the most widely
consulted resources. Most respondents also appeared to agree that the information
needed to manage their REMS was readily available. What was somewhat surprising,
however, were the findings that other types of documents such as those available from
CIOMS and the EMA were not frequently consulted, indicating that REMS concepts are
not being viewed in a more holistic framework of risk management. Perhaps more needs
to be done to encourage industry personnel to recognize the broader implications and
usefulness of the REMS program as part of overall risk management for the company.

130
Although respondents appear overall to be satisfied with the written materials and
guidance relevant to maintaining the REMS program, they were less satisfied with the
transparency of FDA’s actions and decisions. In particular, participants commonly
suggested that FDA actions had been opaque when REMS programs were to be modified
or rescinded or when changing regulatory expectations related to the REMS program as a
whole. Comments ranged from modest expressions of dissatisfaction with a perceived
lack of clarity to quite detailed grievances. Comments such as: “The FDA does not have
a model as to how REMS should operate”; “FDA could do a better job explaining…”;
and “FDA provides comments in a piecemeal fashion...” reflect the kinds of views that
seemed typical of respondents here.
Although the present survey was directed at understanding the experience and challenges
with REMS programs after they had been approved and implemented, it was instructive
that several comments also addressed transparency and consistency during earlier stages
of the REMS implementation process. The comment that highlighted “No objective
approaches to risk management by the FDA, leading to judgement calls on all sides to
require a REMS, and further, what to include in a REMS” suggests that FDA could do a
better job. These results might be an important signal to FDA that more work is needed
to improve transparency at all stages of implementation. This might include not only
better explanations to industry regarding the way that FDA reviewers make decisions but
also attention internally to the training and collaboration between reviewers in different
divisions, to encourage more transparency as well as consistency (see below) of
processes and advice for different product REMS.

131
At the same time, criticism should not be unfairly directed at a process such as REMS,
that requires some degree of judgement and at the same time confidentiality to protect the
proprietary interests of the companies. It is easiest to be transparent when a particular
program has consistent, reproducible rules that leave little room for judgement (Ball,
2009). However, the decisions that must be made in the course of REMS approvals and
modifications cannot always be predictable and consistent, in part because the risk
problem differs from product to product. REMS programs must be tailored to specific
risks, specific products and/or specific populations. Hence the programs by which they
are managed may not lend themselves to a standardized approach, as was clearly
acknowledged by many respondents. Nevertheless, most respondents did not seem to
know, for example, about the rules that FDA uses to decide on the rescission of a REMS,
and further most of those with a rescinded REMS were not told why. I believe that more
transparency around this issue would be helpful.
Another reason why the decisions and actions related to the modifications and
rescindment of a REMS program are not clear relates to ambiguities surrounding the
goals and effectiveness of the REMS programs more generally. FDA is clearly aware
that some form of a REMS program might be useful to manage risk but has no clear
metrics to use when deciding if a program is reaching a satisfactory level of risk
reduction. This confusion seems apparent with statements such as: “I have seen both
sides, FDA typically provides clear guidance, however, I have seen case where they do
not” and “we have had examples where we were unexpectedly released from REMS, and
another that despite good effectiveness results for the 18-mo and 36-mo assessments, our

132
request to remove the survey assessment requirement (on a medguide only rems) was
denied”.
Perhaps to address these concerns, FDA has published a number of recent guidance
documents and reports to provide the industry with its current thinking on various
subjects, including how to determine when a REMS is necessary, how to format REMS
documents, and how to use tools for electronic submission. Most of these documents still
focus on the requirements for setting up a REMS rather than taking it away. However,
because most of these documents are still in their draft form, the industry may have the
opportunity to work with the FDA to finalize them in ways that would address some of
the concerns raised here.
Transparency has become the lead philosophy governing biomedical research and
regulation (Wechsler, 2010) and FDA even launched a Transparency Initiative as early as
2010 to provide the public with more information on its policies and practices as a way to
bolster confidence in its decisions. This initiative might be usefully extended to clarify
some of the polices and decision-making processes used with regard to REMS, as was
done previously, for example, to clarify the overall benefit-risk assessment that is made
during the review and approval of the new drug applications (Food and Drug
Administration, 2013b).
5.3.2 Collaboration Across the Key Stakeholders
The REMS program, and especially the difficult ETASU program, requires the industry
to interact closely with various stakeholders. The survey, therefore, included questions to
explore the perceived effectiveness of communications and collaborations between the

133
company and other key stakeholders- health care providers, pharmacies, and patients- as
REMS programs are being modified. Most of the respondents appear to share the view
that those collaborations are sufficient. Perhaps the best insight into this question was not
that captured by the admittedly vague multiple-choice question relating to collaborations
(Do you feel you that your company collaborates sufficiently with stakeholders (FDA,
Health care Providers, Pharmacies, Patients) to modify REMS programs?) but rather the
text responses. For example, it appears that sponsors are communicating with health care
providers, pharmacies and patients early in the process before they reach out to the FDA
with proposed changes. Some are even including these stakeholders in their discussions
with the FDA when they feel that that FDA’s comments and requests are not appropriate
under conditions of current medical practice or sales environment. Their responses align
with social science observations that communication among stakeholders is essential in
terms of evidence, concerns, and preferences in addition to the implementation of risk
minimization (Wu, 2016). The comments here suggest that sponsors understand the
importance of the voices of patients and healthcare providers where REMS are
concerned.
5.3.3 Harmonization/ Consistency
The last key determinate of success utilized in the research framework was
“harmonization”, interpreted for this work as consistency across FDA product divisions
and drug products. Challenges with consistency overlapped to some extent with those of
transparency, discussed above. That inconsistency might be an issue of concern was
clear even before the survey was disseminated, as identified to some extent in chapter 2.

134
The literature abounds with examples of what some commentators have identified as
inconsistencies such as those associated with products such as acne treatments (Kerr &
Wosinska, 2017), pregnancy termination medications (Mifeprex REMS Study Group,
2017), oncology drugs, and weight loss products (Boudes, 2017).
Two examples may be sufficiently instructive to preface considerations of the views of
respondents with regard to this issue. One has been the handling of risks related to
embryo-fetal toxicity. To address this important recognized risk, oral retinoids to treat
severe and refractory acne pose risk and potentially prescribed for young females of
reproductive age have a REMS with ETASU in place (Orfanos & Zouboulis, 1998).
Further, a REMS with ETASU is required for thalidomide and lenalidomide, known
teratogens indicated for multiple myeloma, even though that disease rarely occurs in a
population likely to become pregnant. In contrast, embryo-toxic chemotherapeutic drugs
included in certain multiple drug regimens for the treatment of that same indication,
multiple myeloma, are not subject to a REMS with ETASU.
Another example of inconsistency seems apparent even across individual indications for
a single drug, as evidenced by topiramate, which is indicated for the treatment of
migraine as well as management of obesity. This drug is known to be associated with a
low risk of birth defects. Although both indications are prevalent in women of child
bearing potential, the obesity indication has a REMS and the migraine indication does not
(Boudes, 2017). Results in the research reported here show that regulatory and clinical
teams are aware of, and are disturbed by, such inconsistencies. Typically, respondents
appeared to feel that FDA has taken steps to standardize the language and REMS

135
elements, but that more needs to be done to explain why apparently dissonant decisions
have been made.
Another area of inconsistency that appears to cause problems for companies is that
associated with the often varying and demanding requirements for the training and
certification of healthcare providers. It was instructive that the respondents ranked most
highly “the need for health care providers to take training and obtain certification to
prescribe” when asked to rank order items presenting the most burden on the healthcare
delivery system; the closely ranked second choice, the “lack of harmonization across
REMS programs”, may to some degree be linked. Health care providers and pharmacies
must undergo many different training and certification programs even for closely related
products (Food and Drug Administration, 2014c). These requirements have been
identified by the companies and health care professionals alike as often burdensome and
confusing (Denvir, 2011; Waldman, 2016). Oncologists in particular prescribe many
drugs that are risky and have REMS programs in place, each of which has its own
particular rules and training requirements that are not integrated with other similar
products (Denvir, 2011). However, while respondents agree that the lack of
harmonization is a burden, their numerous text box entries suggest that harmonization
may not always be possible, especially when dealing with drugs from very different
families with different types of risks. Respondents provided comments such as,
“harmonizing for products in the same class is fairly straight forward…it becomes more
challenging across therapeutic areas and nearly impossible across all programs” and “you
can’t develop a REMS across therapeutic areas since it is product specific”. Individual

136
REMS programs could have different goals even within the same therapeutic area
because the risks are often product- as well as indication-specific.
Rather than harmonizing REMS, respondents had a recurring suggestion of sharing
information in a sort of “lessons learned” approach, so that sponsors developing or
modifying REMS can learn from previously implemented systems. That said, the sharing
of information may be complicated given that many companies are reluctant to share
information that is not publicly available for competitive reasons; REMS materials, in
some instances, are even copyrighted (Shire-NPS PHarmaceuticals, 2017). While the
respondents of this survey may view such sharing as a good idea, their opinions may not
reflect the views of industry as a whole, and in particular the views of the commercial
department can see the strategic management of a REMS as a competitive tool
(Sarpatwari, Avorn, & Kesselheim, 2014).
5.3.4 Other Areas of Concern
Other comments by the respondents provided additional insights into the challenges faced
by the sponsors working in REMS. One that was somewhat unexpected was a repeatedly
stated concern about the rules defining the use of vendors for certain REMS ETASU
programs. Specifically, some were bothered that only one FDA-endorsed vendor can be
used by sponsors to assure a closed distribution system. A closed distribution system is
one that is typically set up through a specialty pharmacy that controls the release of drug
only to parties that have the required safe-handling and other procedures in place (Barlas,
2016). The respondents commented that the REMS system becomes costlier and is prone
to quality issues when a single monopolistic situation is set up in which vendors do not

137
need to compete. When utilizing a singular pharmacy management system, sponsors are
forced to buy the entire bundle of services from that vendor instead of contracting for
only the service components that are needed. Thus, some sponsors are finding that the
services that they purchase are subpar or unnecessary.
5.3.5 Modifying REMS
Perhaps the most surprising set of responses received in this survey related to the views
of respondents on the value of REMS in general. When asked, in their opinion, if REMS
should be “kept as is”, “modified”, or “eliminated”, not a single respondent believed that
REMS should be eliminated. Even in the face of frustration, REMS management teams
appear to regard REMS programs as playing an important role in ensuring safe use of
drugs. At the same time, however, the value that most see it offering is not considered to
come from its role in assuring market access. This result was unexpected. The reason for
putting REMS programs in place at the outset of the FDA’s REMS initiative was
specifically to assure market access for products whose risk profiles otherwise would
make product commercialization too great a threat to patient health (Worthy, 2016). The
responses here suggest that the focus on patient safety by the REMS team may lead those
teams may blind them somewhat to the associated commercial implications of such
programs. It would be interesting to explore the views on this issue held by those in other
departments of the same companies, to see if these views differed. The role played by
REMS in the eyes of these companies is particularly relevant now, given the considerable
attention recently focused on REMS as a way to discourage generic drug entry and to
capture market share (Food and Drug Administration, 2018b; Hoffman, 2018).

138
5.4 Conclusions and Future Directions
Not surprisingly, the results of this survey show more systematically an overall level of
dissatisfaction with complicated REMS programs, a dissatisfaction already suggested by
anecdotal reports in the literature. Given that much of the frustration was centered
around issues of transparency and consistency, it would seem useful to focus some effort
on improving the transparency and consistency of these programs to make them more
successful. In particular, it might be helpful to have clearer guidance on how programs
are assessed and why they are terminated. It may also be helpful to consider why
inconsistencies exist, particularly in similar products, and to find ways to increase the
consistency of approaches across different requirements by FDA. Implementing a REMS
program can be a huge and expensive burden for many of its key stakeholders, so
demonstrating its effectiveness in a transparent manner will be important to gain the
support of the stakeholder community.

139
REFERENCES

Arora, D. (2012). Pharmacovigilance - An Industry Perspective: Navi Mumbai, India:
Pharmapublisher.
Ball, C. (2009). What Is Transparency? Public Integrity, 11(4), 293-308.
Barlas, S. (2016). Generics Companies Press for REMS Relief: Potential Law Would
Clarify When Brand-Name Manufacturers Must Supply Drug Samples. Pharmacy
and Therapeutics, 41(10), 601-644.
Barrett, M. S., et al. (2004). Blowing the Whistle on Fen-phen: An Exploration of
MeritCare's Reporting of Linkages Between Fen-phen and Valvular Heart
Disease. The Journal of Business Communication, Oct 2004.
Boudes, P. F. (2017). Risk Evaluation and Mitigation Strategies (REMSs): Are They
Improving Drug Safety? A Critical Review of REMSs Requiring Elements to
Assure Safe Use (ETASU). Drugs R D, 17, 245-254. doi:10.1007/s40268-017-
0175-y
Bouvy, J. C., De Bruin, M. L. & Koopmanschap, M. A. (2015). Epidemiology of
Adverse Drug Reactions in Europe: A Review of Recent Observational Studies.
Drug Safety, 38(5), 437-453.
Brennan, Z. (2018). REMS Draft Guidance: FDA Looks to Deal With Exploited
Requirements. Regulatory Focus Retrived from https://www.raps.org/news-and-
articles/news-articles/2018/5/rems-draft-guidance-fda-looks-to-deal-with-exploi.
Craig, D. S. (2010). The Food and Drug Administration Risk Evaluation and Mitigation
Strategy. Journal of Pain and Palliative Care Pharmacotherapy, 24(2), 145-148.
doi:10.3109/15360281003713834
Cronin, M., BA, Berger, Susan, Seligman, Paul J,. (2014). Risk Evaluation and
Mitigation Strategies WIth Elements to Assure Safe Use: Alignment of the Goals
WIth the Tools to Manage Risk. Therapeutic Innovation & Regulatory Science,
48.6, 724-733.

140
Dabrowska, A. (2018). FDA Risk Evaluation and Mitigation Strategies
(REMS): Description and Effect on Generic Drug Development. Retrieved from
https://fas.org/sgp/crs/misc/R44810.pdf
Denvir, D. (2011). Value-Based Care in Myeloma. Retrieved from
http://valuebasedmyeloma.com/vbcc-issues/2011/april-2011-vol-2-no-2/496-
vbcc-496
Eaton Baier, V., March, J. G.; Saetren, H. (1986). Implementation and Ambiguity.
Scandanavian Journal of Management Studies, 2(3 - 4), 197-212.
European Medicines Agency. (2015). Guideline on Good Pharmacovigilance Practices
(GVP). Retrieved from
www.ema.europa.eu/ema/index.jsp?curl=pages/regulation/document_listing/docu
ment_listing_000345.jsp
European Parliament. (2012a). Directive 2012/26/EU of the European Parliament and of
the Council. Retrieved from
ec.europa.eu/health//sites/health/files/files/eudralex/vol-1/dir_2012_26_en.pdf
European Parliament. (2012b). Regulation (EU) No 1027/2012 of the European
Parliament and of the Council of 25 October 2012. Retrieved from
ec.europa.eu/health//sites/health/files/files/eudralex/vol-
1/reg_2012_1027/reg_2012_1027_en.pdf
Fiedler, J. H. (2005). The Vioxx Debacle. Engineering in Medicine and Biology
Magazine, IEEE, 24(2), 106-109. doi:10.1109/MEMB.2005.1411356
Fischhoff, B., Brewer, N, , Downs, J. S. (2011). Communicating Risks and Benefits: An
Evidence-Based User's Guide. Retrieved from
www.fda.gov/downloads/AboutFDA/ReportsManualsForms/Reports/UCM26806
9.pdf
Fixsen, D. L., Naoom S. F., Blase, K. A., Friedman, R. M., Wallace, F. (2005).
Implementation Research: A Synthesis of Literature. Tampa, FL: University of
South Florida, Louis de la Parte Florida Mental Health Institute, The National
Implementation Research Network. Retrieved from
ctndisseminationlibrary.org/PDF/nimmonograph/pdf

141
Florko, N. (2017). House Oversight Probes How Voluntary REMS-Like Systems Affect
Generics. InsideHealthPolicy.com's FDA Week. Retrieved from
https://insidehealthpolicy.com/daily-news/brands-use-voluntary-restricted-
distribution-much-rems-slow-generics
Food and Drug Administration. (2018). Approved Risk Evaluation and Mitigation
Strategies (REMS). Retrieved from
https://www.accessdata.fda.gov/scripts/cder/rems
Food and Drug Administration. (2009a). Guidance for Industry Format and Content of
Proposed Risk Evaluation and Mitigation Strategies (REMS), REMS Assessments,
and Proposed REMS Modifications.
Food and Drug Administration. (2009b). Prescription Drug Information for Patients:
History. Retrieved from
http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMateria
ls/RiskCommunicationAdvisoryCommittee/UCM150262.pdf
Food and Drug Administration. (2009c). Zyprexa Relprevv NDA Approval Letter.
Retrieved from
http://www.accessdata.fda.gov/drugsatfda_docs/appletter/2009/022173s000ltr.pdf
Food and Drug Administration. (2010). Risk Evaluation and Mitigation Strategy (REMS)
Public Meeting. Retrieved from www.fda.gov/drugs/newsevents/ucm210201.htm
Food and Drug Administration. (2011a). Guidance Medication Guides - Distribution
Requirements and Inclusion in Risk Evaluation and Mitigation Strategies
(REMS). Retrieved from
www.fda.gov/downloads/Drugs/.../Guidances/UCM244570.pdf
Food and Drug Administration. (2011b). Supplement Approval Release REMS
Requirement NDA 021995/S-017. Retrieved from
http://www.accessdata.fda.gov/drugsatfda_docs/appletter/2011/021995s017ltr.pdf
Food and Drug Administration. (2012a). Risk Evaluation and Mitigation Strategy
(REMS) Assessments: Social Science Methodologies to Assess Goals Related to
Knowledge. Retrieved from
www.fda.gov/downloads/Drugs/NewsEvents/UCM310935.pdf

142
Food and Drug Administration. (2012b). Rosiglitazone REMS Program. Retrieved from
www.fda.gov/downloads/drugs/drugsafety/postmarketdrugsafetyinformationforpa
tientsandproviders/ucm255624.pdf
Food and Drug Administration. (2012c). Summary Minutes of Meeting of the Anesthetic
and Analgesic Drug Products Advisory Committee. Retrieved from
http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMateria
ls/Drugs/AnestheticAndAnalgesicDrugProductsAdvisoryCommittee/UCM336475
.pdf
Food and Drug Administration. (2013a). Actemra REMS. Retrieved from
http://www.fda.gov/downloads/Drugs/DrugSafety/PostmarketDrugSafetyInformat
ionforPatientsandProviders/UCM202044.pdf
Food and Drug Administration. (2013b). Structured Approach to Benefit-Risk
Assessment in Drug Regulatory Decision-Making. Retrieved from
https://www.fda.gov/downloads/ForIndustry/UserFees/PrescriptionDrugUserFee/
UCM329758.pdf
Food and Drug Administration. (2014a). Meeting of the Drug Safety and Risk
Management Advisory Committee (DSaRM). Retrieved from
www.fda.gov/downloads/AdvisoryCommittees/Committees/CommitteesMeeting
Materials/Drugs/DrugSafetyandRiskManagementAdvisoryCommittee/UCM4567
81.pdf
Food and Drug Administration. (2014b). Report: Standardization of Risk Evaluation and
Mitigation Strategies. Retrieved from
www.fda.gov/downloads/ForIndustry/UserFees/PrescriptionDrugUserFee/UCM4
15751.pdf
Food and Drug Administration. (2014c). Summary Minutes of the Drug Safety and Risk
Management Advisory Committee Meeting. Retrieved from
http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMateria
ls/Drugs/DrugSafetyandRiskManagementAdvisoryCommittee/UCM456780.pdf
Food and Drug Administration. (2014d). Zyprexa Relprevv REMS. Retrieved from
http://www.fda.gov/downloads/Drugs/DrugSafety/PostmarketDrugSafetyInformat
ionforPatientsandProviders/UCM202330.pdf

143
Food and Drug Administration. (2015a). Actemra Supplement Approval. Retrieved from
http://www.accessdata.fda.gov/drugsatfda_docs/appletter/2015/125276s106,1254
72s014ltr.pdf
Food and Drug Administration. (2015b). Food and Drug Administration Safety and
Innovation Act (FDASIA). Retrieved from
www.fda.gov/RegulatoryInformation/Legislation/SignificantAmendmentstotheFD
CAct/FDASIA/ucm20027187.htm
Food and Drug Administration. (2015c). History. Retrieved from
www.fda.gov/AboutFDA/WhatWeDo/History
Food and Drug Administration. (2015d). Risk Evaluation and Mitigation Strategies
(REMS): Understanding and Evaluating Their Impact on Health Care Delivery
System and Patient Access. Retrieved from
http://www.fda.gov/Drugs/NewsEvents/ucm441308.htm
Food and Drug Administration. (2015e). Risk Evaluation and Mitigation Strategies
(REMS): Understanding and Evaluating Their Impact on the Health Care
Delivery System and Patient Access. Retrieved from
www.fda.gov/Drugs/NewsEvents/ucm441308.htm
Food and Drug Administration. (2015f). Risk Evaluation and Mitigation Strategies:
Modifications and Revisions Guidance for Industry. Retrieved from
www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidan
ces/ucm441226/pdf
Food and Drug Administration. (2015g). Supplement Approval for Avandia, Avandamet,
and Avandaryl. Retrieved from
http://www.accessdata.fda.gov/drugsatfda_docs/appletter/2015/021071Orig1s050,
021410Orig1s039,021700Orig1s022ltr.pdf
Food and Drug Administration. (2016a). FDA Basics Webinar: A Brief Overview of
Risk Evaluation and Mitigation Strategies (REMS). Retrieved from
www.fda.gov/downloads/AboutFDA/Transparency/Basics/UCM328784.pdf
Food and Drug Administration. (2016b). REMS Data Files and Historic REMS
Information. Retrieved from

144
http://www.accessdata.fda.gov/scripts/cder/rems/index.cfm?event=RemsData.pag
e
Food and Drug Administration. (2016c). REMS Integration Initiative. Retrieved from
http://www.fda.gov/ForIndustry/UserFees/PrescriptionDrugUserFee/ucm350852.
htm
Food and Drug Administration. (2017a). Criminal and civil actions filed against Aegerion
Pharmaceuticals Inc. Retrieved from
https://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm577101.h
tm
Food and Drug Administration. (2017b). FDARA: Making a Difference for Industry and
Patients. Retrieved from https://blogs.fda.gov/fdavoice/index.php/2017/08/fdara-
making-a-difference-for-industry-and-patients/
Food and Drug Administration. (2018a). Approved Risk Evaluation and Mitigation
Strategies (REMS). Retrieved from
https://www.accessdata.fda.gov/scripts/cder/rems
Food and Drug Administration. (2018b). Statement from FDA Commissioner Scott
Gottlieb, M.D., on New Policies to Reduce the Ability of Brand Drug Makers to
Use REMS Programs as a Way to Block Timely Generic Drug Entry, Helping
Promote Competition and Access [Press release]. Retrieved from
https://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm609365.h
tm
Hoffman, M. (2018). Gottlieb Announces New FDA Policies to Reduce Pharma
Exploitation of REMS. Retrieved from http://www.mdmag.com/medical-
news/gottlieb-announces-new-policies-to-reduce-pharma-use-of-rems-program
Hyman, P. M., P.C. (2016). REMS Tracker. Retrieved from www.fdalawblog.net
Janssen, W. F. (1981). The Story of the Laws Behind the Labels. FDA Consumer(June
1981).
Jarrell, K. (2012). Regulatory history: Elixir of Sulfanilamide. Journal of GXP
Compliance, 16(3), 12-14.

145
.
Juni, P., Narty Dip, L., Reichenbach, S., Sterchi, R., Dieppe, P., Egger, M.. (2004). Risk
of Cardiovascular Events and Rofecoxib: Cumulative Meta-analysis. The Lancet,
364(9450), 2021-2029. doi:10.1016/S0140-6736(04)17514-4
Karha, J. T., Eric J. (2004). The Sad Story of Vioxx, and What We Should Learn From it.
Cleveland Clinic Journal of Medicine, 71(12), 933-939.
Kerr, K. W., & Wosinska, M. E. (2017). Patient Access in Restrictive Risk Management
Programs: The Case of iPLEDGE. Therapeutic Innovation & Regulatory Science,
51(1), 16-23. doi:10.1177/2168479016663266
Kim, J. H. , Scialli, A.. (2011). Thalidomide: The Tragedy of Birth Defects and the
Effective Treatment of Disease. Toxicological Sciences, 122(1), 1-6.
doi:10.1093/toxsci/kfr088
Lindquist, M. (2010). Celebration and Change at CIOMS. Uppsala Reports. Retrieved
from https://www.who-umc.org/media/2650/ur49_web_smallest.pdf
Mifeprex REMS Study Group. (2017). Sixteen Years of Overregulation: Time to
Unburden Mifeprex. New England Journal of Medicine, 376(8), 790-794.
Nicholson, S. C., Peterson, J., Yektashenas, B. (2012). Risk Evaluation and Mitigation
Strategies (REMS): Educating the Prescriber. Drug safety : an international
journal of medical toxicology and drug experience, 35(2), 91-104.
Office of the Inspector General. (2014). FDA Lacks Comprehensive Data to Determine
Whether Risk Evaluation and Mitigation Strategies Improve Drug Safety 2013.
Retrieved from https://oig.hhs.gov/oei/reports/oei-04-11-00510.pdf
Food and Drug Adminstration Safety and Innovation Act (FDASIA), (2012).
Orelli, B. (2013). Why You Should Care About This Diabetes Drug Saga. Retrieved
from www.fool.com/investing/general/2013/06/08/why-you-should-care-about-
this-diabetes-drug-saga.aspx

146
Orfanos, C., & Zouboulis, C. (1998). Oral Retinoids in the Treatment of Seborrhea and
Acne. Dermatology, 196, 140-147.
Pecoraro, J. (2012). Survey fatigue. Quality Progress, 45(10), 87.
Pitts, B. G., Crosby, R., Laufenberg, S., Meidinger, G., & Monson, N. (1998). The Use of
Clinical Practice Guideline to Manage and Verify Weight Loss Outcomes of
Patients Treated with Fen-phen in Primary Care Settings. Nutritions in Clinical
Practice, 13, 241-250.
Rabin, B. A., Brownson, R.C., Kerner, J.F., Glasgow, R.E. (2006). Methodologic
Challenges in Disseminating Evidence-based Interventions to Promote Physical
Activity. am J Prev Med, 31(4S), S24-S34. doi:10.1016/j.amepre.2006.06.009
Rasmussen, N. P. (2008). America's First Amphetamine Epidemic 1929-1971. Am J
Public Health, 98(6), 974-985. doi:10.2105/AJPH.2007.110593
Routledge, P. P. M. (1998). 150 Years of Pharmacovigilance. The Lancet, 351(9110),
1200-1201. doi:10.1016/S0140-6736(98)03148-1
Sarpatwari, A., Avorn, J., & Kesselheim, A. S. (2014). Using a Drug-Safety Tool to
Prevent Competition. The New England Journal of Medicine, 370, 1476-1478.
doi:10.1056/NEJMp1400488
Sheehan, K. B. (2001). E-mail Survey Response Rates: A Review. Journal of Computer-
Mediated Communication, 6(2), 0-0. doi:10.1111/j.1083-6101.2001.tb00117.x
Shire-NPS PHarmaceuticals, I. (2017). Natpara Terms of Use. Retrieved from
https://www.natpara.com/terms-of-use.html
Smith, M. Y., & Morrato, E. (2014). Advancing the Field of Pharmaceutical Risk
MInimization Through Application of Implementation Science Best Practicces.
Drug Safety, 37, 569-580. doi:10.1007/s40264-014-0197-0
Solberg, M., & Richmond, F. J. R. (2012). Transparency in Drug Submission Processes
of 3 Asian Countries: A Survey of Industry Views. Drug Information Journal,
46(216-225). doi:10.1177/0092861512436583

147
Trotter, R. T. (2012). Qualitative Research Sample Design and Sample Size: Resolving
and Unresolved Issues and Inferential Imperatives. Preventative Medicine, 55,
398-400.
Tuffs, A. (2007). Thalidomide: The True Story? BMJ, 334(7600), 933.
doi:10.1136/bmj.39199.637986.59
Waldman, K. (2016). Challenge of Integrating REMS Elements into Pharmacy Systems.
Retrieved from https://www.jhsph.edu/research/centers-and-institutes/center-of-
excellence-in-regulatory-science-and-
innovation/training/Waldman_CERSI%20Writing%20Competition_2016.pdf
Wechsler, J. (2010). Transparency Initiative to Provide More Information on FDA
Regulation and Drug Development. Formulary, 45(7), 232-233.
Williams, R. (2012). The Nazis at the Heart of the Worst Drug Scandal of All Time:
Revelations of Their Connections With the Makers of a Deadly Drug. Newsweek,
160(12).
Worthy, S. L. (2016). Don't Sell Out Safety: A Call to Preserve Risk Evaluation and
Mitigation Strategies to Reduce Harm to Patients and the Public in the U.S.
Journal Pharmaceutical Policy and Practice, 9(2). doi:10.1186/s40545-016-
0051-0
Wu, J. (2016). The US Food and Drug Administration's Risk Evaluation and Mitigation
Strategy (REMS) Program - Current Status and Future Direction. Clinical
Therapeutics, 38(12), 2526-2532.

148
APPENDIX A. FINAL VERSION OF SURVEY

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Benefits-risk frameworks: implementation by industry

PDF

The impact of FDA Patient-Focused Drug Development (PFDD) on US drug development strategies: a survey of views from pharmaceutical product companies

PDF

Risk approaches and standards used in hospitals: a survey of industry views

PDF

Regulatory dissonance in the global development of drug therapies: a case study of drug development in postmenopausal osteoporosis

PDF

Examining the regulatory framework for drug compounding: industry views and experiences

PDF

Incentivizing quality in the manufacture of pharmaceuticals: manufacturers' views on quality ratings

PDF

Regulatory harmonization in a resource-limited setting: the World Health Organization Collaborative Procedure for Accelerated Registration

PDF

Evaluation of FDA-sponsor formal meetings on the development of cell and gene therapies: a survey of industry views

PDF

FDA influence on advisory committees through documentation: a content analysis and survey of industry views

PDF

Implementation of tobacco regulatory science competencies in the tobacco centers of regulatory science (TCORS): stakeholder views

PDF

Effect of GDUFA legislation on the development and approval of generic drugs: a survey of industry views and experiences

PDF

Regulatory CMC strategies for gene and cell therapies during mergers and acquisitions: a survey of industry views

PDF

Examining the cord blood industry views on the biologic license application regulatory framework

PDF

Implementation of unique device identification in the medical device industry: a survey of the change management experience

PDF

An integrated framework to evaluate customer service delivery: a study of electronic systems at FDA's Los Angeles Import Operations Branch

PDF

Regulatory team development in post-merger integration: a survey of views from medical product companies

PDF

Implementation of risk management in medical device companies: a survey analysis of current practices

PDF

Experience with breakthrough therapy designation: an industry survey

PDF

Risk management and recalls: a survey of medical device manufacturers

PDF

A survey of US industry views on implementation of decentralized clinical trials

Asset Metadata

Creator McDowell, Alicia Marie (author)

Core Title Challenges in the implementation of Risk Evaluation Mitigation Strategies (REMS): a survey of industry views

School School of Pharmacy

Degree Doctor of Regulatory Science

Degree Program Regulatory Science

Publication Date 10/16/2018

Defense Date 06/15/2018

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag biologics,drug safety,OAI-PMH Harvest,pharmaceuticals,REMS,risk evaluation mitigation strategy,risk management,RiskMAP,Safety

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Richmond, Frances (committee chair), Kuo, Chiaoyun (committee member), Pacifici, Eunjoo (committee member), Whalen, Ellen (committee member)

Creator Email acolaros@usc.edu,aliciamarieb73@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c89-80021

Unique identifier UC11670538

Identifier etd-McDowellAl-6850.pdf (filename),usctheses-c89-80021 (legacy record id)

Legacy Identifier etd-McDowellAl-6850.pdf

Dmrecord 80021

Document Type Dissertation

Format application/pdf (imt)

Rights McDowell, Alicia Marie

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA