University of Southern California Dissertations and Theses

Implementation of risk management in medical device companies: a survey analysis of current practices

(USC Thesis Other)

Implementation of risk management in medical device companies: a survey analysis of current practices

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content
IMPLEMENTATION OF RISK MANAGEMENT IN
MEDICAL DEVICE COMPANIES:
A SURVEY ANALYSIS OF CURRENT PRACTICES

by

Tony C. Chan

__________________________________________________________________

A Dissertation Presented to the
FACULTY OF THE USC SCHOOL OF PHARMACY
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF REGULATORY SCIENCE

December 2012

Copyright 2012 Tony C. Chan

ii
DEDICATION
I would like to dedicate this dissertation to my beloved family members. First
and foremost are the two most influential ladies in my life. My dedicated and loving
wife, Nancy, has always been supporting me tirelessly throughout all my post-
graduate academic pursuits. My faithful and trusting mother, Sap Kan Leung, who
always allows me the freedom to pursue my dreams. My adorable daughter, Esther,
provided me with many editorial comments and inspirations for my writing. Last but
not the least are my siblings, John Chan, May Chan, Patcy Chan, and Cleo Chan-
Lam, who are my loyal supporters and cheer leaders for all of my endeavors.

iii
ACKNOWLEDGEMENTS
This dissertation is a product of more than five years of study, research, and
writing. During that time, I was assisted by many people, and would like to take this
opportunity to express my sincere gratitude.
First, my advisor, Dr. Frances Richmond at the University of Southern
California, Director of the International Center for Regulatory Science, was
extremely supportive and influential. I sincerely thank her for all her guidance,
encouragement, support, patience, and numerous hours of advice. I have benefited
from her expertise in the regulatory science, global regulatory affairs and risk issues
as related to health products. And I am especially thankful for her help in framing the
dissertation and commenting extensively on all the drafts of each chapter. Her
unbelievable level of enthusiasm filled with much positive energy and never ending
support of her students has been an inspiration to me.
I would also like to thank other University of Southern California professors
and my dissertation committee members for their very helpful insights, comments
and suggestions. Additionally, I would like to acknowledge Dr. Michael Jamieson,
Dr. Kathy Rolle, Kimberly Horwood, our very patient and supportive doctoral
program managers along with Aditya Garg for his technical support, Christine
Browning for her administrative support, and Erin Chow for her logistic support on
virtual meetings. And finally thanks to the University of Southern California School
of Pharmacy colleagues who provided early research support for this work;
particularly, Ms. Pamela Corey at the library research department.

iv
TABLE OF CONTENTS
Dedication ii
Acknowledgements iii
List of Tables vi
List of Figures vii
Abbreviations x
Abstract xii
Chapter 1: Overview of the Study 1
1.1 Introduction 1
1.2 Statement of the Problem 2
1.3 Purpose of the Study 3
1.4 Importance of the Study 4
1.5 Limitations, Delimitations, Assumptions 5
1.6 Organization of the Study 7
Chapter 2: Literature Review 8
2.1 A Historical Perspective on Risk 8
2.2 Risk as a Set of Triplets (Risk Triplets) 9
2.3 An Evidence-Based Approach for Risk Decisions 13
2.4 Early Contributions by the U.S. National Research Council 15
2.5 Risk Management Frameworks Relevant to Medical Devices 20
2.6 Lessons Learned from Disasters and Failures of Risk Management 30
2.7 The Research Model 36
Chapter 3: Methodology 44
3.1 Introduction 44
3.2 Stage I: Creation of Survey Instrument 44
3.3 Stage II: Confirmation of Survey Instrument By Focus Group 45
3.4 Stage III: Administration of the Survey Instrument – Data 46
Collection & Analysis
Chapter 4: Results 48
4.1 Focus Group 48
4.2 Survey 50
4.3 Comparisons 89

v
Chapter 5: Discussion 95
5.1 Summary 95
5.2 The Dual Triad – “Behavior+Capability” Research Model 100
5.3 Additional Insights and Implications 110
5.4 Future Directions 114
5.5 Conclusions 116
Glossary 118
Bibliography 120
Appendices 126
Appendix A: Risk Management Failure Lessons 126
Appendix B: Survey Instrument 140
Appendix C: Q.1 Comments on Changes 150
Appendix D: Q.3 Comments on Perceptions 152
Appendix E: Q.15 Comments on Challenges 156
Appendix F: Summary of Comparisons 159

vi
LIST OF TABLES
Table 4.1. Focus Group Participants 48
Table A.1. Lessons from NASA: The Challenger Crisis 127
Table A.2. Lessons from NASA: The Columbia Disaster 129
Table A.3. Lessons from Sendai Earthquake/Nuclear Disaster 130
Table A.4. Lessons from Financial Crisis 132
Table A.5. Lessons from 911 Security Crisis 136
Table A.6. Lessons from Hurricane Katrina 138
Table A.7. Lessons from British Petroleum (BP) Oil Spill 139
Table F.1. Comparison: T30’s vs “Others” 160
Table F.2. Comparison: Risk Management Staff vs The Executive Group 163
Table F.3. Comparison: Smaller Size (<1,000 employees) versus Larger 167
Size Companies (>5,000 employees)
Table F.4. Comparison: High-Level versus Low-Level of Uncertainty 171
Avoidance

vii
LIST OF FIGURES
Figure 2.1. Kaplan’s evidence-based decision structure 14
Figure 2.2. Framework for Environmental Health Risk Management 19
Figure 2.3. FDA’s view on Managing The Risks of Pre-Market and 24
Post-Market Product Use
Figure 2.4. FDA’s Role in Medical Product Risk Management 25
Figure 2.5. ISO 14971 Risk Management Process 27
Figure 2.6. The behavior triangle 37
Figure 2.7. The capability triangle 41
Figure 2.8. Research model for medical device risk management 42
implementation
Figure 4.1. Affiliation and job level of respondents 52
Figure 4.2. Profile of participating companies 53
Figure 4.3. Level of uncertainty avoidance 55
Figure 4.4. Extent of changes in the last two years 55
Figure 4.5. To what extent do you agree or disagree with the following 58
statements that describe a possible perception of your
division’s risk management system
Figure 4.6. Please rank from most effective to least effective the 61
following methods that management uses to inform about
risk issues within your division/company
Figure 4.7. Effectiveness of communicating risk information 62
Figure 4.8. Structural features of organization 64
Figure 4.9. Title of process owner 64
Figure 4.10. Risk management process owner’s other responsibilities 65

viii
Figure 4.11. Process owner’s supervisor 66
Figure 4.12. How would you describe the following aspects of your 67
division’s risk management system?
Figure 4.13. Have you observed your division to use the following 69
techniques when analyzing or assessing product or process
risks?
Figure 4.14. To what extent do you agree or disagree on the following 70
major challenges when you implement your risk management
system within the quality management system?
Figure 4.15. Which risk activities/processes does your division primarily 73
outsource?
Figure 4.16. Why does your division outsource risk management activities? 74
Figure 4.17. In what functional area does the risk management budget reside? 76
Figure 4.18. Please rank the following risk management activities according 76
to the amount of time spent
Figure 4.19. How would you describe the impact of the current economic 77
climate for investment in risk management in your division?
Figure 4.20. Please rank the following perceived business benefits that 78
your divisional management considered to be important when
they invest in risk management system
Figure 4.21. What are the primary challenges for your risk management 79
organization over the next two years?
Figure 4.22. What are your division’s minimum qualifications for those 80
who conduct risk management activities?
Figure 4.23. What kind of risk management training has your division 81
offered for key employees who conduct risk management
tasks?
Figure 4.24. What kind of regulations and standards training have those 82
responsible for your division’s risk management activities
obtained?

ix
Figure 4.25. What kind of training on risk management techniques do 84
you know to be offered to those responsible for your division’s
risk management activities? Check all that apply
Figure 4.26. How/When does your division plan to continue educate and 85
train the risk management workforce?
Figure 4.27. Which of the following risk management incidents have 86
happened in your division in the last twelve months?
Figure 4.28. How likely do you think that the following type of incident 87
might happen in your division over the next twelve-month
period?
Figure 4.29. Regarding your division’s Corrective Action System 88

x
ABBREVIATIONS
Acronym Full Phrase or Meaning
A.D. Anno Domini (Latin); In the year of the Lord (English)
AAMI Association for the Advancement of Medical Instrumentation
AFD Anticipatory Failure Determination
ATSDR Agency for Toxic Substances and Disease Registry
B.C. Before Christ
BP British Petroleum
C. Centigrade
CAIB Columbia Accident Investigation Board
CAPA Corrective And Preventive Action
CFR Code of Federal Regulations
CGMP Current Good Manufacturing Practices
Ch. Chapter
CPSC Consumer Product Safety Commission
CRMPG Counterparty Risk Management Policy Group
Deg. Degree(s)
DoD Department of Defense
EC European Communities
EN European Norm
EPA Environmental Protection Agency
et al. And others
EU European Union
F&E Tree Fault Tree and Event Tree
FAA Federal Aviation Administration

xi
FCIC Financial Crisis Inquiry Commission
FDA Food and Drug Administration
FMEA Failure Mode and Effect Analysis
GHTF Global Harmonization Task Force
GMP Good Manufacturing Practices
HAZOP Hazard and Operability Study
HROs Highly regulated, high-risk, or high-reliability organizations
IEC International Electrotechnical Commission
ISO International Organization for Standardization
NASA National Aeronautics and Space Agency
NIEHS National Institute of Environmental Health Sciences
NIOSH National Institute for Occupational Safety and Health
NRC National Research Council
NSTS National Space Transportation System
OSHA Occupational Safety and Health Administration
p. Page
pp. Pages
QRA Quantitative Risk Assessment
SRB Solid Rocket Booster
STS Space Transportation System
TRIZ Acronym for Russian phrase meaning “Theory of the Solution of
Inventive Problems”
UA Uncertainty Avoidance
U.S. United States

xii
ABSTRACT
This survey analysis examined current practices related to the implementation
of risk management in medical device companies. Twenty-six of the Top 30 (by
global market revenues) and twenty-seven other medical device companies with
direct product sales in the U.S. market participated in the study. Through a literature
analysis, the researcher developed a systematic framework, the
“Behavior+Capability” model, to explore risk management attitudes and practices of
a medical device company according to six dimensions. This framework was used to
construct a forty-question survey instrument for the study. Most respondents to the
survey classified their companies as relatively risk averse. Most viewed their risk
management systems as satisfactory but some elements, such as risk communication,
independence of risk managers, breadth of risk tools and methods to capture lessons
learned, were not well-developed. Respondents in most companies and particularly
the largest companies judged their risk management systems as relatively immune to
the current economic climate. Most systems appeared to be highly driven by
regulatory requirements rather than business imperatives.
Results suggested that initiatives to achieve best practices in risk
management might be improved by incorporating more advanced approaches and
applications of risk management tools and techniques. More research on specific
aspects of weakness identified for some companies might elucidate root causes for
the challenges. Risk management systems might be enhanced further in medical
device companies by expansion of training for typical in-house activities to include

xiii
outside certification or graduate programs, or by introducing risk management as
part of curricula in engineering or regulatory science programs.

1
CHAPTER 1
OVERVIEW OF THE STUDY
1.1 Introduction
For every dollar that an American consumer spends, 25 cents will pay for
FDA-regulated products (FDA 2011). The oversight of FDA-regulated products is
more stringent than that of most other product types because FDA’s mission, stated
as “Protecting and Promoting Public Health”, is operationalized by requirements to
demonstrate safety and effectiveness. In the medical device sector, the FDA began
requiring evidence of safety and effectiveness before a medical device could be put
on the market for commercial use as early as 1968. However, until 1996, the primary
tool to ensure product safety was the enforcement of regulation requiring Good
Manufacturing Practices (GMP). In the mid-to-late 1980s, it became apparent that
such an approach had limitations. FDA discovered that more than 40% of post-
market product problems came from design flaws rather than manufacturing
problems associated with the medical device (FDA 1990). These design problems
then led the FDA to promulgate a new regulation, 21 CFR 820 – Quality Systems
Regulations, published on October 7
th
, 1996, that embraced a design control strategy
with a broader focus on the whole product life cycle.
One of the stipulations of the new Quality System Regulation was a
requirement for risk analysis. A specific comment (# 83) in the preamble of the
regulation further identified that risk analysis should be performed according to an
ISO 14971-1 standard, titled “Application of Risk Analysis to Medical Devices” (EU

2
2002). The preamble also stated that medical device manufacturers should update
their systems as this particular risk-related standard evolved over time (FDA 1996).
As of March 9
th
, 2010, a second edition of the ISO 14971 Standard, ISO 14971:2007,
has been adopted as the prevalent version (EU 2007). As part of this standard,
regulatory expectations have evolved, so that the standard no longer focuses on risk
assessment, but rather embraces a wider system of risk management.
It is challenging for companies to establish a risk management system that
has evolved from a simple need for analysis to a more comprehensive, action-
oriented, and problem-solving activity. Since 1996, the regulatory expectations in
managing medical device risks during the total product life cycle have significantly
increased. It is evidenced not only by the aforementioned need for risk management
during the design and development phase (FDA 1997). The company is also
expected to integrate risk management into the quality management system (GHTF
2005), and in addition, into such activities as outsourcing (GHTF 2008) and
corrective and preventive actions (CAPA) (GHTF 2010).
1.2 Statement of the Problem
The need for risk management is not a new requirement for medical device
companies, but the evolving standards and uneven interpretation of expectations are
reflected in anecdotal evidence; companies vary greatly in the extent and methods by
which they operationalize risk management activities. Typically literature relating to
risk management as carried out by industry uses a case study or descriptive
methodology. It is therefore not clear what constitutes “usual practice” or “best

3
practice” across a spectrum of companies. In addition, the recognized and
harmonized EN/ISO 14971 “Application of Risk Management to Medical Devices”
standard has become the framework for acceptable industry practice for more than
ten years, a long enough period for companies to develop reasonably mature
practices and operational experience. However, we do not have a systematic
approach to understand what methods are used to conduct risk management
activities, what issues are most problematic in achieving good results, and what
elements are not captured by the ISO framework but nevertheless impact the success
of the company in avoiding safety problems with the products that they produce.
1.3 Purpose of the Study
This exploratory study used survey methods to understand how companies
conduct risk management. By surveying a number of companies of differing sizes
and sales volumes, it attempts to identify current practices in the industry, including
the extent to which these processes are similar or different, and examines the
challenges that companies encounter when trying to implement an effective risk
management process within their quality management systems.
The study begins with a literature analysis to understand what is seen by
previous individuals as important elements in a risk management system and
develops a systematic framework for areas of risk management practice that are
important for the successful implementation of ISO 14971. This framework, which
looks at both the behavior and capability of the company to conduct risk
management, was used to guide the construction of the survey so that the survey

4
addressed more comprehensively those areas suggested to be important in previous
studies of deficient systems. The survey was distributed to 80 risk management
practitioners at different organizational levels in medical device companies with a
strong U.S. presence, that were differentiated by whether they belonged to the “Top
30” companies by sales volume or to “Other” companies.
1.4 Importance of the Study
Understanding the strengths, weaknesses, and challenges of risk management
may help medical device companies to improve their risk management systems.
Such an understanding may help the companies to implement better systems by
avoiding implementation pitfalls and by learning from the best practices of others.
The ultimate goal and benefit of improving a risk management system is to improve
medical products, by reducing cost, eliminating waste, curtailing the number and
severity of adverse events.
Another potential benefit of the research to be undertaken is to provide
information to guide training initiatives within a medical device company. Training
initiatives often attempt to deal with a variety of issues, some of which are redundant
for most personnel. Thus, precious time is spent on materials that the trainees
understand well already and less time is available to train in areas of deficit. Such a
broad based approach often results in dissatisfaction with the training program. A
well-focused training program that concentrates on issues known to be of particular
concern would increase the effectiveness and decrease the time spent on unfruitful or

5
unhelpful training. Such focus might help to make the training more valuable and
palatable.
A final area of significance of the research shown here is to contribute to the
development of a snapshot that sets a benchmark against which future systems can
be compared. By having a systematic and well-considered survey tool, a later survey
can be conducted to see if systems have shown improvements over time. Individual
companies can study the practices of others and potentially use the survey as a tool
with which to compare their own individual activities.
1.5 Limitations, Delimitations, Assumptions
A number of limitations and delimitations must be acknowledged that may
affect the collection of data and the compilation of study results. This study will
focus on medical device companies that have direct product sales in the U.S. market
and thus are subject to the U.S. FDA’s medical device rules and regulations.
Medical device companies that do not sell products to U.S. are not included in the
scope of this study. In addition, original equipment manufacturers of medical devices
are not included in the scope of this study. Moreover, the researcher is limited by the
time available under reasonable academic constraints for doctoral research, so the
project is, by necessity, a snapshot in time.
There are concerns typical for any type of survey research regarding control,
reliability and validity of the instrument, sampling bias and sample size constraints
on the study. A new research model (Behavior+Capability) is developed in this work
based upon thorough review of literature; no previous information using this model

6
is available to assure its validity and this may reduce the confidence that we can have
in its value as a comparative or validation benchmark. Challenges of using this
model are to some degree mitigated by the fact that some elements of the model have
been used previously by other researchers. In addition, the researcher will use the
reviewed literature to support the construction of each survey question in order to
enhance the reliability and validity of the instrument. Another approach that the
researcher will add to improve the face validity of the instrument is the use of a focus
group feedback to fine-tune the instrument.
In a quantitative research approach, researchers always strive to ensure
representative samples of their research populations. However, representative
samples may not exist or be feasibly identified in a non-homogeneous medical
device industry. A homogeneous sample might be achieved by limiting the study to
only companies of a certain size or product line. However, such a sample would
represent only a small subsector of the industry, whose views may not extrapolate to
the entire industry. Therefore, this study will largely focus on the qualitative patterns
of risk management activities as an exploratory study with the hope that findings can
provide insights about risk management that could be helpful to the medical device
industry in general, and could shape further more specific research.
This study faces additional challenges that are common to all exploratory
research using survey methodologies. The researcher anticipates that respondents
will be the owners of the risk management process in the surveyed company. The
process owner may not have specific knowledge regarding each of the areas of

7
concern in the survey. Furthermore, the process owner may hold only one
perspective in the organization that may not be a true or accepted view for other
individuals in the same company. Memory of change in the risk management system
is further made difficult by employee turnover. Recently hired respondents may not
be able to recognize previous experiences or transitions. The respondents may also
be hesitant to answer questions, especially on sensitive issues related to product
safety. One way to encourage true responses is to guarantee confidentiality and
anonymity to the respondents, but this may not be sufficient to satisfy some
respondents. The researcher assumes respondents will participate in the study with
open and honest attitude and also believes the survey instrument is constructed with
bias reduced to a minimum. However, these assumptions may not prove to be fully
justified and the researcher will be vigilant to keep bias to a minimum.
1.6 Organization of the Study
In the chapters that follow, the researcher presents the study on the
implementation of risk management in medical device companies. Chapter 1
provides an overview of the problem and an introduction to the research. Chapter 2
reviews the current state of knowledge in this field by studying the available
literature relating to risk management in general and to standardized frameworks in
particular. Chapter 3 outlines the methods used to guide the analysis of rigor of
oversight of a survey analysis. Chapter 4 presents the findings from analysis of the
survey and additional insights, and Chapter 5 then discusses the results and their
implications.

8
CHAPTER 2
LITERATURE REVIEW
2.1 A Historical Perspective on Risk
…..but from the tree of the knowledge of good and evil you shall not
eat, for in the day that you eat from it you shall surely die.
— Genesis 2:17 (Ryrie 1978)

For as long as there has been recorded history, there has been substantial
concern about the challenges of living in a risky world where the certain outcome is
eventually death. It is not clear that mortality derived from a single biblical act. Yet,
whatever its origin, part of the human condition is to extend this limited lifetime for
as long as possible, in an environment where scarce resources, disease, accidents and
natural disasters challenge the quality of that life. Thus, the management of risk has
been an important part of survival and prosperity throughout history. According to
Grier (Grier 1981), a group of shamans called the Asipu, who lived close to the
reputed Garden of Eden in the Tigris-Euphrates valley around 3200 B.C., served
their community by guiding “risky” decisions based on systematically collected
evidence or signs. Before pursuing an uncertain venture, such as a proposed marriage
arrangement or a favorable building location, the community would invite a member
of the Asipu to identify possible actions and alternatives (Covello and Mumpower
1985). The Asipu would gather data and interpret signs from the gods. The priest-
like Asipu would then systematically enter the positive or negative signs into a
ledger that documented all of the identified alternatives. The Asipu would analyze

9
the data, interpret the results and recommend the best solution to the requestor. The
final report was issued on an etched clay tablet (Oppenheim and Reiner 1977).
The Asipu’s practices appear to be an early, simple form of risk analysis.
They paint a reasonably complete picture of risk analysis procedures much like those
that we would recognize today. Their description provides evidence that civilizations
have long been dealing with the problems of risk in a sophisticated manner.
However, what have changed are the efforts to formalize the processes so that a
standardized approach to risk assessment and management can be used consistently
and the development of a standardized terminology for universal understanding.
2.2 Risk as a Set of Triplets (Risk Triplets)
In order to understand and communicate effectively about risk, it is necessary
to have a clear and complete definition of the risk that is confronted. One approach
to the definition of risk is described as a “set of triplets” (Kaplan and Garrick 1981).
The “Set of Triplets” prompts the risk analyst to answer three questions:
What can happen? i.e., What can go wrong? What is the hazard or hazardous
situation?
How likely is it? i.e., What is the frequency or probability of the hazard or
hazardous situation that may occur?
What are the consequences? i.e., What is the damage or harm caused by the
hazard or hazardous situation? What is the extent of the damage or harm?

10
Risk analysts can also formulate these triplets mathematically in terms of a
scenario or description, a probabilistic analysis of that scenario, and a quantitative
metric for the level of damage or harm that can occur consequent to that scenario.
Describing the Hazard
In many archeological finds from ancient Assyrian, Egyptian, and Sumerian
sites, predecessors of modern dice called “tali”, shaped and polished from the
“knucklebone” or heel of deer, horses, oxen, or sheep, were uncovered together with
tomb illustrations and scoring boards that confirm the use of tali for gaming. The
Arabic word for dice, “al zahr”, was referred to as present day “hazard” (Bernstein
1996). Today risk is analyzed by considering a specific scenario within which both
the consequence or harm and uncertainty are initiated by a hazard, defined as “a
source of danger” (Webster Dictionary).
Quantifying Risk
In order to make sensible decisions about risk, it is important to predict the
likelihood that the situation producing harm will occur. Records from early
Mesopotamia clearly suggest that even early civilizations understood the need to
estimate the probability of risk. The quantification of risk may link back to man’s
fascination with games of chance, a fascination almost as old as man himself (David
1962).
Expressing probability in a quantitative way, however, is a much more recent
science in the western world. According to Covello and Mumpower (Covello and
Mumpower 1985), contemporary risk analysis is difficult to separate from

11
probability theory because quantitative risk analysis requires mathematical notions of
probability. Mathematical theories that encompassed the essential elements of
frequency, betting, randomness and probability only appeared 1500 years after the
Mesopotamian empire, when they were captured by the work of Pascal in 1657,
Halley in 1693, Bernoulli in 1696, Bayes, Price and LaPlace in the 1770s, and others.
The emergence of probability theory in the 17
th
century set the stage for quantitative
risk assessment (QRA). QRA is the application of a mathematical framework using
probability theory to aid decision-making regarding the safety of complex
technological systems (Apostolakis 2004).
Interestingly, the mathematical approach also had a longer history.
According to Grier (Grier 1981), clear evidence of a probabilistic basis for risk
evaluation is seen in the work of Arnobious, a prominent figure in a pagan church of
North Africa in the 4
th
century A.D. Arnobious was led to develop a theory of
probability after his denouncement of pagan beliefs in favor of Christianity. The
bishop of the Christian church challenged Arnobious’ Christian faith. Rejected for
baptism, Arbnobious made an effort to prove his real conversion by publishing an
eight-volume monograph entitled “Against the Pagans”, in which he introduced an
argument based on probabilistic risk analysis. He discussed the risks and
uncertainties associated with one’s decisions that could affect one’s soul. He used a
two-by-two matrix to argue the two alternatives of “accept Christianity” or “remain a
pagan”. He also debated two possibilities of state of affairs: “God exists” or “God
does not exist”. He concluded that if God did not exist, there would be no difference

12
between whether one lived a godly life or a secular one, except that Christians would
be making fools of themselves by denying pleasures of the flesh. However, on the
other hand, if God did exist, the eternal benefits of being a Christian would be far
better than a pagan’s eternal suffering. Arnobius’ presentation had marked the first
recorded demonstration of the “Dominance Principle”, a common heuristic method
in modern decision science using conditions of risk and uncertainty. Arnobius had
not only won his argument to prove his Christian faith, but also eventually his theory
was accepted by the mainstream of Christian theology and intellectual thought. In
1657, Pascal demonstrated his first application in probability theory by extending
Arnobius’ matrix. His conclusion was that the expected value of being a Christian
outweighed that of being an atheist.
Linking Consequences through Causality
Whether to take a risk depends not only on probability, but also on severity of
harm that might be associated with the situation. This requires the demonstration of
a causal relationship between the adverse consequence and the hazard. The
importance of causality could be recognized as a focus in medical diagnosis, even at
the time that mathematical descriptions of risk were being formulated in the 17
th

century. John Evelyn (1620-1706), for example, postulated that smoke in London
had caused respiratory problems that were endemic at the time, and also correlated
scrotal cancer with occupational exposures to soot in chimney sweeps. Based on
such associations, Edmond Halley proposed life-expectancy tables in 1693 that
eventually became widely used for insurance purposes.

13
The concepts of causality and incidence go hand-in-hand and today rely
heavily on statistical estimation. Thomas Bayes’ work on a system of inference,
published in 1761, was a seminal contribution to qualitative analysis. Bayes’
Theorem links the uncertainty of a probability model before and after observing the
modeled system and is now used extensively in quantitative risk assessment. Richard
Price, founding father of actuarial science, published his well-known book titled
“Observations on Reversionary Payments” in 1771 (Bernstein 1996). This book was
regarded as the bible of actuarial science that marked the beginning of applying
probabilistic calculation in the field insurance. In 1792, Pierre Simon de LaPlace
linked a first-of-its-kind quantitative risk analysis with the probability of mortality
with and without smallpox vaccination, bringing together risk and causality (Molak
1997) in a way that is typical for risk analysis today.
2.3 An Evidence-Based Approach for Risk Decisions
The ultimate reason to analyze risk is to make a decision about the best
option given a set of alternatives based on the available evidence. Thus, a risk
decision (Haimes, Kaplan et al. 2002) requires three elements: a set of alternatives,
an evaluation of the outcomes of each alternative, and a value judgment on each
outcome. Since the truth of every outcome typically has some level of uncertainty,
Bayes’ Theory can be applied on the entire body of available evidence. Kaplan
(Kaplan 1997) pulled these concepts together in a framework for evidence-based
decision-making (Figure 2.1).

14

Figure 2.1. Kaplan’s evidence-based decision structure

15
Risk analysts may apply a Bayesian probabilistic approach on the “Risk
Triplets” to formulate a QRA system model for complex and large-scale systems to
identify, prioritize, assess, and manage risk scenarios (Haimes, Kaplan et al. 2002).
This process is typically recognized in such applications as high reliability or safety
operations. For example this approach is commonly applied in aerospace, aviation,
and nuclear industries. However, it is not clear to what extent this approach is
commonly used in the medical device industry. One area of interest in the present
research is the extent to which more complex, mathematical tools are used to
quantify risk or whether risk is still characterized or done by applying simpler
estimation methods.
2.4 Early Contributions by the U.S. National Research Council
The growing recognition that risk management was an important part of
product management was clearly apparent in the second half of the twentieth century
in sectors as diverse as train systems, nuclear power generation and space travel.
Thus, it is not surprising that parallel discussions in the field of health product
development could be recognized. These were largely spurred by concerns
regarding interactions between health, safety, and the environment, as technologies
such as nuclear power and air pollution heightened public awareness of potential
health issues. Among the federal agencies that became advocates of risk science
were the Environmental Protection Agency (EPA), Occupational Safety and Health
Administration (OSHA), Consumer Product Safety Commission (CPSC), National
Institute of Environmental Health Sciences (NIEHS), National Institute for

16
Occupational Safety and Health (NIOSH), Food and Drug Administration (FDA),
and Agency for Toxic Substances and Disease Registry (ATSDR). One of the first
steps in systematizing such studies were carried out by Mantel and Bryan (1961),
who performed risk estimation studies by testing animals with different dosages of
drug substances and then using bioassay data to estimate dose-related risk – a step
critically important for risk evaluations. Later, studies of potential chemical hazards
as diverse as asbestos, formaldehyde, pesticides, and saccharin, provided evidence to
justify stricter government regulation (National Research Council (U.S.). Committee
on Risk Assessment Methodology. and National Research Council (U.S.). Board on
Environmental Studies and Toxicology. 1993). However, it was also clear that
regulations could not be used to control every risk, no matter how small. What were
regulators to do?
In order to guide regulatory decision-making, the U.S. National Research
Council undertook a series of initiatives to understand how to evaluate risk and use it
to make better regulatory decisions. The seminal initiative was the formation of the
first committee – Committee on Institutional Means for Assessment of Risks to
Public Health in 1983. The Committee was charged with considering the current
practice of risk assessment and its relation to the process of regulating human health
hazards. The Committee was also given responsibility to suggest improvements to
those risk assessment procedures. From its work derived three seminal reports.
The first report, titled “Risk Assessment in the Federal Government:
Managing the Process” (more commonly called the Red Book) (National Research

17
Council (U.S.). Committee on the Institutional Means for Assessment of Risks to
Public Health. 1983) emphasized the need to improve risk assessments and
associated decision-making in the government. It defined steps important in risk
assessment, and provided a generally accepted nomenclature for risk assessment. In
this report, attention was drawn to risk communication, which was highlighted as a
key element in a democratic society. To explore this aspect of risk management
further, a second committee – Committee on Risk Perception and Communication
was formed and met six times between May 1987 to June 1988, to develop a report
titled “Improving Risk Communication” (National Research Council (U.S.).
Committee on Risk Perception and Communication. 1989). The Committee
recommended several requirements for effective communication, including setting
realistic goals, safeguarding openness, safeguarding balance and accuracy in risk
messages, and fostering competence, as essential elements for improving risk
communication.
A third committee, the Committee on Risk Characterization of the National
Research Council was also formed and met in 1987. Its work was then somewhat
delayed until 1994 after which it produced a report titled “Understanding Risk” in
1996. This report proposed the necessity to re-conceive risk characterization in order
to increase the likelihood of achieving sound and acceptable decisions in managing
risk. It was viewed as important because risk characterization was often
misperceived to be useful only to provide a summary or a translation of technical

18
results to a decision maker, instead of relating to the overall process of
comprehending and dealing with risk.
The pioneering efforts by the U.S. National Research Council and its
committees were important in establishing risk management as a science-based,
academic and applied discipline in the healthcare sector. The three landmark studies
described above presaged the current framework of risk management by examining
six dimensions to be considered when managing risk; i.e. risk assessment, risk
management, risk perception, risk communication, risk characterization, and risk
decision-making.
The reports by the National Research Council ultimately contributed to the
1997 report of the U.S. Presidential/Congressional Commission on Risk Assessment
and Risk Management. This report emphasizes a dynamic risk communication
process involving the ongoing engagement of stakeholders. The process was named
“The Commission’s Framework for Environmental Health Risk Management”.
The framework developed by the Presidential Commission, illustrated in
Figure 2.2, was designed to help all types of risk managers, including government
officials, individuals employed by private sector businesses, and members of the
public, to make good risk management decisions when dealing with any type of
environmental health risk. The framework is general enough to work in many
situations in which the degree of effort could be scaled to the importance of the
problem, the potential severity and economic impact of the risk, the level of
controversy surrounding the risk, and resource constraints. The framework is

19
intended primarily for risk decisions related to setting standards, controlling
pollution, protecting health, and cleaning up the environment. Three key principles
emphasized as important to implement this framework successfully included:
(i) Adopting a broad context for risk assessment, instead of evaluating
single risks associated with single agents in single environmental
medium;
(ii) Involving stakeholders at all phases of the process; and
(iii) Adopting an iterative approach, so that any new information or
perspectives that may emerge may be taken into account by revisiting
early stages of the process.

Figure 2.2. Framework for Environmental Health Risk Management

20
The development of these frameworks and associated principles and
guidelines has brought an element of clarity to the field of risk assessment and risk
management. Risk management principles can be of value in assigning priorities to
important risk issues competing for attention and resources, in reaching decisions in
the face of scientific uncertainty about the level of risk associated with health
hazards, in balancing benefits and risks, and in acknowledging social and cultural
considerations in risk management. Without such guidance, risk management
decision-making can be highly complex, raising difficult questions to which there are
often no easy answers (Krewski 2011). Nevertheless, in the early 1990s, the U.S.
based considerations of risk management remained relatively unfocused and difficult
for general practitioners to implement easily.
To assist in the development of a more specific framework for risk
management in the medical devices arena, many risk managers looked both to the
FDA and to organizations at the international level, particularly to the standard-
setting bodies such as the International Organization for Standardization and the
Global Harmonization Task Force. In the descriptions that follow, the development
of standards and guidance documents for risk management are examined in the
particular context of medical devices.
2.5 Risk Management Frameworks Relevant to Medical Devices
Much of the thinking captured in the reports described above reflects the
complexity of the risk management process—its dependence on context, clear
concepts and useful definitions, and its prioritization based on severity and

21
prevalence. Thus, a systematic evaluation of risk will depend on having an arsenal of
techniques or tools with which the various hazards and situations associated with a
medical device can be identified, evaluated, and reduced. From such an analysis
comes an often difficult decision about “How safe is safe enough?” (Derby and
Keeney 1981). The decision may result in a conclusion regarding the acceptability
of the risk and may ultimately keep the product from entering the market. The
decisions will require an analysis of medical benefits versus risks that will be posed
to patients, users, care-takers, and other stakeholders such as the public, society, or
even the environment. Therefore, a comprehensive risk management system or
framework is essential to provide an effective and efficient system that meets many
needs; it enables a company to produce safe and effective medical products, fulfill
regulatory requirements, satisfy stakeholders’ concerns, and ensure business
longevity.
Therefore, in medical product industries, specific requirements were sought
that would assure medical product safety and efficacy before a company put its
product into commerce. Some of the early efforts were captured in two sets of
frameworks that are in many ways similar, but come from federal and international
sources. Nationally, the U.S. Food and Drug Administration (U.S. FDA) introduced
a regulatory risk management framework for medical product use. Internationally,
the International Organization for Standardization (ISO) and the Global
Harmonization Task Force (GHTF) published risk and safety related standards and
guidance documents through their expert committees. These standards and guidance

22
documents, described in more detail below, were considered to reflect state-of-the-
art, best practices for industry. Their usefulness as benchmarks is reflected in the
fact that they were used to offer protection to medical product manufacturers from
liability suits in a court of law; U.S. courts often accepted compliance with an
appropriate safety standard as a suitable defense that a medical product company had
been diligent in carrying out its obligation to keep patients or users free from
unacceptable risks. A more detailed description of these standards is therefore
relevant to understanding the framework that governs risk management in the
medical device sector today.
A. The U.S. FDA Risk Management Framework
In the medical device sector, concern about the safety of medical devices
became evident from a regulatory viewpoint from the mid-1960s, when the first
regulations were promulgated to require evidence of safety and effectiveness before
a medical device could be put on the market for commercial use. At that time, FDA
used the Good Manufacturing Practices (GMP) regulations to achieve safety and
effectiveness through careful management of product manufacturing and inspection
of the devices themselves. However, in the mid-to-late 1980s, the agency was
disturbed by its findings that more than 40% of the post-market product problems
that came to their attention came from faults in the actual design of medical devices.
These design problems led to the creation in 1996 of a new regulation, 21 CFR 820 –
Quality System Regulation, that embraced a design control strategy with enhanced
focus on risk. Design controls required medical device manufacturers marketing

23
products in the U.S. to institute a systematic, risk-based system of product design in
order to reduce the large numbers of product defects seen on the market as a result of
poor design practices.
Within the published Quality System Regulation, a requirement for risk
analysis was stipulated first time in U.S. medical device history. However, the
methodology to assure that risks were appropriately assessed and managed was in its
infancy. To improve this situation, the U.S. Department of Health and Human
Services, Food and Drug Administration (U.S. FDA) formed a committee whose
report was titled “Managing The Risks From Medical Product Use”. (FDA 1999)
This report suggested the need for a systemic framework to structure the risk
management of medical products. Figure 2.3 shows this framework and indicates
how various stakeholders were seen to be involved in managing medical product
risks.
The report also describes the roles of all participants in the medical product
development life cycle and delivery system (Figure 2.4). The ultimate aim of clearly
defined roles was seen to maximize benefit and minimize risk. Every participant in
this process would have a clearly defined role and a shared responsibility to
safeguard a threshold in which benefits outweighed risk. This goal was to be
achieved by ensuring medical products were evaluated for risk throughout their
lifecycles – i.e. through development, testing, manufacturing, labeling, prescribing,
dispensing, and usage to improve patient health.

24

Figure 2.3. FDA’s view on Managing The Risks of Pre-Market and Post-Market
Product Use

Source: U.S. Department of Health and Human Services, Managing The Risks From
Medical Product Use

25

Figure 2.4. FDA’s Role in Medical Product Risk Management

Source: U.S. Department of Health and Human Services, Managing The Risks From
Medical Product Use

At the same time, those who drafted the Quality Systems regulations were
also aware of considerable international attention to this issue. Comment # 83 in the
preamble of the regulation discussed how risk analysis should be performed
according to the ISO 14971-1 Standard, “Application of Risk Analysis to Medical
Devices”. It also stated that medical device manufacturers should stay current with
the development of this particular risk-related standard. This latter instruction was
significant because the ISO standard was not produced by the FDA itself, and was a
rather special departure from the more usual approach of the FDA to write US-
specific guidelines. Since that time, FDA has recognized the use of ISO 14971 as a
harmonized international standard to guide the risk management activities of medical

26
device companies. Thus, companies began to study carefully the documents
produced by the International Organization for Standardization (ISO) for guidance
on risk management.
B. The International Risk Management Framework: ISO 14971
The development of ISO 14971 was not carried out in a vacuum, but instead
was shaped by precedent activities in the European Union (EU) to establish risk
management standards quite early in the 1990s. The first step made by the EU was
the publication in 1997 of a European Standard, EN (European Norm) 1441 on Risk
Analysis for Medical Devices (CEN 1997). The framework focused on using a
specific set of methods called Failure Mode and Effect Analysis (FMEA), for
medical device risk analysis. ISO adopted this European Standard in 1997, when it
essentially duplicated its publication under the title “Medical Device – Risk
Analysis”, ISO 14971-1. This standard was useful for prioritizing activities but
remained incomplete because of its focus on the particular use of a single approach,
namely FMEA. Thus, a working group was formed to incorporate risk management
practices already defined in the IEC 60601 Standard Series, a set of standards
focusing on medical electrical equipment, into its new international standard,
designated as ISO 14971, in 2000. This standard then replaced the ISO/IEC 14971-1
standard, and was recognized as both an European-harmonized standard and an
FDA-recognized standard. It emphasized a more complete risk management
framework, shown in Figure 2.5 (ISO 2000).

27

Figure 2.5. ISO 14971 Risk Management Process

RISK ANALYSIS

•
INTENDED USE identification
•
HAZARD identification
•
RISK estimation
•
HACCP plus ISO/IEC 14971

RISK EVALUATION

•
RISK acceptability decisions

RISK CONTROL

•
Option analysis
•
Implementation of measures
•
Critical Control Points
•
RESIDUAL RISK EVALUATION
•
Overall RISK acceptance

Post-production Information

•
Post-production experience
•
Review of RISK MANAGEMENT
experience
Risk
Assessment
Risk
Management

RISK ANALYSIS

•
INTENDED USE identification
•
HAZARD identification
•
RISK estimation

RISK EVALUATION

•
RISK acceptability decisions

RISK CONTROL

•
Option analysis
•
Implementation of measures
•
RESIDUAL RISK EVALUATION
•
Overall RISK acceptance

Post-production Information

•
Post-production experience
•
Review of RISK MANAGEMENT
experience
Risk
Assessment
Risk
Management

28
The ISO 14971 Standard includes all the basic principles of risk management
applicable to medical devices. It specifies the steps required for identifying hazards,
estimating risk, evaluating risk, and controlling risk. A medical device
manufacturer’s risk management process must include these steps. The new standard
has two important aspects that deserve special mention. First, the standard expands
the scope and potential approaches to the risk management process (Figure 2.5) that
enables manufacturers to satisfy the essential requirements of the medical device
directives of the EU. These directives have similar authorities in the EU as the U.S.
Code of Federal Regulations in the U.S.. Essential requirements are necessary
elements for protecting the public interest in the EU, are mandatory for product
compliance before they are put into commerce, and must be applied as a function of
the hazards inherent to a given product (Lemmel 2000).
Second, the standard deemphasized the use of FMEA, which is recommended
only if it fulfills appropriately the particular risk analysis task under study; many
other analytical approaches are acknowledged to potentially be suitable to achieve
the same end. The expanded scope of the standard emphasizes management
responsibilities. The standard stipulates that management responsibility is the initial
and key requirement. More precisely, management must incorporate the following
tasks:
• Defining policies for determining acceptable risk;
• Ensuring the provision of adequate resources, including the identification
of a risk management team;

29
• Ensuring the assignment of trained personnel to perform risk assessment
and management activities; and
• Reviewing the results of risk management activities at defined intervals to
ensure the effectiveness of the risk management process.
The expanded scope also mandates implementation of the following records:
• Risk management plan;
• Risk management file;
• Risk management report; and
• Post-production information.
These requirements are interesting because they draw attention to the fact that
risk management rests on more than having a framework, but also on a human
dimension where the activities of people must be directed and organized. Currently,
the ISO 14971 Standard has just gone through a second edition that incorporated
only minor changes from the initial edition; however, the volume of explanation of
the stipulated framework in the second edition has increased almost three-fold (from
22 pages to 65 pages). The ISO 14971:2007 “Application of Risk Management to
Medical Devices” stipulates an expectation of a risk management framework that
considers the total lifecycle of the product. It includes a risk management process
(See Figure 2.5), and defines executive responsibilities, personnel qualifications for
performing risk management activities, and the documentation needed to provide a
record of risk management activities. Limitations in the document are still perceived
by some to exist. For example, it does not provide guidance with respect to clinical

30
decisions that may be a significant part of the risk/benefit analysis. In fact, the EU is
currently challenging the committee on the sufficiency of the guidance with regard to
risk/benefit analysis (King 2011). In addition, some readers of the standard can
develop the false impression that it only addresses the design control elements of a
quality management system because manufacturing only appears once in the text
although it is in fact mentioned a further twenty times in the annexes of the
document which are provided for explanations.
In summary, the ISO Standard provides a framework to the medical device
manufacturers for effective management of the risks associated with the use of their
products and within which experience, insight, and judgment are applied
systematically to manage these risks. It also helps practitioners to identify processes
by which they can identify hazards associated with a medical device, estimate and
evaluate the risks associated with these hazards, control these risks, and monitor the
effectiveness of that control.
2.6 Lessons Learned from Disasters and Failures of Risk Management
Failure is the Mother of Success.
— a Chinese Proverb

Having a framework to analyze risk is not always sufficient to ensure that no
problems happen. Much can be learned about the limitations and challenges of
current risk management approaches by looking at recent history, when hazards were
not managed adequately and had serious negative outcomes despite the seeming
presence of a robust risk management system. In order to gain insight into other

31
aspects of risk management that might also be important in the success of a risk
management program, a subset of disasters or crises were chosen for review with the
aim to learn why risk management failed and what other elements or attributes would
be necessary to have a successful program.
James Reason (Reason 1997) suggested that organizational accidents have
multiple causes. A single accident or disaster is likely associated with a subset of
problems; however, many elements that can contribute to problems are unlikely to be
evident in a single case study. Thus, by examining a range of scenarios, it is possible
to gain insight into those factors that consistently or commonly contribute to the
failures. The information gleaned from such an analysis can be useful to guide the
formulation of an appropriate framework for the research. Here the researcher has
examined seven situations where failures in risk management were evaluated in
detail with the aim of extracting general lessons. These are summarized in Appendix
A, and certain lessons learned from two such examples are discussed below because
they give important general lessons about the elements beyond the use of a risk
management framework that are important to ensure the success of a risk
management process.
A. Lessons from NASA: The Challenger Crisis
On January 28
th
, 1986, an organization considered to be a leader in reliability
and risk management, the National Aeronautical and Space Agency (NASA), was
presented with a problem of unprecedented magnitude. Approximately one minute
after takeoff, the Space Shuttle Challenger disintegrated over the Atlantic Ocean

32
leading to the deaths of its seven crew members. The Challenger disaster was the
25
th
launch after 24 successful Shuttle missions since the Columbia’s maiden voyage
of the reusable National Space Transportation System (NSTS) Program that began in
1981. In the particular launch under discussion, the schedule of the launch had
changed 13 times from its original date on January 22
nd
for various reasons including
previous Space Transportation System (STS) flight delays, bad weather conditions,
and hardware problems. The launch was carried out on an unusually cold morning
with temperatures close to 31 deg. F (-1 deg. C), the lowest temperature limit
designated for such a launch. The catastrophic failure occurred because an O-ring
seal in its right solid rocket booster (SRB) failed at liftoff because of the extreme low
temperature. The O-ring failure caused a breach in the SRB joint it sealed, allowing
pressurized hot gas from within the solid rocket motor to reach the outside and
impinge upon the adjacent SRB attachment hardware and external fuel tank. This led
to the separation of the right-hand SRB’s aft attachment and the structural failure of
the external tank. Aerodynamic forces promptly broke up the shuttle (Committee on
Shuttle Criticality Review and Hazard Analysis Audit 1988).
In response to the accident, President Reagan appointed William P. Rogers to
lead a Presidential Commission charged with investigating what went wrong (The
Rogers Commission). The Rogers Commission recommended NASA to review the
risk assessment efforts of its Space Transportation System (STS) and to identify
elements that must be improved before flight to ensure mission success and flight
safety. In addition, The Rogers Commission recommended an audit panel be

33
appointed by the National Research Council (NRC) to verify the adequacy of the
effort and report directly to the Administrator of NASA. Thus, The Committee on
Shuttle Criticality Review and Hazard Analysis Audit (The Committee) was formed
to carry out this investigation.
The Committee, with staff support from the Space Applications Board of
Commission on Engineering and Technical Systems of the National Research
Council, published the report “Post-Challenger Evaluation of Space Shuttle Risk
Assessment and Management” (Committee on Shuttle Criticality Review and Hazard
Analysis Audit 1988). The report provided 17 recommendations on improving
project management, technical approaches, decision-making and organizational
structure. Its report made clear that the Challenger disaster was not just a
technological failure, but also a failure of the risk management process. NASA had a
comprehensive risk management framework that included all the necessary elements
of a QRA approach before the accident.
The pre-launch conditions in many aspects were not favorable to Challenger.
The launch was delayed 13 times due to hardware malfunctions, heavy winds, and a
variety of other causes. Particularly, the unusually cold temperatures had caused
some concerns from engineers at Morton Thiokol, the contractor responsible for the
construction and maintenance of the shuttle’s SRBs. However, the engineers failed to
win their argument that the launch should be stopped even though the O-ring seal
that ultimately failed was already classified as “Criticality 1” – the highest risk

34
category that could have postponed the launch, but was vetoed by the flight
management.
Therefore, despite having a comprehensive QRA approach, the Rogers
Commission pointed out that the disaster was the consequence of organizational
behavior attributed to many aspects of NASA’s history, culture, organizational
structure, communications, and other operational weaknesses that an outsider might
not have expected to be associated with this highly respected world-renowned space
agency.
B. Lessons from NASA: The Columbia Disaster
On February 1
st
, 2003, 16 minutes before a scheduled landing, the Space
Shuttle Columbia was destroyed in a disaster that claimed the lives of all seven
members of her crew. Columbia was named after the first American vessel to
circumnavigate the Earth more than 200 years ago. In 1981, Columbia became the
first space shuttle to fly in Earth’s orbit and successfully complete 27 missions over
more than two decades. She traveled more than six million miles in 16 days.
Within two hours of the loss, the Columbia Accident Investigation Board
(CAIB) was formed in accordance with procedures previously established by NASA
following an accident connected to the space shuttle, Challenger, 17 years earlier in
1986. The CAIB was composed of thirteen members from Department of Defense
(DoD), Federal Aviation Administration (FAA), National Aeronautics and Space
Administration (NASA), and various academic institutions. It considered itself an
independent and public institution, accountable to multiple stakeholders including

35
the American public, the White House, Congress, the astronaut corps and their
families, and NASA. The CAIB had a broad scope of responsibilities not only to
investigate the accident but also to examine more holistically NASA’s operation of
the Space Shuttle Program. The CAIB presented a report (United States. Columbia
Accident Investigation Board 2003) whose comprehensive revelations provided a
rich insight into the technical, organizational, and cultural state of the organization
and the recommendations given were highly impactful for future launches.
The report summarized the technical and organizational causes that initiated
the disaster. From a technical point of view, it gave twenty detailed
recommendations, related for example, to the Thermal Protection System, Imaging,
Orbiter Sensor Data, Bolt Catchers, Closeouts, Micrometeoroid and Orbiter Debris,
and Foreign Object Debris. However, these issues are beyond the scope of the thesis
developed here. More pertinently, however, the report also identified at least a
dozen organizational issues related to project management and human behavior that
were rooted in NASA’s history and culture. The CAIB report recommended the
following major organizational changes and additions:
• A robust and independent technical authority over the program, with
complete control over specifications and requirements, and waiver
authority with respect to them;
• An independent safety assurance organization with line authority (e.g. a
program manager) over all levels of safety oversight; and

36
• An organizational culture that reflected the best characteristics of a
learning organization.
2.7 The Research Model
The CAIB report highlighted many of the same deficiencies that were present
in the NRC report two decades earlier. It suggested that NASA had difficulty to
incorporate the lessons learned from the previous Challenger failure, especially in its
organizational and cultural features. Chief amongst these problems were the lack of
objective and independent authorities. Managers could bypass the regular process
and outcomes of risk management activities and unduly reduce the ability of those
with concerns from influencing decisions to proceed down a hazardous path. These
observations raised questions about the importance of human behavior and other
elements in shaping risk management activities. They suggest that any investigation
of risk management activities in a corporate environment must reach beyond the
establishment of a framework and process, and include other elements that shape its
organizational safety behavior, and, in particular, an organization’s ability to
memorialize and learn from previous mishaps and mistakes.
If we are to study the way that medical device companies are currently
equipped to carry out risk management activities, we need to consider carefully
which elements of those activities should be evaluated. Thus, a review of literature
that addresses the breadth of issues important for risk and safety management was
carried out. Two papers in particular appeared to go beyond the anecdotal

37
suggestions of risk management elements of importance, and attempted to generate a
more synthetic framework to guide the systematic analysis of comparable systems.
The first approach was advanced by Guldenmund (Guldenmund 2010) who
tried to enunciate a more holistic and pragmatic framework to describe the safety
culture of an organization. This framework is summarized below in Figure 2.6.

Figure 2.6. The behavior triangle

Guldenmund’s framework describes three important features of an
organization that shape a desired level of safety behavior, and interact dynamically to
influence all risk management activities. These three elements include the structure,
culture, and processes of an organization.

38
Organizational Structure can be defined as “the division of authority,
responsibility, and duties among members of an organization.” (Whittington and
Panny 2004). Structure primarily refers to the formal framework of an organization,
that is, how the work is done and by whom (Hopkins 2006). From the point of view
of management, an efficient structure facilitates both effective coordination and
communication (Mintzberg 1983). This element is captured in the ISO 14971
considerations of management responsibility that were described earlier.
Processes are the patterns of activity taking place throughout an organization,
often divided into three levels: the primary processes, which deal with the main
output(s) of an organization, such as software integration and hardware installation;
the secondary processes, which support the primary ones, for example, management
and quality control; and the tertiary processes, for example, formulations of policies
and strategies designed to drive and support both the primary and secondary
processes (Guldenmund 2010). Into this area presumably would fit the incorporation
of a risk framework and associated formal activities, such as formal meetings and
reviews, as well as policies to enable such activities.
Culture underscores the concept that the organization’s beliefs and attitudes
are manifested in actions, policies, and procedures that will also affect its safety
performance (Ostrom, Wilhelmsen et al. 1993). This is an area that was not well
described by risk management standards but was clearly seen to be important in the
NASA crises, when a culture of taking chances seemed to play importantly into
decisions regarding space launches.

39
The work of Guldenmund provides an interesting research framework to
study risk management as well as safety, as might be expected from the close
relationship between the two. However, is it sufficiently complete to encompass all
of the elements that might affect the success of risk management? A second useful
paper to complement the work of Guldenmund is that of Sullivan and Beach
(Sullivan and Beach 2009), who emphasized the importance of resources to underpin
the ability to carry out effective programs such as risk management programs at a
corporate level. Sullivan and Beach used the term “organizational capability” to
describe the set of resources and competences available to a company, the tools
necessary to accomplish certain tasks and the ability to use those tools effectively.
They pointed out that all organizations require resources such as time, money,
people, technology, and information in order to function effectively. This might be
depicted as a two-element “framework” as shown in grey, in Figure 2.7.
Resources describe all of the investments indispensible in highly regulated,
high-risk, or high-reliability organizations (HROs). The costs of non-compliance,
liability, and accidents far outweigh the costs of proactive and preventive measures
(Roberts and Bea 2001). Many HROs are painfully aware of these significant safety
and compliance investments (Weick 2004) (Hopkins 1999). Adequate resources are
always prerequisites for medical device companies to comply with CGMP
regulations and international standards (FDA 1996) (ISO 2003) (ISO 2000).
Competence refers to the requirements that individuals working in HROs
attain high levels of proficiency (Sullivan and Beach 2009). A medical device

40
company’s ability to accomplish risk management tasks relies heavily on trained
staff. Exceptional competence has often been considered key to averting disaster,
while lack of competence is very common in evidence when disasters occur (Roberts
and Bea 2001). Competence is prerequisite for medical device companies to comply
with CGMP regulations and international standards. The need to demonstrate
appropriate and relevant qualifications, experience, and training are stipulated in the
medical device regulatory requirements (FDA 1996) (ISO 2000) (ISO 2003).
An additional element, however, was added after inspecting the two
frameworks and comparing the various spheres with the findings and
recommendations of the NASA reports described above, and with the reports
collated in Appendix A for five other disasters of similar magnitude. The one
additional element that was perhaps not seen to be covered adequately by the two
frameworks was that of corporate memory and learning.
Memory is a concept that describes the importance of capturing previous
experience in a way that can inform future risk decisions. Two capabilities may be
implied by this requirement, one to manage information that is captured from past
events, and one to ensure that the lessons learned are not archived so deeply that they
cannot contribute to current and future activities. In such a case, learning would not
occur. A significant challenge that impacts memory and learning is reflected by the
grim statistics for employee retention. According to one recent study, almost one
third of employees have held their present positions for less than 6 months and they
are already searching for another new job. With 10- 15% annual attrition, many

41
companies turn over 60% or more of their entire talent base within 4 years (Stein and
Christiansen 2010). Thus, most employees are not in a given company long enough
to develop a significant history of experience with its previous risk management
challenges.
This additional, memory/learning, element of organizational capability was
more than amply in evidence in the NASA reports described above (United States.
Columbia Accident Investigation Board 2003) (Committee on Shuttle Criticality
Review and Hazard Analysis Audit 1988). Thus, the researcher has expanded the
two element framework developed from the work of Sullivan and Beach to add a
third element that creates a “capability triangle” with the third element, short-formed
as “memory”, in white as shown in Figure 2.7.

Figure 2.7. The capability triangle

42
Now, by examining the results of the very thorough reports subsequent to the
NASA crises and five other disasters listed in Appendix A, it is difficult to find a
recommendation or problem not subsumed under the various sectors identified in one
of the two frameworks identified above. The researcher sees the frameworks as
complementary, in which the behavior and capability environments overlay one
another. The researcher proposes to use these two interlocking frameworks to ensure
that the survey developed in the present study will capture a range of elements
considered to be important in shaping a medical device company’s safety behavior
and its capability to manage risk.

Figure 2.8. Research model for medical device risk management implementation

43
In the present research the researcher is interested in the way that companies
prepare themselves for risk management. Using the above model, the researcher will
survey a sample of medical device companies that are obligated by their regulatory
requirements to have a robust risk management system. The researcher will
compose the survey questions based on covering the elements in Figure 2.8 so that
current practices in the medical device industry can be explored and benchmarked.
From this study, the researcher hopes to identify the current state of risk management
activities and areas in which gaps are perceived.

44
CHAPTER 3
METHODOLOGY
3.1 Introduction
This exploratory study had three stages. In the initial stage, a draft survey
instrument was created to examine the implementation of risk management in
medical device companies based on the “Behavior+Capability” model as shown in
Figure 2.8. In the second stage, a focus group of seasoned practitioners in product
risk management was convened to advise on the draft survey instrument. In the last
stage, the finalized survey instrument was used to collect and analyze data from
targeted respondents.
3.2 Stage I: Creation of Survey Instrument
The survey instrument had 40 questions, divided into three parts: (A)
Demographic Information; (B) Elements that shape a medical device company’s
behavior; and (C) Elements that enable a medical device company’s capability.
Questions in the Demographic Information section aimed to assess the background
profile of the respondents, their associated companies, the products that they
produce, and their willingness for further research participation. Each of the two
other elements were queried with approximately the same number of questions.
Further, each element had three sub-categories of questions. For (B) Behavior, the
three sub-categories were (B-1) Culture, (B-2) Structure, and (B-3) Process; for (C)
Capability, the three sub-categories were (C-1) Resources, (C-2) Competence, and
(C-3) Memory. The survey was constructed by questions of the following types:

45
• A set of options from which to choose;
• A rated continuum from strongly disagree (1) to strongly agree (5) or
from extremely unlikely (1) to extremely likely (5); and
• A question requesting a text entry to expand on information relevant to
the choice-based questions and scales.
3.3 Stage II: Confirmation of Survey Instrument By Focus Group
The researcher created a series of survey questions by reference to the
“Behavior+Capability” model developed in Chapter 2. An initial focus group of ten
members was invited to examine and discuss the quality of the survey questions prior
to circulation of the survey. Members of the initial focus group were selected from
the USC DRSc program, standards committees and organizations, regulatory
agencies, and a bank of experienced risk management practitioners known to the
researcher. The researcher assessed the availability and willingness of each potential
member to participate by contacting each individual either by face-to-face meeting,
phone, internet, or e-mail. The participants in the focus group were invited to meet
for a two-hour session at the USC Health Science Campus. The researcher provided
participants with the electronic draft survey in advance and a hard copy of the survey
at the time of the session. The USC Regulatory Program office graciously provided
light refreshments during which the members of the focus group could be introduced.
The researcher introduced and explained the study and then initiated a discussion of
each question sequentially, allowing an average of about five minutes per question.

46
The researcher consolidated the focus group feedback and suggestions; then, he
proceeded to finalize and administer the survey instrument as described below.
3.4 Stage III: Administration of the Survey Instrument – Data Collection &
Analysis
The administration of the survey instrument involved three major activities.
First, the researcher gathered the necessary contact information and profiles of
potential participants by reference to the membership database of a trade association.
An email address list was compiled by including individuals whose profiles indicated
that they worked in quality or risk management in a medical device company. Then,
the researcher verified the distribution logistics provided by the Qualtrics Software in
two separate tests. In the initial test, an invitation letter, thank you note, reminder
note, and the final confirmation letter were created and distributed through the web-
link in the Qualtrics system. In the second test, the researcher distributed a draft
survey to ten e-mail addresses. The returned surveys were used to test the capability
of the software to collate and analyze the mock data. The researcher then sent an
invitation letter to the potential participants. After the receipt of confirmation from
willing participants who met inclusion criteria, the researcher then distributed the
survey link through the Qualtrics Software.
After the initial data was collected and a preliminary evaluation of the
recipients was carried out, additional recipients were identified by specifically
enriching the sample with individuals known to the researcher or colleagues as being
employed in one of the “Top 30” medical device companies. In 2008, 30 medical

47
device companies (hereafter, called Top 30) accounted for 89% of the estimated US
$210 billion in global sales revenue in the worldwide medical device market (WHO
2010). Since there were about 27 000 medical device companies in the world, the
remaining 11% of global sales revenue was shared by a predominance of
manufacturers in the small and medium enterprise (Others) category.
The researcher employed a web-based survey tool, Qualtrics (http://
www.qualtrics.com/), to create both the initial survey instrument for the focus group
and the finalized version for the study of medical device companies. This software
not only provided a platform for designing, distributing, and evaluating survey
results, but also helped to remove bias by randomizing the questions before delivery
to each respondent. Qualtrics gathered and stored survey results electronically. The
researcher collected, managed, and analyzed the survey data where possible using
the Qualtrics software platform. Since the survey instrument was constructed using a
qualitative rather than a quantitative approach, graphical representations and
tabulations in percentages and absolute numbers were typically employed to evaluate
the data.

48
CHAPTER 4
RESULTS
4.1 Focus Group
The major objective of the focus group was to aid the researcher in
confirming and enhancing the survey instrument to produce a tool appropriate for the
purpose of the study. In order to obtain a broad perspective, the focus group involved
both academic and industry practitioners as shown in Table 4.1.

Table 4.1
Focus Group Participants
Participant Profile
Institution Job Title
Large (Top Thirty) Medical Device
Companies
(1) Director of Regulatory Affairs
(1) Manager of Radiation Safety
Medium Size Medical Device Company (1) Director of Regulatory Compliance
Pharmaceutical Product Companies
(2) Managers of Pharmaceutical
Product
Medical Product Consultants
(1) Clinical Affairs Consultant
(1) Quality System Consultant
Aerospace/Electronic Consultant (1) Executive Consultant
USC Professors
(1) Director of Biomedical Engineering
(1) Director of Drug Development
(1) Research Advisor
Total Number of Participants: (11) Eleven

49
The researcher conducted the focus group on a Friday in mid-January from
12:00 to 2:00 pm in three segments. First, the researcher gave an overview of the
research topic as participants ate lunch. Then, the researcher introduced the
Behavior+Capability Model and the structure of the survey instrument in a semi-
formal classroom setting. The focus group then spent majority of the time
commenting on each of the survey questions.
An obvious insight from the focus group was that good and clear questions
would yield better and more useful results. Every question and its answers were
evaluated to ensure that responses would yield clear and unambiguous data. The
focus group emphasized the importance of anonymity and sense of security to
encourage participation and honest answers to questions.
The group identified that a more insightful study might result if management
and practitioner perspectives could be compared. Thus attention was paid to defining
the job level of the participant in the survey. Concern was expressed that small
medical device companies might not be able to provide all of the solicited
information in this survey. A suggested alternative proposed by the focus group was
to have separate studies of the topic in small and large medical device companies, so
that the size of company was also defined in a specific question. Most of the
remaining suggestions were alterations to the choice of words or the number of
alternative answers to questions. The finalized survey is shown in Appendix B.

50
4.2 Survey
A total of 850 invitations to participate in the survey were sent in mid-
February, 2012, to the initial cohort of individuals on the extracted recipient list.
Two hundred and fifty potential recipients did not receive the invitation as indicated
by a bounce-back message. The remaining 600 surveys prompted expressions of
interest from approximately 50 individuals, who were considered to be the first
contact list. The survey was distributed in late-February to the agreeing respondents
and to an additional set of 36 names known to the investigator or to the initial
respondents.
A. Demographic Information
Eighty individuals responded to the survey instrument. Fifty-one of these
respondents were employed by twenty-six of the “Top 30” companies, and provided
perspectives on the implementation of risk management in their own division or
company. Eighteen respondents from the Top 30 were senior management who had
titles of Director or above. Thirty-three respondents from the Top 30 were technical
staff and line managers. Of the remaining twenty-nine respondents from the other
companies not in the Top 30, ten were at the level of director and above whereas
nineteen were technical staff and line managers.
Of the eighty respondents, five completed the survey only in part. Seventy-
four respondents provided their job functions and titles. Most (over 64%) were
professionals in Quality Risk Management, about one-quarter (26%~28%) were from
the R&D and Regulatory functions, and the remaining (8%) respondents were

51
primarily associated with other functions such as Clinical Affairs, Medical Affairs,
and Systems Engineering (Figure 4.1a). No participant was from the manufacturing
function. Some respondents identified affiliations with more than one function. Thus
the number of reports exceeds the number of respondents in Figure 4.1. Two-fifths
of the participants had a title of Director or higher. The remaining three-fifths were
staff practitioners and line managers engaged in risk management activities (Figure
4.1b).
The profile of participants included companies that ranged widely in size and
nature of commercialized products. Sixty percent came from companies or divisions
of more than a thousand employees, and the remainder came from companies or
divisions of less than a thousand employees (Figure 4.2a). These companies designed
and manufactured a spectrum of medical devices that included FDA Class I, II, and
III products (Figure 4.2b) and together had experience in almost all FDA product
categories, except pathology (Figure 4.2c). Most companies designed or
manufactured more than one FDA Class and one FDA Category of products.
Forty-five of the eighty respondents were willing to participate in further
research, thirty-two of them in an interview and forty in a follow-up survey.

52

A: Departments(s) of affiliation (n=74)

B: Job titles (n=74)
Figure 4.1. Affiliation and job level of respondents

53

A: Company/Division Size

B: FDA Product Classifications

C: Product Category
Figure 4.2. Profile of participating companies (n=74)

54
B. Behavior
Seventeen questions were used to assess the elements shaping a medical
device company’s behavior. Six questions inquired about the culture (B-1), five
questions about the staffing structure (B-2), and six questions about the processes (B-
3) used in risk management activities in the respondent’s company.
B.1 Culture
For this subcategory, respondents gave feedback on his/her company’s
willingness to take risk, on changes with respect to risk management in the past two
years, and on the effectiveness of the risk management system and its associated
mechanisms for risk communication.
Respondents had varying views about their organizations’ risk culture (Figure
4.3). More than half (54%) felt that their organizations tend to avoid uncertainty
(unwilling to take risk) at a moderate level and more than a quarter (27%) of the
respondents characterized their organizations as having a high level of uncertainty
avoidance; less than one-fifth of respondents identified that their companies had a
low level of uncertainty avoidance.
Respondents generally identified that the medical device companies with
which they are associated had changed in the way that they handle risk management
in the last two years (Figure 4.4). These respondents characterized the changes as
minor changes (42%), or major changes (55%). Only two (3%) of the seventy-six
respondents identified no change in their risk management systems.

55

Figure 4.3. Level of uncertainty avoidance (n=74)

Figure 4.4. Extent of changes in the last two years (n=76)

56
The survey also provided an opportunity for the respondents to comment on
the nature of those significant changes made to the risk management system for the
last two years. There were forty comments reported by the respondents (Appendix
C). These comments suggested that the changes were motivated by at least one of the
following three considerations: (i) compliance; (ii) specific improvements; and (iii)
general process improvements. More than a third of the comments (14/40) attributed
changes to the desire to meet FDA’s expectations and requirements, to satisfy the
requirements of ISO 14971, IEC 62366, and IEC 60601 standards. About a quarter
of the comments (10/40) suggested that changes were related to the improvement of
specific risk assessment and feedback procedures, such as risk definitions and post-
market information, and other forms of specific improvements, particularly the
implementation of certain new software tools. The remaining comments (16/40)
were related to changes on general process improvements for the risk management
system. Examples are given below and a full listing of comments can be found in
Appendix C.
(i) Compliance (14/40)
This group of comments indicated the changes were mainly motivated to
satisfy regulatory or standard requirements. Selected examples were as follows:
FDA published a guidance and increased the requirements for injectable
auto injector devices. This required “us” to focus more on risk management
specially usability aspect.

Complete overhaul to ensure adherence to latest FDA expectations.

57
Updated to meet ISO 14971 requirements.

Inclusion of IEC62366 Requirements.

(ii) Specific Improvements (10/40)
Respondents in this group pointed out the specific changes that they made in
their risk management systems. Specific examples included the following:
Complete revamp. Centralized risk management system “Software (B)”
implementation. we have Neuromodulation devices, class IIII.

Introduction of new tools; realignment of scales & definitions.

Better connection between pre- and post-market systems.

Add robust means of incorporating complaint information.

(iii) General Process Improvements (16/40)
This group of comments mentioned some general process improvements
within the risk management system. Examples were as follows:
New interfaces to design control relative to process risk assessment. How to
ensure significant specifications of the product within the process risk.

Improved procedures, major revisions to two important risk management
files.

Continuous improvement after years of stagnant.

Continuous improvements continue to expand the reach and depth of risk
management.

Many choices reflected a positive feeling about the value and adequacy of
their risk management systems. Almost all respondents (95%, 72/77) considered that

58
risk management added at least some level of value to their divisions’ deliverables;
over half (57%) agreed strongly with this view (Figure 4.5). Over three-quarters
(76%) of the respondents held the view that the current risk management system is
adequate. Two-thirds of the respondents recognized risk management as a part of
their organizations’ strategic plan. A little less than two-thirds (64%) of the
respondents felt that their organizations had enough tools to perform risk
management activities. However, only about half (51%) of the respondents perceived
they had adequate risk management training.

Figure 4.5. To what extent do you agree or disagree with the following statements
that describe a possible perception of your division’s risk management system
(n=77)

59
On the more negative side, more than one-third (37%) of the respondents felt
that risk management was perceived by their colleagues to be burdensome to their
organizations and a little less than a third (30%) felt that risk management was
perceived by colleagues to be the responsibility of others (see examples in next
section paragraph i).
Respondents were also requested to provide additional opinions. Forty-six
respondents provided their comments on how risk management was perceived within
their organizations. The numbers of comments reflected three different views in
relatively similar proportions. Examples are given below and a full listing of
comments can be found in Appendix D.
(i) A “Necessary Evil” or “Lagging Behind” View (16/46)
This group of comments suggested that the company did not understand
sufficiently how to tie risk management into other elements of the quality system.
Selected comments were as follows:
Project managers saw risk management as a burdensome activity.

R&D is the most prone to viewing risk management activities as burdensome
followed by Marketing.

Overall the perception is that it is a required function but it brings very little
value to the process.

Risk management is often thought of as a file you go to, not an activity.

60
(ii) A “Necessary Good” or “Catching-Up” View (13/46)
This group of respondents appeared to recognize the importance of risk
management to their organizations’ strategic and operational objectives.
There are many risk assessment tools. It is difficult to know which tools to
use in specific situations. Could benefit from additional training at the
associate level.

Post market risk management is not up to the mark.

Risk management tools are established and their use is not standardized.

(iii) An “Ahead-Of-The-Curve” View (17/46)
This group of comments reflected positive attitudes that reflected a desire to
meet the challenges. The respondents realized the importance of proactive activities.
Risk Management is continuing to expand into a discipline for management
of all forms of uncertainty.

Risk Management is fully accepted in my business unit. It is well known in all
sections of our business unit.

Systematic risk management supports a major goal of our products’ quality:
safety. Our brand is connected to the customer expectation of superiorly safe
products.

Risk management is embedded in everything from environmental controls,
design control and validation.

Of the seven suggested ways (Figure 4.6) for an organization to inform their
employees about risk issues, a majority (84%) of the respondents ranked “Meetings”
to communicate risk in the top three; more than half (55%) ranked “Training” in the
top three; and about half (50%) ranked “Product Risk Report” or “Peer Informal

61
Communication” in the top three. The “Web Site” appeared to be at the bottom of
the ranking as a method to inform employees about risk issues.

Figure 4.6. Please rank from most effective to least effective the following methods
that management uses to inform about risk issues within your division/company
(n=76)

62
When questioned about the effectiveness of communicating risk information,
a majority (72%) responded that they felt that their company was moderately
effective and less than a fifth (18%) of the respondents felt risk communication
methods were very effective (Figure 4.7). Only a few respondents characterized
their companies as ineffective in communicating risk.

Figure 4.7. Effectiveness of communicating risk information (n=77)

63
B.2 Staffing Structure
For this subcategory, the researcher queried a respondent on his/her
company’s risk management structure, process owner, his/her additional
responsibilities, and his/her supervisor. Medical device companies typically
organize their risk management functions in one of four different ways: Integrated
(All divisions follow a single structure reporting to the same executive in the
corporation); Divisionalized (Each division has independent structure);
Departmentalized (Different departments organize risk management differently); and
Individualized (Individuals organize their own risk management activities). Of the
seventy-five respondents, only one responded that activities were “Individualized”.
Less than half responded that an “Integrated” and “Divisionalized” approach was
used (43% and 40% respectively). The remaining small group (16%) responded that
a “Departmentalized” structure was used (Figure 4.8).
From the responses, the process owners of Risk Management had titles that
spanned throughout the organizational hierarchy from specialist to CEO/President
(Figure 4.9). One specialist (<1%), three engineers/scientists (4%), eleven managers
(14%), thirteen directors (17%), thirty-three vice presidents (43%), five Chief
Officers (7%), five CEOs/Presidents (7%), and five other titles owned the risk
management process.

64

Figure 4.8. Structural features of organization (n=75)

Figure 4.9. Title of process owner (n=76)

65
Six of the risk management process owners had no other responsibility except
managing risk. The remaining seventy respondents most commonly self-identified
that they had additional responsibilities (Figure 4.10). About three-quarters (51/70,
73%) had responsibility in Quality Assurance, a little less than half (32/70, 46%) had
responsibility in Regulatory Affairs, a little more than a third (26/70, 37%) had
responsibility in R&D, and about a fifth (15,70, 21%) had responsibility in
Manufacturing. About a quarter of the respondents also identified a further diversity
of responsibilities such as reliability engineering (3), clinical (3), medical (3),
regulatory compliance (1), and others (6).

Answer

Response %
Regulatory

32 46%
Quality

51 73%
R&D

26 37%
Manufacturing

15 21%
Others

16 23%

Figure 4.10. Risk management process owner’s other responsibilities (n=70)

The person who supervised the risk management process owner also spanned
the organizational hierarchy. Most (67/76, 88%) reported to someone at the director

66
level or higher, whereas only five (7%) reported to managers (Figure 4.11). In
addition to the traditional positions reflected in Figure 4.11, the process owner may
also report to a team or to the Board of Directors In this survey, one process owner
reported to the Board of Directors, and another reported to the design and
development team.

Figure 4.11. Process owner’s supervisor (n=76)

B.3 Processes
For this subcategory, the researcher queried a respondent on his/her
company’s risk management system procedures (Figure 4.12), techniques used,
implementation challenges, and outsource activities. At least nine areas were

67
identified in chapter 2 as areas where a risk management system should be carried
out formally. Most respondents identified that formal rather than informal procedures
existed for most of these areas, as shown in Figure 4.12; the areas in which risk
management seemed least often conducted was in the area of product disposal, where
the majority of respondents identified that they either did not know of activities or
that the company only had informal procedures.

Figure 4.12. How would you describe the following aspects of your division’s risk
management system? (n=76)

68
As identified in Chapter 2, a number of techniques are often added to the
toolbox of risk managers in addition to the well-accepted tools of fault tree analysis
and failure mode and effect analysis that are typical for risk analysis. Of these
techniques, shown in Figure 4.13, only one of these methods, functional analysis,
appeared to be used quite commonly as a formal tool by the respondent’s division
(34/71, 48%). Less than a third (22/70, 31%; 18/70, 26%; & 22/74, 30%) of the
respondents had formal procedures on HAZOP, HACCP, or PRA respectively. None
had formal procedures on “Markov Analysis” and less than half (30/69, 43%) of the
respondents had heard of this technique. Only one respondent had a formal
procedure on “Delphi Technique” and nearly half (32/69, 46%) of the respondents
had never heard of this technique. About half (36/69, 52%; 35/69, 51%; 32/70, 49%)
of the respondents recognized but did not use Monte Carlo Analysis, Weibull
Analysis, and Bayesian Analysis respectively. A quarter (17/69, 25%) of the
respondents had never heard of these three techniques. There were twenty-eight
other comments on techniques (Q.13). Nineteen (19/28) of these comments
specifically mention FMEA, eleven (11/28) mention FTA, three (3/28) mention
hazard analysis, and a few others mention unidentified methods in the survey, such
as five-whys, brainstorming, root cause analysis, and software assurance case
development.

69

Figure 4.13. Have you observed your division to use the following techniques when
analyzing or assessing product or process risks? (n=74)

Respondents were presented with ten possible challenges (Figure 4.14) when
implementing risk management within their quality management systems. They
identified schedule pressures, resource constraints, and fluctuating priorities as the
top three concerns; a majority (55/76, 72%; 53/75, 71%; and 45/75, 60%) of the
respondents had agreed and strongly agreed on the issues respectively. A rather even
mix of respondents identified that they agreed or disagreed (36/74, 49%; 33/75, 44%;
32/75, 43%; and 32/75, 43%; respectively) that reliance on previous approaches,

70
organizational barriers, lack of integrated management, and stifled professional
differences were issues that affected the effectiveness of the risk management
system. A smaller but still significant number of respondents identified that their top
three concerns included compromises to gain management approval, characterization
of risk management as a part of quality system rather than a system on its own, and
lack of agreed vision.

Figure 4.14. To what extent do you agree or disagree on the following major
challenges when you implement your risk management system within the quality
management system? (n=75)

71
In addition to the previous list of ten challenges, respondents were also
requested to provide additional opinions. Twenty-nine respondents provided their
comments. The numbers of comments could be grouped into three areas: knowledge
and understanding, integration, and organization. Examples are given below and a
full listing of comments can be found in Appendix E.
(i) Knowledge and Understanding Challenges (7/29)
This group of comments referred mainly to the concerns of knowledge and
understanding of some specific areas of a risk management system. The following is
a selected list of comments:
Risk Analysis versus Risk Management understanding.

Overall Residual Risk Evaluation Details with respect to IEC 60601-1.

There is a strong element of having to guess some risks particularly for a new
product.

Acceptance for all products.

(ii) Integration Challenges (10/29)
Respondents in this group indicated the challenges were mainly related to the
integration of sub-systems into a single cohesive holistic system. Examples were
selected as follows:
Integration of premarket and post-market risk management is very poor.
Identification of risk on product design defect during development is not very
well connected.

Lack of integration of risk management into the quality system due to lack of
system integration knowledge.

72
Lack of a detailed division-wide risk acceptance criteria.

Difficult to carry out from beginning to end due to complexity.

(iii) Organizational Challenges (12/29)
This group of respondents commented on organizational issues, such as
resources, attitudes, and behaviors during their risk management implementation.
Some comments were shown below:
Update to risk management improvement is sometimes driven by compliance
observations rather than proactively.

Need to get an engineering person hired who can take over this function,
rather than always hiring it out to a consultant.

Risk management implementation occurred many years ago.

There is a lack of support from many other departments.

Respondents were also queried about the use of outsourcing for various
aspect of risk management. The top three activities that were identified as
outsourced were training (20/72, 28%), verification and validation (12/72, 17%) and
review (14/72, 19%) (Figure 4.15). Most of the outsourced review activities (11/14,
80%) were undertaken to obtain medical expert or legal opinions. Only a small
minority (10/72, 14%) of respondents outsourced no activities.

73

Figure 4.15. Which risk activities/processes does your division primarily outsource?
(n=72)

The two primary reasons identified for outsourcing related to the need for
expert or specialized knowledge (Figure 4.16) provided for outsourcing. The reason
identified as least likely to explain outsourcing was the lower cost of outsourcing.

74

Figure 4.16. Why does your division outsource risk management activities? (n=55)

C. Capability
The researcher composed sixteen questions to assess the elements that
enabled a medical device company’s capabilities. Five questions inquired about
resources (C-1), five questions inquired about competence (C-2), and six questions
inquired about the memory (C-3) of a respondent’s company.

75
C.1 Resources
For this subcategory, the researcher queried a respondent on his/her
company’s risk management budget allocations, time spent on risk management
activities, trends in support for risk management, perceived priority of benefits for
risk management investment, and risk management organizational challenges.
Participants queried about the functional area controlling the budget for risk
management universally identified that the budget was not totally independent. Most
commonly, the risk management budget was reported to reside in Quality
departments (43/73, 59%) (Figure 4.17). It was associated with R&D (29/73, 40%),
or Regulatory Affairs (26/73, 36%) in most of the other companies. A small number
of companies derived the budget from Manufacturing (9/73, 12%). A few
companies (8/73, 11%) were reported to have no budget at all for risk management.
Respondents asked to rank time spent on five risk management activities
most commonly identified Design Control activities as their first or second choice
(53/68, 78%) (Figure 4.18). Half (34/68, 50%) ranked Post-Production data analysis
as the first or second choice and one third (23/68, 34%) ranked process risk control
activities as first or second. One-fifth (14/68, 21%) of the respondents ranked Design
Transfer risk management activities as first or second choice, and only about
Production Data Analysis was listed first or second by the least number of
respondents (12/68, 18%).

76

Figure 4.17. In what functional area does the risk management budget reside?
(n=73)

Figure 4.18. Please rank the following risk management activities according to the
amount of time spent (n=68)

77
When questioned about the impact of the current economic climate on
investment in risk management, half (37/73, 51%) of the respondents reported no
real impact at all (Figure 4.19). About a quarter (17/73, 23%) suggested that budget
constraints threatened to reduce the level of investment in risk management. The
other quarter of respondents saw an opposite pattern of increased or potentially
increasing investment in risk management activities.

Figure 4.19. How would you describe the impact of the current economic climate
for investment in risk management in your division? (n=73)

Respondents queried about benefits of risk management investment typically
focused on the ability to prevent regulatory non-compliance. A majority (61/72,
85%) of respondents ranked this benefit as a first or second choice (Figure 4.20). A
majority of the respondents ranked prevention of market/operational losses and about
a third ranked enhancement to market reputation/perception as their first or second

78
choice. Seldom selected as first or second choice were the benefits of increasing
corporate learning and memory.

Figure 4.20. Please rank the following perceived business benefits that your
divisional management considered to be important when they invest in risk
management system (n=72)

Seven primary challenges to the risk management organization in the next
two years were offered as choices for respondents to rank, as shown in Figure 4.21.
The most commonly selected challenge (32/74, 43%) was the availability of tools
and techniques to meet their needs. Most of the other choices were selected with

79
similar frequency. The least commonly identified (13/74, 18%) challenge for their
risk management organization was cost reduction. In an associated comments field,
the respondents identified a diversity of other challenges, such as the need to keep
the system updated to meet current demands from authorities, to align risk
management activities with changing business structure, to implement new
processes, to maintain and distribute relevant risk management data, and to improve
the ease with which risk management procedures could be used.

Figure 4.21. What are the primary challenges for your risk management
organization over the next two years? (n=74)

C.2 Competence
For this subcategory, the researcher queried a respondent on his/her
company’s minimum qualifications, training, and continued education of risk
management personnel. When asked about the minimum qualifications for those who

80
conduct risk management activities, most (46/73, 63%) reported a Bachelor of
Science as the minimum educational level; only about a quarter (20/73, 27%) of the
respondents reported that Post-Graduate education was required in their organization
(Figure 4.22). About half (35/73, 48%) reported that minimum preparation was
experience in medical devices of less than 3 years, whereas a slightly smaller number
(30/73, 41%) reported a requirement for more than three years of medical device
experience.

Figure 4.22. What are your division’s minimum qualifications for those who
conduct risk management activities? (n=73)

There were at least three different ways that organizations appeared to train
key employees who conducted risk management tasks. More than four-fifths (62/74,
84%) employed internal subject matter expert(s) to train their peers, and a further
three quarters (57/74, 77%; 56/74, 76%) engaged third parties for risk management
training and on-the-job training respectively (Figure 4.23). Only about a tenth of the

81
respondents (8/74, 11%) identified that other ways of risk management training were
used. Identified methods included e-learning, coaching and mentoring programs,
and risk manager educational programs. Third party programs included FDA, PDA,
AAMI, CfPIE webinars, conferences, and other programs.

Figure 4.23. What kind of risk management training has your division offered for
key employees who conduct risk management tasks? (n=74)

Most medical device companies are generally required or expected to follow
certain regulations, directives, standards, and regulatory guidance. Most (69/74,
93%; 68/74, 92%; 65/74, 88%) of the respondents reported that individuals
responsible for risk management activities had obtained training on the three key

82
standards and regulations important for risk management in the US, including ISO
14971 Application of Risk Management to Medical Devices, ISO 13485 Quality
Management Systems for Medical Devices and 21 CFR 820 Quality System
Regulation, respectively (Figure 4.24).

Figure 4.24. What kind of regulations and standards training have those responsible
for your division’s risk management activities obtained? Check all that apply (n=74)

More than three-quarters (58/74, 78%) of the respondents reported that
individuals responsible for risk management activities had obtained training on both
General Requirements of GMP and European Medical Device Directives, and more
than half (46/74, 62%; 39/74, 53%) of the respondents reported that those
responsible for risk management activities had obtained training on Human Factors

83
and IEC 60601 Medical Equipment Safety respectively. A significant minority of
the respondents (35/74, 47%; 31/74, 42%; 30/74, 41%) reported that those
responsible for risk management activities had obtained training on ISO 62366
Usability, IEC Software Life-Cycle, and ISO Sterilization Standards respectively.
About one-seventh (10/74, 14%) of the respondents reported that those responsible
for risk management activities had obtained training on other requirements or
standards. Standards mentioned as subjects of these less common training activities
included the Canadian GMP, Pharmaceutical Risk Management, and a few other
international standards.
In a comprehensive list of twenty available risk management techniques
useful to medical device companies, almost all respondents (67/72, 93%) reported
that Failure Mode and Effect and/or Criticality Analysis (FMEA or FMECA)
training had been offered to those responsible for risk management activities, and
over two-thirds (50/72, 69%) reported that Fault Tree Analysis (FTA) training had
been offered (Figure 4.25). About two-thirds (45/72, 63%) of the respondents
reported that Statistical Process Control (SPC) training had been offered. A list of
additional training opportunities selected by about half of the respondents (40/72,
56%; 40/72, 56%; 40/72, 56%; 38/72, 53%; 37/72, 51%) included training in Cause
and Effect Analysis, Preliminary Hazard Analysis (PHA), Process Capability
Analysis, Design of Experiment (DOE), and Analysis of Variance. Less than half
(32/72, 44%; 24/72, 33%; 22/72, 31%) of the respondents reported that Functional
Analysis, Event Tree Analysis or Hazard Analysis and Operability Study (HAZOP)

84
training had been offered to those responsible for risk management activities. Least
commonly identified were training in Hazard Analysis and Critical Control Point
(HACCP) or Probabilistic Risk Assessment (PRA) (19/72, 26%; 18/72, 25%),
Weibull Analysis (12/72, 17%), Monte Carlo Analysis (7/72, 10%), Bayesian
Analysis, Bow Tie Analysis, Layer of Protection Analysis (LOPA) and Delphi
Techniques (6/72, 8%; 3/72, 4%; 2/72, 3%; 2/72, 3%). One respondent reported that
5-Why’s training had been offered to those responsible for risk management
activities, and no respondent reported that Markov Analysis training had been
offered.

Figure 4.25. What kind of training on risk management techniques do you know to
be offered to those responsible for your division’s risk management activities? Check
all that apply (n=72)

85
When questioned about continuing education and training of the risk
management workforce, the most commonly identified option was in-house training
(61 respondents) (Figure 4.26). A much smaller number of respondents identified
external training (42), certificate courses (28), or graduate degrees (22). A majority
of the respondents reported that their organizations had planned to have in-house
training (53, 87%) or external training (30, 71%) for the risk management workforce
in the next twelve months. In contrast, less than a third (9, 31%) of the respondents
reported that their organizations had planned on certificate courses for the risk
management workforce in the next twelve months, and less than a tenth (2, 9%)
reported planning for graduate degrees.

Figure 4.26. How/When does your division plan to continue educate and train the
risk management workforce? (Number of Respondents (n) = 61, 42, 28, & 22
respectively from top to bottom.)

0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%

Graduate
Degrees
(2/22)

Cer;ficate
Course
(9/28)

External
Training
(30/42)

In-‐House
Training
(53/61)

86
C.3 Memory
Respondents were queried about whether certain types of risk incidents had
occurred in their companies over the last twelve months, whether such events might
be predicted to occur in future and how such events were handled (Figure 4.27).
When questioned about risk management incidents within the last twelve months,
more than three-quarters (52/68, 76%) of the respondents reported that a Reportable
Medical Device Reporting (MDR) incident had occurred. Such incidents are defined
by significant failures or adverse events associated with a marketed product that can
have a serious negative effect on patient safety (DHHS 1996). A majority (41/68,
60%) of the respondents also reported a recall and about half (35/68, 51%; 35/68,
51%) of the respondents reported field corrections or the need to stop shipments in
the last 12-month period. It was less common (16/68, 24%; 15/68, 22%) for
respondents to report project cancellations because of unacceptable product risk and
agency/patient safety concerns prompting label changes.

Figure 4.27. Which of the following risk management incidents have happened in
your division in the last twelve months? Check all that apply (n=68)

87
When questioned about the type of incident that might be anticipated in the
next twelve months (Figure 4.28), the most likely event, anticipated by about two-
thirds (46/71, 65%) of the respondents, was a reportable medical device report
(MDR). About one-third of respondents anticipated a field correction, recall or stop-
shipment order (Figure 4.28). Much less frequently (10/69, 14%) did respondents
expect a safety concern that would prompt label changes. Only a few (9/66, 14%)
respondents thought that projects might be cancelled due to unacceptable product
risk.

Figure 4.28. How likely do you think that the following type of incident might
happen in your division over the next twelve-month period? (n=70)

88
Not all participants responded to the question regarding their organizations’
corrective action systems (Figure 4.29). The number of responses ranged from sixty-
four to seventy (64, 65, 66, 67, & 70). Three-fifths (43/70, 61%) of the respondents
reported that their organizations did not have a library of lessons learned. Close to
two-thirds (41/64, 64%) of the respondents reported that their libraries of lessons
learned were not easy to access. Only a little more than half (35/67, 52%) of the
respondents reported that their people use the lessons learned. About half (35/66,
53%; 34/65, 52%) reported that their organizations’ libraries had a repository of root
cause analyses, and these root cause analyses were helpful.

Figure 4.29. Regarding your division’s Corrective Action System. (Number of
Respondents (n) = 70, 67, 64, 65, 66 respectively from top to bottom.)

89
4.3 Comparisons
A. “Top 30” Medical Device Companies versus “Others”
Differences between response patterns for Top 30 companies and for Others
were compared by cross-tabulating responses of the two sets of respondents (see
Appendix F). Respondents in the Top 30 and Others were generally similar in their
views, with a few exceptions examined below and summarized in Appendix F-1.
Differences in the response patterns of more than 10 % were highlighted for
discussion because they may suggest areas for further exploration in future, but such
differences may not be especially meaningful given the relatively small sample sizes.
In most aspects the larger, Top 30 companies appeared to be more actively engaged
in risk management. With regard to Behavioral elements, more Top 30 companies
characterized changes in the risk management system as significant rather than minor
over the last two years whereas companies in the “Others” group had more minor
changes than significant changes. Most respondents from Top 30 companies did not
characterize risk management activities as burdensome and most felt that the risk
management training available to them was sufficient. More respondents from the
“Others” companies felt that risk management activities were burdensome and more
than half felt that insufficient training was offered. Top 30 companies appeared to
have formalized risk management systems in place for a broader range of activities,
such as the management of product disposal risks. As far as investments in risk
management were concerned, respondents from Top 30 companies appeared to enjoy
better commitment and investment from management. Fewer of the respondents

90
from the Top 30s had no budget for risk management and fewer reported to
manufacturing than respondents in the Others. Respondents from Top 30s appeared
generally more interested in lessons learned and found their root cause analyses more
helpful than the Others.
However, not all feedback from Top 30 companies was more positive than
that from Others. For example, when implementing a risk management system, more
respondents from Top 30 companies appeared to be challenged by a lack of agreed-
upon vision and characterization of the risk management system than was typical for
the Others group. More respondents from the Top 30s had challenges in
collaborating with other business entities and in cost reduction than the Others.
B. Risk Management Staff versus The Executive Group
The viewpoints of “Risk Management Staff” who were on the front line in
the delivery of risk management activities did not differ much from those of the
“Executive Group” who managed the overall resources. However, there were a few
areas in which these two groups appeared to have slightly opposing views. In the
cultural domain, a third of the Risk Management Staff ranked Top Management
Broadcast as one of the top 3 methods judged to be most effective for
communicating risk information, while more than a third of the Executive Group
ranked it as one of the bottom 3 methods (Appendix F-2, Q.4). A majority of the
Risk Management Staff felt their organization had a moderate rather than high level
of uncertainty avoidance, while a higher proportion from the Executive Group
tended to view the uncertainty avoidance as high. From a process implementation

91
perspective, relatively more Risk Management Staff saw challenges related to the
lack of agreed-upon vision, organizational barriers, and stifled professional
difference of opinions than the Executive group (Q.14). Risk Management Staff
ranked more time spent in manufacturing and less time spent in post-production data
analysis; while the Executive Group had the opposite pattern (Q.18). Relatively more
of the Risk Management Staff than the Executive Group anticipated that they would
face challenges in aligning with overall corporate business strategy, collaborating
with other business entities and cost reduction in the next two years; fewer
respondents from Risk Management Staff than the Executive Group saw the
availability of tools and techniques as a major challenge.
Risk Management Staff expected recalls and field corrective actions to be
more unlikely than likely, but “stop shipments” to be more likely than unlikely. The
Executive Group had the opposite expectations in the same areas. More respondents
from Risk Management Staff than the Executive Group responded negatively on
using lessons learned, on having a repository of root cause analyses and on the
perceived helpfulness of those lessons learned repositories.
C. Smaller Size (< 1,000 employees) versus Larger Size (> 5,000 employees)
Companies
Cross-tabulation of responses from smaller versus larger companies
illuminated only modest differences (Appendix F-3). Respondents from smaller
companies more often reported that their risk management system underwent minor
changes rather than significant changes over the last two years, whereas respondents

92
from larger companies characterized changes in risk management systems more
commonly as significant rather than minor. Respondents from smaller companies
were quite evenly split on their perceptions that risk management activities were or
were not burdensome but larger companies typically characterized the activities as
not burdensome (16/31 not burdensome vs 10/31 burdensome). More respondents
from smaller companies claimed a low level of uncertainty avoidance rather than a
high level of uncertainty avoidance compared to respondents from larger companies
(8/30 low-level for smaller companies vs 13/31 high-level for larger companies).
Respondents from smaller companies appeared to have approaches that were more
informal than those of larger companies in managing product disposal risks.
Respondents from smaller companies were equally split in their views on
compromise to gain management approval (Appendix F-3, Q.14).
In terms of capabilities, respondents from larger companies identified that
their companies were making more investments in risk management activities than
those from smaller companies. In the next 2 years, respondents from smaller
companies anticipated more challenges in gaining needed tools and techniques,
whereas respondents from larger companies more commonly focused on challenges
anticipated as a result of talent acquisitions or collaborations with other business
entities. Smaller companies appeared to rely more on third-party training, whereas
larger companies appeared to rely more on internal expertise for training key
employees. Respondents from smaller companies also expected recalls, fields
corrections, and “stop shipments” in future to be more unlikely than likely whereas

93
large companies expected recalls, fields corrections, and “stop shipments” more
likely than unlikely. More respondents from smaller companies appeared interested
in using lessons learned (17/28 used lessons learned vs 11/28 not used) and about
half of them had a repository of root cause analyses; respondents from larger
companies were split on the use of lessons learned and more had repository of root
cause analyses (16/29 had root cause repositories vs 13/29 had no root cause
repository).
D. Uncertainty Avoidance (UA)
Respondents varied in their assessments of their companies’ level of risk
avoidance (Appendix F-4). Most respondents (16/20, 80%) who self-identified as
being from companies with a high level of uncertainly avoidance (H-UA) were from
companies of sizes more than a thousand employees whereas a majority (8/12, 67%)
of those in the low-level uncertainty avoidance group (L-UA) were from companies
with fewer than a thousand employees. In regard to Behavior questions, half of the
respondents in the L-UA group reported that their colleagues perceived risk
management activities to be burdensome whereas more than half (13/20, 65%) of
those in the H-UA group had an opposite perception. Only half of those in the L-UA
group used risk management as a part of their strategic plan, whereas most (16/20,
80%) of those in the H-UA group had risk management as part of this plan. When
implementing risk management within a company’s Quality Management System,
resource constraints, fluctuating priorities, and compromises to gain management
approval were the major challenges of most (10~13/14, 70%~93%) in the L-UA

94
group. In terms of managing risk management activities, many (6/14, 43% and 8/14,
62%) of those in L-UA companies identified that design transfer and product
disposal risks were managed informally without any procedures whereas a majority
of the H-UA managed design transfer and product disposal risks formally with
procedures (16/20, 80% and 12/20, 60% respectively); while. Fewer L-UA
respondents had formal procedures in 8 of 9 techniques than H-UA respondents.
In the areas of Capability, respondents from the L-UA group (0/12) identified
that there had been no discussions or evidence of resource increases and no
enunciated plan for an increase in resources for risk management in the next six
months. In contrast, a small number in the H-UA group reported that investment in
risk management had increased (5/20), was currently in discussion (1/12) or would
be increased in the next six months (1/20). Further those in the L-UA group
identified that general budget constraints and cost cutting programs might reduce the
level of investment (5/12, 42%). In the area of Competence, (7/12, 58%) of the L-
UA had occasionally sought objective expert opinions for outsourcing whereas a
majority (9/14, 64%) of the H-UA had never sought any objective expert opinions
for outsourcing. In the area of Memory, both H-UA and L-UA had similar responses
on having a library of lessons learned. The H-UA had more people using the lessons
learned (12/18, 67% vs 3/12, 25%); more people indicated ease of access to the
library (9/17, 53% vs 2/11, 18%); and more people agreed that their root cause
analyses were helpful (11/18, 61% vs 2/11, 18%) than the L-UA. Details of these
cross-tabulations are presented in Appendices.

95
CHAPTER 5
DISCUSSION
5.1 Summary
This research project has studied how medical device companies conduct risk
management activities. Such activities are mandated by regulatory agencies in
different countries under specific, often unique sets of regulations and standards. In
the U.S., these requirements are described in 21CFR 820, Quality Systems
Regulations, which identifies a need for risk management activities as part of design
controls. In contrast, in Europe and Japan, quality systems are assessed by reference
to a different standard, ISO 13485, Quality Management Systems for Medical
Devices. Although both 21CFR 820 and ISO 13485 draw extensively on the same
risk management standard, ISO 14971, it is possible that the way that risk
management is implemented by companies in different countries could vary based on
the different regulatory and cultural environments surrounding these activities.
Further, many countries have medical device regulatory systems that are poorly
developed, and we might anticipate that risk management activities might differ, or
even be lacking, in the smaller companies serving these markets where requirements
for risk management may not be mandated by law. In order to delimit the study to a
set of companies that are relatively homogeneous in regulatory requirements, the
decision was made to examine risk management activities in medical device
companies with direct product sales in the U.S. market, that are all subject to the
U.S. FDA’s medical device rules and regulations. This analysis therefore may not

96
represent the thinking or management activities in other countries, but may serve as a
benchmark against which the medical device companies in other countries may be
compared. To the knowledge of the researcher, no studies of medical device
regulation of the type presented here have been conducted in any constituency
previously, so that the opportunity to compare the U.S. results to those in other
countries will depend on future research, preferably using the same or similar
research tool.
A challenge presented by this type of study was to acquire information that is
normally held by a specialized subset of individuals in a company. Gaining access to
such individuals, and then obtaining their cooperation, was perhaps the most difficult
aspect of the research, both because identifying such individuals can be difficult, and
because there is no assurance that they will want to participate once contacted. To
ensure that the study sampled from companies representing a substantial proportion
of the marketplace, the researcher targeted the Top 30 medical device companies,
measured by market size as in sales. The fact that twenty-six of the Top 30
companies responded, together with twenty-seven medical device companies
additional to those in the Top 30, ensured that the surveyed companies not only
covered at least 90% of the market but produced products that covered nineteen of
the twenty FDA product categories and all three FDA product classifications.
However, despite the wide coverage that this sampling provided, care must be taken
to recognize that absolute numbers are relatively small, and small differences in
subpopulations could be misleading. In most cases, a difference in opinion of 15%

97
typically was the result of differences in the responses of three or four individuals.
Therefore, both absolute numbers as well as percentages were reported in the
appendices in order that the reader could make an independent assessment of the
weight that differences might represent.
The two methods used in this work, first to research the individuals holding
risk management roles by inspecting the membership list of the trade association and
second by identifying specifically individuals who held these roles in targeted Top
30 companies, yielded two samples with different characteristics and different
response rates. The response rate of 59/600 (10%) observed when a large group of
companies were targeted only by email request, contrasted with the much higher rate
of 21/30 (70%) that was gained by contacting the representatives of the Top 30
companies more personally and in some cases multiple times through e-mails and
follow-through phone calls. In addition, the researcher has been actively involved in
the risk management standard development for more than ten years and most of his
acquaintances are in the Top 30 companies. Many of the Top 30 companies tend to
have representatives in the standard activities. The former rate is lower than typical
of that seen when surveys are distributed electronically in other research situations.
For example, online surveys analyzed by SuperSurvey (Hamilton 2009) yielded the
following response rates: (i) average survey response rate – 32.52%; (ii) median
survey response rate – 26.45%; and (iii) total response rate – 13.34%. On the other
hand, a much lower response rate of 1% was identified by Hoffman in an email
survey to a group of individuals in the pharmaceutical sciences (Hoffman 2010).

98
The lower response rate might be attributed to the sensitivity of industry employees
to solicitation from unknown sources, especially when the survey might threaten to
ask questions about proprietary information. Such a concern might be particularly
anticipated when respondents were surveyed about their management of risks. The
researcher received several (5~6) queries about the source of the survey (unpublished
observations), possibly reflecting concerns of many of the invitees. Another possible
reason for the low response rate was because of the security filters employed by
companies. Many companies’ server security systems might have prevented
invitations from reaching the target respondents. About a third of the Top 30 targeted
respondents had not received the e-mails with the web-link to the Qualtrics software
because the e-mails were either being transferred to a “junk” folder or eliminated by
the server security systems. The researcher had to resend the web-link to different,
usually personal, e-mail addresses. Even the researcher’s own server had classified
the invitation as junk mail when it was sent for validation purposes to the researcher.
That 26 of the Top 30 medical device companies contributed to the results
was felt to increase the validity of the sample from the largest companies that
presumably have resources and manpower sufficient to institute “best practices” in
risk management. These companies also control 89% of the $210 billion medical
device worldwide sales revenues estimated in 2008 (WHO 2010). The data collected
from these respondents were supplemented by respondents from 27 of the nearly
27,000 medical device companies in the global market that were designated as
“Others”. Although the number of respondents was relatively small compared, for

99
example to a survey of political views or a survey of preferences with regard to a
common consumer product such as toothpaste, it must be recognized that the
numbers of risk managers in the U.S. is not large. Typically a company may employ
only one or a couple of such targeted individuals, and they can be hard to find if they
do not belong to a trade organization. Nevertheless, representation of the industry
quite broadly was to some extent confirmed by the fact that together, respondents
from these fifty-plus companies manufactured products in all three FDA Medical
Devices Classifications and nineteen of the twenty FDA Product Categories.
It was of some interest to compare the views of the Top 30 and the rest of the
companies. The fact that most answers of respondents from the Top 30 companies
were not greatly different from those of other large companies, and were also similar
in many respects to those of smaller companies, may relate to the need for all device
companies to comply with the same regulatory requirements. Further, the often
similar responses may provide evidence that the sample of companies that was
polled electronically had reasonable external validity. The broad general similarities
between responses of the Top 30 companies versus the Others, and between larger
and smaller companies further suggests that the general lessons suggested by this
research may have reasonable validity to represent to U.S. medical device industry in
general. However, despite the wide coverage that this sampling provided, care must
be taken in interpreting the results to recognize that absolute numbers are relatively
small.

100
5.2 The Dual Triad – “Behavior+Capability” Research Model
Without any prior comprehensive research data available on how medical
device companies perform risk management activities, the research goal of the
present study was modest: to collect such information more systematically using a
comprehensive survey tool. The study could not rely on a strong history of research
methodology in this field. Thus, it was challenging to structure the research in a way
that would assure the capture of most elements important to risk management, and
not focus too heavily on some aspects to the detriment of others. The approach that
was selected was based on a dual triad model, the “Behavior+Capability” model as
explained in chapter 2. In the opinion of the researcher, the framework was very
useful to facilitate a fairly comprehensive view of risk management activities and
attitudes in a snapshot of time corresponding to 2011.
A. The “Behavior” Triad
The “Behavior” Triad is the “shaping” arm of the Behavior+Capability
Model. Its three behavioral subcategories – culture, structure, and processes –
together appeared to be useful in probing attitudes and organizational reporting
structures for carrying out risk activities. More specifically, they provided insights
into the behavioral norms, chains of command and communication, and procedural
practices that link risk management activities with the overall mission and business
objectives of the medical device company. The risk culture of a company is
reflected in its attitude toward risk tolerance and risk management. Risk
management has been a required element for medical product design and

101
commercialization for nearly a decade, and thus it may not be surprising that most
companies appreciate the need to carry out risk management activities and seem
committed to such activities. However, what also seems clear is that risk attitudes do
vary in companies and this can affect the approach to risk management.
It is difficult to say whether variability in risk attitudes is entirely due to
differences in company culture or whether it reflects the particular perspective of the
participant, who may not have a full appreciation for company views as a whole.
However, if the assumption is made tentatively that the respondent in this study is in
fact reporting accurately on the state of his or her company, then it is interesting to
explore whether the variations might be correlated with other features of the
company. For example, we could speculate that risk tolerance might be affected by
the nature of the products that are sold by the company. Medical devices vary
greatly in risk. This variability is recognized by regulatory agencies that have
developed a risk-based classification scheme to ensure that risky devices, such as
long term implantable or life-sustaining devices (designated as Class III devices in
the U.S. and the EU), are subjected to greater scrutiny than devices with less risk,
such as hospital supplies or simple instruments (designated as Class I). Because
respondents in this survey were able to provide information on the classes of devices
made by their companies, it was hoped that a comparison could be made between
responses of those making Class I devices and those making Class III devices.
However, companies making only one or the other category of devices were small in
number; many companies had a broader range of products spanning different device

102
classes. Thus cross-comparisons could not be usefully made and the question will
have to be explored in future with a more targeted set of respondents.
Alternatively, we might postulate that larger companies have less risk
tolerance for several reasons. It appeared to be the case that respondents from larger
companies more frequently classified their companies as more formal, resourceful,
and self-reliant than smaller companies. Large companies are often publicly traded
companies for whom the consequences of problems with widely distributed products
are particularly serious. Such companies often have much greater vulnerability to
legal liability because of their “deep pockets” and are guarded by larger internal
quality assurance teams with more numerous methods of quality oversight. However,
we must be cautious to recognize that the size of the company and the nature of its
products may be related.
In this study, it was interesting, though perhaps not surprising, to find that
medical device companies characterized as having a higher level of uncertainty
avoidance reported a more common use of formal procedures and policies. Risk
attitudes of the respondents appeared to be consistent with one of the findings in The
GLOBE Study (House and Global Leadership and Organizational Behavior
Effectiveness Research Program. 2004) on uncertainty avoidance, which showed that
“the more uncertainty avoiding a society is in its national character, the more its
people prefer champions to enact organizational rules, norms, and policy when
promoting innovation”. Substituting the word for “company” instead of “society”
would make this sentence appropriate for the work described here.

103
Results also suggested that risk averse companies tended not to outsource risk
management activities. The use of formal procedures and in-house management of
risk management would tend to increase the level of control over product
development and production, leaving less room for potential problems (Donovan and
McDermott 2010). Most respondents appeared to be satisfied with the risk
approaches and culture in the company, but one area of potential weakness that of
risk communication, stood out. Responses suggested that many companies did not
have strong and effective systems for risk communication; most respondents (55/77,
71%) rated the effectiveness of risk communication as moderate. Risk
communication is an area that has been identified previously as one that is both
important and difficult to manage (National Research Council (U.S.). Committee on
Risk Perception and Communication. 1989). It is important because failing to
communicate risk (National Commission on Terrorist Attacks upon the United
States. 2004; Stulz 2008; Stulz 2009) within a medical device company might
handicap the ability of individuals working with a product to recognize and manage
knowable and concealed risks in an effective way. Such failures could result in
product problems with subsequent expensive recalls or patient injury.
Communicating risk may be seen as a risky process to be avoided by some because
of liabilities involved in documenting or making visible problems that could be
discoverable in court. However, the liability associated with failures to communicate
problems could also be sufficiently serious to affect the survival of the company
(Stulz 2008; Stulz 2009). Perhaps a useful area for further study would be to

104
characterize successful models that have been developed within particular companies
whose employees feel that this activity is done well, in order to help risk managers in
their efforts to design better communication systems. The early guidance from the
Committee on Risk Perception and Communication on risk communication for
medical device companies is helpful in this regard because it provides insights on
common misconceptions and challenges, and it also provides guidelines on the
process of risk communication. However, experts trained and experienced in risk
analysis or risk assessment may not be always successful in communicating risk in
an effective manner. Risk communication deserves more serious attention within a
medical device company, and this research suggests that in many companies, process
owners of risk management may need better communication and legal training.
The way in which risk management is handled and prioritized in a company
is affected by its organizational structure. In the companies under examination here,
it was unusual for responsibilities to manage risk to be vested in an independent job
function, as reflected by the fact that there were only three Chief Risk Officers and
only six dedicated process owners (6/76) amongst the respondents in the fifty-plus
companies. Instead, most of the risk management process owners (70/76) had other
responsibilities. A more specific effort to designate specialist functions in risk
management might be appropriate for some medical device companies, particularly
large companies with many products. It was notable that one (but only one) of the
“Top 30” companies in this study had an entire group dedicated to risk management.
Without the opportunity to focus exclusively on the risk management function, one

105
might be concerned that individuals with multiple responsibilities might have less
time to educate themselves and to evaluate risk in a proactive manner because of
conflicting time commitments for other tasks.
It is interesting to consider how behavior related to risk management might
be modified by looking to other types of companies for ideas regarding “best
practices”. If medical device companies were to benchmark against the financial
sector, for example, where highly visible failures have caused considerable
rethinking of the behavioral aspects of culture relating to risk, they might want to
implement a few key principles recommended for this industry (Blankfein 2009).
These include: a) ensuring that risk and control functions are independent from the
business unit; b) placing risk managers in positions of at least equal stature (status,
hierarchical position) with Operational Managers; and c) ensuring that the judgments
of risk managers prevail when there is disagreement on risk acceptability.
A lesson learned from the experiences of the NASA (United States.
Columbia Accident Investigation Board 2003) and the financial sector (Blankfein
2009) was that both sectors outsourced their risk management and did not take
sufficient ownership of their own risk analyses. Both NASA and selected financial
organizations admitted that the practice of outsourcing risk management activities
might have contributed to their failures. The present study indicates that larger
medical device companies tended not to outsource risk management and also used
internal experts for training, but smaller companies often relied on outsourcing or
consultants. The practices in small companies may be related to the fact that few

106
individuals with suitable skills were employed in the small company, owing to its
limited resources for staffing. In addition a small company might have more modest
needs for risk management activities if it only produces a few low-risk products.
The outsourcing could also reflect the fact that few well-trained risk management
practitioners are to be found in the general talent pool. Typically risk management is
not taught in engineering programs. It might be suggested, given the importance of
risk management across many disciplines, that training in risk science should be
introduced as a standard component of some curricula at academic institutions. One
might even argue that risk science is at the heart of regulatory science and could be
used as a basis for understanding why and how regulations and practices have
evolved.
It appeared that many of the lessons learned from previous disasters and
failures of risk management might still be of value to medical device companies. If
medical device companies were to benchmark against these previous “lessons
learned”, some areas in the behavior side of the model would need to be enhanced.
These would include the need to: a) improve risk communication from a moderate to
very effective level; b) assign a dedicated risk management process owner; c)
institute formal procedures in all aspects of a total lifecycle risk management
process, and more specifically, in design transfer, outsourcing, supplier control, and
product disposal.

107
B. The “Capability” Triad
The “Capability” Triad is the “enabling” arm of the Behavior+Capability
model. The three arms of this triad – resources, competence, and memory – seemed
to be useful as a framework to determine the means, aptitude, and learning
capabilities of the medical device company, and provided insights on how a medical
device company’s financial investments, cultivation of human talent, and
mechanisms to retain learning might empower and improve risk management
practices over time. Answers to questions related to the “Capability” arm suggest
that organizations typically place their risk management activities in the hands of
individuals with good general levels of education but with relatively narrow or
company-based training in risk management methods. Most companies appeared to
have little training in more advanced methods that go beyond standard FMEA and
FTA techniques. In this study it was assumed that individuals in the risk
management function would be familiar with and be applying a range of methods
because they are standard to the field and are called out in virtually all standards and
guidance documents related to risk management (IEC/ISO 2009). Thus, it was
surprising to find that many of the respondents did not recognize or use many of the
advanced methods that are proposed as useful adjuncts particularly when trying to
quantify risks or evaluate trends (Kaplan and Garrick 1981).
“How many and what tools are needed for risk management?” This is a
practical question to challenge the process owner of the risk management system. A
blacksmith, goldsmith or watch-smith has a unique toolbox, but what are the central

108
tools of a risk-smith? Has risk become a “trade” in the medical device industry yet?
From the results of this survey, a relatively narrow set of tools was employed to
study risk. The respondents in this study likely represent some of the most advanced
practitioners in U.S. companies. Yet relatively large numbers of respondents either
did not recognize or did not employ techniques such as HACCP, Weibull, Markov
and Delphi techniques, that are discussed in the risk standard, IEC/ISO 31010. Why
this may be the case is an interesting question. It may mean that the current
requirements to demonstrate compliance with regulations are relatively simple and
easily met so that a more advanced expertise is not seen by companies as a priority.
Both the FDA Quality System Regulation and the ISO 14971 Standard stipulate that
medical device companies must allocate sufficient resources and assign qualified and
trained personnel to perform risk management activities. However, it is difficult for
any external assessor or auditor to identify deficiencies in this area, because the
auditors themselves are not experienced in advanced risk management methods and
typically do not make a judgment about whether the particular approaches employed
for one product or another are suitable for that product and have been carried out in
an effective way. It would require considerable judgment on the part of an auditor to
assess whether a pro forma FMEA analysis is sufficient for effective risk analysis of
a particular medical device. FDA issues a warning letter when a company fails to
demonstrate adequate risk analysis as required by 21 CFR 820.30(g). However, the
criticism usually identifies the lack of a system approach that links to other risk

109
management activities (Cassens 2012) or a serious deficiency in the risk analysis
approach (Silverman 2012).
Alternatively, the results may suggest that for most companies, some of the
more specialized tools, such as Bow-Tie Analysis Layer of Protection Analysis, or
the Delphi Technique are not considered sufficiently helpful in improving insights
into risk problems to justify their adoption. Some of these methods are challenging
to employ because they require advanced engineering or statistical skills and access
to well-constructed datasets. It may also reflect the lack of formal training
opportunities for practitioners, if training is limited to in-house or condensed short-
course methods, as suggested by the responses accumulated here. Further, future
formal academic risk management education did not appear to be generally
supported by respondents’ organizations. There may be an opportunity for academic
institutions to formalize such education in undergraduate engineering studies in order
to prepare future risk management practitioners for medical device companies. An
important avenue for future research may be to explore further the tools for risk
management and to understand why different tools have such incomplete penetration
into risk management operations.
Previous investigations of the disasters and failures of risk management had
identified the importance of the memory/learning dimension as an indispensible
element of competence in the enabling-arm of the risk-management model. One way
to prevent risk management failures is not to repeat the same mistakes by
remembering how to avoid them. A minority of companies appeared in this study to

110
have developed a library of lessons learned. However, not every medical device
company surveyed had such provisions. There may be an opportunity for medical
device companies to institute formally a corporate memory or learning repository
system for risk management experiences. However, before the development of such
a repository is accepted as a “best practice”, it may be necessary to further explore
why companies have to date been slow to implement such a library. Does this failure
reflect a lack of sophistication or knowledge about useful practices for risk
management, or does it imply that something in the way that such libraries are
developed and used makes them unwieldy or unhelpful for most organizations?
Some of the responses to questions in the “Capability” arm pointed to areas
in which further improvements could be made in many companies. These might
include the following suggestions that come from comparing current activities to best
practices in other types of companies or to suggestions in the risk standards
themselves: a) allocating an independent budget for risk management activities; b)
training risk management staff comprehensively on many more tools and techniques
rather than focusing on FMEA, FTA, and PHA; c) instituting a comprehensive
corrective action system that would allow corporate learning and retention of
experience for new and next generation staff.
5.3 Additional Insights and Implications
From the behavioral perspective, there were three cultural elements that
appeared to deserve attention. First, the risk attitudes of medical device companies
appeared to be consistent with the GLOBE Study results on uncertainty avoidance,

111
i.e. “the more uncertainty avoiding a society (company) is in its national
(organizational) character, the more its people prefer champions to enact
organizational rules, norms, and policy when promoting innovation”. Second, many
medical device companies viewed risk management a necessary evil – a regulatory
compliance activity only. Third, risk communication appeared to be a weaker area in
medical device companies. Structurally, only a few medical device companies had
executives with dedicated roles and rarely had an independent structure for risk
management. Risk management processes in medical device companies focused
mainly on the design control activities of the product lifecycle but not on later stage
activities such as product disposal activities.
From the capability perspective, the current economic climate did not seem to
affect most medical device companies’ resource allocation for risk management
activities. Medical device companies only focused on using a few tools for managing
risks and seemed to be satisfied with their existing level of competence. The
retention of previous experience in terms of lessons learned and root cause analyses
appeared not to be a universally respected area of attention.
Studies such as this one to scan the environment of risk management
activities also provoke questions about how to enhance risk management practices in
ways that would decrease regulatory actions or reactions to problems in the field. In
this study, more than half of the respondents reported significant recent risk
incidents, such as recalls, MDRs, field corrections, and requirements to stop
shipments. More than a third of the respondents anticipated that similar significant

112
risk incidents would happen within the next twelve-month period. It may be
beneficial for these medical device companies to enhance their risk management
practices constructively and proactively. Yet the question must be asked, “How
much risk management is good enough for a medical device company?” This is a
question that has no universal answer, since risk activities must be graded and
prioritized according to the specific nature of the product under consideration and the
history and capabilities of the company producing it. Nonetheless, based on this
study, one might suggest that companies look again at their practices. If the time is
right for some medical device companies to realize better risk management practices,
they could potentially use some or all of the six elements of the Behavior+Capability
Model to gauge their progress either longitudinally over time or in reference to the
responses of other companies examined concurrently here.
A great concern when implementing any required program in a company is to
achieve an understanding and appreciation for the importance of that activity if it is
to be done well. The following comments (Appendix D Q.3), provided by two
individuals from two different companies, appear to summarize the opposing
sentiments of different individuals with regard to risk management.
Comment 1: Although the ISO guidance and ICH Q9 have been issued
mandating the requirements for industry to perform risk management as part
of the lifecycle development for medicinal products, the enforcement of risk
management compliance is still at its infancy. Until we are seeing more
agency inspection and non-compliance citations along this area, it will take a
while for companies to agree to implement the risk management program full
scale.

113
The comment from this individual reflects views expressed by a few other
respondents that risk management is an exercise in fulfilling compliance obligations.
While the need to meet regulatory expectations as a primary driver would certainly
provide direction for a company’s behavior when dealing with risk management
activities, it is not clear that the role played by these activities in the achievement of
the company’s core goals is appreciated.
Comment 2: A) There is the need to align risk rating system between R&D
Design Hazard Analysis, Design and Process FMEAs, Software risk
assessment, CAPA Impact Assessment, and post-market Risk Evaluations,
and update impacted documents as applicable. B) We need to create
linkages between existing risk management tools to ensure alignment and
maintainability (Design FMEA, Process FMEA, Fault Tree Analysis,
software risk assessment, etc.) within revised Risk Management Process.
Include process for definition of critical quality attributes. C) Ensure
medical opinions regarding harm are documented in risk management file
and referenced in hazard and risk analyses. Assess and implement
improvements as necessary in establishment of risk benefit analyses for
products.

The comments from this individual are much more sophisticated and suggest
a larger and more integrated approach to risk management across the product
lifecycle. As more advanced practitioners become experienced in applying risk
management, they can provide valuable insights into places where a good set of tools
and activities can be made better. Several similar comments regarding challenges
with definitions or connections between various pieces of risk management activities
suggest that risk managers may not be altogether comfortable with their approaches
even when they have a good grasp of the methods. This suggests some scope for
research. It raises the question of whether risk can be a formal discipline of science

114
in decision making while at the same time being used in a practical way to enhance
products throughout their life cycle.
5.4 Future Directions
This is the first time that a Dual Triad Model has been used as a research
framework to assess risk management in medical device companies. This initial
model is by no means perfect. However, it does provide a way to capture a broader
range of topics than might be considered in the absence of a formal framework, and
it allowed the researcher to decompose a relatively complex activity into manageable
parts for analysis and discussion. If researchers in the future consider it worthwhile
to pursue improvement on the model, further quantitative research instruments might
be developed to enhance or modify the six subcategories. There is much potential for
future research in the risk management field; the fact that more than half of the
respondents (45/80) were willing to continue the research either by interview (32/45)
or by survey (40/45) may suggest an interest in sharing personal perceptions and risk
methods that would be valuable to the industry as a whole.
The literature review in Chapter Two has suggested that throughout recorded
history, substantial concern has focused on the challenges of living in a risky world
where the certain outcome of life is eventually death. This same concern persists
today and is particularly apparent in the requirements to manage the risks of medical
devices. However, the approach to understand and tackle these concerns has evolved
from shaman practices to scientific approaches and more formal frameworks. The
critical developments in risk management frameworks during the latter half of 1990s

115
included the initial draft EN/ISO 1441 – Risk Analysis Standard for Medical Devices
in 1996, and the Risk Management Framework on Medical Product Use by the FDA.
In the same timeframe, Kaplan received his distinguished 1997 award (by the
Society of Risk Analysis) on a paper titled “The Words of Risk Analysis”. Sixteen
years have passed; ISO 14971 has been revised twice, the original EN/ISO 1441 has
been updated three times. Nevertheless, medical device companies often seem
content with the original EN/ISO 1441 approach that focuses almost exclusively on
the use of the FMEA tool. From the results here, it may be time for risk professionals
in the medical device field to consider a more advanced approach that will require
both experience and knowledge about tools and techniques. To this end, it may be
beneficial for one or more champions (one of the regulatory science academic
institutions, the U.S. FDA, or a tripartite partnership between government, university
and industry) to advocate a risk science program within engineering or regulatory
science curricula to meet the challenges of the next generation of emerging medical
device companies. First steps in the direction have in fact been made. For example,
the University of Southern California has a graduate program in risk management
called the Certificate in Patient and Product Safety program
(http://regulatory.usc.edu) and Virginia Tech University has had an MS program in
Health Product Risk Management (http://vto.vt.edu/risk/graduate_degree.html).
However, these programs have relatively modest enrollments and a more systematic
promulgation of risk education may be needed as a core component of biomedical

116
engineering programs if sufficient well-trained individuals are to be provided to the
medical device industry.
5.5 Conclusions
Medical devices provide great benefit to patients, but they can also cause
unwarranted injuries or even death. Medical device companies are responsible for
preventing or eliminating health risks associated with using the products, and
simultaneously maintaining financial solvency. One of the ways in which medical
device companies can achieve these dual goals is through better risk management to
reduce the risks and financial costs associated with regulatory actions, liability and
damage to reputation that occurs when safety or function is compromised. From the
work presented here, it would appear that most medical device companies have
implemented the international standard ISO 14971 and they may also have been
aware of the FDA’s risk management framework. However, these documents were
written with a limited scope and with a regulatory focus. The research model in this
study provides a more comprehensive and complementary approach to characterize
important elements of a functional risk management system. This disciplined
approach aims to help medical device companies to better understand, monitor,
evaluate, and improve their course of actions associated with risks. It is based on a
review of literature on project risk management, high reliability organizations, and
lessons learned from disasters and failures of risk management.
Risk management is the skeleton that supports a total lifecycle quality
management system of a medical device company. If a risk management system is

117
insufficient or lacking, it is more likely that the company may encounter a safety
crisis that will cripple operations or even eliminate the existence of the medical
device company. The English word “Crisis” is the same as the two Chinese
characters - . They are pronounced as “weiji”. They literally mean “Danger
and Opportunity”. The development of an effective risk management culture, by
using best practices and enhancing the study of problems using disciplined and
appropriate methods, can help a company to avoid danger, and rather capitalize on
the opportunities provided in the marketplace for safe, effective and reliable products
whose regulatory risks and financial uncertainties have been minimized.

118
GLOSSARY
A. § ISO 14971:2007 Terms and Definitions
1. Harm means physical injury or damage to the health of people, or damage to
property or the environment
[ISO/IEC Guide 51:1999, definition 3.3]

2. Hazard means potential source of harm
[ISO/IEC Guide 51:1999, definition 3.5]

3. Hazardous Situation means circumstance in which people, property, or the
environment are exposed to one or more hazard(s)
[ISO/IEC Guide 51:1999, definition 3.6]

4. Risk is the combination of the probability of occurrence of harm and the
severity of that harm
[ISO/IEC Guide 51:1999, definition 3.2]

5. Risk Analysis means systematic use of available information to identify
hazards and to estimate the risk
[ISO/IEC Guide 51:1999, definition 3.10]

6. Risk Management means systematic application of management policies,
procedures and practices to the tasks of analyzing, evaluating, controlling and
monitoring risk.

B. § 21 CFR 820.3 Definitions
1. Establish means define, document (in writing or electronically), and
implement.

2. Management with executive responsibility means those senior employees of a
manufacturer who have the authority to establish or make changes to the
manufacturer’s quality policy and quality system.

3. Quality means the totality of features and characteristics that bear on the
ability of a device to satisfy fitness- for-use, including safety and
performance.

4. Quality System means the organizational structure, responsibilities,
procedures, processes, and resources for implementing quality management.

119

5. Specification means any requirement with which a product, process, service,
or other activity must conform.

6. Validation means confirmation by examination and provision of objective
evidence that the particular requirements for a specific intended use can be
consistently fulfilled.

7. Process Validation means establishing by objective evidence that a process
consistently produces a result or product meeting its predetermined
specifications.

8. Design Validation means establishing by objective evidence that device
specifications conform with user needs and intended use(s).

9. Verification means confirmation by examination and provision of objective
evidence that specified requirements have been fulfilled.

C. § Other terms used in the text
1. Top 30 represents the thirty medical device companies that had 89% of the
worldwide market revenues in 2008.

2. “Others” represents medical device companies other than the thirty medical
device companies that had 89% of the worldwide market revenues in 2008.

120
BIBLIOGRAPHY
Apostolakis, G. E. (2004). “How useful is quantitative risk assessment?” Risk
Analysis 24(3): 515-520.

Bernstein, P. L. (1996). Against the Gods. New York, John Wiley & Sons, Inc.

Blankfein, L. (2009). Do not destroy the essential catalyst of risk. Financial Times,
Feb 8, 2009 (accessed July 9, 2012 at http://www.ft.com)

Cassens, B. (2012). Warning Letter to Acclarent, Inc. DHHS. San Francisco,
FDA.June 14, 2012

CEN (1997). “Medical Devices - Risk Analysis.” British Standards Institute (Ref.
No. EN 1441:1997 E).

Committee on Shuttle Criticality Review and Hazard Analysis Audit, S. A. B.,
Commission on Engineering and Technical Systems, National Research
Council (1988). Post-Challenger Evaluation of Space Shuttle Risk
Assessment and Management. Washington, D.C., The National Academy
Press.

Covello, V. T. and J. Mumpower (1985). “Risk analysis and risk management: An
historical perspective.” Risk Analysis 5(2): 103-120.

CRMPG III (2008). Containing systemic risk: The road to reform. Counterparty Risk
Management Policy Group III. (accessed, July 9, 2012 at
http://www.crmpolicygroup.org)

David, F. N. (1962). Games, Gods and Gambling: The Origins and History of
Probability and Statistical Ideas from the Earliest Times to the Newtonian
Era. London, Charles Griffin.

Derby, S. L. and R. L. Keeney (1981). “Understanding ‘How safe is safe enough?’”
Risk Analysis 1(3): 217-224.

Lowe, N. and Scott, W. (1996). Medical Device Reporting for User Facilities:
DHHA pp. 11-14. (access on July 9, 2012 at www.fda.gov)

Donovan, C. J. and W. E. McDermott (2010). Negotiating medical device
outsourcing agreements. Medical Product Outsourcing, MPO, Jan/Feb, 2010.
(accessd on July 9, 2012 at www.mpo-mag.org)

121
EU (2002). “Commission communication in the framework of the implementation of
Council Directive 93/42/EEC of 1 June 1993 in relation to medical devices
and Directive 98/79/EC of the European Parliament and of the Council of 27
October 1998 on in-vitro diagnostic medical devices.” Official Journal of the
European Communities (C182).

EU (2007). “Commission communication in the framework of the implementation of
the Council Directive 93/42/EEC concerning medical devices, European
Union.” Official Journal of the European Union (C186).

FCIC (2011). The Financial Crisis Inquiry Report: Final Report of the National
Commission on the Causes of the Financial and Economic Crisis in the
United States. Washington D.C., U.S. Government Printing Office.

FDA (1990). Device recalls: A study of quality problems. DHHS Publication FDA
90-4235

FDA (1996). 21 CFR Parts 808, 812, 820; Medical Devices: Current Good
Manufacturing Practice (CGMP) Final Rule; Quality System Regulation.
Vol. 61, No. 95. Federal Register.

FDA (1997). Design Control Guidance for Medical Device Manufacturers.
Government Printing Office. (accessed on July 9, 2012 at www.fda.gov)

FDA (1999). Managing the Risks from Medical Product Use: Creating a Risk
Management Framework. Government Printing Office. (accessed on July 9,
2012 at www.fda.gov)

FDA (2011). Strategic plan for regulatory science. Government Printing Office.
(accessed on July 9, 2012 at www.fda.gov)

GHTF (2005). Implementation of Risk Management Principles and Activities within
a Quality Management System, Global Harmonization Task Force,
SG3/N15R8:2005. (accessed on July 9, 2012 at www.ghtf.org)

GHTF (2008). Quality Management System - Medical Devices - Guidance on the
Control of Products and Services Obtained from Suppliers, Global
Harmonization Task Force. SG3/N17R9:2008. (accessed on July 9, 2012 at
www.ghtf.org)

122
GHTF (2010). Quality Management System - Medical Devices - Guidance on
Corrective Action and Preventive Action and Related QMS Processes, Global
Harmonization Task Force. SG3/N18:2008. (accessed on July 9, 2012 at
www.ghtf.org)

Grier, B. (1981). The early history of the theory and management of risk. Judgment
and Decision Making Group Meeting. Philadelphia, Pennsylvania.

Guldenmund, F. W. (2010). “(Mis)understanding safety culture and its relationship
to safety management.” Risk Analysis 30(10): 1466-1480.

Haimes, Y. Y., S. Kaplan, et al. (2002). “Risk filtering, ranking, and management
framework using hierarchical holographic modeling.” Risk Analysis 22(2):
383-397.

Hamilton, M. B. (2009). Online Survey Response Rates and Times: Background and
Guidance for Industry. Ipathia, Inc. (accessed on July 9, 2012 at
www.supersurvey.com)

Hoffman, M. (2010). “New data provide insight into Pharma-Industry daily lives.”
Pharmatech 34.

Hopkins, A. (1999). “For whom does safety pay? The case of major accidents.”
Safety Science 32: 143-153.

Hopkins, A. (2006). “Studying organizational cultures and their effects on safety.”
Safety Science 44(10): 875-889.

House, R. J. and Global Leadership and Organizational Behavior Effectiveness
Research Program. (2004). Culture, Leadership, and Organizations: the
GLOBE Study of 62 Societies. Thousand Oaks, Calif., Sage Publications.

IAEA (2011). IAEA International Fact Finding Mission of the Nuclear Accident
Following the Great East Japan Earthquake and Tsunami Tokyo, Fukushima
Dai-ichi NPP, Fukushima Dai-ni NPP and Tokai NPP, Japan., International
Atomic Energy Agency. (accessed on July 9, 2012 at
http://www@pub.iaea.org)

IEC/ISO (2009). IEC/ISO 31010:2009 Risk Management: Risk Assessment
Techniques. Geneva, Switzerland., International Electrotechnical
Commission.

123
ISO (2000). ISO 14971 Medical Devices - Application of risk management to
medical devices. Geneva, Switzerland., International Organization for
Standardization. 2000-12-15

ISO (2003). ISO 13485 Medical Devices - Quality Management Systems - System
requirements for regulatory purposes. Geneva, Switzerland., International
Organization for Standardization.

Kaplan, S. (1997). “The words of risk analysis.” Risk Analysis 17(4): 407-417.

Kaplan, S. and B. J. Garrick (1981). “On the quantitative definition of risk.” Risk
Analysis 1(1): 11-27.

King, R. (2011). “European Commission questions standard.” AAMI News 46(1):
13.

Krewski, D. (2011). Risk Assessment, Risk Management. Encyclopedia of Public
Health.

Lemmel, M. (2000). Guide to the implementation of directives based on the new
approach and the global approach. Office for Official Publications of the
European Communities. (accessed on July 9, 2012 at http://europa.eu.int)

Mintzberg, H. (1983). Structures in Five: Designing Effective Organizations.
Englewood Cliffs, NJ, Simon & Schuster.

Molak, V. (1997). Fundamentals of Risk Analysis and Risk Management. Boca
Raton, Lewis Publishers.

National Commission on Terrorist Attacks upon the United States. (2004). The 9/11
Commission report: final report of the National Commission on Terrorist
Attacks upon the United States. New York, Norton.

National Research Council (U.S.). Committee on New Orleans Regional Hurricane
Protection Projects. (2009). The New Orleans Hurricane Protection System:
Assessing Pre-Katrina Vulnerability and Improving Mitigation and
Preparedness. Washington, D.C., National Academies Press.

National Research Council (U.S.). Committee on Risk Assessment Methodology.
and National Research Council (U.S.). Board on Environmental Studies and
Toxicology. (1993). Issues in Risk Assessment. Washington, D.C., National
Academy Press.

124
National Research Council (U.S.). Committee on Risk Perception and
Communication. (1989). Improving Risk Communication. Washington, D.C.,
National Academy Press.

National Research Council (U.S.). Committee on the Institutional Means for
Assessment of Risks to Public Health. (1983). Risk assessment in the Federal
Government: Managing the Process. Washington, D.C., National Academy
Press.

Oppenheim, A. L. and E. Reiner (1977). Ancient Mesopotamia: Portrait of a Dead
Civilization. Chicago, University of Chicago Press.

Ostrom, L., C. Wilhelmsen, et al. (1993). “Assessing safety culture.” Nuclear Safety
34(2): 163-173.

Reason, J. T. (1997). Managing the Risks of Organizational Accidents. Aldershot,
Hants, England; Brookfield, Vt., USA, Ashgate.

Roberts, K. H. and R. Bea (2001). “Must accidents happen? Lessons from high
reliability organizations.” Academy of Management Executive 15(3): 70-79

Ryrie, C. C. (1978). The Ryrie Study Bible: New American Standard Translation.
Chicago, Moody Press.

Silverman, S. (2012). Warning Letter to Amplivox Limited. DHHS. Silver Spring,
FDA. May 2, 2012.

Stein, M. A. and L. Christiansen (2010). Successful Onboarding: A Strategy to
Unlock Hidden Value within your Organization, McGraw-Hill.

Stulz, R. M. (2008). “Risk management failures: What are they and when do they
happen?” Journal of Applied Corporate Finance 20(No. 4): pp. 39-48.

Stulz, R. M. (2009). “Six ways companies mismanage risk.” Harvard Business
Review 87(3): 86-94.

Sullivan, J. and R. Beach (2009). “Improving project outcomes through operational
reliability: A conceptual model.” International Journal of Project
Management 27(8): 765-775.

United States. Columbia Accident Investigation Board (2003). Columbia Accident
Investigation Board Report. Washington, D.C., National Aeronautics and
Space Administration.

125
Weick, K. E. (2004). “Normal accident theory as frame, link, and provocation.”
Organization & Environment 17(1): 27-31.

Whittington, R. and K. Panny (2004). Principles of Auditing and Other Assurance
Services. Boston, McGraw-Hill.

WHO (2010). Medical Devices: Managing the Mismatch. Geneva, Switzerland.,
World Health Organization.

Winston, A. (2010). Five lessons from the BP oil spill, Harvard Business Review.
Blogs Network (accessed on July 9, 2012 at chemeng.nmsu.edu)

126
APPENDIX A
RISK MANAGEMENT FAILURE LESSONS
The researcher examined seven reports of risk management failures. This
review used the Behavior+Capability model to reveal the reasons for failure in
managing risk and identified many cross-related elements of the model for a
successful risk management program. Both the lessons learned and
recommendations were direct quotes from each of the seven reports and each
lesson/recommendation was then coded as appropriate with one or more elements of
the proposed Behavior+Capability model. These were tabulated under this appendix
in the next section.

127
Table A.1
Lessons from NASA: The Challenger Crisis (Committee on Shuttle Criticality
Review and Hazard Analysis Audit 1988)

Research Model
Behavior:
B1-Cultural; B2-Structural; B3-Process Behavior+Capability
Challenger
Crisis

Capability:
C1-Resource; C2-Competence;
C3-Learning/Memory B-1, 2. 3 C-1, 2, 3
Category # Lessons/Recommendations Behavior Capability
1 Formal, objective criteria for approving or rejecting
proposed critical item waivers;
3
2 Modification of the criticality definitions in terms of
the probability of failure, probability of worse-case
effects and rationale that produced the priority rating;
3
3 Failure Mode and Effect Analysis (FMEA) and Critical
Item List (CIL) be used as the inputs for quantitative
risk assessment to provide the basis for risk acceptance
and the control of residual hazards;
2
4 Linkages between the formal risk assessment process
and the STS engineering change activities;
3
5 Mission anomalies and their dispositions be
documented, justified by the launch decision makers
and fed into the formal risk assessment and
management processes for action before committing to
the next flight;
3
6 Statistical sciences and probabilistic risk assessment
approaches be applied at the earliest possible
opportunity in the design, development, test,
manufacturing, and operations;
2
7 Top-down and bottom-up risk assessment approaches
be brought together in a coherent manner to ensure
completeness and effectiveness of an integrated
systems engineering and system safety analyses;
3 2
8 Independent approval process for hardware
certification and software verification and validation;
3

128
Table A.1, continued
9 Creation of a list of mandatory Launch Commit Criteria
that cannot be waived under any circumstances;
3
10 FMEA to include human factors as potential causes of
failure modes and provide linkages to the hazard analysis;
2
11 Adequate funding for the procurement and repair of spare
parts and treat cannibalization of parts as potential causes
of failures;
1
12 Clear roles, responsibilities, and authorities for those who
are final decision makers to avoid “collective
responsibilities” where “everybody’s business becomes
nobody’s business”;
2
13 A comprehensive plan for conducting periodic inspection
and maintenance of the structure of each STS throughout
her service life;
3
14 FMEA on software to identify and predict fault and error
modes with periodic independent review and oversight;
3 2
15 Strong central program direction for integrating all aspects
of the NSTS program to avoid differences in procedures
that would lead to imbalances among various center
authorities;
1, 3
16 Application of practicable Non-Destructive Evaluation
(NDE) techniques to the Solid Rocket Motor (SRM) at the
launch facility, at the highest possible level of assembly
(SRM stack configuration), and emphasize development of
improved NDE methods;
2
17 A focused agency-wide Systems Safety Engineering (SSE)
function, at both Headquarters and the centers responsible
for:
The validation, qualification, and certification of design
and development, activities;
Full systems approach to the continuous identification of
safety risks and the objective quantitative evaluation of
such safety risks;
Provision of outputs to the NASA Program Directors for
their risk management efforts; and
Provision of assurance that their systems are ready for final
safety certification to the previously established risk
acceptable levels by the NASA Administrator
2

129
Table A.2
Lessons from NASA: The Columbia Disaster (United States. Columbia Accident
Investigation Board 2003)

Research Model
Behavior:
B1-Cultural; B2-Structural; B3-Process Behavior+Capability
Columbia
Disaster

Capability:
C1-Resource; C2-Competence;
C3-Learning/Memory B-1, 2. 3 C-1, 2, 3
Category # Lessons/Recommendations Behavior Capability
Organizational 1 Original compromises to gain approval for the Space
Shuttle Program
1
2 Resource constraints 1
3 Fluctuating priorities 1
4 Schedule pressures 1

5 Mischaracterization the Shuttle as operational rather
than developmental
3

6 Lack of an agreed national vision for human space
flight
1

7 Reliance on past success as a substitute for sound
engineering practices
1

8 Organizational barriers that prevent effective
communication of critical safety information
1
9 Stifled professional differences of opinions 1

10 Lack of integrated management across program
elements
2, 3

11 Evolution of an informal chain of command and
decision-making processes that operated outside the
organization’s rules.
1, 3
Physical

The CAIB provided detailed recommendations on 20+
of the physical related issues regarding the Thermal
Protection System, Imaging, Orbiter Sensor Data, Bolt
Catchers, Closeouts, Micrometeoroid and Orbiter
Debris, and Foreign Object Debris.
1, 2

130
Table A.3
Lessons from Sendai Earthquake/Nuclear Disaster (IAEA 2011)

Research Model
Behavior:
B1-Cultural; B2-Structural; B3-Process Behavior+Capability
Sendai Disaster

Capability:
C1-Resource; C2-Competence;
C3-Learning/Memory B-1, 2. 3 C-1, 2, 3
Category # Lessons/Recommendations Behavior Capability
Strengthen
preventive
measures
against a severe
accident
1 Strengthen measures against earthquakes and tsunamis 1
2 Ensure power supplies 1
3 Ensure robust cooling functions of reactors and PCVs 1
4 Ensure robust cooling functions of spent fuel pools 1
5 Thorough accident management (AM) measures 3
6 Response to issues concerning the siting with more than
one reactor
3
7 Consideration of NPS arrangement in basic designs 3
8 Ensuring the water tightness of essential equipment
facilities
1
Enhancement of
response
measures
against severe
accidents
9 Enhancement of measures to prevent hydrogen explosions
10 Enhancement of containment venting system 1
11 Improvements to the accident response environment 3
12 Enhancement of the radiation exposure management
system at the time of the accident
3
13 Enhancement of training responding to severe accidents 2
14 Enhancement of instrumentation to identify the status of
the reactors and PCVs
1
15 Central control of emergency supplies and equipment and
setting up rescue team
3 1

131
Table A.3, continued
Enhancement of
nuclear
emergency
responses
16 Responses to combined emergencies of both large-scale
natural disasters and prolonged nuclear accident
3
17 Reinforcement of environmental monitoring 1
18 Establishment of a clear division of labor between
relevant central and local organizations
2
19 Enhancement of communication relevant to the accident 1
Enhancement of
nuclear
emergency
responses
20 Enhancement of responses to assistance from other
countries and communication to the international
community
1, 3
21 Adequate identification and forecasting of the effect of
released radioactive materials
3 2
22 Clear definition of widespread evacuation areas and
radiological protection guidelines in nuclear emergency
3
Reinforcement
of safety
infrastructure
23 Reinforcement of safety regulatory bodies 2, 3
24 Establishment and reinforcement of legal structures,
criteria and guidelines
2, 3
25 Human resources for nuclear safety and nuclear
emergency preparedness and responses
1, 2
26 Ensuring the independence and diversity of safety
systems
2
27 Effective use of probabilistic safety assessment (PSA) in
risk management
2
Thoroughly
instill a safety
culture
28 Thoroughly instill a safety culture 1

132
Table A.4
Lessons from Financial Crisis

Research Model
Behavior:
B1-Cultural; B2-Structural; B3-Process Behavior+Capability
Financial
Crisis

Capability:
C1-Resource; C2-Competence;
C3-Learning/Memory B-1, 2. 3 C-1, 2, 3
Category # Lessons/Recommendations Behavior Capability
(FCIC 2011) 1 Widespread failures in financial regulation, including the
Federal Reserve’s failure to stem the tide of toxic
mortgages;
3
2 Dramatic breakdowns in corporate governance including
too many financial firms acting recklessly and taking on too
much risk;
1
3 An explosive mix of excessive borrowing and risk by
households and Wall Street that put the financial system on
a collision course with crisis;
3
4 Key policy makers ill prepared for the crisis, lacking a full
understanding of the financial system they oversaw; and
2
5 Systemic breaches in accountability and ethics at all levels. 1
(Stulz 2008) 6 Use the wrong risk metrics and thus, wrongly measuring
risks.
2
7 Fail to communicate risk assessments to guide top
management and boards.
1, 3
8 Fail to monitor risks appropriately and maintain the
company’s targeted risk positions.
2, 3
(Stulz 2009)
Relying on
historical data
9 Risk management modeling involves extrapolating from
the past, but rapid financial innovation in recent decades
has made history an imperfect guide.
2, 3
Example: Historical data were of little use in estimating the
impact of the recent fall in house prices, because those data
did not cover a period during which the market saw a
downturn while a large number of subprime mortgages
were outstanding.

133
Table A.4, continued
Focusing on
narrow
measures
10 Many financial institutions use daily measures to track risk.
These underestimate a company’s exposure, because they
assume that assets can be sold quickly, limiting the
company’s losses within a day.
2
Example: Financial crises involve a dramatic withdrawal of
liquidity from securities markets, leaving firms exposed for
weeks or months on positions they cannot easily unwind.

Overlooking
knowable
risks
11 Risk managers simply overlook many types of risk and
sometimes even create them.
2
Example: Investors in Russia tried to hedge the risk of a
collapse in the ruble by taking currency positions with
Russian banks. But they failed to recognize that a shock to
the banking system would threaten those banks’ ability to
meet their commitments.

Overlooking
concealed
risks
12 People responsible for incurring risk often do not report it,
sometimes deliberately, but often unintentionally.
Organizations have a tendency to expand unreported risks.
1
Example: If traders receive a share of the profits they
generate but do not have to defray the losses, they have an
incentive to take risks, which is easier to do if the risks are
unmonitored.

Failing to
communicate
13 Risk management systems will provide little protection if
risk managers do not communicate clearly.
1, 3
Example: The Swiss bank UBS attempted to explain its
subprime and housing exposures in an overly complex way
and to the wrong audience.

Not managing
in real-time
14 Risks can change sharply and quickly with daily
fluctuations in the stock market.
2
Example: A manager holding a barrier call option who does
not check the risk throughout the day may fail to put
appropriate hedges in place.

134
Table A.4, continued
(Blankfein
2009)
15 Risk Management should not only be predicated on
historical data.
2, 3
16 Too many people outsourced their risk management and
didn’t do their own analysis,
2, 3
17 Size matters – absolute exposure matters. 2
18 Hedging your risks needs more careful thought. 2
19 Everything must be on the table – hidden risks can be
critical to a complete picture.
3 2
20 Complexity outstripped the operational capacity to manage
the risks.
2
21 Risks need to be properly valued. 3
22 Risk and Control Functions must be independent from
business units.
2
23 Risk Managers need to have at least equal stature (status,
hierarchical position) with Operational Managers.
2
24 When there is disagreement on Risk Acceptability, Risk
Managers should prevail.
2
25 External regulation is critical – self regulation is
inadequate.
1, 2, 3
(Blankfein
2009)
26 Regulators should implement more robust information
sharing to monitor systemic risks.
2, 3
27 Removing risk entirely will decrease growth and reduce
well-being that flows from it.
3
(CRMPG III
2008)
Corporate
Governance
28 Risk management activities cannot just rely on quantitative
metrics. Risk management must rely on judgment,
communication and coordination, across the organization
and reaching to the highest levels of management.
2, 3
29 The culture of corporate governance in an institution must
ensure real independence of risk management function
from other functions, not only in a reporting context but
also in a decision-making context.
1
30 Financial institutions must examine their framework of
corporate governance in order to ensure that it is fostering
the incentives that will properly balance commercial
success and disciplined behavior over the cycle while
ensuring the true decision-making independence of key
control personnel from business unit personnel.
1

135
Table A.4, continued
Risk
Monitoring
31 Timely access to real-time information is critical to risk
monitoring of proper signals. Models and metrics are only
as good as the institutional ability to monitor real-time
positions and risk exposures.
2
32 Financial institutions must have, or be developing, the
capacity (1) to monitor risk concentrations to asset classes
as well as estimated exposures, both gross and net, to all
institutional counterparties in a matter of hours and (2) to
provide effective and coherent reports to senior
management regarding such exposures to high-risk
counterparties.
1, 2
Risk Appetite 33 Estimating acceptable thresholds of risk appetite is more an
art than a science. Both scenario analyses and stress tests
together with quantitative inputs are necessary.
3 2
34 Financial institutions must periodically conduct
comprehensive exercises aimed at estimating risk appetite.
The results of such exercises should be shared with the
highest level of management, the board of directors and the
institution’s primary supervisor.
1, 3
Contagion 35 Contagion (channels and linkages through which local
financial disturbances can take on systemic characteristics)
are likely unpredictable; however, their basic causes are
reasonably well known and recognized. Therefore,
financial institutions should build into the risk management
frameworks ongoing analysis and brainstorming about
contagion risks.
2, 3
Oversight 36 Board of Directors must oversee the company’s goal of
maximizing shareholder value over time. With the
complexity and scarce resource, the Board must know how
to ask the right questions based on proper information.
2
37 In a financial institution, the highest-level officials from
primary supervisory bodies should meet at least annually
with the boards of directors of large integrated financial
intermediaries. The purpose of the meeting would be for
the supervisory authorities to share with the board of
directors and the highest levels of management their views
of the condition of the institution with emphasis on high-
level commentary bearing on the underlying stability of the
institution and its capacity to absorb periods of adversity.
2, 3

136
Table A.5
Lessons from 911 Security Crisis (National Commission on Terrorist Attacks upon
the United States 2004)

Research Model
Behavior:
B1-Cultural; B2-Structural; B3-Process Behavior+Capability
911 Security
Crisis

Capability:
C1-Resource; C2-Competence;
C3-Learning/Memory B-1, 2. 3 C-1, 2, 3
Category # Lessons/Recommendations Behavior Capability
Terrorist 1 Leaders able to evaluate, approve, and supervise the
planning and direction of a major operation;
2, 3
2 Personnel system able to recruit candidates,
indoctrinate them, vet them, and give them the
necessary training;
3 1
3 Communications sufficient to enable planning and
direction of operatives and those who would be
helping them;
3
4 Intelligence effort capable to gather required
information and form assessments of enemy
strengths and weaknesses;
3 2
5 Able to move people great distances; and 2
6 Able to raise and move the money necessary to
finance an attack.
2
7 Sophisticated, patient, disciplined, and lethal; 1
8 No distinction between military and civilian targets; 1
9 Collateral damage is not in its lexicon; 1
10 Hostility towards U.S. and the western values is
limitless and unimaginable;
1
11 No religious and political pluralism, plebiscite, and
equal rights for women; and
1
12 Increasing support from Arabs and Muslims against
the U.S..
1

137
Table A.5, continued
United States 13 Relatively not serious and ignorant about the enemy 1
14 Much intelligence had already been known but not
actively or adequately acted on for a long time
1, 2, 3
15 The U.S. security agencies did not understand the
scope of the potential threat and had no intention to
adjust any deterrence policies, plans, and practices
for defeating the threat.
2
16 Breaking down of interagency communications in
confronting various security threats. Security
agencies operated in silos.
1

138
Table A.6
Lessons from Hurricane Katrina (National Research Council (U.S.). Committee on
New Orleans Regional Hurricane Protection Projects. 2009)

Research Model
Behavior:
B1-Cultural; B2-Structural; B3-Process Behavior+Capability
Hurricane Katrina

Capability:
C1-Resource; C2-Competence;
C3-Learning/Memory B-1, 2. 3 C-1, 2, 3
Category # Lessons/Recommendations Behavior Capability
Risk
Communication
1 It is important to communicate regularly and
consistently the risks of hurricanes and storm surge to
residents in layman’s terminology in order to enhance
the appreciation of the risks involved
2
Independent
Review for
Engineering and
Design
2 Periodic external reviews in the design, construction,
and maintenance of large, complex civil engineering
projects such as hurricane protection system are
necessary. The review team should be objective and
independent of the authority that initiated the review
such that all the technical assessments are reliable,
methods are credible, designs are adequate and safe,
potential biases are minimized, and political
sensitivity/correctness are avoided.
2, 3 2
Periodic
Assessments and
Updates of
Concepts,
Methods, and
Data
3 Changes in either or both physical and political
environmental conditions can influence hurricane and
flood protection projects. The effective and efficient
deployment of the most current scientific and
engineering approaches must be synchronized with
these conditions. Therefore, periodic scientific reviews
are necessary to take into account relevant updated
information and data appropriate to render certain
assumptions on which these projects were based partly
or fully obsolete.
2, 3 2
The Future of
Hurricane Risk
Analysis for New
Orleans and the
Gulf Coast
Region
4 It is important that the experience and lessons learned
from the disaster to be institutionally memorized for the
future. A publicly accessible archive of all data,
analytical models, analysis results, and model products
from the IPET projects is necessary for “institutional
learning and memory”.
3

139
Table A.7
Lessons from British Petroleum (BP) Oil Spill (Winston 2010)

Research Model
Behavior:
B1-Cultural; B2-Structural; B3-Process Behavior+Capability
BP Oil Spill
Crisis

Capability:
C1-Resource; C2-Competence;
C3-Learning/Memory B-1, 2. 3 C-1, 2, 3
Category # Lessons/Recommendations Behavior Capability
Relying on old,
fossil-fuel based
technologies is
devastating for
the planet, for
society, and for
business.
1 He asserted that oil spills was unavoidable events inherent
to the technology itself. He anticipated more regulatory
controls would be instituted for the oil industry to commit
in mitigating the myriad risks of catastrophic failures.
2, 3
Preparing for a
world where
things only go
right is extremely
dangerous.
2 He observed the oil companies had invested heavily in
exploration technologies, finding ways to do things —
like dig a mile under water — that were only space-age
fantasies until recently. But the oil companies had done
relatively very little to invest in the technologies to avoid
spills, contain them, and clean them up.
1, 2
Downplaying
corporate
mistakes is a big
mistake.
3 He compared the CEO of British Petroleum (BP), Tony
Hayward’s infamous and inappropriate public remarks
(the spill was “relatively tiny” compared to the “very big
ocean”, “I’d like my life back.”, etc…..) as childish versus
the Johnson & Johnson executive decisions on the
handling a disaster when it dealt with the poisoning of
Tylenol (and thus murder of some of its customers) in
1982. The massive, and immediate, recall was
unprecedented and set the standard for corporate behavior
in the face of existential threats to a business.
1
Environmental
risks can threaten
the viability of a
business.
4 BP has lost over a third of market value, worth about $70
billion. The New York Times went so far as to suggest
that BP could be vulnerable to takeover once all its
liabilities for this spill are accounted for.
1
Companies can
lose the reputation
as a sustainability
leader very fast.
5 BP’s reputation has been shattered. In the 1990s, the CEO
at the time, Lord John Browne, set BP on a path to go
“beyond petroleum.” The money BP saved through
carbon reductions. For years, the sustainability
community has praised BP as best-in-class. Warren
Buffett famously said, “It takes 20 years to build a
reputation and five minutes to ruin it.” Having a
reputation as a sustainability leader is valuable, but it’s a
tenuous thing, and it can be lost very fast.
1 3

140
APPENDIX B
SURVEY INSTRUMENT

141

142

143

144

145

146

147

148

149

150
APPENDIX C
Q.1 COMMENTS ON CHANGES
Q.1
Some significant changes have been made to our risk management system for the last two
years in my division.
(14) Compliance
i) Revision to Severity rank - changed from 4 level to 5 levels, Integration with IEC 62304,
62366, and 60601. More training on risk management processes, we also revised post market
risk management process. We also revamped the processes to integrate post market with
premarket risk management process. In radiotherapy devices, integration is very important.
We learnt a lot from it.
i) FDA published a guidance and increased the requirements for injectable auto injector
devices. This required to focus more on risk management specially usability aspect.
i) Complete overhaul to ensure adherence to latest FDA expectations
i) New standards and regulations
i) Risk management SOP introduced
i) Procedural changes to ensure consistency with requirements
i) Updates to better align with ISO 14971
i) Updated to meet ISO 14971 requirements
i) Currently working on significant changes to system to be more compliant with ISO 14971
i) A new company and UL 3rd edition
i) Inclusion of IEC62366 Requirements
i) Incorporated EN62366
i) More usability risk
i) Incorporation of usability / task analysis
(10) Specific Improvements
ii) We fully validated and implemented a “Software (A)” tool to support our Device Risk
Management Process and Device Risk Files in our Bioscience Division. We also launched
the Risk Management for Therapeutics.
ii) Complete revamp. Centralized risk management system “Software (B)” implementation. we
have Neuromodulation devices, class IIII
ii) New system of risk calculation
ii) Alignment with ISO14971 definition of risk
ii) Introduction of new tools; realignment of scales & definitions

151
ii) We unified the risk estimation table for our all affiliate companies.
ii) Post-market surveillance
ii) Better connection between pre- and post-market systems.
ii) Add robust means of incorporating complaint information.
ii) Added Design and Process FMEAs
(16) General Process Improvements
iii) New interfaces to design control relative to process risk assessment. How to ensure
significant specifications of the product within the process risk
iii) We have been using risk management for several years so changes are always made but it is
a mature system so changes are not significant
iii) Improved procedures, major revisions to two important risk management files
iii) New procedures launched
iii) Great emphasis
iii) Continuous improvement after years of stagnant
iii) Integration in development model
iii) SOP updated to include more details in risk.
iii) Continuous improvements continue to expand the reach and depth of risk management
iii) Process Improvement
iii) Harmonized system
iii) Re-designed system
iii) New Risk Management Process and Implementation
iii) Integrating dissimilar practices among various sites to 1 process
iii) Integration activities
iii) Some minor revisions in software risk management

152
APPENDIX D
Q.3 COMMENTS ON PERCEPTIONS
Q.3 Provide comments on your division’s perception on the risk management system.
(16) A Necessary Evil or Lagging Behind View
i) My division is in the medical product development business, so we are required by the
FDA and our quality system to have a safety risk management process in place. Over the
years, we’ve put a pretty good system in place, but PM’s still see it as a burden sometimes,
and it usually falls to the SE’s to “take care of it”.
i) R&D is the most prone to viewing risk management activities as burdensome followed by
Marketing. Quality Regulatory and Operations are fairly accepting of the value and are
conversant with the tools and practices of Risk Management. In general, the company is
much more prone to thinking about financial risk management than it is product risk
management.
i) Overall the perception is that it is a required function but it brings very little value to the
process.
i) Risk management is often thought of as a file you go to , not an activity.
i) Although the ISO guidance and ICH Q9 have been issued mandating the requirement for
industry to perform risk management as part of the lifecycle development for medicinal
products, the enforcement of risk management compliance is still at its infancy. Until we
are seeing more agency inspection and non-compliance citations along this area, it will take
a while for companies to agree to implement the risk management program full scale.
i) Many other quality systems are used to mitigate “common” type hazards such as
biocompatibility so it seems redundant to assess this type of hazard within an FMEA.
Often times it seems that the risk analysis is a “paper” exercise because we are aware of
many hazards (do not need an FMEA to identify) and take them into account during new
product development.
i) Nowadays, risk management requires a lot of data mining, pre and post market integration,
and feedback to development projects. This requires a good automated software tool. I
think we need to purchase one. Radiation therapy is a high risk product, and it requires
significant risk management effort. Older products does not have very diligent risk
management. I think it was due to the ISO 14971:2000, which was not that stringent for
use error and integration with other standards. So, my company did not consider use error
as an issue.
i) It is seen as Regulatory’s responsibility to maintain the system even though we may not
have the expertise required for all areas of the risk analysis. There is reluctance to add
outside sources as resources even though this may be more efficient than ultimately
missing important risk factors which result in product issues at a later date.
i) Personnel appreciate that a risk-based approach (activities commensurate with risk) should
be employed for activities, but generally lack the discipline to follow a methodical process
for implementing risk management to arrive at accurate outcomes.

153
i) System deliverables are too time consuming to create. FMEAs have 100’s of entries; too
complex. Engineers are not using the FMEAs as an effective tool consistently. It is
difficult to keep risk assessment for like products consistent; no risk management software.
i) Division understands the importance of risk management, however the challenge is to
adequately balance risk management and other competing demands, especially for resource
needs in the immediate term since risk management usually does not provide immediate
short-term benefits.
i) Most all involved in the process realize the value of risk management and would
characterize it as a necessary evil.
i) Role of Risk Management is perceived primarily as a regulatory requirement and not a
development tool.
i) I would not say risk management is part of the strategic plan, it is a requirement for
medical devices that is becoming more and more a focus of the agencies.
i) In some cases I get the feeling it is a matter of filling out the Risk Management forms, and
not seeing the true benefit of what the process/forms can show you in of risk avoidance.
i) Risk management results are mostly ignored until they impact the design effort or field
action.
(13) A Necessary Good or Catching-Up View
ii) A) There is the need to align risk rating system between R&D Design Hazard Analysis,
Design and Process FMEAs, Software risk assessment, CAPA Impact Assessment, and
post-market Risk Evaluations, and update impacted documents as applicable. B) We need
to create linkages between existing risk management tools to ensure alignment and
maintainability (Design FMEA, Process FMEA, Fault Tree Analysis, software risk
assessment, etc.) within revised Risk Management Process. Include process for definition
of critical quality attributes. C) Ensure medical opinions regarding harm are documented
in risk management file and referenced in hazard and risk analyses. Assess and implement
improvements as necessary in establishment of risk benefit analyses for products.
ii) Risk management and continuous improvement is embedded into the framework and
organization of my department. However, emphasis is placed on training that is typically
online and systems-based for added efficiency, and does not include components that allow
for questions and answers, or opportunity for user feedback. The fact that an adequate
feedback loop does not exist may reduce staff engagement, and hinders the development of
appropriate processes. Also, processes are sometimes rushed to implementation, thus
reducing valuable upstream cross-functional feedback.
ii) Understanding how risk management ties into other elements of the quality system to
enable decision making, is not well understood. For example, there is a lack of
understanding from the Sr. Management down to associates handling complaints. A large
degree of risk management decisions are not based on outputs of the risk file. Decisions
regarding product recalls, complaints, and non-conformances do not link back to the risk
file in >80% of the cases.

154
ii) Risk Management process have been developed and implemented into the design and
development portions of the business including periodic and annual reviews. The business
is currently trying how to better integrate the production and post production information
into the process including linkage to other quality system decision points (supplier, CAPA,
change, etc.)
ii) As a regulatory consultancy, risk management is central to our activities on behalf of our
clients, so we have to be at the top of the game with this. This situation may not be typical
of manufacturers.
ii) People realized that the application of risk management is a no going back path and it is
very important to support decision-making aimed at efficacy and safety of medical
products but most involved do not understand the scope of its application and how to run it.
ii) Risk Management is the province of a small group of individuals which provide an analysis
for the reminder of the group to review and/or accept. The tools and training available for
this small group is key to the success of the project.
ii) There are many risk assessment tools. It is difficult to know which tools to use in specific
situations. Could benefit from additional training at the associate level.
ii) Risk management is a fundamental part of our operating plan, strategic plan though, I am
not sure if it falls into that.
ii) The new requirements of 60-601-1 edition 3 for electromechanical devices adds an entirely
new level of complexity to risk management assessment.
ii) Post market risk management is not up to the mark.
ii) Critical system to assess product risk and address concerns in a timely manner.
ii) Risk management tools are established and their use is not standardized
(17) An Ahead-Of-The-Curve View
iii) We have an entire Risk Management Department dedicated to risk. They take their jobs
serious. They provide training to all employees and ensure that risk documents are
routinely reviewed and updated to cover recent events or incidents. Unfortunately, this is
an area where Top Management is not as visible as I would like to see. Communication on
risk only comes from the Risk Management Department and is not shared or broadcasted
by others in senior executive level positions.
iii) The QA group got the ball rolling pretty strong in 2009. Got the Exec. VP of Engr and
Ops to go to training, and the chief design engineer. The VP wrote the policy and
procedures. Then it came time to get 2 important product line files updated. QA had to
take initiative to get this started. Had to use a consultant whose efforts are going to be used
in future updates. The consultant was also retained to do the risk mgmt planning and
design FMEA’s for new product development projects. His work is giving us a template to
follow for doing these things ourselves in the future.
iii) As the head of the Quality Management System and being very proactive in the Risk
environments, I have recently introduced the requirements for Human Factors which is also
known as Usability. Both Usability and Risk management overlap but within an increased
focus upon the user, the Risk evaluation tasks have grown.

155
iii) Risk management is an important part of our design process and it has been woven into
other reviews and processes such as management reviews, CAPAs, design changes, etc.
It has definitely proven its value to us, but it is a time-consuming effort. We are always
looking at ways to do it better and ways to lean it out.
iii) Risk management is considered essential in all aspects of the medical equipment:
development, production and even after placing on the market. It is required to satisfy ISO
14971 and IEC 60601 (3rd edition), otherwise we have no market access. Moreover, the
risk management process makes us feel more confident on the quality of our systems.
iii) Risk management is included both in the development process of new products and in the
product change control process
iii) We have a broad and well understood program and orientation towards the need and
importance of risk management.
iii) Risk Management is continuing to expand into a discipline for management of all forms of
uncertainty
iii) Risk Management is fully accepted in my business unit. It is well known in all sections of
our business unit.
iii) Risk management is one of important tool to keep safety for users. We have special
organization and risk managers and also we have training program.
iii) Systematic risk management supports a major goal of our products’ quality: safety. Our
brand is connected to the customer expectation of superiorly safe products.
iii) Actually it is routine, while the start (bout 10 years ago) required major one time effort.
Today risk management is a central process throughout product lifecycle.
iii) It has become part of our culture.
iii) FMEA and SHUMA activities are well established to ensure proper level of risk
management.
iii) Risk management is essential to the effective application of our (quality) management
system.
iii) Has grown to have traces to all major parts of the development process.
iii) RM is embedded in everything from environmental controls, design control and validation

156
APPENDIX E
Q.15 COMMENTS ON CHALLENGES
Q.15 What are the challenges of your division’s risk management system implementation?
(7) Knowledge and Understanding Challenges
i) There is no limit to the expectation of FDA regarding the extent of activities that should be
undertaken with regard to risk management. Also, many FDA investigators seem to take
the approach that no residual risk is acceptable.
i) Lack of comprehensive itemization of known risks/hazards for our product type(s).
Decades of the results of product use of competitive products can’t be located easily to aid
in proper analysis and risk assessment and mitigation.
i) Challenge of implementing the new 3rd edition 60-601 electrical safety standard that is
much more complex.
i) Risk Analysis versus Risk Management understanding
i) Overall Residual Risk Evaluation Details with respect to IEC 60601-1
i) There is a strong element of having to guess some risks particularly for a new product.
i) Acceptance for all products
(10) Integration Challenges
ii) The main challenge is that different manufacturing sites (under the parent company) have
their own customized ways of doing quality risk assessments and risk management. There
is no consistent scale used for Severity, Probability and Detection (FMEA) across sites,
making it difficult for different risk assessments to “speak” to one another. Many of the
corporate quality risk assessments are structured more like “GMP gap” analysis; rating
only against a severity and control scale (i.e.. Risk = S x P is not always done). They look
for gaps in policy and procedure against regulatory requirements, and rate it as high, med,
low risk.
ii) Different sites around the world incur different requirements for updating and maintaining
risk assessments. Difficult to keep risk assessments consistent across product families due
to global structure of business, i.e. similar products in Europe and US may have risk
assessments that differ. Need software but it is expensive and requires resources to
implement and maintain.
ii) Some challenges include integrating Risk with other quality sub-systems , such as CAPA,
FCA and Complaint Management. We have accomplished that, for the organizations that
are beginning to implement Risk Management, recommendation could be to take a
Systems approach and once system is in place look for a Tool to digitize and automate the
process, and create libraries.
ii) Maintaining a reasonable and cogent summary for management and customers and
connecting that to and maintaining a comprehensive and cohesive analytic basis (incl.
traceable probability values/calcs and reasonable representative severities across the risk
spectrum)

157
ii) Integration of premarket and post-market risk management is very poor. Identification of
risk on product design defect during development is not very well connected.
ii) Changing cultural expectations and creating effective tools to drive compliance and
consistency across a complex network of facilities.
ii) Lack of integration of risk management into the quality system due to lack of system
integration knowledge
ii) Lack of a detailed division-wide risk acceptance criteria.
ii) Complexity of our products and their applications.
ii) Difficult to carry out from beginning to end due to complexity.
(12) Organizational Challenges
iii) The current focus upon Risk Management and other areas being folded into the overall
Risk Plan (Usability) yet maintaining distinction between each major category of risk
concern is a challenge mainly because the regulatory community is raising the expectations
of the entire documented Risk Management activities. As a small company with limited
resources, this has the potential to be an area requiring significant focus. With a mentality
specific to “would you use this device on your family members”, our attitude is that good
enough is not good enough!
iii) Risk is taken very seriously in my organization. The product lifecycle bakes in risk
activities throughout the process and product development cannot advance to the next stage
unless all risk activities have been completed and implemented. The only challenge is
adding extra time, money, and resources to ensure all risk activities have been completed
during product development.
iii) Dealing with systematic (rather than random) factors contributing to hazardous situations
(mainly related to human interaction, error) for the prediction of the probability of
occurrence of harm. Dealing with regulatory bodies not being always systematic (in the
sense of a formal risk management process) when they assess acceptable risk.
iii) It is seen as a “necessary evil” rather than a tool for designing products/processes rather
than a tool for making sound decisions based on identification of potential areas where
issues may arise. Not always as thorough as needed because of the rush to get product
designed, approved and to market.
iii) Not sure I understood the question above; not clear if question is around challenges to
implementing the system or on a project/product basis
iii) Need to get an engineering person hired who can take over this function, rather than always
hiring it out to a consultant
iii) Update to risk management improvement is sometimes driven by compliance observations
rather than proactively.
iii) Lack of acts/behavior of top management consistent with his support laid down in written
procedure.
iii) Risk management implementation occurred many years ago.
iii) Resources

158
iii) It is a process that is required. I don’t see it as a challenge.
iii) There is a lack of support from many other departments

159
APPENDIX F
SUMMARY OF COMPARISONS
The researcher compared the results in four ways:
1. Top 30 Companies versus the “Others” Companies;
2. Risk Management Staff versus the Executive Group;
3. Smaller Size (<1,000 employees) versus Larger Size Companies (>5,000
employees);
4. High-Level versus Low-Level of Uncertainty Avoidance.
The comparisons were summarized with tabulated results under this appendix
in the following sections. Results are shown only for those questions in which
potential differences existed, usually with differences of 15 % or more, between the
two compared groups.

160
Table F.1
Comparison: T30’s vs “Others”
Model Category “Top 30’s” (n=50) “The Others” (n=26)
B-1 System Changes Majority had significant changes More than half had minor changes
B-1 Perceptions Generally did not feel risk
management activities
burdensome. Most felt RM
training was sufficient
Some felt risk management activities
burdensome, more than half felt
training insufficient
B-3 Formal Procedures Managed risks of product disposal
with procedures
Did not appear to manage these risks
formally.
B-3 Lack of Agreed Vision More concerned More not concerned
B-3 Characterization of
RM as QM rather than
an Integrated System
More of concern Less of concern
C-1 RM Investment A quarter had increased. One
would increase in 6 months. A
few were in discussion
Only a couple in discussion.
C-1 RM Budget A few had no budget and a few
reported to Mfg.
Several had no budget and several
reported to Mfg.
C-1 Challenges Over Next
2 Years
Many more had challenges when
collaborating with other business
units or divisions and some more
challenges in cost reduction
Less had challenges when collaborating
with other business units or divisions
and less had challenges in cost
reduction
C-3 Lessons Learned &
Root Cause Analyses
Appeared generally interested in
lessons learned and root cause
analyses were generally helpful.
Appeared somewhat interested in
lessons learned and root cause analyses
were considered not so helpful.

B-1 Culture: Some differences between the two groups in the system changes and perceptions.

B-3 Process: Some differences in processes: Top 30 appeared to be more formal in managing product disposal
risks, were more concerned about agreed vision, and characterized RM as part of QMS rather than an integrated
system.

C-1 Resources: Top 30 seemed to invest more in RM than the Others and had more challenges in collaborating
with other units and trying to obtain cost reduction than the others.

C-3 Memory: Top 30 appeared to be more conscientious in the lessons learned and root cause analyses than the
others.

161
Table F.1, continued

162
Table F.1, continued

163
Table F.2
Comparison: Risk Management Staff vs The Executive Group
Model Category RM Staff (n=46) Executive (n=28)
B-1 Risk
Communication
Methods
A third ranked Top Mgmt
Broadcast top 3
More than a third
ranked Top Mgmt
Broadcast inbottom 3
B-1 Uncertainty
Avoidance
Majority (66%) felt moderate
level
Many (39% each) felt
moderate or high
B-3 Challenges on RM
Implementation
Many felt lack of agreed vision Many disagreed that
company lacked agreed
vision
B-3 Challenges on RM
Implementation
Equally agreed and disagreed
(35% ea) on org barriers that
prevented understanding of
regulatory expectations
Majority (56%)
disagreed
B-3 Challenges on RM
Implementation
Equally agreed and disagreed
(37%/39%) on lack of integrated
mgmt across system elements
More than half (52%)
disagreed.
B-3 Challenges on RM
Implementation
Equally agreed and disagreed
(33%/39%) on stifled
professional difference of
opinions
More than half (52%)
disagreed.
C-1 Amount of Time
Spent
<1/2 (44%) ranked time spent in
Mfg top 2 and 1/3 for bottom 2
Big majority (73%)
ranked time spent in
Mfg bottom 2
C-1 Challenges Over
Next 2 Years
Many saw challenges in aligning
with overall corporate business
strategy, collaborating with other
business entities and cost
reduction;
Not as many saw challenges in
the availability of tools &
techniques.
Many less saw the same
challenges;
Half saw challenges in
the availability of tools
& techniques.

164
Table F.2, continued
C-3 RM Incidents in
Next 12 Months
Recalls more unlikely (44%)
than likely (25%)
Fields Corrections more unlikely
(37%) than likely (24%)
Stop more likely (38%) than
unlikely (33%)
Recalls more likely
(43%) than unlikely
(35%)
Fields Corrections more
likely (41%) than
unlikely (30%)
Stop Shipments more
unlikely (43%) than
likely (28%)
C-3 Corrective Action
System
A bit more people do not use
lessons learned
More do not have a repository of
root cause analyses (55% vs
45%)
More felt root cause analyses not
helpful (60% vs 40%)
More people (62%)
used lessons learned
More have a repository
of root cause analyses
(52% vs 48%)
More felt root cause
analyses helpful (58%
vs 42%)

B-1 Culture: Some differences between the two groups in risk communication methods and perceived
risk attitudes. RM Staff ranked top management broadcast top 3 in effectiveness; however, Executive
ranked otherwise. Majority of the RM Staff felt a moderate level of uncertainty avoidance; Executive
group felt either moderate or high level of uncertainty avoidance.

B-3 Process: Opposing views on challenges when implementing the RM system. The Executive group
more often disagreed on challenges such as lack of agreed vision for implementation, organizational
barriers preventing understanding of regulatory expectations, lack of integrated mgmt across system
elements, and stifled professional difference of opinions; RM Staff were split (similar %) on these
issues.

C-1 Resources: RM Staff (< ½) ranked time spent in Mfg top 2, while the Executive group (73%)
ranked time spent in Mfg bottom 2. RM Staff anticipated challenges in aligning with overall corporate
business strategy, collaborating with other business entities and cost reduction in the next 2 years;
smaller proportion of Executive group shared the same concerns. Executive group had more concerns
than RM staff about availability of tools and techniques.

C-3 Memory: RM Staff was generally optimistic in recalls and field actions, while the Executive
group was more pessimistic except in the area of stop shipment. In the area of corrective action
system, the RM Staff was generally responded with a negative view of the usefulness of lessons
learned, repository of root cause analyses; Executive group responded more positively.

165
Table F.2, continued

166
Table F.2, continued

167
Table F.3
Comparison: Smaller Size (<1,000 employees) versus Larger Size Companies
(>5,000 employees)
Model Category < 1,000 Employees (n=30) > 5,000 Employees (n=31)
B-1 System Changes More minor changes than
significant changes
More significant changes than
minor changes
B-1 Perceptions Equally agreed and disagreed
that RM activities were
burdensome. More felt not
enough RM training
More disagreed on risk
management activities were
burdensome. Majority felt
enough training
B-1 Uncertainty
Avoidance
More rated low level than high
level
Many more rated high level than
low level
B-3 Formal Procedures More managed risks of product
disposal informally without
procedures
More managed risks of product
disposal formally with
procedures
B-3 Challenges on RM
Implementation
Similar numbers
agreed/disagreed on compromise
to gain mgmt approval
A few more disagreed on
compromise to gain mgmt
approval
B-3 Challenges on RM
Implementation
Close to half (45%) disagreed Equally agreed and disagreed
(33% ea) that org barriers
prevented understanding of
regulatory expectations
B-3 Challenges on RM
Implementation
Similar numbers agreed and
disagreed (43%/40%) on
Characterization of RM as QM
rather than an Integrated Syst
Close to half (43%) disagreed
and a third agreed.
C-1 RM Investment A few increased already. None
would increase in 6 months. A
few in discussion
Several had increased. One
would increase in 6 months.
Only a couple in discussion.
C-1 Challenges Over
Next 2 Years
Primary challenges in
availability of tools &
techniques
Primary challenges in
collaborating with other
business entities and acquisition
of resources/talents
C-2 RM Training for Key
Employees
More rely on third-party More rely on internal expert

168
Table F.3, continued
C-3 RM Incidents in Next
12 Months
Recalls, Fields Corrections, and
Stop Shipments more unlikely
than likely.
Recalls, Fields Corrections, and
Stop Shipments more likely
than unlikely.
C-3 Lessons Learned &
Root Cause Analyses
More interested in lessons
learned. Equal number had
repository of lessons learned
Equal number responded people
use lessons learned.
More appeared had repository
of lessons learned

B-1 Culture: Some differences between the two groups in the system changes, perceptions, and risk
attitudes. Smaller size companies had more minor changes than significant changes; larger companies
had more significant changes than minor changes. Smaller size companies equally agreed or disagreed
that risk management activities were burdensome. Smaller size companies more commonly claimed
low level than high level of uncertainty avoidance; larger size companies most commonly claimed
high level of uncertainty avoidance.

B-3 Process: Smaller size companies appeared to be more informal than larger size companies in
managing product disposal risks. Smaller size companies split on importance of challenges associated
with compromise to gain mgmt approval and characterizing RM as part of QMS rather than an
integrated system; more larger size companies disagreed on these two issues. Equal number (1/3) of
larger size companies identified that organizational barriers did or did not prevent understanding of
regulatory expectations; close to half of the smaller size companies disagreed on this issue.

C-1 Resources: Larger size companies seemed to have invested more commonly than the smaller size
companies in RM. In the next 2 years, small companies anticipated to have more challenges in
availability of tools & techniques; more large companies anticipated having challenges when
collaborating with other business entities and acquisition of resources/talents.

C-2 Competence: Smaller size companies appeared to rely more on third-party training; larger size
companies appeared to rely more on internal expertise for training key employees.

C-3 Memory: Smaller size companies expected recalls, fields corrections, and stop shipments more
unlikely than likely; larger size companies expected recalls, fields corrections, and stop shipments
more likely than unlikely. More smaller size companies appeared interested in lessons learned and
equal numbers had or did not have a repository of lessons learned; larger size companies had equal
number of responses saying that people used or did not use lessons learned and more had repository of
lessons learned.

169
Table F.3, continued

170
Table F.3, continued

171
Table F.4
Comparison: High-Level versus Low-Level of Uncertainty Avoidance

Model Category
Self-declared High-level of Uncertainty
Avoidance
Self-declared Low-level of
Uncertainty Avoidance
A Company Profile Larger company size (> 1000
employees)
Smaller company size (< 100
employees)
C-1 Resource Constraints Appeared to be less prohibitive for
implementation
Appeared to be a definite concern
B-3 Fluctuating Priorities Appeared to be less concerned Appeared to be a definite concern
B-3 Compromise to gain
Mgmt Approval
Did not appear to be a concern Appeared to be a concern
B-1 Perception of
Colleagues
Majority did not feel risk management
activities burdensome
Generally felt risk management
activities burdensome
B-1 Part of Strategic Plan Significant majority used RM as a part
of the strategic plan
RM was generally not in the
strategic plan
C-1 RM Investment Current economic climate did not
appear big impact on investment.
Evidence of past, present, and future
investment.
Current economic climate
appeared curtailed investment.
Appeared lack of investment in
the past, present, & near future.
B-3 Formal Procedures Managed risks of product transfer and
disposal more formally with procedures
Did not appear to manage these
risks formally.
B-3 Use of Techniques Appeared to have more formal
procedures in managing product and
process risks.
Significantly less formal
procedures.
B-3 Seeking Outside
Expert Opinions
Rarely sought outside expertise. Frequently sought outside
expertise.
C-3 Lessons Learned &
Root Cause Analyses
Appeared significantly more interested
in lessons learned and root cause
analyses.
Appeared much less interested in
lessons learned and root cause
analyses.

B-1 Culture: Slightly different between the two groups: strategic planning & perception of risk management.

B-3 Process: Obviously different in the RM process: more focus & formality in high-level of uncertainty
avoidance group.

C-1 Resources: The high-level uncertainty avoidance group appeared to be more resourceful for risk
management implementation than the low-level uncertainty avoidance group.

C-3 Memory: The high-level uncertainty avoidance group appeared to be more conscientious in the lessons
learned and root cause analyses than the low-level uncertainty avoidance group.

172
Table F-4, continued

173
Table F.4, continued

174
Table F.4, continued

Abstract (if available)

Abstract This survey analysis examined current practices related to the implementation of risk management in medical device companies. Twenty-six of the Top 30 (by global market revenues) and twenty-seven other medical device companies with direct product sales in the U.S. market participated in the study. Through a literature analysis, the researcher developed a systematic framework, the “Behavior+Capability” model, to explore risk management attitudes and practices of a medical device company according to six dimensions. This framework was used to construct a forty-question survey instrument for the study. Most respondents to the survey classified their companies as relatively risk averse. Most viewed their risk management systems as satisfactory but some elements, such as risk communication, independence of risk managers, breadth of risk tools and methods to capture lessons learned, were not well-developed. Respondents in most companies and particularly the largest companies judged their risk management systems as relatively immune to the current economic climate. Most systems appeared to be highly driven by regulatory requirements rather than business imperatives. ❧ Results suggested that initiatives to achieve best practices in risk management might be improved by incorporating more advanced approaches and applications of risk management tools and techniques. More research on specific aspects of weakness identified for some companies might elucidate root causes for the challenges. Risk management systems might be enhanced further in medical device companies by expansion of training for typical in-house activities to include outside certification or graduate programs, or by introducing risk management as part of curricula in engineering or regulatory science programs.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Risk management and recalls: a survey of medical device manufacturers

PDF

Implementation of unique device identification in the medical device industry: a survey of the change management experience

PDF

Risk approaches and standards used in hospitals: a survey of industry views

PDF

Current practices in biocompatibility assessment of medical devices

PDF

Regulatory team development in post-merger integration: a survey of views from medical product companies

PDF

Using telemetry to ensure safe and reliable medical device operation: experience with defibrillators and infusion pumps

PDF

Design control for software medical devices: an industry survey of views and experiences

PDF

“Regulatory” due diligence: a survey investigation of best practices in the medical products industry

PDF

Organizational communication of regulatory intelligence: a survey of the medical device industry

PDF

Continuity management in biobank operations: a survey of biobank professionals

PDF

FDA influence on advisory committees through documentation: a content analysis and survey of industry views

PDF

Reprocessing of single-use medical devices: a survey investigation comparing the views of three unheard stakeholders

PDF

Current practices in pharmaceutical container closure development

PDF

Promotion of regulated products using social media: an industry view

PDF

The impact of incomplete monographs on the OTC drug industry: a survey investigation of industry views

PDF

Challenges in the implementation of Risk Evaluation Mitigation Strategies (REMS): a survey of industry views

PDF

Views on global harmonization of pharmacopeial standards: a survey of key stakeholders

PDF

A survey analysis of transparency in three Asian regulatory agencies responsible for medical products

PDF

The role of universities in the commercialization of medical products: a survey of industry views

PDF

Validation master plans: progress of implementation within the pharmaceutical industry

Asset Metadata

Creator Chan, Tony C. (author)

Core Title Implementation of risk management in medical device companies: a survey analysis of current practices

School School of Pharmacy

Degree Doctor of Regulatory Science

Degree Program Regulatory Science

Publication Date 07/27/2012

Defense Date 07/24/2012

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag analysis,behavior,capability,healthcare management,implementation,Management,medical device,OAI-PMH Harvest,practices,risk,survey

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Richmond, Frances J. (committee chair), Booth, Thomas (committee member), Davies, Daryl L. (committee member), Loeb, Gerald E. (committee member)

Creator Email AGSM_TCHC@earthlink.net,tchan.agsm@me.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-70589

Unique identifier UC11290209

Identifier usctheses-c3-70589 (legacy record id)

Legacy Identifier etd-ChanTonyC-1039.pdf

Dmrecord 70589

Document Type Dissertation

Rights Chan, Tony C.

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA