University of Southern California Dissertations and Theses

Using telemetry to ensure safe and reliable medical device operation: experience with defibrillators and infusion pumps

(USC Thesis Other)

Using telemetry to ensure safe and reliable medical device operation: experience with defibrillators and infusion pumps

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

USING
TELEMETRY TO ENSURE SAFE AND RELIABLE MEDICAL DEVICE OPERATION:
EXPERIENCE WITH DEFIBRILLATORS AND INFUSION PUMPS

by

Carolyn M. Wright

A Dissertation Presented to the
FACULTY OF THE USC SCHOOL OF PHARMACY
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF REGULATORY SCIENCE

August 2021

Copyright 2021 Carolyn M. Wright

ii
ACKNOWLEDGEMENTS
This thesis and the entire doctorate program have been a joy to complete. I have learned so much
from those in the program and those that have supported me with this thesis. I would especially like to
thank Michael Jamieson, DRSc for encouraging me when I underwent a job and industry change that
could have completely derailed this work. I would also like to thank Dr. Frances Richmond for your
guidance and feedback throughout the process. I would like to thank those who recommended me for the
program, Barb Copeland, Jeff Dornoff and Tom Truszkowski. Your letters to the USC admission
department expressed such confidence in my capability that I’ve always felt obligated to live up to your
endorsements. Finally, I would like to thank my husband, children, parents and my sister for their
continued encouragement throughout the years.

iii
TABLE OF CONTENTS
Acknowledgements ............................................................................................................. ii
List of Tables ...................................................................................................................... v
List of Figures .................................................................................................................... vi
Abstract….. ...................................................................................................................... viii
Chapter 1. Overview ........................................................................................................... 1
2.1. Introduction .......................................................................................... 1
2.2. Statement of the Problem ..................................................................... 3
2.3. Purpose of the Study ............................................................................ 3
2.4. Importance of the Study ....................................................................... 4
2.5. Limitation, Delimitations, Assumptions .............................................. 5
2.6. Organization of Thesis ......................................................................... 6
2.7. Definitions ........................................................................................... 7
Chapter 2. Literature Review.............................................................................................. 9
2.1. Evolution of Technology and Medical Device Regulation ................. 9
2.1.1. Defining Medical Devices as a Product Class ......................... 9
2.1.2. Medical Device Amendments of 1976 .................................. 12
2.1.3. Safe Medical Devices Act ..................................................... 14
2.1.4. Software as a medical device ................................................ 20
2.2. Medical Devices and Software Innovation ........................................ 22
2.1.5. Software that is a Mobile Medical Application ..................... 24
2.1.6. Software Used to Control a Medical Device Directly ........... 26
2.3. Medical Device Risk Management .................................................... 29
2.4. Postmarketing Surveillance and Device Performance Monitoring ... 30
2.1.7. Infusion Pump Creates a Network Security Risk .................. 32
2.1.8. CT-Tube Monitoring Predicts Failures ................................. 32
2.1.9. Use of Telemetry to Detect Device Failures ......................... 33
2.5. Cyber threats: a growing concern for device performance ................ 36
2.1.10. Content of Premarket Submissions for Management of
Cybersecurity in Medical Devices – Draft Revision 2018 .... 37
2.1.11. Postmarket Management of Cybersecurity in Medical
Devices - Guidance for Industry and Food and Drug
Administration Staff 2016 ..................................................... 39
2.1.12. Cybersecurity Practices from Standards Committees ........... 40
2.6. Medical Device Telemetry Monitoring ............................................. 41
2.1.13. Medical Device Telemetry Literature Search ....................... 42
2.1.14. Telemetry for Cyber Threat Monitoring and Protection ....... 44
2.1.15. Equipment Monitoring and Maintenance .............................. 46
2.1.16. Expectations for medical device telemetry usage ................. 50
2.7. Exploring Implementation of Telemetric Monitoring ....................... 51

iv
2.1.17. Exploration – Phase 1 ............................................................ 53
2.1.18. Design and Development – Phases 2 & 3 ............................. 54
2.1.19. Business Sustaining – Phase 4............................................... 55
Chapter 3. Methodology ................................................................................................... 56
2.8. Survey Development ......................................................................... 56
2.9. Survey Validation .............................................................................. 56
2.10. Survey Deployment ........................................................................... 58
3.1.1. Participant Selection .............................................................. 58
2.11. Administration, Data Collection and Analysis .................................. 59
Chapter 4. Results ............................................................................................................. 60
2.12. Participant Responses ........................................................................ 60
2.13. Profiles and Background of Participants ........................................... 60
2.14. Types and Uses of Collected Data ..................................................... 65
2.15. Device Performance LTM Implementation ....................................... 71
4.1.1. Exploration – Phase 1 ............................................................ 72
4.1.2. Design and Development – Phase 2 & 3 ............................... 79
4.1.3. Business Sustaining – Phase 4............................................... 88
2.16. Device Performance Telemetry ......................................................... 91
4.1.4. External Defibrillators ........................................................... 91
4.1.5. Implantable Defibrillators ..................................................... 93
4.1.6. Infusion Pumps ...................................................................... 95
2.17. Cyber Threat Telemetry ................................................................... 104
2.18. Data Collected from Participants not working with Targeted
Device Types ................................................................................... 112
Chapter 5. Discussion ..................................................................................................... 116
2.19. Methodological Considerations ....................................................... 116
5.1.1. Overview of Research ......................................................... 116
5.1.2. Delimitations ....................................................................... 116
5.1.3. Limitations ........................................................................... 119
2.20. Considering Results ......................................................................... 122
5.1.4. Telemetry for Device Performance ..................................... 122
5.1.5. Cybersecurity as a Subset of Device Performance .............. 124
5.1.6. Device Performance LTM Implementation ........................ 126
5.1.7. Device Performance Monitoring Expectations ................... 130
2.21. Conclusion and Future Direction ..................................................... 134
References….. ................................................................................................................. 136
Appendicies..................................................................................................................... 145
Appendix A: Focus Group Preparation Materials ..................................... 146
Appendix B: Survey Updates – Post Focus Group .................................... 162
Appendix C: Skip Logic Testing ............................................................... 181

v
List of Tables
Table 1 Computer-related recalls broken down by failure category ..................................36
Table 2 Recognized Information Technology (IT) & Device Security Consensus
Standards ....................................................................................................41
Table 3 Medical Device Computer Related Recalls and Adverse Event Reports .............52
Table 4 Focus Group Participants ......................................................................................57
Table 5 “Other” Product Types with which Respondents had Experience .......................62
Table 6 “Other” Marketed Product Countries ...................................................................63
Table 7 “Other” Roles within the Organization .................................................................65
Table 8 “Other” Data Collected by Medical Devices ........................................................66
Table 9 Data Types Logged or LTM’d by Device Type ...................................................67
Table 10 Performance Data Types by Use and Device .....................................................68
Table 11 Design Control Process Participation by Product Type .....................................69
Table 12 Data Collection Incorporated During Design Control ........................................70
Table 13 “Other” Data Collection Incorporated During Design Control ..........................70
Table 14 “Other” Implementation Stage ............................................................................71
Table 15 Implementation Stage by Product Type..............................................................72
Table 16 “Other” Challenges Seen During Exploration ....................................................77
Table 17 “Other” Challenges Seen during Design and Development ...............................83
Table 18 “Other” Reasons for Delays ................................................................................88
Table 19 Items Provided to Business Sustaining Team .....................................................90
Table 20 “Other” Performance Data logged by External Defibrillators ............................92
Table 21 “Other” Performance Data collected by Implantable Defibrillators ...................94
Table 22 “Other” Performance Data collected by Infusion Pumps ...................................96
Table 23 “Other” Methods of Presenting Performance Data .............................................98
Table 24 Performance Data Reviewers by Device Type ...................................................99
Table 25 “Other” Performance Data Use .........................................................................101
Table 26 “Other” Requirements that Telemetry Performance Monitoring Satisfies .......103
Table 27 Cyber Threat Data that were Logged, Transmitted and Monitored ..................104
Table 28 Expectations for the Monitoring of Cyber Threats ...........................................105
Table 29 Methods of Presenting Cyber Threat Data to Users .........................................106
Table 30 Use of Cyber Threat Monitoring Data ..............................................................107
Table 31 Demographics of Non-Infusion Pump and Defibrillator Participants ..............112
Table 32 “Other” Roles of Non-Infusion Pump and Defibrillator Participants ...............112

vi
List of Figures
Figure 1 Application of Design Controls to Waterfall Design Process .............................18
Figure 2 A Schematic Representation of the Risk Management Process ..........................20
Figure 3 Medical Device Risk Management Process Stage Focus ....................................30
Figure 4 Number of Computer-related and Non-related Recall Events Per Year. ............34
Figure 5 Computer-Related Medical Device Recalls, Impacted Devices, 2006-2011 ......35
Figure 6 Software Risk Assessment Process .....................................................................38
Figure 7 Maintenance Evolution ........................................................................................47
Figure 8 Implementation Stages and Supporting Drivers ..................................................53
Figure 9 Years of Experience ............................................................................................61
Figure 10 Product Type with which Respondents had Experience ...................................62
Figure 11 Geographic Reach of Marketed Products ..........................................................63
Figure 12 Company Size Represented by Respondents ....................................................63
Figure 13 Company Size for Different Product Types ......................................................64
Figure 14 Organizational Roles Held by Respondents ......................................................65
Figure 15 Data Types Logged or LTM’d ..........................................................................66
Figure 16 Implementation Stage of Devices on which Respondents Reported .................71
Figure 17 Exploration Stage Document Usefulness ..........................................................74
Figure 18 Challenges for Telemetry Exploration ..............................................................76
Figure 19 Items Created Before Proceeding to the Design & Development Stage ...........79
Figure 20 Importance of Factors associated with Design and Development .....................81
Figure 21 Difficulties during the Design & Development Stage .......................................83
Figure 22 Valuable Actions and Items During Implementation ........................................85
Figure 23 Incidence of Project Delays or Cancellations ....................................................86
Figure 24 Magnitude of Implementation Delay.................................................................86
Figure 25 Reasons for Delays in Implementation ..............................................................88
Figure 26 Data Types collected by External Defibrillator Companies ..............................92
Figure 27 Data Types collected by Implantable Defibrillator Companies ........................94
Figure 28 Data Types collected by Infusion Pump Companies .........................................96
Figure 29 Performance Data Presented to the User ...........................................................97
Figure 30 Expectations for the Review of Performance Data ...........................................98
Figure 31 Performance Data Usage .................................................................................100
Figure 32 Benefits from Performance Data .....................................................................102
Figure 33 Requirements Satisfied by Telemetered Performance Monitoring .................103
Figure 34 Should Telemetry Cyber Threat Monitoring Be Required by Regulators? .....108
Figure 35 Products That Should Utilize Cyber Threat Monitoring .................................109

vii
Figure 36 Benefits of Cyber Threat Monitoring ..............................................................110
Figure 37 Cyber Threat Monitoring Provides or helps to Satisfy ....................................111
Figure 38 “Other” Cyber Threat Data Logged or LTM’d ...............................................113
Figure 39 Does Increased Understanding of Device Performance Justify Cost? ............113
Figure 40 Processes that Should Be Considered During the Risk Assessment ...............114
Figure 41 Additions to Postmarket Surveillance Programs for Telemetry Capable
Devices .....................................................................................................115

viii
ABSTRACT…..
Most medical product regulations focus on premarket activities, to assure that innovative health
care tools enter the market only after their risks and benefits have been assessed. However, those products
must also be monitored after market entry to ensure that they remain safe and effective over time. A
special set of risks and opportunities exist for electronic devices that can transmit data by telemetry. Such
devices are vulnerable to mechanical problems, software errors or cyber-attack but these risks can often
be monitored and mitigated using telemetered data to trigger corrective actions. This exploratory study
assessed how device telemetry is currently being collected and monitored by defibrillator and infusion
pump companies to ensure the safe and effective operation of their devices. A survey was designed based
on a modified implementation framework and was disseminated to medical device industry professionals
involved in different aspects of product development or field operations. The results from this survey
indicate that many manufacturers are monitoring performance aspects of their products using telemetry
data without specific requirements from regulators. However, adoption of this capability is not universal.
Further, telemetered data appears to be rarely used to detect cyber intrusions, despite much regulatory
attention to this vulnerability. Results suggest that regulators could add value by clarifying and perhaps
expanding their expectations related to device telemetry usage related to risk management, cybersecurity
and post-market surveillance.

1
Chapter 1. Overview
2.1. Introduction
Medical devices have been used to improve human health for more than a century. However,
their regulatory history is much shorter. The landmark Medical Device Amendments (MDA) of 1976
updated the Food Drug and Cosmetics Act, so that the United States Food and Drug Administration
(FDA) would have the authority to classify and control the distribution of medical devices based on their
complexity and risk to public health. (94th U.S. Congress, 1975) At that time, however, most marketed
devices were only beginning to incorporate sophisticated elements such as electronic components and
software. Today, a broad range of medical devices contain and are controlled by electronics and software.
In some cases, devices communicate interactively with other devices or remote sites, with the more
advanced devices having the ability to be adjusted from those remote sites based on device telemetry.
Fourteen years after the MDA were enacted, the Food and Drug Administration (FDA) took an
important step to draw attention to the safety challenges associated with complex medical devices by
publishing the 1990 report titled “Device Recalls: A Study of Quality Problems”. In that report, FDA
identified that approximately 44% of medical device recalls could be attributed to faults in the design of
the device. (DeMarco, 2011) The report prompted the development of the “Design Control Guidance for
Medical Device Manufacturers” which became effective in 1994. This guidance clarified several
expectations associated with the design, risk management, control and revision of product specifications
and implementations. It laid the groundwork for what regulatory bodies expected of companies in the
design phase, although it stopped short of providing rules to specify what was needed from a design
standpoint. These guidelines were not directed specifically at software; instead they were generalized to
include the development and integration all parts of the medical device. (FDA, 1997)

2
The rapid advancement of radio frequency (RF) biomedical technology, the availability of cloud-
based data storage, and the low cost of increasingly capable sensors encouraged a new facet of industry,
called the Internet of Things (IoT). Medical device industries were not excluded from the use of these
new and powerful tools introduced by the IoT movement. IoT technology allowed devices to “talk to the
cloud”, in order to send data on their location, performance, consumer and patient biometrics and
behaviors. High-powered analysis tools then enabled companies to evaluate the much larger collections of
data housed in the cloud. The phrase, “data is the new gold”, has been coined to capture the importance of
this transmitted telemetry to improve capability, performance and profitability. The use and sale of such
data for targeted consumer marketing is perhaps the most visible part of the telemetry utilization.
However, its health-related uses include a role in patient, device and security monitoring, in order to
ensure safe and reliable patient care.
The area that has had the most attention from regulators is that of data confidentiality, reflected in
the Health Insurance Portability and Accountability Act (HIPAA) in the U.S. and the General Data
Protection Regulation (GDPR) in Europe. These rules limit infringement on personal privacy and set
expectations on acceptable and appropriate use of this newly available data. However, under these rules
the protections to assure confidentiality only extend to personally identifiable information or a person’s
health records. For other aspects of medical device telemetry, such as software malfunctions and cyber
threats, the industry has been guided at a high level by regulations, guidance documents and international
standards for design, risk, complaint and corrective and preventive action (CAPA) management.
(European Commission, 2016) It was not until 1990 that the FDA started to clarify their expectations for
manufacturers to take steps to protect the medical devices from cybersecurity risks. The requirements of
the Safe Medical Devices Act led FDA to update its guidance document, “Content of Premarket
Submission for Management of Cybersecurity in Medical Devices”, to assist industry regarding its
thinking in this regard. Although these documents are helpful to establish approaches to improve the

3
safety of medical devices, they provide relatively little guidance on when and how to use available
telemetry to ensure that a device continues to meet its intended use as it ages.
2.2. Statement of the Problem
Marketed devices are now capable of sending data to cloud-based storage locations. This
telemetry includes user identification and biomedical information, habits, device performance, cyber
signals, alerts and many other items that can be analyzed by those with access to the data. The need to
manage this information effectively is made clear in the currently available device regulations, guidance
documents and standards. However, those documents are written at a relatively high level and thus leave
medical device manufacturers with little specific information on the regulatory expectations for collecting
and proactively monitoring telemetry from their devices as well as what they should do to manage and
protect that information.
FDA’s Adverse Event database for the years 2006-2011 has shown that, of the total 1,210
computer-related recalls of medical devices that affected 12,024,863 devices, 35.7% of the recalls were
associated with non-software fault causes which impacted a total 9,721,395 devices (81%). (Alemzadeh et
al., 2013) It is therefore reasonable to ask whether extending postmarket monitoring requirements to
include medical device telemetry would improve the performance and safety of those devices and thus the
safety of patients using them. The research presented here sought to understand what types of useful
medical device telemetry was available and being collected as well as how the data was being used to
maintain and protect equipment over its lifecycle.
2.3. Purpose of the Study
The purpose of this study is to gain a more complete understanding of the types of device
telemetry that were being collected, monitored, analyzed and used to assure the effectiveness of devices
by medical device manufacturers and health care providers. This study has two parts. First, an

4
examination of existing literature establishes a baseline reflecting the current state of knowledge and
views about the use of transmitted data to monitor patient health and device performance. Second, a
survey, built on this foundation, was developed to explore further if and how well companies were using
their data to monitor device performance.
The survey was developed by referencing a modified framework developed by Fixsen (Fixsen et
al., 2015) to structure questions about the efforts and challenges that occurred as companies considered
and then in some cases advanced the implementation of telemetry monitoring to address postmarket
performance. History has shown that two particular types of medical devices, defibrillators and infusion
pumps, have been particularly vulnerable to failures and recalls. Thus, those products were used as test
cases, and the survey was then distributed to professionals who worked for defibrillator or infusion pump
manufacturers over the last 3 years. The targeted participants included software, system, service, clinical,
regulatory and quality engineers who supported the product development or business sustaining efforts
for medical device manufacturers. Given that different subsectors of the device industry have different
constraints on the use of such data and concerns related to the risks of their specific products, respondents
were sub-stratified by the size of the company at which they were employed and the type and
characteristics of the products on which they had worked.
2.4. Importance of the Study
The results from this study are valuable to 4 groups of stakeholders:
• risk management experts in the medical device area who put into practice the use of device
telemetry to manage risk;
• companies that currently have developed or are in the process of developing medical devices that
incorporate telemetry-based software or those that have software that is classified as a medical
device.

5
• patients who are concerned about how the data from their wireless medical device is being used;
• regulators who are trying to guide companies on best practices related to the use of medical
device telemetry in support of risk-based monitoring and postmarket surveillance.
The information collected in this study helps to illuminate the current practices and techniques for
using telemetry so that staff involved in the development of medical device products can better
understand the options available for telemetry monitoring. The work involved in establishing a robust
monitoring process is difficult and can only be accomplished if there is a substantial potential benefit to
the patient or medical staff. This survey sought to understand whether companies were realizing this
benefit and using the new capabilities in the postmarket surveillance process to reduce patient risk.
2.5. Limitation, Delimitations, Assumptions
This study had several delimitations. First, it focused on medical devices included in the scope of
devices called out in the FDA guidance document “Postmarket Management of Cybersecurity in Medical
Devices”:
any marketed and distributed medical device including: 1) medical devices that
contain software (including firmware) or programmable logic; and 2) software that is
a medical device, including mobile medical applications. In addition, this guidance
applies to medical devices that are considered part of an interoperable system and to
“legacy devices,” i.e., devices that are already on the market or in use. (FDA, 2016)
Therefore drug, biologic and combination products that were not capable of transmitting
telemetry were excluded from this survey. The survey was also delimited to companies that manufactured
or distributed defibrillators and infusion pumps in the United States. The primary target recipients of this
survey were the staff and leadership of the medical device industry (employees, contractors or
consultants) from the software, system, service, clinical, regulatory and quality departments who support
product development or business sustaining efforts that were knowledgeable about the product’s
telemetry, or postmarket surveillance.

6
Limitations on the effectiveness of this survey included the concern that detailed information on
telemetry from medical devices might be considered part of a company’s intellectual property and
therefore might not be published or shared by the survey respondents. Data Science, as a discipline, is
also relatively new in the medical products industry so this limited the amount and types of information
that were available in the literature, particularly since the industry practitioners were not typically
encouraged to write and publish scholarly work. This limited the foundational information available to
inform the research and the construction of the survey. Further, surveys are often not popular amongst
busy professionals, the target audience, so response rate might have been limited by the interest and
participation of industry members requested to complete the survey. Because I anticipated that
participants had limited time to allocate to a survey, the length of the survey was constrained to reduce
survey fatigue which could have limited the ability to probe deeper into certain areas. A survey offers
only a limited opportunity to explore the views and challenges facing respondents, because the length of
the survey and the detail of the questions was restricted. Thus, the depth of answers may have been
reduced. However, text boxes were included to solicit further comment. It was also possible that some
respondents may have been uncertain as to whether they should answer the questions from the point of
view of the company or from their own personal experiences. A focus group of experts was used to
critique the survey tool and address ways to minimize these latter limitations to the extent possible.
2.6. Organization of Thesis
The outlined research contains five (5) chapters. Chapter 1 provides an overview of the
background leading to the research question as well as the overall research approach. Chapter 2 provides
a detailed review of the literature that describes the history and present state of medical device
development, risk management and telemetry monitoring for computer-related medical devices. This
chapter highlights currently documented regulations and standards as well as industry technology
advances that have enabled the transmission of medical device telemetry. It provides examples of the use

7
of telemetry as a tool for device performance monitoring. Chapter 3 outlines the methods used to evaluate
the telemetry monitoring methods in the medical device industry. Chapter 4 provides results of the
research. Chapter 5 contains a discussion of the research results, conclusions, and the path forward for
additional research.
2.7. Definitions
Cloud-based Data Storage - is a model of data storage in which the digital data is stored on physical
servers (often multiple servers in remote locations), which are typically owned and managed by a hosting
company. Manufacturers often buy or lease data storage capacity from these hosting companies who are
responsible for keeping the data available and accessible.
AE – is an acronym for adverse event, which is any undesirable experience associated with the use of a
medical product in a patient.
CAPA – is an acronym for corrective and preventive action
Electromechanical – is a mechanical device that is electrically operated.
IoT – Internet of Things – is an industry accepted definition of products that can transmit data, telemetry,
to the internet or a storage location.
LTM – is abbreviation for the process of logging, transmitting and monitoring of data. LTM is analogous
to telemetry.
Performance Monitoring – is the process of systematic data collection and evaluation to identify changes
in the performance of a system or its components, such that remedial action may be planned in a cost-
effective manner to maintain reliability.
QSR – is an acronym for quality system regulation

8
Telemetry - is the automatic recording of instrument readings and the wireless transmission of that data to
remote locations.
WI – is the abbreviation for weighted index, a method of identifying the average score given different
weights to each score.

9
Chapter 2. Literature Review
2.1. Evolution of Technology and Medical Device Regulation
Medical devices are capable of great good but also great harm. Today, the risks associated with
such products are magnified if the devices incorporate sophisticated elements such as software. These
kinds of advancements were unimaginable when the earliest medical products regulations were
developed. For example, the first federal food and drug law, the Food and Drugs Act of 1906, banned
adulterated or misbranded food or drugs from interstate commerce, but failed to include medical devices
altogether.
The earliest medical devices on the market were typically mechanical in nature because power
was not available in homes or offices. By the late 1800s, the discovery of electricity and development of
infrastructure ushered in the second industrial revolution, which allowed engineers to build machinery for
mass production. Nevertheless, industries were not highly mechanized. Equipment was simple and thus
relatively easy to maintain and repair. Health care facilities at this time used machines in a limited way
for the treatment of their patients. These early medical devices were managed with simple maintenance
and repair strategies designed to replace worn parts or lubricate joints, for example.
2.1.1. Defining Medical Devices as a Product Class
In 1938, the Federal Food, Drug, and Cosmetic Act gave the government expanded authority to
regulate foods and drugs, including the expectation that the FDA would require premarket approval of
new drugs to substantiate their safety. In addition, and for the first time, the law authorized the regulation
of cosmetics and medical devices. This Act included the definition of a "device" to mean any…
…instrument, apparatus, or contrivance, including any of its components, parts, and
accessories, intended for use in the diagnosis of disease or other conditions, or in the
cure, mitigation, treatment, or prevention of disease in man or other animals, or
intended to affect the structure or any function of the body of man or other animals.
(FDA, 2017b)

10
The inclusion of medical devices as a separate category was intended to provide oversight for
both “quack” and legitimate devices; however, no requirement was put into place for industry to have
premarket interactions with the regulators. This absence was not particularly surprising. Most legitimate
devices on the market were relatively simple. A reviewer with a basic knowledge of science or
engineering could easily recognize an adulterated or malfunctioning device. When the FDA’s staff
attended to medical devices at all, they were primarily concerned with the commercial availability of
grossly hazardous or misbranded products. Further, the FDA could only exercise its oversight after the
devices were introduced to the market and had been transported across state lines, something that was
only beginning to happen through catalogue sales. (ISO, 2015)
The situation began to change in the middle of the twentieth century. The widespread availability
of reliable power in the late 1940s was important because it allowed a renaissance in computing that
advanced the technological capabilities of industry, and by extension, of medical products. A “third
industrial revolution” (Radziwill, 2018) began at the end of the 1960s with the invention of the
programmable logic controller. This critical computing component became one of the primary
advancements in the development of digital computers, the internet and eventually the availability of e-
commerce sites. (Radziwill, 2018) Progress in design engineering, coupled with new developments in the
electronics, plastics, metallurgical and the ceramics industries, drove the invention of a variety of medical
devices that were no longer simple. Some examples include pacemakers, dialysis machines, defibrillators
and heart valves.
The regulatory agencies were challenged to keep pace with the accelerating rate of technology
change in the medical device industry. Regulatory agencies saw a growing gap in their ability to provide
oversight of the medical device industry, putting patient safety at risk. One of the primary reasons that the
oversight gap was growing was that the version of the 1938 Federal Food, Drug, and Cosmetic Act in
place in the late 1960s did not explicitly call out premarket or commercial review requirements for
medical devices. Medical device companies could therefore pursue all kinds of experimental approaches

11
that lacked adequate design review, clinical testing and quality control. This resulted in a series of public
health scandals associated with the use of dangerous products such as intrauterine devices and contact
lenses. (Committee on the Public Health Effectiveness of the FDA 510(k) Clearance Process & Board on
Population Health and Public Health Practice, 2011) In 1969, the Secretary of Health, Education, and
Welfare convened a medical device study group composed of experts in medicine and technology to
assess the hazards associated with this regulation gap. The Cooper Committee, as it became known,
conducted extensive literature searches and held meetings with representatives of the medical profession,
industry, patients, and government agencies. (Study Group on Medical Devices Department of Health
Education and Welfare, 1970) The data collected in this broad assessment provided a graphic picture of
an unsafe medical device environment, in which 10,000 injuries could be related directly to defective or
dangerous medical devices over a ten-year period. Of these injuries, 751 had proved fatal. More
specifically, 512 deaths and 300 injuries could be attributed to heart valves, 89 deaths and 186 injuries to
heart pacemakers, and 10 deaths and 8,000 injuries to intrauterine devices. (94th U.S. Congress, 1975)
The Cooper Committee completed its research in mid-1970 and its report was made public in
September 1970. The report emphasized…
… that problems do in fact exist and that a predictable increase in the complexity and
sophistication of medical devices requires action now to prevent the emergence of
even more serious and complex problems in the foreseeable future .... the study group
believes that present and potential hazards, and the need for reliability and
effectiveness of devices necessitates explicit legislation. (94th U.S. Congress, 1975)
The committee was also tasked with developing a new framework for more comprehensive
medical device legislation. One of the recommendations of the study group was that medical devices be
classified into three regulatory categories based on product complexity and patient risk. Products in
higher risk categories would be subject to more rigorous regulatory controls than those in lower
categories. In addition, the Cooper Committee recommended that a separate department, specific to
medical devices, be created within the FDA. That department should be granted regulatory authority to

12
ensure the efficient and effective implementation of the proposed legislative requirements. Amongst its
suggested responsibilities was ensuring that industry adhered to good manufacturing practices (GMP),
reported product defects, and maintained adequate records. To facilitate that oversight, device companies
would be required to register with the FDA, so that the FDA could carry out inspections of their
manufacturing facilities. Perhaps the recommendation that would prove most impactful was the
suggestion that FDA develop a review mechanism for medical devices that would differ from the process
that was already in place for drugs, with the specific focus of protecting patients and other users. (94th
U.S. Congress, 1975)
2.1.2. Medical Device Amendments of 1976
Soon after the Cooper Committee report was made available, Congress passed the Medical
Device Amendments of 1976. This legislation marked the beginning of a new era in medical device
regulation in the United States. The MDA established for the first time a comprehensive scheme for the
premarket and postmarket regulation of medical devices that classified the devices using a risk-based
tripartite system.
Class I devices would be those with the lowest risk. For these devices, regulators believed that general
postmarketing controls would be sufficient to provide reasonable assurance of safety and effectiveness.
Class III devices would include the most complex and highest risk medical devices that were used for
supporting or sustaining life. These devices would be subject to the most stringent controls and require
FDA Premarket Approval (PMA) prior to commercial sale.
Class II devices were those that did not fall into the Class I or Class III device categories because general
controls were not believed to be sufficient by themselves to provide reasonable assurance of safety and
effectiveness. Nonetheless, the risks associated with the devices could be assessed and managed by
adhering to some form of special control such as an FDA-established performance standard. These class

13
II devices were not considered to be life supporting or sustaining. Proof of compliance with those
standards and with GMPs was required as part of a premarket notification process that was less
demanding that that of premarket approval for Class III devices.
The MDA also authorized the FDA to review all types of medical devices already on the market
in 1976 and to either ban them as unsafe devices or regulate them according to the guidelines above. To
do this, FDA created review panels to evaluate and classify the pre-amendment devices. For those
assigned to Class III, a PMA application would be required in order to demonstrate safety and
effectiveness. For those assigned to Class I or II, postmarket activities would be enough to address safety
and effectiveness issues on a product-specific basis. The expectation was that after all the existing devices
of 1976 were reviewed and classified, new devices would be automatically placed in the Class III
category unless the FDA was convinced that the device provided less risk to the patient and should be
reduced to a Class II or Class I category.
As the implementation of the MDA progressed, the FDA began to recognize the magnitude of the
work imposed on them by the regulation. Particularly worrisome was the time it would take to complete
the classification of all current and new devices. Thus, the FDA introduced modifications built on a risk-
based approach to prioritize activities related to medical device approvals. One specific process that was
adopted for the Class I and Class II products was the prenotification pathway, also known as the “510(k)”
pathway, named after MDA Section 510(k). Using this pathway, a new device could be proven to be
“substantially equivalent” to a pre-amendment device or a post-amendment class I or II device and could,
therefore, enter the market with the same classification and controls as its predicate device. The
Government Accountability Office (GAO) estimates that 31% of the medical devices that entered the
market between the years 2003-2007 use the “510(k)” submission process. (Committee on the Public
Health Effectiveness of the FDA 510(k) Clearance Process & Board on Population Health and Public
Health Practice, 2011) (Shapiro, 2013)

14
2.1.3. Safe Medical Devices Act
As experience was gained with the administration of the MDA, regulators became aware that
certain gaps were still present in the medical device review and monitoring processes. These gaps were
made clearer in February 1989 when a report titled “Medical Devices: FDA’s Implementation of the
Medical Device Reporting Regulation” was released by the General Accountability Office (GAO). This
report, commissioned to examine the FDA’s implementation of the MDR regulation, uncovered several
significant weaknesses in the medical device reporting (MDR) system, mostly related to its capability to
handle the seven-fold increase in complaints associated with the MDR. Some of the major concerns
included the significant delays in complaint reporting and analysis, the low number of complaints being
reported when an injury or death occurred, and the high number of small companies that were not aware
of the reporting requirements. (GAO, 1989) These weaknesses led the GAO to conclude that that the full
potential of the MDA had not been realized. The need for a fuller implementation was also made clear by
a highly publicized deaths associated with the three failures below. (Committee on the Public Health
Effectiveness of the FDA 510(k) Clearance Process & Board on Population Health and Public Health
Practice, 2011)
2.1.3.1. Heart Valve Failures
The Bjork-Shiley Convexo-Concave heart valve was a medical product whose failure created a
major public health crisis. This product had gone through a number of design changes; one of particular
importance occurred in the late 1970s when the device structure was reengineered with lighter and more
fragile struts connecting the valve ring to the moveable leaflet of the valve. This change was made to
improve the manufacturability of the heart valve and to reduce the possibility of thrombus formation.
(Sun, 2012) The new device was approved because the design change promised to reduce the product’s
overall adverse events profile. However, the hidden danger of metal fatigue in the struts was not
anticipated until these struts began to fracture catastrophically in the implanted patients. Data from 2012
indicates that approximately 82,000 valves had been implanted worldwide, and of these, over 500 cases of

15
fractures were reported, and two thirds of those fractures resulted in patient death. Of particular concern
was the fact that the patients who had received this particular design were not tracked and therefore could
not be differentiated from others with different types or brands of heart valves. (Sun, 2012)
2.1.3.2. Apnea monitors
Apnea monitors were first introduced pre-MDA in the mid-1960s, for the management of apnea
in premature infants. The hypothesis that apnea was a precursor to Sudden Infant Death Syndrome (SIDS)
was first proposed in 1972. Manufacturers were encouraged to develop such products to monitor infants
in hopes of preventing SIDS. However, performance failures that caused infant electrocution raised
concerns about the design of these infant monitors. (Flannery, 1991) At the same time, the usefulness of
those monitors came into question. A task force on Prolonged Infantile Apnea was therefore created in the
early 1980s to evaluate the efficacy of these monitors at preventing SIDS. That task force concluded in a
1985 statement that “a causal relationship between prolonged apnea and SIDS has not been established.”
Therefore, in addition to the design flaw that posed serious health risks, the apnea monitors could not
even claim to be effective in preventing SIDS. (Committee on Fetus and Newborn, 2003) (Chapman,
1994, Flannery, 1991)
2.1.3.3. Faulty Jaw Implants
Faulty jaw implants were one of the most prevalent reports to come to the attention of regulators
in 1991. These medical devices were made by a company called Vitek that manufactured their implants
out of a fluorocarbon polymer, Proplast. These devices had not been reviewed prior to entering the market
in 1983 because they were considered to be substantially equivalent to a product that was already on the
market. The patients were told that these implants would relieve their joint pain, but many of these
defective implants disintegrated in response to the motion produced when the patient chewed.
(Uzdavines, 2018) Unfortunately, sixty to eighty thousand patients who already suffered from

16
temporomandibular joint disorders were at further risk of having a more serious loss of jaw function due
to these low-quality implants. (Forman, 1992)
In response to these and a growing number of other device concerns, Congress introduced the
Safe Medical Devices Act of 1990. This Act expanded the authority of the FDA to require device recalls
of dangerous products. It also increased tracking, surveillance and reporting requirements for
manufacturers of high-risk devices and added healthcare facilities and distributors to the list of entities
that had obligations to report medical device injuries and malfunctions. (Flannery, 1991)
Some of the events described above also highlight deficiencies in the oversight of the medical
device product development process. Until the 1990s, medical device production was expected to adhere
to GMPs, that were originally developed to assure the quality of manufactured drugs. Thus, the quality
system that was imposed for medical devices was limited to manufacturing only. These quality standards
were not able to provide oversight to the design process of medical devices which started with the
identification and combination of components and ended after the manufacturing processes was
completed. A report titled “Device Recalls: A Study of Quality Problems'', published by the FDA,
described an evaluation of medical device recalls that occurred from October 1983 through September
1989. In this study, the FDA found that approximately 44 percent of the quality problems that led to recall
actions were attributed to design errors or deficiencies. These design-related defects involved both
noncritical devices (e.g., patient chair lifts, in vitro diagnostics, and administration sets) and critical
devices (e.g., pacemakers and ventilators). Similar conclusions were drawn in a second report published
in 1990 by the Inspector General in the Department of Health and Human Services titled ``FDA Medical
Device Regulation from Premarket Review to Recall''. Importantly for this review, the data on which they
based some of their conclusions came from an evaluation specific to software-related recalls from 1983 to
1991. That evidence suggested that over 90 percent of computer-related device failures were due to
design-related errors, often related to a failure to validate software prior to routine production. (FDA,
1996)

17
In order to eliminate these types of design failures, the Safe Medical Devices Act instructed the
FDA to update its device GMP requirements by including requirements for preproduction design
validation. The FDA formally implemented this requirement as Title 21of the Code of Federal Regulation
part 820 Quality System Regulation (QSR) which was broader than conventional drug GMPs. (Flannery,
1991) The FDA had identified, in the QSR preamble, the areas of Design Control and Corrective and
Preventative Action as two of the top four deficiency areas found during inspections regardless of
company size. (FDA, 1996) The specific need to control the design and manage the risk of a medical
device was included in section 820.30 of Title 21. It required manufacturers to establish and maintain
procedures to create design and development plans, to control and verify device design and to ensure that
the design requirements were met. The manufacturer was also expected to establish and maintain
procedures to ensure that the design requirements addressed the intended use and needs of both the users
and patients. This process was expected to include risk assessments relevant to users and patients. Further,
manufacturers were required to establish and maintain procedures to ensure that design outputs, which
were the prototypes or implementations from the design process, were verified to meet the design input
requirements and software validation success criteria. All of these new requirements were explained more
fully in a guidance document, issued in 1997, titled “Design Control Guidance for Medical Device
Manufacturers”. (FDA, 1997)
The Corrective and Preventative Action section (820.100) of the Code of Federal Regulations
(CFR) was at the same time revised to make clear that it was the manufacturer’s responsibility to assure
device quality over its entire life cycle. This implied that devices would have to be monitored and subject
to preventative and corrective actions in order to ensure that aging devices would continue to meet their
intended use. For those devices containing software, this section eventually became the foundation for
today’s cybersecurity, data protection and monitoring requirements.
The new quality systems regulations also espoused a concept that was gaining traction not just in
the US but internationally, that interventions to manage quality should be tailored to the risks associated

18
with that product. The “Design Control Guidance for Medical Device Manufacturers” was issued in 1997
to assist users as they implemented these new quality requirements by clarifying how user requirements
would be refined into system requirements and then into specifications for the product. When the design
was finally implemented in a production environment, the user requirements would define the
requirements for validations that were conducted to assure that the device met its intended use prior to
being marketed.
Figure 1 Application of Design Controls to Waterfall Design Process
(FDA, 1997)

The Design Control Guidance for Medical Device Manufacturers also laid out the expectation
that risk management should begin early in the development of the product when the user requirements
were just being created. It defined risk management as…
…the systematic application of management policies, procedures, and practices to the
tasks of identifying, analyzing, controlling, and monitoring risk. (FDA, 1997)
The risk management process was expected to become more refined as additional product and
risk information became available throughout the device’s development and commercial life cycle.

19
Perhaps the most influential organizations to support the design control and risk management
effort was the International Organization for Standardization (ISO). ISO has members from 164 countries
and has published over 22,598 International Standards in order “to facilitate the international coordination
and unification of industrial standards”. (ISO, 2019) One of the particularly significant standards to this
topic has been “ISO 14971 Medical Devices – Application of risk management to medical devices”.
(Dolan, 2004) This standard describes a process for hazard identification and risk estimation, risk
evaluation, risk control and the continual re-review of the process throughout the product lifecycle, shown
below in Figure 2 A Schematic Representation of the Risk Management Process. (ISO, 2007)

20
Figure 2 A Schematic Representation of the Risk Management Process
(ISO, 2007)

2.1.4. Software as a medical device
Part of the complexity that characterized many emerging medical devices came from the use of
computer-based technologies and software to extend the capabilities of those devices. Therefore, the
design control and related risk management requirements outlined in the new regulation and guidances
could not have come at a better time.
Risk Analysis
Risk Evaluation
Risk Control
Evaluation of Overall
Residual Risk
Acceptability
Risk Management
Report
Production and Post-
production Information

21
The very first software programs developed by industry were typically used to perform
mathematical equations at a faster rate than humans could achieve. The software functioned when it was
incorporated into specifically assembled pieces of equipment called computers. By the 1940s, computers
were being used by the government to calculate artillery ballistic tables for the U.S. Army. This
programming was done with IBM punch cards and a team of people who flipped manual switches to
change electrical signals within the computer. Depending on the complexity of the calculation, a single
program could take weeks to map, a full month to load into the computer and several additional days to
verify and debug a single calculation. (Slaughter, 2014)
The early programming languages (often called “machine language”) attempted to reproduce the
punch card process by using software instructions that would automatically flip switches but could only
be executed on specific computers. The development of a human readable programming language called
FORTRAN in 1954 marked a turning point in software functionality. Written using simple English-like
instructions, FORTRAN required a compiler to convert the instructions into machine language. This
opened the use of software to a larger group of computer users. By the mid-1970s, hundreds of software
companies were producing and selling application-specific software packages that could be bought Off-
The-Shelf (OTS). The personal computer, introduced in the 1980s, further expanded the software market
and in 1989, Tim Berners-Lee created the concept of the World Wide Web, which allowed people to
move text and data easily from one computer to another across a new communication platform called the
Internet.
The introduction of the Internet fueled software innovation like no other advancement because
now information could be passed around the world freely using online service providers. Another step
forward occurred when a Finnish university student named Linus Torvalds developed a free open-source
software operating system called Linux. This new operating system allowed individuals to create and
exchange new software without the need for a highly qualified team of paid programmers.

22
Advancements in technology did not go unnoticed by the health care industry. As early as the
1980s, medical device manufacturers began incorporating software into their devices. Software
applications that were used in these early medical devices came from other industries and were applied to
the medical device market as an afterthought. In some cases, the software was an integral part of the
equipment that added new or improved performance features. Some examples of medical devices that
contained computerized software were respirators, cardiac monitors and ultrasound equipment.
(Chapman, 1994) In other cases, the software could be used as a standalone tool and exists as a medical
device by itself, in-vitro diagnostic (IVD) software.
Medical device companies also began to recognize the usefulness of the internet as a way to
solve many problems, from connecting users with instruction manuals and product feedback, to
submissions of materials and information through email and newsletters, and eventually for the
programming and control of the medical devices themselves. However, as these early experiments in
internet usage were taking place, no regulations differentiated the specific use of the internet for medical
device applications from other uses. Instead, the guide rails for best practices developed more generally
through experience gained from software industries elsewhere.
2.2. Medical Devices and Software Innovation
It is beyond the scope of any dissertation to evaluate the full impact of computerized applications
in the universe of medical devices. In this research project, one particular application of computer-related
technology is of particular interest, and that is the way in which software and the internet are used to
communicate with devices remotely in order to monitor the performance of the device. This kind of
communication is part of an emerging field called the “internet of things”, a term that has had various
definitions:
The internet of things, or IoT, is a system of interrelated computing devices,
mechanical and digital machines, objects, animals or people that are provided with

23
unique identifiers (UIDs) and the ability to transfer data over a network without
requiring human-to-human or human-to-computer interaction. (Rouse, 2019)
The internet of things (IoT) is a computing concept that describes the idea of everyday
physical objects being connected to the internet and being able to identify themselves
to other devices. The term is closely identified with RFID as the method of
communication, although it also may include other sensor technologies, wireless
technologies or QR codes. (Technopedia, 2019)
…the networking capability that allows information to be sent to and received from
objects and devices (such as fixtures and kitchen appliances) using the Internet.
(Mirriam Webster, 2019)
The “internet of things” is a relatively novel concept. In 1990, 313,000 computers and 3 million
people had online access. However, the internet was without “things” until the Interop trade show when
the Sunbeam Radiant Control Toaster became the first “thing” connected to the internet. In a
demonstration at that show, a user in a remote location was able to activate the device using software to
send a signal to a relay switch over the internet. For the demonstration, a toaster was plugged into a
software-controlled relay switch that could provide power to the toaster when it was in the “on” position.
This action would automatically lower the bread and begin toasting. When the relay switch changed to the
“off” position, the power to the toaster would be cut off and the appliance would automatically raise the
toast. (Romkey, 2017)
Inspired by this demonstration, University of Toronto Professor Steve Mann created software that
was used with a wearable camera and computer system that assembled and transferred the images to a
remote computer that displayed the images. (Mann, 1995) These simple software applications drove a
wave of interest and invention among designers in the medical device industry that resulted in a range of
innovations associated with internet connectivity.
The numerous resulting uses of medical device software could be characterized into the following
types:

24
• Software used for manufacturing process control
• Software used for the management of medical data, transmission, viewing and storage
• Software that is a mobile medical application that only processes data and does not have a
hardware component
• Software used to control a medical device directly
• Software purchased OTS and used as a medical device
As noted above, software can be used for many purposes. However, this research focuses on a very
specific medical device function that is generally controlled by software that is either a mobile medical
application or software that controls a medical device directly. The function that is of interest in this work
is the transmission and use of medical device telemetry.
2.1.5. Software that is a Mobile Medical Application
The healthcare industry has discovered both medicinal and financial value in allowing patients to
be mobile as they use their medical products. (Odesile, 2017) Patients can now leave the hospital while
having their biomedical signals monitored or receive medical treatment such as insulin delivery or nerve
simulation from their device. This new capability has allowed patients to take part in managing their own
care while minimizing the need for repeated office or hospital visits.
FDA issued a guidance in 2015 on “Mobile Medical Applications” (FDA, 2015) to give direction
to manufacturers of mobile medical applications that fall under its regulatory purview. The guidance
includes appendices that provide 22 types of devices that fall under the definition of mobile medical
application and gives multiple examples of those that do not. The fundamental thought behind the
guidance is that the oversight of the device is not determined by the platform but instead by the function
and intended use of the application. The guidance also recommends that the manufacturers follow
appropriate QSRs, 21 CFR part 820, when designing, developing and performing postmarket surveillance
for medical devices. Under QSR, companies are also expected to validate their software and these

25
expectations are further detailed in the 2002 guidance document, the “General Principles of Software
Validation”. This guidance document specifies that the intended use, safety and risk associated with the
software should determine the validation approach, techniques and the level of effort to be applied.
Software is not a physical entity and does not deteriorate. In fact, software is more likely to
improve with age, as defects are discovered and removed during the supported life of the software.
However, with software revision comes the risk of introducing new defects that may not be detected
immediately, and which may require deep software understanding to correct. In addition, as software ages
the manufacturers of OTS software may determine that the current version is no longer supported, leaving
it vulnerable to security risks and viruses. For this reason, FDA identified that the development and
commercial change process should be controlled even more tightly for software than for hardware. Some
of the formally required steps in the software validation process include: creation of a software
requirements and specifications; design reviews; risk and traceability analysis; methods and techniques to
detect and prevent software errors; software validation plans; procedures; attention to revalidation for post
commercialization changes; and an independent review of validation components for high risk devices.
(FDA, 2002a)
Remote monitoring of medical devices using software employs methods to connect the devices
that can leave the software vulnerable to cyber threats such as viruses and worms. These threats can risk
the safe and effective operation of the medical device. In order to protect against cyber attacks, networked
software typically must undergo maintenance upgrades (patches) throughout its life cycle. The guidance,
“Cybersecurity for Networked Medical Devices Containing Off-The-Shelf (OTS) Software” (FDA,
2005b), issued in January 2005, focused specifically on the device manufacturer’s long-term
responsibility related to the use of OTS software in their medical devices. The principles of this guidance,
of course, additionally apply to a manufacturer’s responsibility for internally developed software. The
guidance also reflects FDA’s concern about the exploitation of cybersecurity vulnerabilities and identifies
that the medical device manufacturers must be responsible to provide necessary updates to protect their

26
device software. The cybersecurity guidance reaches back to predecessor regulations and highlights
exactly where it is stipulated that manufacturers are in fact responsible for patch updates.
The need to be vigilant and responsive to cybersecurity vulnerabilities is part of your
obligation under 21 CFR 820.100 to systematically analyze sources of information
and implement actions needed to correct and prevent problems...
Under 21 CFR 820.30(g), design validation requires that devices conform to defined
user needs and intended uses, including an obligation to perform software validation
and risk analysis, where appropriate. Software changes to address cybersecurity
vulnerabilities are design changes and must be validated before approval and
issuance.(FDA, 1996)
In an effort to reduce the burden on manufacturers and to encourage timely software patches, the
FDA has also clarified that preapproval may not be needed for software changes to address cybersecurity
vulnerabilities so long as the updates do not change device performance and are not expected to have an
adverse effect on the safety and effectiveness. It is, however, necessary for the manufacturer to document
the change, the level of associated risk and the nature of design validation conducted in the Device
History File. Class III medical devices also need to have documentation of the change in the product’s
Annual Report. (FDA, 2005b)
2.1.6. Software Used to Control a Medical Device Directly
Software has been incorporated in medical devices for decades, primarily as elements embedded
in the “hardware”; however embedded software had not come under regulatory scrutiny until the 1990s.
Then, FDA performed an analysis of medical device recalls between 1992 and 1998. Its study revealed
that about 8% of the recalls could be attributed to software failures, mostly caused by changes to the
initially approved software. (FDA, 2000) Since that report, technology has increased in complexity and
frequently manufacturers are integrating OTS and open-source software with their application-specific
software. Software that is used to control a medical device can be one of the most complicated types of

27
software in the medical device universe. It is therefore subject to the basic regulatory requirements
outlined above, but also has additional, more stringent, requirements, some of which are listed below.
The FDA guidance, “Content of Premarket Submissions for Software Contained in Medical
Devices”, (FDA, 2005a) is one of the primary documents to help manufacturers to navigate the
requirements associated with the development, approval, and update of software-containing devices. It is
a broad guidance that applies to devices that are regulated under Premarket Notification (510(k),
Premarket Approval (PMA), Investigational Device Exemption (IDE) and Humanitarian Device
Exemption (HDE) pathways. This document makes clear FDA’s opinion that finished product validation
or testing is insufficient to establish safety and effectiveness. Rather, they must be assured that the
product is safe and effective through thoughtful design, systematic risk management and rigorous
verification processes. The “Content of Premarket Submission for Software Contained in Medical
Devices” is aligned with the standard, “ISO 14971 Medical Devices – Application of risk management to
medical devices”. However, it adds a new risk metric called the “Level of Concern”. This metric ranks the
potential impact of faulty software on the severity of harm, either directly or indirectly, to a patient or
operator, and helps to determine the level of software-related documentation to be included in a premarket
submission (510(k), PMA, IDE or HDE). (FDA, 2005a, ISO, 2007)
The Level of Concern should be determined through hazard analysis prior to the implementation
of any activities to mitigate those hazards. According to the FDA, the Level of Concern will be Major if a
failure or latent flaw associated with function or data transmission could result directly in death or serious
injury of the patient or operator. The Level of Concern would be considered Moderate if such a failure or
latent design flaw could result directly in a minor injury and it would be considered Minor if the failures
or flaws would be unlikely to cause injury. (FDA, 2005a)
The guidance further advises that updates to software will almost certainly be needed over the life
span of a medical device. Requirements for submission and validation of the updated software should

28
then be based on the previously established Level of Concern. The guidance also explains that the FDA
Special 510(k) program can be used by manufacturers of devices cleared under the 510(k) program when
they make a change to their medical device software, if the “modification does not alter the intended use
or the fundamental scientific technology of the device.” (FDA, 2005a)
A growing number of more advanced medical devices are being designed to use telemetry so that
device data can be shared with the manufacturer or the patient’s medical support team. Generally,
telemetry is “the automatic measurement and wireless transmission of data from remote sources”. (Rouse,
2005) A variety of approaches, including cellular telephones, computer networks or messaging, can be
used to receive and then analyze certain health or treatment parameters for patients. For example, the
insulin pump system works as an “artificial pancreas” with a blood glucose sensor that can measure and
send data, telemetry, to other system components, as well as an insulin pump that translates that data into
a command to increase or decrease the rate of insulin delivery. (Leavitt, 2010) Telemetry can also be used
to communicate data related to the performance of the device. For example, Medtronic's MyCareLink is a
system that enables healthcare providers to track the battery status and general functional state of
implanted pacemakers and cardiac resynchronization therapy pacemakers. (Kelly, 2019)
As software becomes more complex, it becomes more central to the risk profile of the device. The
FDA believes that the majority of software-related device problems now occur after the software has been
revised subsequent to commercialization. (FDA, 2005a) This concern will be an increasing topic of future
guidance because advances in technology and software are driving new and more frequent needs for
software updates and cybersecurity protection. The assurance of device functionality and safety has
become increasingly important as wireless, networked devices and the electronic movement of medical
device-related health information become more prevalent. To address this increased concern, the FDA
discusses software interfaces, networking and their component infrastructure in a 2018 draft updated
guidance, titled “Content of Premarket Submission for Management of Cybersecurity in Medical
Devices”. (FDA, 2018a) There, the FDA clarifies its expectation that manufacturers consider the

29
capabilities and limitations of these interfaces when they conduct their hazard analysis during risk
assessment and then plan deliberately to mitigate any issues that are revealed.
2.3. Medical Device Risk Management
It is clear from recent studies of recalls related to software that design issues are commonly the
principal culprit. The “Design Control Guidance for Medical Device Manufacturers” makes it clear that
the software associated with medical devices brings additional risks, so the hazards related to use of
software should be included in the risk assessment process. (FDA, 1997) A key standard to frame risk
management is “ISO 14971, Medical Device – Application of risk management to medical devices”.
(ISO, 2007) The standard notes that the analyses included in the risk plan should evolve over time as the
design is refined and prototypes are tested.
Because software is always in evolution, different considerations can be important at different
points in time. At the early design stage, FDA acknowledges in their “Guidance for Off-the Shelf
Software” that level of risk associated with software hazards will be difficult to estimate because software
failure rates will not be available. The Center for Devices and Radiological Health (CDRH) therefore
suggests that early risk management activities should focus on the severity of the harm that could result
from the software failure. Risk control measures should then be implemented in the design process to
reduce the risk of harm, should this event occur and, if possible, to begin to estimate the probability of
such an occurrence. ISO 14971 specifically calls out the need to incorporate additional controls into the
design to detect conditions of potential harm, particularly when the design cannot be improved to prevent
the hazardous condition. (ISO, 2007) Risks should then be reviewed as human factors and clinical
experience is obtained. These updated assessments should be incorporated into later risk management
reviews to ensure that the new risks are being controlled. (FDA, 1999)

30
Figure 3 Medical Device Risk Management Process Stage Focus
(ISO, 2007)

2.4. Postmarketing Surveillance and Device Performance Monitoring
Most of the literature described above identifies how to plan for a safe and efficacious product.
However, such a desirable objective cannot always be attained. In 2011, FDA published a report titled
“Medical Devices and the Public's Health: The FDA 510(k) Clearance Process at 35 Years”. In this
report, the committee documented their opinion that there were substantial weakness in the postmarketing
surveillance of medical devices. Its Recommendation 7-2 states explicitly:
Early Development
Focus of Risk
Management
Later Development
Focus of Risk
Management
= Probability x Severity of the harm
Risk Analysis
Risk Evaluation
Risk Control
Evaluation of Overall
Residual Risk
Acceptability
Risk Management
Report
Production and Post-
production Information

31
The Food and Drug Administration should develop and implement a comprehensive
strategy to collect, analyze, and act on medical-device postmarket performance
information. (Committee on the Public Health Effectiveness of the FDA 510(k)
Clearance Process & Board on Population Health and Public Health Practice, 2011)
The review committee further emphasized that the FDA should give high priority to postmarket
surveillance to improve the oversight medical device safety and effectiveness over the short and long
term. (Committee on the Public Health Effectiveness of the FDA 510(k) Clearance Process & Board on
Population Health and Public Health Practice, 2011)
If products do not perform well after they are sold, how is a company to find out about those
deficiencies as quickly as possible? Currently medical device surveillance, as conducted by the FDA, is
based on the feedback of manufacturers, caregivers or patients regarding device performance. The
methods used to collect this data typically fall into one of 3 main categories. The first is passive
surveillance, which relies on feedback from manufacturers, physicians and patients to identify patient
injury or device malfunction. This data can be used by both the manufacturer and the FDA to determine if
new actions are needed. The second category is derived from post approval clinical studies where safety
can be assessed more proactively and formally as clinical studies progress. The third is the use of real-
world data from a variety of sources including but not limited to medical product registries, social media
analyses or insurance claims databases. (Rajan Prashant et al., 2015)
These three methods provide valuable surveillance data; however, they are all reactionary and
often can only be assessed after device failure. In order to understand fully the performance of a device, it
would seem important to monitor how the unit is operating compared to its system performance
specifications during normal operation instead of waiting for a failure to occur. The manufacturing and
aerospace industries have become proactive in monitoring the performance of equipment responsible for
high precision or safety critical functions. The performance data from their equipment is analyzed and
compared to system specifications and historical performance. (Microsoft, 2018) The feedback allows
manufacturers to observe performance degradation and predict failure in advance of a breakdown, so that

32
intervention can be scheduled before the equipment breaks. (Microsoft, 2018) Similar approaches have
recently been taken for some medical products, as illustrated below.
2.1.7. Infusion Pump Creates a Network Security Risk
Infusion pumps are used extensively to provide life-supporting and life-saving medications in
many settings. However, these devices are not immune to cyber threats and need to be monitored for
improper performance. In this case, the hospital used its cybersecurity software to detect that the Alaris
infusion pump was vulnerable to a set of software weaknesses, labeled the “URGENT/11” family. These
vulnerabilities could allow hackers at remote sites to take control of the medical device and change or
block its function, introduce malware, or permit information to be leaked. The hospital’s cybersecurity
platform used telemetric signals monitored from the device to feed analytic software that could detect
unusual behavior suggesting a risk from attack. (Torrey, 2019) Without this type of detection software,
devices could be compromised for long periods of time before detected. (O'Dowd, 2020)
2.1.8. CT-Tube Monitoring Predicts Failures
As medical centers strive for better efficiency, a premium is placed on the having uninterrupted
access to their much-needed medical devices. One example is illustrated by the management of
computed/computerized tomography (CT) scanners. These are expensive pieces of equipment on which
hospitals rely for a wide range of important diagnostic applications, so it is important that they do not go
out-of-service. For this reason, medical device manufacturers have started to monitor the performance of
their CT-tubes in order to predict incipient failures before any disruption occurs. This predictive
monitoring enables the systems to be repaired at a convenient time when a change in performance
indicates the need. (Goyen, 2020) However, identifying and monitoring performance variables in order to
predict a device failure is technically challenging and not possible in all cases. High levels of skill and
experience are needed to establish product specific telemetry systems and also to recommend actions
based on the telemetry. (Fiix Inc., 2019)

33
2.1.9. Use of Telemetry to Detect Device Failures
The above examples make clear that the use of telemetry to monitor postmarket device
performance is gaining significant traction. This technological capability comes at critical time because
the medical device industry continues to see device failures that proactive monitoring might help to
identify and prevent. Alemzadeh analyzed 5,294 recalls and 1,154,451 adverse events reported to the
FDA between 2006 and 2011. She further analyzed a subset of these failures that she named computer-
related failures, and defined as “any event causing a computer-based medical device to function
improperly or present harm to patients/users”. These failures were categorized as being caused by device
software, hardware, input/output (I/O), battery or other reasons that could not be easily categorized in the
previous four categories. In this 6-year period, 1,210 (23%) of the medical device recalls could be
attributed to computer-related medical device failures (Figure 4). This number compares to previous
studies that were conducted over the years of 1983-1997 and 1999-2005 where the number of computer-
related device failures was 6% and 33.4% respectively. (Alemzadeh, 2016)

34
Figure 4 Number of Computer-related and Non-related Recall Events Per Year.
(Reprinted with permission from Data-driven resiliency assessment of medical cyber-physical
systems(Alemzadeh, 2016))

The 1,210 (23%) computer-related medical device recalls fell into multiple failure categories and
examples of the key words used to categorize the various fault categories are provided in Table 1 below.
In total, these 1,210 recalls issued impacted 12,024,836 medical devices. When the recovery actions for
these failed devices were examined, Alemzadeh found that 17.8% (2,145,087) required replacement parts
or complete removal of the device. The majority of the devices that needed replacement parts needed new
batteries (1,135,478) and the remainder needed other hardware replacements (805,868). When
specifically looking for the percentage of failures that occurred in these devices because of software
failures as compared to non-software failures, she also noted that 64% of the recalls were related to
software failures. Those recalls only impacted 19% of the units in the field. The remaining 81%
(9,721,395 devices) of the computer-related medical devices that were impacted by the recalls had other
types of failure modes, such as battery or hardware failures (Figure 5). While software failures draw most
of the attention, they impact only a small number of the devices in the market. The theoretically good
news in these statistics is that the other 81% of the medical devices with failures may have the potential to

35
be monitored telemetrically so that failures can be detected before they impact patient health. The large
number of these medical devices show the importance of non-software-related failures in terms of higher
cost for manufacturers, caregivers and patients. (Alemzadeh, 2016)
Figure 5 Computer-Related Medical Device Recalls, Impacted Devices, 2006-2011
(Alemzadeh, 2016)

35%
24%
20%
19%
2%
- 1,000 2,000 3,000 4,000 5,000
Hardware
Other
Batery
Software
Input/Output
Number of Computer-Related Medical Devices Impacted by Recalls
Thousands
Failure Category
Computer-Related Medical Devices Impacted by Recalls
2006-2011

36
Table 1 Computer-related recalls broken down by failure category
(Alemzadeh, 2016)
Failure
Categories
Example Keywords to Describe the Failure Components
Number of
Recalls
Software
Software, application, function, code, version, backup, database,
program, bug, Java, run, upgrade
778 (64.3%)
Hardware
Board, chip, hardware, processor, memory, disk, PCB, electronic,
electrical, circuit, leak, short-circuit, capacitor, transistor, resistor
179 (14.8%)
Other
Error, system, fail, verification, self-test, reboot, Web, robotic,
calculation, document, performance, workstation
142 (11.7%)
Battery Battery, power, supply, outlet, plug, power-up, discharge, charger 70 (5.8%)
Input/Output
Sensor, alarm, message, screen, signal, interface, monitor, connect,
button, scanner, key, speaker, wireless, terminal, communication
41 (3.4%)

Performance failure is an expensive problem. Maisel estimated that even two decades ago,
implanted cardioverter-defibrillators and pacemakers alone cost the healthcare system approximately $87
million per year to manage recalls and safety alerts. (Maisel et al., 2001) The adoption and use of
telemetric diagnostics from medical devices could enable a predictive approach to identifying incipient
problems at an earlier stage when cost of remediation would be lower. For example, in May of 2019,
three adverse events were reported to the FDA in which the batteries in Medtronic pacemakers became
fully depleted sooner than expected and without warning. Medtronic had developed a telemetric
monitoring system, the MyCareLink system, to track battery status and general function of the implanted
pacemaker, but the system failed to identify the depletions appropriately. One device failure was
identified prior to implantation, but the other two, were responsible for the death of one patient and the
hospitalization of the other. (Kelly, 2019)
2.5. Cyber threats: a growing concern for device performance
In 2002, more than 7,000 new viruses, Trojan horses, and computer worms were
reported, but by 2012 the number of new malicious computer programs has grown
exponentially, with reports of 200,000 new viruses detected every day! (Slaughter,
2014)

37
As more and more medical devices rely on software, an emerging risk of some concern is the
potential for malicious damage to equipment, not through wear or component failure, but through cyber
attacks. The loss of equipment functionality after such an attack can harm a patient or reduce the
availability of the equipment for important diagnostic and treatment functions. (Fu and Blum, 2013)
These risks are well illustrated by the experiences of the Department of Veterans Affairs (VA), whose
records show that at least 327 devices had been infected by malware at VA hospitals since 2009. In these
records, the VA identified more than 40 viruses that infected a range of devices including X-ray machines
and laboratory equipment. In one case, in January 2010, a VA catheterization laboratory closed
temporarily after malware-infected computer equipment could no longer be used for procedures needed to
open blocked arteries after a patient experienced a heart attack. (Weaver, 2013)
It is easiest to address cybersecurity provisions during the design of a new product. However,
cyber attack tools continue to evolve and require that even newer equipment be monitored and upgraded
throughout its life cycle to protect against attacks. (Fu and Blum, 2013) To advise on such needed
modifications, FDA has published 2 guidance documents, one to address what they think to be
appropriate actions to protect devices from cyber risks as they are being designed, and the other to
introduce cybersecurity measures for products that have already been approved or cleared.
2.1.10. Content of Premarket Submissions for Management of Cybersecurity in Medical Devices
– Draft Revision 2018
This 2018 draft guidance was originally issued in 2014 and is in the process of undergoing
revision. The updated document recommends that manufacturers address cybersecurity during the design
and development of the medical device, because this early attention can result in more a robust and
efficient mitigation of patient risks. The guidance introduced risk tiers for cybersecurity considerations
because the standard classification of devices into 3 risk categories by the MDA did not correlate with the
types of cybersecurity protection that some products needed. For purposes of addressing cybersecurity,

38
software was therefore graded in risk from highest, called Tier 1 to lowest, Tier 3. Interventions at the
lowest tier were relatively modest. However, manufacturers of Tier 1, “Higher Cybersecurity Risk”
devices had more requirements, and they were advised to establish design inputs for their device related
to cyber risks and address cybersecurity weaknesses as part of the software risk assessment process. As
those devices were designed, FDA recommended that developers appropriately address the following
elements: (1) identification of assets, threats, and weaknesses (current or future); (2) assessment of the
impact of identified threats and weaknesses on device functionality; (3) analysis of the likelihood that the
threat or weakness could be exploited; (4) determination of risk levels and mitigation strategies; (5)
assessment of residual risk after mitigation and the establishment of risk acceptance criteria.
Figure 6 Software Risk Assessment Process
Risk Analysis
Risk Evaluation
Risk Control
Evaluation of Overall
Residual Risk
Acceptability
Detect and Respond to
Dynamic Cybersecurity
risks
Identification of assets,
threats, and weaknesses
Assessment of the likelihood
of a threat and of a
vulnerability being exploited
Assessment of the impact of threats and
vulnerabilities on device functionality
and end user/patients.

The 2018 Draft Guidance specifically recommends that manufacturers refer to the FDA list of
recognized consensus standards to address cybersecurity concerns in the Information Technology (IT) and

39
medical device areas (Table 2). The guidance also reinforces the need for companies to utilize the
National Institute of Standards and Technology (NIST) “Framework for Improving Critical
Infrastructure” to manage their own cybersecurity risks. By following the NIST framework companies
can adjust their behavior so that they leverage only trusted devices and networks for their development,
manufacturing and post commercialization processes. The guidance further clarifies that in order for a
network or device to be considered trusted they must have a process for (1) device asset protection; (2)
ensuring trusted content; and (3) safeguarding data.
2.1.11. Postmarket Management of Cybersecurity in Medical Devices - Guidance for Industry
and Food and Drug Administration Staff 2016
The 2016 postmarket guidance continues the recommendations for cybersecurity monitoring and
protections into later phases of the product life cycle, no matter how much time has passed since product
approval. The FDA guidance discusses preventative measures that should be taken to reduce the risk of
cyber attacks for connected medical devices. The primary actions that are recommended by the guidance
include: designing the medical device using the recommendations outlined in the “Medical Device
Premarket Cybersecurity Guidance”; implementing the NIST “Framework for Improving Critical
Infrastructure Cybersecurity”; participating in an Information Sharing Analysis Organization (ISAO); and
implementing a cybersecurity risk management program. The guidance further clarifies that a
cybersecurity risk management process should include risk analysis, risk evaluation, risk control, and
incorporation of production and post-production information.
The first process that is required, risk analysis, contains 3 subcomponents: identification of
intended use; identification of hazards; and estimation of risk for each hazard. The identification of
hazards, specifically cyber hazards, is critical for this discussion because a method is needed to monitor
cyber activities in order to identify vulnerabilities and risks.

40
Manufacturers are encouraged to actively identify cybersecurity signals that might
affect their product, and engage with the sources that report them. (FDA, 2016)
When vulnerabilities to cyber threats are detected and mitigations are identified, the guidance
informs manufacturers that preapproval of these “patches” would not be required. Devices that have
periodic reporting requirements under 21 CFR 814.84 should include information concerning
cybersecurity vulnerabilities and any changes that are made to control risk. The report should, however,
include a description of the vulnerability, how the manufacturer became aware of the concern, and the
date and name of the ISAO to which the vulnerability was reported, if any.
2.1.12. Cybersecurity Practices from Standards Committees
Guidance documents are limited in the degree of granularity that they can provide for the many
types of connected devices and situations that might confront manufacturers. The FDA has recognized
that many domestic and international consensus standards address aspects of safety and/or effectiveness
relevant to medical devices. In fact, staff from the CDRH have in many cases participated in the
development of these standards which can address particular challenges to devices and propose more
detailed approaches to manage them. CDRH believes that conformance with recognized consensus
standards can support a reasonable assurance of safety and/or effectiveness for many applicable aspects of
medical devices. Some of the standards that have specific application to the cybersecurity area are listed
in Table 2. Conformity with recognized standards is generally recommended unless a manufacturer can
document another more appropriate method to address the issue. (FDA, 2007) Nevertheless, conformance
with recognized consensus standards is voluntary.

41
Table 2 Recognized Information Technology (IT) & Device Security Consensus Standards
FDA Recognized Consensus Standards can be found at
http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfStandards/search.cfm.
Org. Num./ Rev. Title
AAMI TIR57:2016 Principles for medical device security - Risk management.
AAMI
ANSI UL
2800-1: 2019
(American National Standard) Standard for Safety for Medical
Device Interoperability
ANSI UL
2900-1 First Edition
2017
Standard for Safety Standard for Software Cybersecurity Network-
Connectable Products Part 1: General Requirements
ANSI UL
2900-2-1 First
Edition 2017
Standard for Safety Software Cybersecurity for Network-
Connectable Products Part 2-1: Particular Requirements for
Network Connectable Components of Healthcare and Wellness
Systems
CLSI AUTO11-A2
Information Technology Security of In Vitro Diagnostic
Instruments and Software Systems; Approved Standard - Second
Edition
IEC
80001-1 Edition 1.0
2010-10
Application of risk management for IT-networks incorporating
medical devices - Part 1: Roles responsibilities and activities
IEC
82304-1 Edition 1.0
2016-10
Health software - Part 1: General requirements for product safety
IEC
TR 80001-2-2
Edition 1.0 2012-07
Application of risk management for IT Networks incorporating
medical devices - Part 2-2: Guidance for the disclosure and
communication of medical device security needs risks and
controls
IEC
TR 80001-2-3
Edition 1.0 2012-07
Application of risk management for IT Networks incorporating
medical devices - Part 2-3: Guidance for wireless networks
IEC
TR 80001-2-5
Edition 1.0 2014-12
Application of risk management for IT-networks incorporating
medical devices - Part 2-5: Application guidance - Guidance on
distributed alarm systems
IEC
TR 80001-2-8
Edition 1.0 2016-05
Application of risk management for IT-networks incorporating
medical devices - Part 2-8: Application guidance - Guidance on
standards for establishing the security capabilities identified in
IEC TR 80001-2-2
IEC
TR 80001-2-9
Edition 1.0 2017-01
Application of risk management for it-networks incorporating
medical devices - Part 2-9: Application guidance - Guidance for
use of security assurance cases to demonstrate confidence in IEC
TR 80001-2-2 security capabilities
IEC ISO
29147 First edition
2014-02-15
Information technology - Security techniques - Vulnerability
disclosure
IEC ISO
30111 First edition
2013-11-01
Information technology - Security techniques - Vulnerability
handling processes

2.6. Medical Device Telemetry Monitoring
The discussion above identifies that medical devices can fail not only because of inherent flaws
or wear, but also because of damage from outside agents engaged in cyber intrusion. To date the reported
adverse events associated with medical devices have been dominated by a multitude of device

42
malfunctions and ransomware events, but no reports have yet identified a serious injury or death due to a
cyber attack. Nonetheless an appropriate consideration of risk should reflect and address all types of
foreseeable hazards even before they cause damage. (Alexander et al., 2019) The sections below provide
two examples of performance monitoring that are currently used in the medical device industry and the
potential benefits that can be achieved if used in a proactive manner. The sections also highlights
situations where monitoring can prevent or minimize medical device adverse events or the health effects
of delayed treatment. The summary is based on an extensive search of the literature that yielded many
articles but almost all deal with its use to gather some type(s) of clinical measures reflecting the state of
the patient. In contrast, relatively little material of any depth could be found that examines the use of
telemetry for monitoring the performance of the device itself.
2.1.13. Medical Device Telemetry Literature Search
A three-phase review of current literature was conducted to assess documentary evidence related
to the use of medical device telemetry for device performance and cyber threat monitoring. The first
review was a broad search of Google Scholar using the terms “Medical Device” and “Telemetry” over the
date range of 2016-2020. This search returned 24,900 patents and citations, sorted by relevance. A review
of the most relevant 151 results included 117 patents and 34 publications. Four of these patents referenced
the transmission of performance telemetry data, one patent referenced the use of telemetry to evaluate
cybersecurity and 2 publications discussed the reaction method a device could utilize to transmit
telemetry in the case of a device transmission failure. I could find no patents or publications describing
how telemetry was used to provide feedback on cyber activity of medical devices.
A follow-up search was done using the USC library for publications from 2016 – 2020 that
contain the words “Medical Device”, “Telemetry” and “Performance”. It was determined from previous
undocumented searches that this was the broadest set of search terms that could be used to capture the
intent of the literature review without unintentionally excluding critical literature. This search resulted in

43
a total of 977 unique results that included 594 newspaper articles, 217 articles, 118 dissertations, 15 text
resources, 13 conference proceedings, 5 books, 7 book chapters, 5 reviews and 3 reference entries. The
217 articles were further sorted to identify those that were under the subject of “Medical Equipment”
resulting in 26 articles. The 594 newspaper articles were also sorted to on-line; however, the subject was
expanded to include those that were concerned with “Medical Equipment”, “Telemetry” or “Medical
Technology”. The newspaper articles that had the subject of “EU Patent Office Publications” were
excluded because that content was already captured by the Google Scholar patent search. The final USC
search yielded a total of 595 documents, including 212 newspaper articles, 217 articles, 118 dissertations,
15 text resources, 13 conference proceedings, 5 book, 7 book chapter, 5 reviews and 3 reference entries.
Fourteen percent (81/595) of the documents discussed the use of telemetry for physiological monitoring
of patients. Only 2% (14/595) of the documents discussed the use of telemetry data related to medical
device performance monitoring. However, none of the documents provided any details beyond identifying
the capability to do so. Finally, in the area of cyber threat detection only 1 document (0.2%) in the USC
library search results discussed the capability of a medical device to utilize telemetry for this purpose.
After the survey was completed and during the documentation of the discussion section a third
literature review was conducted. The counsel of a USC librarian was employed for this review to
recommend a database to be searched as well as the search criteria. At the librarian’s recommendation 4
additional multidisciplinary science and engineering databases were quired to identify literature on the
topic of telemetry usage for performance and cyber threat monitoring. The databases Scopus, Web of
Science, IEEE Explore and Engineering Village were searched to identifying related publications. The
search terms for this review were “Infusion Pump” or “Infusion Pumps” or “Defibrillator” or
“Defibrillators” and “Transmit” or “Transmitting” and “Measure” or “Measuring” and “Monitor” or
“Monitoring”. The date range of 2016-present was applied. The results of this search included 287
magazines, journals, early access and conference publications of which 39% percent (112/287) discussed
physiological monitoring of patients while 3% percent (9/287) discussed the use of medical device

44
performance monitoring and 5% percent (13/287) discussed the possibility of using telemetry for cyber
threat monitoring. However, the medical devices that were discussed in the cyber threat publications were
secondary devices that were located outside the body intended to support the security of the implanted
medical device. (Newaz et al., 2020, Rathore et al., 2017)
2.1.14. Telemetry for Cyber Threat Monitoring and Protection
The last two decades have been a golden age for telemetry. By 2006, 50% of the medical devices
on the market contained software that helped to determine the performance of a device or was itself a
medical device (called “standalone software”). The increasing complexity of these devices caused many
researchers and patients to question the security and safety of medical devices. (Burns et al., 2016)

2.1.14.1. Medical Device Network Security
Information transmitted on medical computer networks is sensitive and warrants an efficient
security system that does not impede the primary functionality of the medical devices. We know that
device functionality can be corrupted or frozen by malicious attacks, either to hold the device hostage
until a ransom is paid, or to hurt patients. Large medical device companies have been able to implement
intrusion detection software as well as anti-malware software globally but these types of software can be
expensive and labor intensive to implement. Nevertheless, the need to monitor the integrity of the
computer-based systems is critical for some forms of equipment and medical devices. For example, in
2017, a novel “cryptoworm” invaded hundreds of thousands of computers and their associated medical
devices over a period of hours in more than 150 countries. The intruders used a “backdoor” in Microsoft
software code to deliver the WannaCry ransomware to every unpatched computer on the network.
Microsoft had, just two months prior, released a security patch to protect a user’s system against this
software weakness. Many individuals and organizations did not update their operating systems and so
were left exposed. (Kaspersky, 2019) The crisis only lasted about 7 hours, from 3:30 a.m. eastern time

45
(ET) on Friday to about 10 a.m. ET, before the “cryptoworm” was shut down, sparing many US computer
networks.
Recorded Future, a vendor of threat intelligence software, was able to document 117 ransomware
incidents that targeted healthcare providers in the United States using methods much like that underlying
the WannaCry event, which affected the ability of the users to access patient health records. Such service
interruptions risked delays in critical medical procedures which was expected to would motivate
healthcare providers to pay ransoms. (Siwicki, 2019)
Part of the evolution of robust software systems has been the implementation of proactive
protections to guard against cyber threats. Advanced IT systems today have systematic risk management
methods that assure patch management, vulnerability scanning, use of antivirus software, asset
management, and cyber threat monitoring. In such systems, administrators will be notified that a
penetration has been made, so that they can identify and characterize such attempts and then harden the
system against subsequent intrusions. In less mature or “legacy” IT systems, these security solutions may
be absent or custom-crafted by local IT staff and may provide a lower level of protection and monitoring.
(Schwartz et al., 2018)
2.1.14.2. Medical Device Cybersecurity
Cybersecurity methods go beyond assuring that the network is secure to assuring that the linkages
with devices are secure. As an example of such measures, Medtronic has developed a line of
programmers (CareLink and CareLink Encore Programmers) that are used with implanted Medtronic
devices such as pacemakers, implantable cardioverter-defibrillators, cardiac resynchronization devices
and implantable monitors. The programmers utilize telemetry data to allow physicians to understand the
patient’s device performance by checking its battery status and allowing the physician to adjust or
reprogram device settings. (Alexander et al., 2019) Telemetry can also be used to protect a medical device
from power-draining cyber threats. Medical device software with this capability can analyze its telemetry

46
data and implement responsive measures to protect itself. For instance, by monitoring and analyzing the
number of unauthorized telemetry requests received from a particular external device over time, software
algorithms can identify if the request numbers exceed a defined threshold. If deemed necessary, the
software can disable its own pairing process to stop excessive communications and protect its own battery
life. (Yoder et al., 2019)
Radio frequency shielding is another method of telemetry monitoring that a medical device can
use to protect itself against unauthorized access. A shield can monitor and intercept the radio frequencies
of external devices that are trying to communicate with the medical device. The shield can to diagnose
abnormalities in these communications and if necessary, can provide interference as a protective action.
(Burns et al., 2016)
2.1.15. Equipment Monitoring and Maintenance
Mechanical and then electromechanical equipment has been used for industrial and consumer
applications since the late 1800s. In order to keep that equipment working as intended over years of use,
companies must perform maintenance. In early approaches, equipment was fixed after it broke, a form of
reactive maintenance that took the equipment out of use and in some cases affected its safety. As
equipment manufacturers and maintenance engineers gained experience with equipment they began to
implement preventative maintenance strategies, so that key aspects of equipment performance could be
examined and returned to specified conditions at planned intervals. Such approaches allowed maintenance
to be carried out quickly and conveniently, before a failure occurred. It therefore reduced the likelihood of
catastrophic failures and significant downtime events. (Radziwill, 2018)
The introduction of IoT and equipment telemetry has improved the ability of manufacturers to
maintain equipment performance in advance of problems. Sensors on equipment can now provide
feedback about the condition of multiple parameters including, for example, temperature, load and
vibration. By monitoring this feedback data, engineers can maintain the equipment proactively with

47
appropriately scheduled maintenance (based on condition instead of fixed periodicity) or even introduce
temporary adjustments until they can schedule a full maintenance event. (Microsoft, 2018) For a large
hospital, the use of a proactive maintenance strategy can have important cost savings if it even reduces
downtime by 1% . (Stryker, 2019) Further, over time, a proactive maintenance program can become a
predictive maintenance program that recognizes and responds to more gradual types of equipment
degradation, or even a prescriptive program, that enables the device to self-heal, by using software and/or
by adjusting the physical environment. The figure below shows the evolution of maintenance over time as
it becomes progressively more advanced with the last 3 stages utilizes device telemetry to make its
recommendations or changes.
Figure 7 Maintenance Evolution

2.1.15.1. Medical Device Predictive Maintenance
The nature of hardware and electronic functions associated with medical devices differs greatly,
so the potential uses of telemetered data will also vary. Many medical devices- crutches, syringes,
Reactive
Preventative
Proactive
Predictive
Perscriptive
1940 - 1955 1955 - 1975 1975 - 2000 2000 +

48
surgical gowns, gloves, for example- currently have no software and no capacity for telemetry. However,
50% of medical devices currently on the market contain or are themselves software (Burns et al., 2016)
and may have the potential to implement predictive maintenance initiated by telemetered signals.
(Gonnelli et al., 2017)
Telemetry is already known to be used for predictive maintenance in medical devices that contain
batteries. A battery differs from the other device components because of its reduce lifespan compared to
other components. As batteries are stored their energy is dissipated and the internal impedance increases.
Fortunately, impedance is a parameter that medical devices can transmit to their monitoring station so that
battery life can be monitored and predicted so that appropriate intervention can be taken without risk to
the patient. (Untereker et al., 2017)
Other devices such as implantable spinal cord stimulators (SCS) have charging coils so that the
device can be recharged from an external source. Under normal operation, the SCS will transmit and
receive data via the telemetry coil during the charging process. However, the charging coil can also be
used to communicate data such as battery capacity and other pertinent charging variables by modulating
the impedance of the coil to code error signals that define failure modes such as shifts in voltage,
interrupted signals and other indicators. A medical technician can use this information to monitor the state
of operation of the device in order to predict future maintenance needs. (Marnfeldt et al., 2018) (Hu et al.,
2018) However, performance monitoring of implanted medical devices can extend beyond its use to
monitor battery function. For example, a patent for an implanted annotated echocardiogram (AECG)
(Thompson, 2000) indicates that telemetric signals may also be used to communicate status codes related
to the aberrant performance state of the device. This AECG analysis software has the capability to run a
self-diagnostic test and then use the results of that test to predict a device malfunction.
Predictive maintenance can be used for non-implanted devices as well. For instance, the
orthopedic and medical equipment manufacturer, Stryker, uses performance-related telemetric signals

49
from most of their smart medical devices. (Microsoft, 2019) The continuous capture and analysis of this
data allows Stryker to respond more quickly to early failures signals of products in a hospital setting. This
predictive data allows maintenance to be scheduled at a time that does not disrupt the medical staff or
patients. (Stryker, 2019)
2.1.15.2. Medical Device Prescriptive Maintenance
The optimal stage of the maintenance evolution, prescriptive maintenance, involves not only
monitoring a device’s performance but using that monitored information to correct the deficient
performance. Medtronic, Inc. has filed a patent that describes the use of telemetric data from its implanted
medical devices to implement predictive and even prescriptive maintenance. Its inventor, Kurt R.
Linberg, describes software that can access and monitor device performance and then prescribe a remote
solution to repair the defective operation. In the event that a component or software defect cannot be fixed
remotely, the software will broadcast an alert that additional action is needed. As part of its function, the
analysis software systematically reviews the usage logs, error logs and power and battery status updates.
It scans the database for integrity issues and can assess the mean time between failures of the critical
components. (Linberg, 2002)
In some cases, so much reliance is placed on the use of telemetry that its failure can become a
potential source of harm for patients. For example, Smiths Medical was caught off guard when a software
flaw compromised their ability to telemetrically detect battery drainage in their devices It was only when
the company received 74 complaints of a failing low-battery alarm signal that they realized the Medfusion
4000 Syringe Pumps had a serious mechanical issue. In response, on October 28, 2019, Smiths Medical
sent a Recall Notice to its customers informing them about the problem and the need to return affected
units to the manufacturer. (FDA, 2019a)

50
2.1.16. Expectations for medical device telemetry usage
It is clear from the previous case studies that telemetric signals from devices can be used to signal
problems before cyber attacks or device failures impact device performance. The FDA has laid out high-
level expectations with regard to cybersecurity requirements and has discussed the importance of
monitoring postmarket performance of medical devices. However, expectations regarding the use of
telemetric data for performance monitoring have yet to be communicated by the regulatory agencies. The
situation is further complicated because multiple federal agencies have roles in the oversight of medical
device performance, particularly as it relates to cybersecurity. For example, in addition to the FDA, the
Center for Medicare and Medicaid Services (CMS), Department of Defense (DOD), Department of
Veterans’ Affairs (VA) and the Department of Homeland Security (DHS) all currently have
responsibilities for medical device cybersecurity. (Chenok, 2012) Underreporting of cyber attacks is also
likely because such reports could damage reputation and create legal liability. As a result, there may exist
a false sense of security by the oversight bodies and cause them to reduce the priority given to providing
guidance on such issues. (Burns et al., 2016)
The shortage of guidance from regulatory agencies is compounded by the scanty literature
regarding the use of telemetry for managing device performance. Companies may refuse to share publicly
the ways that they use telemetry to avoid disclosing trade secrets or intellectual property. (van den
Bekerom, 2019) Thus, it is unclear how far companies have come in the use of telemetric data to monitor
device performance. However, such uses are likely to be more important going forward. For example, a
2019 survey of healthcare IT executives indicate that executives increasingly direct their organizations to
use wireless biomedical devices for activities such as such as echocardiogram (EKG/ECG), patient
monitoring, and medical imaging, to improve both patient care and operational efficiency. However, few
appeared to possess systems capable of monitoring such devices proactively. (Callisch, 2019) Further, the
executives who responded to that survey were end-users and not manufacturers, so their views provide

51
only indirect evidence about the extent to which monitoring methods have been introduced into medical
devices.
2.7. Exploring Implementation of Telemetric Monitoring
Medical device technologies have changed dramatically in the last 50 years. These improvements
have opened doors to healthcare that were previously inaccessible, providing patients with increased
mobility and the ability to live normal lives. Wireless data transmission has also given the opportunity to
monitor not only the health of the patient but also the health and performance of their medical devices.
However, it is not clear whether and how device feedback data is being used to accomplish this post-
marketing vigilance. In this thesis, I wish to explore how far the medical device industry has progressed
in implementing telemetric monitoring as a tool to assure the reliable ongoing operation of their devices.
Understanding how to use telemetric signals may be one way to address the performance problems that
continue to plague the industry. For example, during 2006–2011, nearly 1,210 unique recall events
involving 12 million medical devices were reported to have failures in software, hardware, batteries or
system interfaces. These deficient devices were examined and it was found that 197 of the 1,210 recalls
were associated with safety critical devices. As the name suggests, safety critical devices are those
devices whose failure have the highest likelihood of sever life-critical consequences to the patients. These
events were identified from class I recalls, class II recalls with a Reason for Recall field that indicated a
patient safety issue such as “injury” or “death”, and class III recalls that had a Failure Mode identified as
“Physical Safety Hazard”. Two particular groups of devices- defibrillators (implantable and external), and
infusion pumps- were responsible for at least 34 recalls and were associated with 332 fatalities over the
time period 2006-2011. (Alemzadeh, 2016) The benefits of achieving better performance behoove a
closer look at the way that performance could be improved by using telemetric monitoring.

52
Table 3 Medical Device Computer Related Recalls and Adverse Event Reports
(Alemzadeh, 2016)
Device Type
(Product Code)
# Recalls # Devices Death Injury Malfunction
Defibrillator External
(MKJ)
17 415,537 16 1 281
Defibrillator Implantable
(NIK/LWS/MRM)
2 170,542 293 14,281 11,028
Infusion Pump
(FRN/LZH/LKK/MEA)
15 945,300 23 574 2,399

Process implementation experts have identified that the full and effective use of any new
program, such as that of performance monitoring studied here, will take time and will occur in stages. A
staged model of implementation such as that developed by Fixsen and his colleagues, the Quality
Implementation Framework, is perhaps the most commonly used framework for studying implementation.
(Meyers et al., 2012) This implementation framework explains that the adoption of a new or changed
program will typically occur in four stages- exploration of its feasibility and value, installation of needed
skill-sets, tools and methodologies, initial implementation of the program, and full implementation of the
new program. (Fixsen et al., 2015) The Quality Implementation Framework describes the interplay
between these different stages in terms of phases; it also includes an element in the fourth phase of
sustaining to reflect the need to support the changed process or program over the longer term. Fixsen and
other researchers have made it clear that the transition from one stage to the next can only be successful if
supported by core implementation “drivers”. These include appropriate and adequate staffing, training,
opportunities for external consultation, staff performance management, information technology systems,
project management, and intercompany support. (Fixsen et al., 2009)

53
Figure 8 Implementation Stages and Supporting Drivers

The survey developed for this research explores, using Fixsen’s model, how industry is
implementing the use of telemetry for performance and cyber threat monitoring. Each implementation
phase has been examined to understand whether and how telemetric implementation has progressed or
stalled, and how the implementation drivers affected the experience and outcomes of that work.
2.1.17. Exploration – Phase 1
The Exploration stage is the period during which the medical device company begins to gather
information to assess if and how such an approach should be implemented. To complete this task the
medical device company would assemble a technical cross functional group to collect and analyze
requirements needed to enable the new capability in their product. The team needs to specify how broad
the implementation should be and what would be included in its scope. Some items to be considered
include, for example, the technical features of the specific product(s), the capabilities of the end user to
take advantage of the telemetric capability, and the value of the new telemetrically-linked feature to the
users and the company. During this period, the team must educate itself about a range of technical
approaches, applicable regulations and known practices within the industry for using performance-related
Design and Development

54
monitoring data. Finally, the required capabilities, resources and time to implement the change would
need to be estimated. The business case for the capability implementation would be reviewed by
executive leadership and a decision to move forward with implementation would be made at the
conclusion of this stage.
2.1.18. Design and Development – Phases 2 & 3
2.1.18.1. Installation Stage – Phase 2
The installation stage begins once the decision has been made to implement the change. Initially
the implementation team will require extra resources for planning, training, coaching, and selecting staff
who will be tasked with implementing the new capability. The product development team must design the
software to transmit the critical data. Technology specialists must characterize the IT systems needed to
collect and analyze the data. Once the device is transmitting data that is being collected by the computer
systems, the data analysis activities begin. Data science experts with training in analysis tools will
develop algorithms, user interfaces, action limits and alerts so that a reaction process to the data can be
developed. These activities would continue until the first complete system is built and considered ready
for installation in commercial products or in clinical research prototypes.

2.1.18.2. Initial Implementation - Phase 3a
Initial implementation begins when the first medical device is able to transmit data to the end user
who will act in response to the data according to a predefined procedure. This stage typically involves a
small pilot group of “beta-test” users who use the application and provide feedback on difficulties or
flaws. It is common in the early stages of implementation to find false outputs or missed opportunities.
Once the team gains confidence in the application and further modifications seem unnecessary, the team
would recommend transitioning the new feature to a full implementation. (Fixsen et al., 2015)

55
2.1.18.3. Full Implementation - Phase 3b
Full implementation is achieved when the new process becomes embedded in the routine use of
the marketed devices. This stage comes with its own resource requirements, Typically, information
technology, data analysis and medical staff are increased to support new capability during the launch of
the full implementation. Once a company has established a full training and support structure for user
implementation, the team defines when full implementation is complete based on expectations and goals
of the company and the medical professions to whom they provide products. (Fixsen et al., 2015)
2.1.19. Business Sustaining – Phase 4
The Business Sustaining stage has been added to this model to describe the continued activity that
will be needed over the life of the medical device in the market. The process of telemetry implementation
will be cyclical as products evolve and technologies improve. The resource allocations required to support
this stage depend on the number and diversity of users that are introduced in the product or the
technology.

56
Chapter 3. Methodology
2.8. Survey Development
This proposed research was based on a mixed methods approach in which a literature review and
analysis was followed by further exploration using survey methods. That survey contained 56 questions
created in a variety of formats including yes/no, multiple-choice and rating scales to measure agreement
or preference. Additionally, open text “comment” boxes were added so that respondents could provide
additional detail that might help to amplify or clarify their views beyond those captured by multiple-
choice questions. Multiple answers were permitted for some questions and for these questions, the
numbers of selections exceeded the number of participants. The survey was broken into four main
elements, respondent profile, implementation of telemetry monitoring, specifics of device performance
telemetry and specifics of cyber threat data telemetry.
An electronic survey platform, Qualtrics (qualtrics.com), was used to structure the survey. This
platform has the capability to disseminate the survey to identified individuals through an internet link and
collect their responses in a way that protects the anonymity of respondents. The survey software is also
able to collate the responses and provide the results in graphical and statistical displays as well as provide
access to raw data.
2.9. Survey Validation
The draft survey was sent initially to a small number of Regulatory Science faculty who
examined it to assure that the software displayed questions and captured answers appropriately. A focus
group of 8 purposefully selected experts with different backgrounds in academia and medical device
companies was then convened to critique the content of the survey and provide suggestions or alternative
approaches to improve the capture of desired information.

57
Table 4 Focus Group Participants
Names Title
James Wabby
Executive Director, Regulatory Affairs (Combination
Products, Devices and Digital Health) at AbbVie
Susan Bain Assistant Professor USC
Benson Kuo Assistant Professor USC
Frances Richmond
Director, D K Kim International Center for Regulatory
Science USC
Homa Alemzadeh Assistant Professor UVA
Michelle Jump
Global Regulatory Advisor - Medical Device
Cybersecurity at MedSec
Darin Oppenheimer Senior Director, US Regulatory Affairs at Medtronic
Doug Christoffel
(provided input in a follow-up meeting)
Principal Engineer, Design Assurance at Baxter
International Inc.

Prior to the focus group meeting, a draft copy of the survey questionnaire, a flow chart showing
the skip logic that was to be used, the Abstract and Chapter 1 of the dissertation were provided to the
focus group participants, so that they could prepare their feedback in advance of the meeting (Appendix
A). Due to COVID-19 social distancing requirements, the focus group meeting was held and recorded
using the Zoom teleconferencing system (zoom.com). The survey was modified according to the
suggestions made by the focus group participants and saved as a Post Focus Group version (Appendix B).
One of the most significant changes to the survey was the addition of questions regarding telemetry
logging for each device type. This change was needed to allow those participants whose devices only
performed the logging function to select an option without choosing the option for telemetry “logging,
routinely transmitting and monitoring”. Another important change that was made to the survey was
related to the implementation phase title names for phase 2, 3a and 3b. These terms did not resonate well
with the focus group members and they were repeatedly confused about the divisions between the stages
of implementation. Therefore, the survey headings were adjusted to merge the Installation, Initial
Implementation and Full Implementation phase questions under the heading of Design and Development.
Skip logic included in the survey was considered to pose a potential risk for error so additional testing

58
was conducted to ensure that the questions were presented or hidden appropriately according to the
selection of certain answers in previous questions (Appendix C). To some extent, these different areas
have then been decomposed in the discussion.
2.10. Survey Deployment
3.1.1. Participant Selection
The survey was sent to professionals who worked for defibrillator and infusion pump
manufacturers or those involved in the service of such devices. The targeted participants included
software, system, service, clinical, regulatory and quality engineers who support the product development
or business sustaining efforts for medical device manufacturers. Participants were identified using
personal referrals, social networking platforms, professional association membership and attendance at
industry meetings. Potential participants were contacted by email or through professional association chat
boards to determine their willingness to participate. Participants were also encouraged to nominate others
to participate, using the snowball technique, to further expand the pool of potential participants. The email
addresses of the identified respondents were entered into the Qualtrics Contact List for survey
distribution.
The survey was then sent electronically to the identified participants with a personal note to
assure the respondents that the results would only be used in a composite manner to protect their
individual identities and the identities of their employers. There was no financial compensation offered or
provided for participation in the survey. It was expected that these professionals were very busy and
might find it difficult to prioritize the survey with their current workload. Therefore, one or two follow up
reminders were sent to remind the potential respondents of their survey commitment. (Sheehan, 2001)

59
2.11. Administration, Data Collection and Analysis
Responses to the survey were collected and stored electronically. The survey remained open until
at least 15 participants from each of the targeted products- external and implanted defibrillators and
infusion pumps- responded to the survey. The survey data was then analyzed using basic descriptive
statistics, graphs or tables as appropriate. The individual entries were presented as entered by the
respondents. The open-ended questions were evaluated by the researcher for trends and commonality. For
questions where a ranked answer was requested, for example, in a stepped fashion from strongly agree to
strongly disagree, a weighted index (WI) was calculated by applying a weight to each response. In a scale
with 5 options, a weight of 1 was assigned to responses of “strongly agree” whereas a weight of 5 was
assigned to “strongly disagree”. The weighted values were summed for all responses and divided by the
total number of respondents. For this calculation, respondents answering “I don’t know” or “cannot
answer” were excluded from the calculation. A report on the findings of the survey was provided to the
survey participants when the study is complete.

60
Chapter 4. Results
2.12. Participant Responses
The survey was open from October 8, 2020 to December 7, 2020. Links to the survey were sent
directly to 97 individuals; 67 fully completed the survey and 6 partially completed the survey, providing
an overall response rate of 75%. The average response rate for individual questions was 44 responses per
participant; this varied according to the number of questions that were masked for different participants
by the skip logic function. In addition, an anonymous link was made available to industry professionals
with appropriate backgrounds through LinkedIn and personal networks; 12 individuals responded to those
links. Combining both groups, 85 respondents completed at least one survey question. The respondents
who self-identified as NOT having experience with defibrillators or infusion pumps selected “OTHER”
instead of one of the three product types under study (Question 3). In a few cases, respondents identified
in the first few profile and background questions that they did not work with defibrillators or infusion
pumps. In these cases, the participants were asked a short set of questions about the data collection of
their device before being led to the end of the survey.
2.13. Profiles and Background of Participants
The first block of questions collected information about the participant’s product experience and
company affiliation. Most (66%, 56/85) had more than 10 years of experience in the medical device
industry. Those with 3-10 years of experience formed the next largest group (28%, 24/85). Only a few
respondents had 0-2 years of experience (6%, 5/85).

61
Figure 9 Years of Experience
Q2 I have worked in the medical industry for …

Infusion pump experience was the most common type of product experience (34%, 29/85).
Smaller numbers reported experience with external defibrillators (25%, 21/85) and implantable
defibrillators (19%, 16/85). The remaining respondents categorized the devices with which they had
experience as “Other” (22%, 19/85), many of which were defined further in Table 5.
N = 85

62
Figure 10 Product Type with which Respondents had Experience
Q3: Please choose one product with which you have telemetry or postmarket experience. This product
should be used as the basis for answering the remaining survey questions.

Table 5 “Other” Product Types with which Respondents had Experience
Patient Monitors Ophthalmic Device Telemetry
Heart Rhythm Monitoring kyphoplasty, diabetes devices Drug eluding stent
insulin pump pacemaker neurostimulator
electrosurgical products,
ventilation products
heart monitor Oxymetry (sic)
body worn sensors Cochlear implant
Biofeedback Telemetry
Device - EMG Software as Medical Device,
Arrhythmia Detection
VAD, inductive coupling Sensor
Bluetooth enabled stethoscopes, Medical Device Data systems, PACS (Picture Archiving and
Communications Systems (Stand-alone software)

N = 85

63
Most of the respondents described their medical devices as being sold globally (71%, 60/85). A
minority were sold only in the US and Europe (20%, 17/85) or the US only (15%, 13/85). Some were sold
in other locations (6%, 5/85) (Table 6) and one respondent indicated that he/she did not know the
locations of product sales (1%, 1/85).
Figure 11 Geographic Reach of Marketed Products
Q4 In which countries are you marketing the product? (Select all that apply).

Table 6 “Other” Marketed Product Countries
US & Canada Europe
KSA EU experience only
Postmarket Only No Marketing

Most of the respondents described their company as large (> 1,000 employees: 60%, 51/85).
Fewer worked in medium sized (201-1,000: 19%, 16/85) or small companies (<200: 21%, 18/85).
Figure 12 Company Size Represented by Respondents
Q5 When I was involved with the above device my organization was...

N = 85
Respondents were encouraged
to select all that applied.
N = 85

64
The size distributions of the companies producing the different product types were compared
(Figure 13). Most companies producing infusion pumps were large (69%, 20/29); 10% (3/29) were
medium in size and 21% (6/29) were small. Similarly, external and internal defibrillator companies were
mostly large (62%,13/21 and 69%,11/16); medium sized companies accounted for 14% and 19% (3/21
and 3/16), and small companies for 24% and 13% (5/21 and 2/16). respectively. The category of “Other”
products had a slightly different distribution, with 37% large companies (7/19), 37% medium sized
companies (7/19) and 26% small companies (5/19).
Figure 13 Company Size for Different Product Types
Q3 Please choose one product with which you have telemetry or postmarket experience. This product
should be used as the basis for answering the remaining survey questions.

Respondents were encouraged to select the roles that described their work. Most common were
roles in Development, R&D or Engineering (52%, 44/85). Many also had roles in Regulatory Affairs
(27%, 23/85) or Quality Assurance / Risk Management (22%, 19/85). Fewer worked in Medical / Clinical
Affairs (14%, 12/85), Product Security (13%, 11/85), Post Market Surveillance (11%, 9/85), Service
Functions (8%, 7/85), or Information Technology (2%, 2/85). Several respondents reported another role
outside of the offered choices (18%, 15/85), shown in Table 7.
5
2
5
6
7
3
3
3
7
11
13
20
Other
Implantable Defibrillator
External Defibrillator
Infusion Pump
Large
Medium
Small
N = 85

65
Figure 14 Organizational Roles Held by Respondents
Q6 When I was involved with the device my functional role in the organization was best described as ...
(Select all that apply)

Table 7 “Other” Roles within the Organization
Executive
Research (2 individuals) Sales, Product Specialist
labelling
Sales (3 individuals) Interaction Design
Technical lead
Design Verification Commercial Development
Sterilization
no direct involvement. Conducted independent study about insulin pump.
I've been in clinical research and RAQA which covered all of the above functions

2.14. Types and Uses of Collected Data
Respondents were asked about the data collected by their device from a series of choices that
included the option of “Other”. The largest percentage identified that their device was capable of logging
performance data (64%, 54/85), and many identified that they logged, transmitted and monitored
(LTM’d) performance data (51%, 43/85). A large number of devices logged (46%, 39/85) and LTM’d
biometric data (54%, 46/85). In comparison, few respondents indicated that their devices logged (9%,
8/85) and LTM’d (9%, 8/85) cyber threat data. Six reported that their devices did not perform any of the
N = 85
Respondents were encouraged
to select all that applied.

66
above functions (7%, 6/85). A few respondents reported another type of data collection in addition to the
offered choices (7%, 6/85) (Table 8). Three respondents indicated that they did not know or could not
answer the question (4%, 3/85).
Figure 15 Data Types Logged or LTM’d
Q7 Which of the following functions does the device perform? (Select all that apply)

Table 8 “Other” Data Collected by Medical Devices
Device Type Description of “Other”
Infusion Pump
Interfacing to infusion pump for logging alarms and some patient
data.
These devices were designed and built in the 1980's, and had little or
now logging capabilities. (sic)
External Defibrillator
Periodic self tests of various complexity
Logging patient data
Implantable Defibrillator
Note that the transmission system is logging cyberattacks, not the
implant
Other Revascularization

The data were cross tabulated to determine how the transmission methods varied from one type of
device to another, as shown in Table 9.

67
Table 9 Data Types Logged or LTM’d by Device Type
Q7 Which of the following functions does the device perform? (Select all that apply). Data in the top
panel identifies the number of responses for each selection, whereas the bottom panel shows the same
responses as a percentage of participants.
Data Functions and Types
Infusion
Pump
External
Defibrillator
Implantable
Defibrillator
Other Total
Log - device performance data 20 16 11 7 54
LTM - device performance data 12 12 16 3 43
Log - biometric data 10 11 10 8 39
LTM - biometric data 12 11 14 9 46
Log - cyber threat data 2 2 3 1 8
LTM- cyber threat data 3 1 3 1 8
None of the above 0 1 0 5 6
Other 2 2 1 1 6
I do not know/cannot answer 1 2 0 0 3
Number of Participants 29 21 16 19 85

% Data Functions and Types
per Participants
Infusion
Pump
External
Defibrillator
Implantable
Defibrillator Other

Log - device performance data 69% 76% 69% 37%

LTM - device performance data 41% 57% 100% 16%

Log - biometric data 34% 52% 63% 42%
LTM - biometric data 41% 52% 88% 47%

Log - cyber threat data 7% 10% 19% 5%

LTM- cyber threat data 10% 5% 19% 5%

None of the above 0% 5% 0% 26%

Other 7% 10% 6% 5%

I do not know/cannot answer 3% 10% 0% 0%

68
The data above suggested that participants who answered “LTM” may have also selected
“logging”, given that this category would be a necessarily part of the LTM process. Data were parsed to
determine how many individuals who selected LTM also selected logging to allow for focus on the
performance data. Table 10 shows that those working with implantable defibrillators never chose logging
alone (0%, 0/16) while 100% indicated that they had incorporated LTM into their device capability
([LTM only, 5 + both log and LTM, 11]/16). For those working with infusion pumps, 38% (11/29) had
devices that were only able to log data, 41% ([LTM only, 3 + both log and LTM, 9]/29) that were capable
of LTM and 21% (6/29) that did not collect any data. For those working with external defibrillators, 24%
(5/21) of devices were reported to only have logging capability while 57% ([LTM only, 1 + both log and
LTM, 11]/21) had LTM capability and 19% (4/21) that did not collect any data.
Table 10 Performance Data Types by Use and Device

Infusion
Pump
External
Defibrillator
Implantable
Defibrillator
Other Total
Log Only 11 5 0 5 21
LMT Only 3 1 5 1 10
Both Log and LMT 9 11 11 2 33
Performance data NOT collected 6 4 0 11 21
Total Respondents 29 21 16 19 85

69
The 66 participants who worked with defibrillators and pumps were offered more detailed
questions regarding their experiences. First, they were asked whether they participated in design control
activities. More indicated that they participated in the design control process than did not (59% versus
41%). This distribution was similar across different device types.
Table 11 Design Control Process Participation by Product Type
Product Type Yes No % Yes % No
Infusion Pump 16 13 55% 45%
External Defibrillator 13 8 62% 38%
Implantable Defibrillator 10 6 63% 38%
Total 39 27 59% 41%

The 39 individuals who participated in the design control process were asked about the types of
data called out in their design requirements; 35 responded to this question and 33 gave feedback on the
data collection incorporated into the design control process. Most commonly these requirements
incorporated performance data logging (82%, 27/33), but more than half also incorporated performance
data LTM (61%, 20/33), biometric logging (55%, 18/33) and biometric data LTM (58%, 19/33).
Relatively few devices incorporated cyber threat logging (21%, 7/33) or cyber threat LTM (15%, 5/33).
Two respondents (external defibrillator: 1 and infusion pump:1) indicated that their devices did not
include requirements to collect biometric, performance or cyber threat data (6%, 2/33). A few respondents
chose “Other” (9%, 3/33) and provided comments, shown in Table 13. Two from the total 35 respondents
could not answer the question (6%, 2/35).
The distribution of choices for this question varied between device types as shown in Table 12.
All of those who worked with implantable defibrillators reported that design requirements included both
performance data logging and LTM. In contrast, requirements for external defibrillators and infusion
pumps more commonly included the performance logging function than LTM.

70

Table 12 Data Collection Incorporated During Design Control
Q13 – Were the following functions incorporated in the early stage requirements of the device? (Select all
that apply)
Multiple answers were permitted therefor, the numbers of selections exceeded the number of
participants.

Infusion
Pump
External
Defibrillator
Implantable
Defibrillator
Total
Log - device performance data 9 10 8 27
LTM - device performance data 6 6 8 20
Log - biometric data 5 7 6 18
LTM - biometric data 7 5 7 19
Log - cyber threat data 3 1 3 7
LTM- cyber threat data 3 0 2 5
None of the above 1 1 0 2
Other 1 1 1 3
I do not know/cannot answer 0 1 1 2
Number of Participants that
answered
15 10 8 33
Log - device performance data 60% 100% 100% 82%
LTM - device performance data 40% 60% 100% 61%
Log - biometric data 33% 70% 75% 55%
LTM - biometric data 47% 50% 88% 58%
Log - cyber threat data 20% 10% 38% 21%
LTM- cyber threat data 20% 0% 25% 15%
None of the above 7% 10% 0% 6%
Other 7% 10% 13% 9%
I do not know/cannot answer 0% 9% 11% 6%

Table 13 “Other” Data Collection Incorporated During Design Control
Device Type Description of “Other”
Infusion Pump
Logging should be considered an indispensable function, but was
not at that time.
External Defibrillator USB connection only - no OS - lay-user defibrillators (think airport)
Implantable Defibrillator
This is exclusive of the remote monitoring system. AICD only.
Cyber security is used for RPM

71
2.15. Device Performance LTM Implementation
Respondents were asked to categorize the stage of implementation that best describes their
device. The responses indicated that most devices were already on the market (65%, 26/40), a minority
were in the design and development stage (23% 9/40) and a few were in a stage not listed as a choice in
the question (8%, 3/40). Stages identified as “Other” are shown in Table 14. Two respondents did not
know or could not answer the question (5%, 2/40).
Figure 16 Implementation Stage of Devices on which Respondents Reported
Q14: What stage of implementation best describes your device, with respect to logging and routinely
transmitting data for performance monitoring?

Table 14 “Other” Implementation Stage
Device Type Description of “Other”
Infusion Pump Both support/sustaining and developing new products
External Defibrillator Implemented
Implantable Defibrillator
I have multiple devices under my tutelage, both post parket, clinical
and preclinical phases (sic)

N = 40

72
Data were stratified by product type to identify whether differences could be identified between
the device groups (Table 15), but no clear differences could be detected in this small population.
Table 15 Implementation Stage by Product Type

Infusion
Pump
External
Defibrillator
Implantable
Defibrillator Total
Exploring the idea 0 0 0 0
Design and development 2 4 3 9
Supporting marketed products 9 6 11 26
Other 1 1 1 3
I do not know/cannot answer 0 1 1 2
Participants 12 12 16 40
Exploring the idea 0% 0% 0% 0%
Design and development 17% 33% 19% 23%
Supporting marketed products 75% 50% 69% 65%
Other 8% 8% 6% 8%
I do not know/cannot answer 0% 8% 6% 5%

4.1.1. Exploration – Phase 1
A series of questions explored the experiences of infusion pump or defibrillator respondents
whose devices used telemetry for performance LTM (40 of 85 respondents). First, they were asked about
the perceived value of various guidances and standards related to the use of performance telemetric
monitoring using a 3-point scale of very useful (weighted as 1), slightly useful (weighted as 2) and not
useful (weighted as 3). Respondents rated international standards most favorably. Of the 33 individuals
who responded, all but one rated international standards as very or slightly useful (73%, [very useful, 17
+ slightly useful, 7]/33, WI=1.4). One respondent indicated that they were “not useful” and nearly a
quarter (24%, 8/33) did not know.
FDA and NIST documents varied in perceived value. Most valuable was the “Design Control
Guidance for Medical Device Manufacturers” (67%, [very useful, 10 + slightly useful, 12]/33, WI=1.7).
Two respondents rated the Design Control Guidance as “not useful” (6%, 2/33) and several indicated that

73
they did not know (27%, 9/33). The “Guidance for the Content of Premarket Submissions for Software
Contained in Medical Devices” was also considered valuable by most (59%, [very useful, 10 + slightly
useful, 9]/32, WI=1.7), but three rated this guidance as not useful (9%, 3/32) and ten participants did not
know (31%, 10/32). Documents related to cybersecurity were rated lower. About half regarded NIST’s
“Guide to Bluetooth Security” as useful (52%, [very useful, 8 + slightly useful, 9]/33, WI=1.9), but some
did not think that it was useful, (15%, 5/33) and several did not know (33%, 11/33). FDA’s “Postmarket
Management of Cybersecurity in Medical Devices” was considered useful by many (50%, [very useful, 6
+ slightly useful, 10]/32, WI=1.7) although one individual considered it not useful (3%, 1/32) and a large
number did not know (47%, 15/32). The documents that were rated as very or slightly useful by a smaller
number of respondents included NIST’s “Vetting the Security of Mobile Applications” (45%, [very
useful, 5 + slightly useful, 10]/33, WI=1.9). Three indicated that it was not useful (9%, 3/33) and many
did not know (45%, 15/33). FDA’s Guidance, “Content of Premarket Submissions for Management of
Cybersecurity in Medical Devices”, was considered useful by some (41%, [very useful, 8 + slightly
useful, 5]/32, WI=1.5); one did not think it was useful (3%, 1/32) and over half did not know (56%,
18/32). FDA’s “Guidance for Industry Cybersecurity for Networked Medical Devices Containing Off-the-
Shelf (OTS) Software” was considered useful by about one-third (36%, [very useful, 5 + slightly useful,
7]/33, WI=1.8), two did not find it useful (6%, 2/33) and most did not know (58%, 19/33).

74
Figure 17 Exploration Stage Document Usefulness
Q16. How useful were these documents? (during the exploration stage)

Respondents were asked to rate a series of challenges using 5 options ranging between “strongly
agree”, weighted as 1, to “strongly disagree”, weighted as 5. Many agreed that gathering competitive
N = 32 or 33
WI = 1.4 WI = 1.7 WI = 1.7
WI =1.9 WI = 1.7 WI = 1.9 WI = 1.5 WI = 1.8
3%
3%
3%

75
intelligence was difficult (40%, [strongly agree, 8 + somewhat agree, 17]/63, WI=2.7). Others did not
agree (24%, [somewhat disagree, 13 + strongly disagree, 2]/63) or neither agreed nor disagreed (22%,
14/63) and some did not know (14%, 9/63). About a third agreed that implementation was made difficult
because of insufficient documents or other information from regulatory agencies (33%, [strongly agree, 3
+ somewhat agree, 18]/64, WI=2.9). A quarter did not agree (23%, [somewhat disagree, 11 + strongly
disagree, 4]/64). Some neither agreed nor disagreed (22%, 14/64) and some did not know (22%, 14/64).
Obtaining sufficient resources was difficult for more than a third (37%, [strongly agree,7 + somewhat
agree,16]/63, WI=3.1) but a similar number disagreed (37%, [somewhat disagree, 13 + strongly disagree,
10]/63). Others neither agreed nor disagreed (13%, 8/63) and some did not know (14%, 9/63). About a
third agreed that implementation was made difficult by insufficient staff with appropriate skills /
knowledge (32%, [strongly agree, 7 + somewhat agree, 13]/63, WI=3.3), but more disagreed (43%,
[somewhat disagree, 11 + strongly disagree, 16]/63). A smaller group neither agreed nor disagreed (16%,
10/63) and some did not know (10%, 6/63). Less than a quarter considered that difficulties were
introduced by an underdeveloped business case (21%, [strongly agree, 6 + somewhat agree, 7]/63,
WI=3.3), whereas a third disagreed (33%, [somewhat disagree, 8 + strongly disagree, 13]/63). Others
neither agreed nor disagreed (27%, 17/63) and some did not know (19%, 12/63). The clarity of project
scope was viewed least commonly as a challenge (22%, [strongly agree, 1 + somewhat agree, 13]/63,
WI=3.5). Instead about half did not see the clarity of the project scope as a challenge (48%, [somewhat
disagree, 17 + strongly disagree, 13]/63), some neither agreed nor disagreed (16%, 10/63) and others did
not know (14%, 9/63). Ten respondents offered comments on challenges during the exploration stage
(Table 16).

76
Figure 18 Challenges for Telemetry Exploration
Q17. The factors listed below are sometimes identified by other, to be challenges to the exploration of
telemetry usage. To what extent to you find them challenging?

N = 63 or 64

77
Table 16 “Other” Challenges Seen During Exploration
Q18. Please describe any other major impediments your team experienced that were not sufficiently
described above.
Making sure that testing satisfied the many variables of FDA requirements
Project timelines were aggressive
First to market challenges meant we helped develop the regulatory requirements.
The problems were that the team had little experience, the consultants had little experience, the
FDA had little experience and the standards did not clearly state what was needed. The standards
inferred, they suggested, and they spoke at high level. Much of it was present information and see if
it was good enough.
Not enough ppl
The transmissions systems were developed in 1985 and evolved as technology improved. Old
devices which precede cyber threats and security requirements are still present in some patients.
Old devices have to be supported by a single transmission system and this limits flexibility. Some
implanted defibrillators last up 15 years. This constrains the ability to change. The most serious
danger is to steal a programmer and nefariously reprogram the implant. That does not require any
cyberattack.
A general lack of related technical knowledge lead to extremely bad choices, from too weak
(MQTT) to too heavy (ISO 11073). Almost no awareness of cybersecurity concerns or mitigations,
especially efficient and effective mitigations.
difficulty making case based on regulatory documentation due to lack of understanding by non-
technical stake holders. all or nothing mindset with regards to implementing cyber security
practices with a leaning towards not implementing practices. My involvement was with post market
risk and threat assessments derived from the evolving landscape and aging technology in order to
proactively address customer concerns and improve product security.
Testing labs were also not familiar with test requirements.
User experience design and user needs when drafting user workflows that interact with this detail of
the product.

78
Survey participants were questioned about the types of deliverables that had been created from
the exploration stage from a list of 8 choices. Most confirmed that high-level process requirements were
created (82%, 27/33); only one indicated that these were not created (3%, 1/33) and a few did not know
(15%, 5/33). Most also agreed that a preliminary risk assessment was created (79%, 26/33); three
identified that it was not created (9%, 3/33) and a few did not know (12%, 4/33). Most also identified that
a project scope was created (76%, 25/33), but three responded that it was not created (9%, 3/33) and a few
did not know (15%, 5/33). Many participants also agreed that a preliminary regulatory assessment was
created (73%, 24/33) but three said this was not created (9%, 3/33), and others did not know (18%, 6/33).
Many participants indicated that the benefits of telemetry were documented (67%, 22/33), three said that
it was not (9%, 3/33) and eight indicated they did not know (24%, 8/33). Predicted implementation costs
were estimated for more than half of the projects (61%, 20/33), but not for a minority (15%, 5/33); nearly
a quarter did not know (24%, 8/33). The remaining choices were known to be implemented by less than
half of the participants. A preliminary post market surveillance plan was created by 45% (15/33) but not
by 27% ( 9/33); the rest (27%, 9/33) did not know. Similarly, 42% created estimates of the long-term
costs to maintain telemetry (14/33) whereas 30% did not (10/33) and a similar number did not know
(27%, 9/33).

79
Figure 19 Items Created Before Proceeding to the Design & Development Stage
Q19. Several types of document are often, but not always, developed before a project is given the
approval to proceed. Which, if any, of the following items were created before you started to design and
develop the telemetric system for performance monitoring?

4.1.2. Design and Development – Phase 2 & 3
As implementation progressed, a new range of factors affecting the success of the product design
and development appeared important to varying degrees. Respondents were offered a number of choices
for factors identified as potentially important, ranging between “very important”, weighted as 1 to “not
very important”, weighted as 3. Most commonly rated as important were leadership support (97%, [very
N = 33
3%

80
important, 24 + somewhat important,7]/32, WI=1.2) and risk management plans (97%, [very important,
22 + somewhat important, 10]/33, WI=1.3). No respondents considered these two factors to be
unimportant and one indicated that she/she did not know (3%, 1/32 and 3%, 1/33 respectively). Nearly as
important were design requirements that included performance monitoring (91%, [very important, 20 +
somewhat important, 9]/32, WI=1.4), although one individual rated it as not very important (3%, 1/32)
and two did not know (6%, 2/32). Adding staff with unique skills was also considered important by most
(88%, [very important, 22 + somewhat important,7]/33, WI=1.4); two individuals considered that it was
not very important (6%, 2/33) and two did not know (6%, 2/33). Most considered identifying the risks of
telemetry as important (84%, [very important,15 + somewhat important,12]/32, WI=1.5) although one
individual did not (3%, 1/32) and a few did not know (13%, 4/32). The availability of training and
materials was considered important by most (78%, [very important,14 + somewhat important,11]/32,
WI=1.7) although not by a few (13%, 4/32) and a few did not know (9%, 3/32). Using telemetry for risk
mitigation was important to most (72%, [very important,6 + somewhat important,17]/32, WI=1.9), but a
few disagreed (13%, 4/32) and a few did not know (16%, 5/32). Vendor partnerships were least
frequently considered as important. Only 59%, [very important, 9 + somewhat important,10]/32, WI=2.0)
considered it as important, several disagreed (28%, 9/32) and a few did not know (13%, 4/32).

81
Figure 20 Importance of Factors associated with Design and Development
Q21. How Important were the following items? (During the Design and Development Stage)

Respondents were asked about seven areas of potential difficulty using a five-point scale from
“strongly agree” (weighted as 1) to “strongly disagree” (weighted as 5). The area most commonly viewed
as difficult was the limited capability of computer systems. Nearly half of the respondents agreed that this
was difficult (46%, [strongly agree, 8 + somewhat agree, 20]/61, WI=2.9), a third did not (33%,
[somewhat disagree 8 + strongly disagree, 12]/61), some neither agreed nor disagreed (13%, 8/61) and a
few did not know (8%, 5/61). Insufficient resources were also viewed as a challenge by 39% of
respondents ([strongly agree, 9 + somewhat agree, 15]/61, WI=3.0), although a similar number, 38%,
disagreed ([somewhat disagree, 14 + strongly disagree, 9]/61), some neither agreed nor disagreed (16%,
10/61) and a few did not know (7%, 4/61). Similarly, deficiencies in the skills and knowledge of staff
were challenging to 40% ([strongly agree, 6 + somewhat agree, 18]/60, WI=3.2), but not to another 40%
([somewhat disagree, 9 + strongly disagree, 15]/60); some neither agreed nor disagreed (13%, 8/60) and a
few did not know (7%, 4/60). Insufficient documents or other information available from regulatory

82
agencies challenged nearly as many (37%, [strongly agree, 5 + somewhat agree, 17]/60, WI=2.8) but
others did not identify this as a challenge (27%, [somewhat disagree, 14 + strongly disagree, 2]/60),
several neither agreed nor disagreed (20%, 12/60) and several did not know (17%, 10/60).
The other areas appeared to be less challenging for this respondent group. The clarity of the
business case was a challenge for 28% ([strongly agree, 4 + somewhat agree, 13]/61, WI=3.4) but about
half disagreed (46%, [somewhat disagree, 14 + strongly disagree, 14]/61), others neither agreed nor
disagreed (16%, 10/61) and some did not know (10%, 6/61). The clarity of the project scope challenged
about a quarter (26%, [strongly agree, 2 + somewhat agree, 14]/61, WI=3.4) but a larger number
disagreed (44%, [somewhat disagree, 13 + strongly disagree, 14]/61), some neither agreed nor disagreed
(20%, 12/61) and some did not know (10%, 6/61). The absence of regulatory requirements was difficult
for only a small number (23%, [strongly agree, 8 + somewhat agree, 6]/61, WI=3.2); nearly a third
disagreed, (31%, [somewhat disagree, 8 + strongly disagree, 11]/61), several neither agreed nor disagreed
(18%, 11/61) and many did not know (28%, 17/61). Additional comments on this topic are shown in
Table 17.

83
Figure 21 Difficulties during the Design & Development Stage
Q22. For your project, how strongly do you agree with the following statement, often identified by others
to be difficulties in the design and development stage of telemetry implementation?

Table 17 “Other” Challenges Seen during Design and Development
Q23. Please describe any other major impediments your team experienced that were not identified in the
previous question.
Schedule
There has never been a cyber attack on a defibrillator. There have been reverse engineered
simulations done. It is easier to steal a programmer than reverse engineering one. Currently the
telemetry systems do not adjust functionality of defibrillators
Difficult to gain buy in from upper management and key stake holders due to lack of understanding
of cyber security and related risk/mitigation models and perceived rigidness of requirements from
regulatory bodies in the implementation of these processes. To unknowledgeable stake holders the
fear of miss handling a cyber security process from a regulatory perspective outweighed their
perceived cyber security risk.

84
Fewer of the survey participants had been involved in the business sustaining stage. Those in this
group were asked to reflect on possible factors affecting the success of implementation by offering them a
number of choices to which they could respond with options including “strongly agree” (weighted as 1) to
“disagree” (weighted as 5). Most participants indicated that user testing helped to identify potential
problems (91%, [strongly agree, 12 + agree, 7 + somewhat agree, 2]/23, WI = 1.8); only one individual
disagreed (4%, 1/23) and one neither agreed nor disagreed (4%, 1/23). The large majority also had active
leadership support (91%, [strongly agree, 9 + agree, 10 + somewhat agree, 2]/23, WI = 2.0); two
individuals disagreed (9%, 2/23) and none neither agreed nor disagreed (0%, 0/23). Reliability/stress
testing provided critical information according to most participants (87%, [strongly agree, 7 + agree, 11 +
somewhat agree, 2]/23, WI = 2.0); no participant disagreed (0%, 0/23) and three neither agreed nor
disagreed (13%, 3/23). Clear goals and timelines for activities were present for most (87%, [strongly
agree, 7 + agree, 9 + somewhat agree, 4]/23, WI = 2.1); none disagreed (0%, 0/23) and three neither
agreed nor disagreed (13%, 3/23). The ability for the implementation team to become fluent with the
analysis tools was considered critical according to most (87%, [strongly agree, 5 + agree, 12 + somewhat
agree, 3]/23, WI = 2.2), none disagreed (0%, 0/23) and three neither agreed nor disagreed (13%, 3/23).
Most also indicated that beta testing was effective (83%, [strongly agree, 6 + agree, 11 + somewhat agree,
2]/23, WI = 2.2); one individual disagreed (4%, 1/23) and three neither agreed nor disagreed (13%, 3/23).

85
Figure 22 Valuable Actions and Items During Implementation
Q25. How would you agree / disagree with the following statements?

86
Respondents were asked whether their teams were able to maintain the telemetry launch schedule
that had been estimated at the beginning of their project. About half of the respondents identified that the
schedule had been delayed (53%, 17/32), but at least a third had met their schedules (38%, 12/32). None
of the telemetry projects had been cancelled and a few did not know or could not answer the question
(9%, 3/32).
Figure 23 Incidence of Project Delays or Cancellations
Q27 Was your project cancelled or delayed compared to the initial estimates at the beginning of the
project?

Of those whose projects were delayed, about half experienced a delay of 2-6 months (53% 9/17),
and the other half, 6-24 months (47% 8/17).
Figure 24 Magnitude of Implementation Delay
Q28 Please indicate the magnitude of the delay in implementing performance monitoring for your device.

N = 32
N = 17

87
Respondents were then queried about the reasons for implementation delays from a list of reasons
that have historically caused project delays, from “strongly agree” (weighted as 1) to “strongly disagree”
(weighted as 5). Most respondents agreed that insufficient resources were a factor (65%, [strongly agree,
2 + somewhat agree, 9]/17, WI = 2.6), but a quarter disagreed (24%, [somewhat disagree, 1 + strongly
disagree, 3]/17) and two neither agreed nor disagreed (12%, 2/17). Most respondents also agreed that
evolving regulations caused delays (53%, [strongly agree, 2 + somewhat agree, 7]/17, WI = 2.4). Only
one disagreed (6%, [somewhat disagree, 1 + strongly disagree, 0]/17) and two neither agreed nor
disagreed (41%, 7/17). Similar numbers of respondents agreed versus disagreed that having staff with
insufficient skills / knowledge contributed to the project delay (41%, [strongly agree, 2 + somewhat
agree, 5]/17; 47% [somewhat disagree, 5 + strongly disagree, 3]/17 WI = 3.1); two neither agreed nor
disagreed (12%, 2/17).
About a third identified that an unclear project scope was a contributing factor (35%, [strongly
agree, 0 + somewhat agree, 6]/17, WI = 3.4) although a half disagreed (47%, [somewhat disagree, 4 +
strongly disagree, 4]/17) and a few more neither agreed nor disagreed (18%, 3/17). Similarly, a third saw
limitations in computing capability as a factor (35%, [strongly agree, 0 + somewhat agree, 6]/17, WI =
3.2) but more disagreed (41%, [somewhat disagree, 4 + strongly disagree, 3]/17, WI) and a few neither
agreed nor disagreed (24%, 4/17). A third also identified that the leadership did not act quickly enough at
the start of the project (35%, [strongly agree, 1 + somewhat agree, 5]/17, WI = 3.0). An equal number
disagreed (35%, [somewhat disagree, 5 + strongly disagree, 1]/17) and nearly a third neither agreed nor
disagreed (29%, 5/17). A lack of clarity in the business case was seen to contribute by relatively few
(18%, [strongly agree, 0 + somewhat agree, 3]/17, WI = 3.7); more disagreed (65%, [somewhat disagree,
7 + strongly disagree, 4]/17) and 3 respondents neither agreed nor disagreed (18%, 3/17). Additional
reasons for project delays were provided in a comment field, shown in Table 18.

88
Figure 25 Reasons for Delays in Implementation
Q29 For your project, do you agree with the following statements, often identified by others to contribute
to implementation delays or cancellations?

Table 18 “Other” Reasons for Delays
Issues were found in device testing that needed to be resolved in the quality system.
FDA required changes

4.1.3. Business Sustaining – Phase 4
Participants were asked about the effectiveness of their sustainability processes, using a ranking
in which a process that was implemented “exactly as needed” was weighted as 1, to “not provided at all”,
weighted as 4. None of the choices were recognized to be exactly as needed by more than about one-third
of respondents.
The process to ensure quality oversight was implemented in a manner that was exactly what was
needed for 35% of respondents (8/23, WI=1.7) but a similar number thought that it needed more detail

89
(39%, 9/23). One respondent indicated that the process did not address the need (4%, 1/23) and another
(4%, 1/23) that no such process had been implemented. A few did not know or could not answer the
question (17%, 4/23).
Fewer regarded the monitoring process and associated procedures as exactly what was needed
(17%, 4/23, WI=2.0) and many more thought that the process required more detail (52%, 12/23). A
couple of respondents indicated that the implemented process did not address the need (9%, 2/23) and one
(4%, 1/23) that no process was provided. Some participants did not know or could not answer the
question (17%, 4/23).
The feedback process to communicate new learnings was viewed as exactly what was needed by
30% (7/23, WI=2.0), but 43% thought that the process needed more detail (43%, 10/23). A few indicated
that the process did not address the need (9%, 2/23), that no process had been implemented (9%, 2/23) or
that they did not know or could not answer the question (9%, 2/23).
Only a few regarded their ongoing staffing and training process as exactly as what was needed
(17%, 4/23, WI=2.1) whereas many thought that it needed more detail (39%, 9/23). A few identified that
the process did not address the need (17%, 4/23) and one (4%, 1/23) that no process had been
implemented. Some participants did not know or could not answer the question (22%, 5/23).
The process for managing staff performance was exactly what was needed for about a quarter of
respondents (26%, 6/23, WI=2.2), but about a quarter thought that it needed more detail (17%, 4/23) and
another quarter that it did not address the need (22%, 5/23). A smaller number (9%, 2/23) indicated that
no process had been implemented and others did not know or could not answer the question (26%, 6/23).
The implemented process for ensuring ongoing involvement of leadership was exactly what was
needed for some respondents (26%, 6/23, WI=2.2) but a similar number identified that it needed more

90
detail (22%, 5/23), a few that the process did not address the need (17%, 4/23) or that there was no
process (13%, 3/23). A few (22%, 5/23) did not know or could not answer the question.
Table 19 Items Provided to Business Sustaining Team
Q26 How effectively were the following items provided to the business sustaining team, if at all.

91
2.16. Device Performance Telemetry
4.1.4. External Defibrillators
Participants working with external defibrillators that were capable of logging and/or logging,
transmitting and monitoring (LTM) performance data were asked about that types of data that were
collected from a list of offered choices. All respondents to this question indicated that all devices could
log and LTM battery capacity. Another parameter frequently logged and LTM’d was impedance of the
paddle or lead (log = 92%, 12/13; LTM = 90%, 9/10). Often logged but less frequently LTM’d were
battery charge time (log = 69%, 9/13; LTM = 50%, 5/10), battery impedance (log = 62%, 8/13; LTM =
30%, 3/10) and time to power up (log = 62%, 8/13; LTM = 30%, 3/10). Data transmission time was being
logged less frequently than it was being LTM’d (log = 54%, 7/13 versus LTM = 70%, 7/10) and time to
upload software was infrequently collected or LTM’d (log = 23%, 3/13 versus LTM = 20% 2/10). Three
respondents logged other types of data (Table 20) and one indicated that their device LTM’d data but did
not know or could not answer the specific question here. That latter response was removed from the
number of individuals that answered the question, and was not included in the above LTM denominator.

92
Figure 26 Data Types collected by External Defibrillator Companies
Q30 & Q31 Choose all types of performance parameters your external defibrillator logs or LTM’s.

Table 20 “Other” Performance Data logged by External Defibrillators
User actions
ECG
Possibly surrounding sound
Patient physiologic data and diagnostic decisions.
We had a black box that logged every user and system action including results of self tests

1
0
2
7
3
3
5
9
10
0
3
3
7
8
8
9
12
13
0% 20% 40% 60% 80% 100%
Do not know/cannot answer
Other
Time to upload software
Data transmission time
Time to power up
Battery impedance
Battery charge time
Paddle impedance
Battery capacity
Log
(N=13)
LTM
(N=10, individuals that answered the question)
External Defibrillator Data Types
Respondents were encouraged
to select all that applied.

93
4.1.5. Implantable Defibrillators
Respondents working with implantable defibrillators were asked about the specific performance
parameters being collected from offered choices. All respondents indicated that their device logged and
LTM’d battery capacity data (log = 100%, 10/10 versus LTM = 100%, 14/14) as well as warnings, alarms
or error messages (log = 100%, 10/10 versus LTM = 100%, 14/14). Lead impedance, battery charge time
and battery impedence were always logged and usually LTM’d (93%, 13/14; 86%, 12/14 and 79%, 11/14
respectively). Other performance parameters were not always collected. The frequency with which the
device connected and disconnected from the network was logged more frequently (60%, 6/10) than
LTM’d (43%, 6/14), data transmission time was logged more frequently (40%, 4/10) than LTM’d (29%,
4/14), total time was logged more frequently (20%, 2/10) than LTM’d (14%, 2/14) and time to upload
software was logged more often (30%, 3/10) than LTM’d (7%, 1/14). Five participants (log = 20%, 2/10
and LTM = 21%, 3/14) indicated they captured other types of data, shown in Table 21. One individual
indicated that their device both logged and LTM’s performance data but did not know or could not
answer the question and was therefore not included in the denominator (number of individuals that
answered the question) above.

94
Figure 27 Data Types collected by Implantable Defibrillator Companies
Q32 & Q33 Choose all types of performance parameters your implantable defibrillator logs or LTM’s.

Table 21 “Other” Performance Data collected by Implantable Defibrillators
Data Logging Data Logging, Transmission and Monitoring
I do not know what total time refers to Last therapy deleivery (sic)
Arrhythmia events, therapy, electrograms
from events, sensing and pacing threshold
Unsure on software upload. Don't know what total
time refers to

R wave, P wave, pacing thresholds, arrhythmia
events, therapy ad effect of therapy
1
3
1
2
4
6
11
12
13
14
14
1
2
3
2
4
6
10
10
10
10
10
0% 20% 40% 60% 80% 100%
Do not know/cannot answer
Other
Time to upload software
Total time
Data transmission time
Frequency of network
Battery impedance
Battery charge time
Lead impedance
Warnings, alarms or error messages
Battery capacity
Log
(N=10, individuals that answered the question)
LTM
(N=14, individuals that answered the question)
Implantable Defibrillator Data Types
Respondents were encouraged
to select all that applied.

95
4.1.6. Infusion Pumps
Participants working with infusion pumps that were capable of logging and/or LTM data were
asked about the performance parameters being collected from a list of offered choices. All or nearly all
indicated that their device logged (94% 17/18) and LTM’d (100%, 10/10) warnings, alarms or error
messages. Many also collected data on the number of resets of the program (log 67%, 12/18; LTM 60%,
6/10), frequency of battery usage (log 61% 11/18; LTM 40%, 4/10), back pressure (log 61%, 11/18; LTM
30%, 3/10), run time (log 56%, 10/18; LTM 40%, 4/10), RPM at various flow rates (log 56%, 10/18;
LTM 40%, 4/10), network connects and disconnects (log 50%, 9/18; LTM 50%, 5/10), alerts per unit of
time (log 50%, 9/18; LTM 40%, 4/10), repeat key strokes (log 44%, 8/18; LTM 30%, 3/10), pressure
difference across the pump (log 44%, 8/18; LTM 30%, 3/10), total lifecycle clock (run time since it was
initially powered up), (log 39%, 7/18; LTM 40%, 4/10), number of times the pump tubing door was
opened and closed per unit time (28% 5/18;LTM 30%, 3/10), energy consumption at different flow rates
(log 28%, 5/18; LTM 30%, 3/10), pump temperature (log 28% 5/18; LTM 10%, 1/10) and pump altitude
(log 11%, 2/18; LTM 0%, 0/10). Six respondents indicated that they captured other types of data (
Table 22). One participant indicated that he/she did not know or could not answer the question, and was
therefore not counted with the others when identifying the number of respondents that answered the
question, in the denominator above.

96
Figure 28 Data Types collected by Infusion Pump Companies
Q34 & Q35 Choose all types of performance parameters your implantable defibrillator logs/LTMs

Table 22 “Other” Performance Data collected by Infusion Pumps
Data Logging
Glucose Sensor readings
Unsure about all features, but answered to the best of my ability
Number and type of interactions with the patient programmer.
Dosing parameters set
Data Logging,
Transmission and
Monitoring
Glucose Sensor readings
Number and type of interactions with the patient programmer.

0
2
0
1
3
3
4
3
3
4
5
4
4
3
4
6
10
1
4
2
5
5
5
7
8
8
9
9
10
10
11
11
12
17
0% 20% 40% 60% 80% 100%
Do not know / cannot answer
Other (please describe)
Pump altitude
Pump temperature
Energy consumption at flow rates
Number of tubing door closures / unit time
Total lifecycle clock
Pressure delta across the pump
Number of repeat key strokes
# alerts / unit time
Frequency of network connects and disconnects
Pump RPM at various flow rates
Total time
Back pressure
Frequency of battery usage
Number of resets of the program
Warnings, alarms or error messages
Log… LTM (N=10)
Infusion Pump Data Types
Respondents were encouraged
to select all that applied.

97
Participants were questioned about the way in which LTM performance data was presented to the
user. Most presented it as warnings, alarms and error messages (88%, 29/33). Less frequently, data were
presented graphically with limits (52%, 17/33) or in the form of summary statistics or analyzed data
(52%, 17/33). About a quarter of respondents indicated that notifications based on alert limits were
provided to the user (45%, 15/33). A few reported that raw performance data were being transmitted
directly from the device to the user (30%, 10/33), and a few shared that total device operation time, (9%,
3/33) as well as performance data graphs without limits (9%, 3/33), was being presented to the user. Four
respondents identified another data presentation format (12%, 4/33), (Table 23) three indicated they did
not know or could not answer (8%, 3/36), 29/33.
Figure 29 Performance Data Presented to the User
Q36 How does the device software present the device performance data to the user? (Select all that apply)
Multiple answers were permitted therefor, the numbers of selections exceeded the number of participants.

3
4
3
3
10
15
17
17
29
0% 20% 40% 60% 80% 100%
I do not know/cannot answer
Other
Graphically without limits
Total time
Raw data
Notifications based on limits
Analyzed Data
Graphically with limits
Alarms/error messages
Respondents were encouraged to
select all that applied.
N = 33 or 36

98
Table 23 “Other” Methods of Presenting Performance Data
total time?
Data goes to a central monitoring system. Then from there the data is sent via the cloud to the
clinical centers. There is no direct connection between the device and the clinic unless a
programmer is used.
related data system provides standardized reports accessable to the customer. Company offers
clinical analysis of data for additional cost. (sic)
Report that can be printed

Most respondents expected hospital staff to review the data transmitted by the devices (91%,
32/35). A majority also believed that the device manufacturing staff (77%, 27/35) should review the
device performance data. A few respondents expected the patient to monitor the data delivered by his/her
device (14%, 5/35) and one participant indicated that the transmitted performance data were not expected
to be reviewed by anyone (3%, 1/35). One respondent selected “other” (3%, 1/35) with the comment that
“device manufacturer staff receives raw and analyzed data if an issue is detected or advanced
troubleshooting is needed”. One respondent did not know or could not answer (3%, 1/36).
Figure 30 Expectations for the Review of Performance Data
Q37 Who is expected to review the transmitted raw or analyzed device performance data? (Select all that
apply)

1
1
1
5
27
32
0% 20% 40% 60% 80% 100%
I do not know/cannot answer
Other
Data is not reviewed
Patient
Device manufacturer staff
Medical staff, clinical engineering,…
Respondents were encouraged to
select all that applied.
N = 35 or 36

99
Participants working with infusion pumps or defibrillators that could log and/or LTM data were
stratified to identify whether differences existed between device types (Table 24). Of those working with
implanted defibrillators, 100% expected that hospital staff would review performance data, but only 53%
expected that it would be reviewed by manufacturing staff. An opposite pattern characterized the views of
those working with external defibrillators; 73% expected that hospital staff would review the data but
91% expected that it would be reviewed by the device manufacturing staff. Almost all of those working
with infusion pumps expected that both hospital staff and device manufacturers would review the
performance data (90% and 90% respectively).
Table 24 Performance Data Reviewers by Device Type
Q37 Who is expected to review the transmitted raw or analyzed device performance data? (Select all that
apply)

100
Participants were asked about the purposes for which device performance data were used from a
list of options. The large majority of respondents said that their company used the data for risk
management activities (89%, 32/36); two (6%, 2/36) disagreed and two did not know or could not answer
(6%, 2/36). Most respondents also said their company used the LTM data for several other functions:
future product development (86%, 31/36); postmarket surveillance (83%, 30/36); and data trending and
analysis (83%, 30/36). For each of these choices only one or two disagreed, and 4-5 did not know or
could not answer. About two-thirds thought that their companies were also using the data to assist in
corrective and preventive actions (CAPA) (69%, 25/36); a few disagreed (11%, 4/36), several did not
know or could not answer (19%, 7/36). Some respondents suggested that they had “other” uses for the
data, specified in Table 25.
Figure 31 Performance Data Usage
Q38 Is the device performance data (raw or analyzed) use in any of the processes below?

N

36
36
36
36
36
5
3%
3%

101
Table 25 “Other” Performance Data Use
Benchmark/analytics
The data is only used for CAPA as needed. Not routinely scanned.
Complaint investigation (PMS)

Participants were asked about benefits associated with telemetered performance monitoring from
a list of suggested options. Most respondents said that feedback for future products was a benefit (92%,
33/36); a few indicated they did not know or could not answer (8%, 3/36). Most also thought that its
support of maintenance or software updates was beneficial (86%, 31/36); a few indicated they did not
know or could not answer (14%, 5/36). Many said direct customer benefits had been produced (78%,
28/36), but a couple disagreed (6%, 2/36) and some indicated they did not know or could not answer
(17%, 6/36). Three quarters agreed that the data improved medical device field management and feedback
for upgrades to current product (78%, 28/36 for each choice); a few disagreed (8%, 3/36 for each choice)
and some indicated they did not know or could not answer (14%, 5/36). About two thirds regarded
reduced performance failures as a benefit (67%, 24/36); a few disagreed (11%, 4/36) and more indicated
they did not know or could not answer than was typical for other choices (22%, 8/36). Some also saw
improved asset management as a benefit (47%, 16/34) but about a quarter disagreed (26%, 9/34) or
indicated they did not know or could not answer (26%, 9/34). One respondent expressed some concerns:
More data with manufucter means more actions needed to be taken. This adds
additional burden and this doesn’t result in any increases sales (sic)

102

Figure 32 Benefits from Performance Data
Q39 Has the analysis or presentation of device telemetry performance monitoring data produced any of
the following benefits?

Benefits were also gauged in a second way, by exploring the requirements that performance data
could satisfy from a list of 5 options. Most respondents identified that it could be used as a risk
management control or detection method (86%, 32/37) or that it was a required product feature (84%,
31/37). About two-thirds also identified that it supported a QSR (59%, 22/37) or legal requirement for
postmarket surveillance (57%, 21/37). Fewer than half identified that it satisfied a CAPA created because
of a previous AE or failure (35%, 13/37). Two respondents (5%, 2/37) noted that this capability satisfied
“other” requirements listed in
Table 26 and one respondent did not know or could not answer (3%, 1/37).
1
16
24
28
28
28
31
33
9
4
3
3
2
4
9
8
5
5
6
5
3
Other (please describe)
Asset management
Reduced medical device performance
failures
Feedback for upgrades to current product
Improved medical device field
management
Customer is directly benefitting from the
technology
Maintenance or software updates
Feedback for future products
Yes No I do not know/cannot answer
N
36
36
36
36
36
36
34
5

103
Figure 33 Requirements Satisfied by Telemetered Performance Monitoring
Q40 Check all of the requirements that telemetry performance monitoring data satisfies for the device.
Multiple answers were permitted therefor, the numbers of selections exceeded the number of participants.

Table 26 “Other” Requirements that Telemetry Performance Monitoring Satisfies
Unsure on CAPA
Improves Patient journey and future devices
Respondents were asked if the telemetered performance data improved their company’s
understanding of their marketed products. Most agreed that it would (89%, 54/61) but 4 disagreed (7%,
4/61) and 3 did not know or could not answer (5%, 3/61). In a follow-up question, participants were asked
if they believed that regulators should require telemetry-capable devices to monitor performance
parameters for postmarket surveillance. Most agreed (78%, 46/59), but a smaller number disagreed (12%,
7/59) or did not know (10%, 6/59). Most participants also agreed that agencies should require this
capability for risk management (78%, 46/59); a minority disagreed (17%, 10/59) and some did not know
(5%, 3/59).

104
2.17. Cyber Threat Telemetry
Only 7 survey participants worked with devices that collected cyber threat data. All had LTM
capability. These respondents identified the types of data collected from their devices (Table 27). The
respondent working with implantable defibrillators identified that data LTM was being conducted for all
of the 4 offered choices: log-ons/unit of time, pairing attempts/unit of time, time to transmit data and time
to upload software. The 3 respondents working with infusion pumps all indicated their devices LTM’d
paring attempts/unit of time. One of the three also indicated their device LTM’d time to upload software.
The other respondent also indicated that log-ons / unit of time and other, authentication failures and
firmware integrity checks, were LTM’d.
Table 27 Cyber Threat Data that were Logged, Transmitted and Monitored
Q45. Choose the type of cyber threat data your device transmits and monitors. (Select all that apply)
Respondents were encouraged
to select all that applied.
Infusion
Pump
Implantable
Defibrillator
Log-ons / unit of time 1 1
Pairing attempts / unit of time 3 1
Time to transmit data 1
Time to upload software 1 1
Other (please describe) 1
I do not know / cannot answer
Total participants 3 1

105
Most respondents believed that both hospital and medical device company staff should review the
cyber threat data. One participant indicated that the data were not reviewed by either hospital or company
staff (Table 28).
Table 28 Expectations for the Monitoring of Cyber Threats
Q47 Who reviews the transmitted raw or analyzed cyber threat monitoring data? (Select all that apply)
Respondents were encouraged to select all that applied.
Infusion
Pump
Implantable
Defibrillator
Medical staff, Clinical Engineering , Hospital biomedical engineers 2 1
Device manufacturer staff 2 1
Data is not reviewed 1
Other
I do not know / cannot answer
Total participants 3 1

106
Participants were offered choices on how cyber threat data were presented to the user.
Respondents identified that data were presented in a variety of forms, including raw data, analyzed data,
data presented graphically with or without limits and as warnings, alarms or error messages (Table 29).
One respondent working with infusion pumps indicated that their devices data were presented in a method
other than the choices provided but that method was not specified.
Table 29 Methods of Presenting Cyber Threat Data to Users
Q48 How does the device software present the cyber threat data to the user? (Select all that apply)
Multiple answers were permitted so the numbers of selections exceeded the number of participants.
N = 4
Infusion
Pump
Implantable
Defibrillator
Raw data 1 1
Analyzed data 1 1
Presented graphically without limits 1 1
Presented graphically with limits

1
Notifications are sent out by the software based on limits

1
Warnings, alarms or error messages 1 1
Other (please describe) 1

I do not know/cannot answer

Respondents 3 1

107
Three respondents identified that data were reviewed for trending and analysis as well as future
product development, but one disagreed. Two respondents said that it was used for postmarket
surveillance and current product CAPAs, but two disagreed. Two respondents indicated that cyber threat
data were used for risk management and was being shared with their ISAO, but one disagreed. One
respondent indicated that the data were being used for another unspecified use.
Table 30 Use of Cyber Threat Monitoring Data
Q49. Is the collected cyber threat monitoring data (raw or analyzed) used in any of the processes below?
N = 4 Yes No
Do not know / cannot
answer
General data trending and analysis 3 1
Future product development 3 1
Postmarket surveillance 2 2
Current product CAPA 2 2
Risk management and reporting 2 1
Information is shared with our ISAO 2 1
Other (please describe) 1

108
Three of four respondents believed that FDA should require a device capable of telemetry
monitoring to use that function to monitor cyber threats in support of postmarket surveillance (75%, 3/4)
and risk management (75%, 3/4). The other respondent did not know.
Figure 34 Should Telemetry Cyber Threat Monitoring Be Required by Regulators?
Q50 Do you think that regulators should expect a telemetry capable device also includes cyber threat
monitoring as part of the risk management and postmarket surveillance processes?

1
1
3
3
0 1 2 3 4
Risk Management
Postmarket Surveillance
Yes No I do not know
N = 4

109
All three respondents agreed that class III medical devices containing software should use the
capability of cyber threat monitoring as part of the risk management (3/3) and postmarket surveillance
(3/3). They also agreed that Class II medical devices capable of cyber threat monitoring should use the
data for risk management (3/3), although only two believed that it should also be required for postmarket
surveillance (2/3). Two believed that medical devices that formed part of an interoperable system or a
mobile medical application should use cyber threat telemetry data for risk management and post market
surveillance. Two said telemetry should be used for risk management (2/3) for all software that is a
medical device and all 3 thought that it should be used for postmarket surveillance (3/3). All respondents
agreed, however, that it was not appropriate to require the use of cyber threat monitoring as part of risk
management or postmarket surveillance for “legacy” (0%, 0/3) and “other” (0%, 0/3) devices.
Figure 35 Products That Should Utilize Cyber Threat Monitoring
Q51 In your opinion, which devices should have cyber threat monitoring as part of the risk management
process? (Select all that apply)
Q52 In your opinion, which devices should have cyber threat monitoring as part of the risk management
process? (Select all that apply)
Multiple answers were permitted therefor, the numbers of selections exceeded the number of participants.

3
2
2
2
3
2
2
2
3
3
0 1 2 3
Other
“Legacy devices” already on the market
All software that is a medical device, except
mobile (smart phone) medical applications
All mobile medical applications
Medical devices considered part of an
interoperable system
Class II medical devices containing software
Class III medical devices containing software
Risk Management Postmarket Surveillance
N = 3
Respondents were
encouraged to select all

110
All four respondents whose devices used cyber threat telemetry regarded its use to assist software
updates in order to ensure continued high-quality operation (4/4) as a benefit. Three agreed and one
disagreed that it could be beneficial for future product development. One respondent suggested “other”
unspecified benefits related to cyber threat monitoring.
Figure 36 Benefits of Cyber Threat Monitoring
Q53 Has the analysis or presentation of cyber threat monitoring data been shown to achieve any of the
following benefits? Multiple answers were permitted therefor, the numbers of selections exceeded the
number of participants.

Respondents were also asked about the extent to which cyber threat monitoring satisfies certain
regulatory requirements and product advantages. Most indicated that it helped to satisfy FDA’s guidance
on “Postmarketing Management of Cybersecurity in Medical Devices” (3/4), as well as the legal
requirement of postmarket surveillance under section 522 of the Federal Food Drug and Cosmetic Act
(3/4). Most also saw this as providing a product feature (3/4). Two of four indicated that the capability
satisfies a European regulatory requirement for cyber threat monitoring (2/4), was utilized for QMS
postmarket surveillance (2/4) or satisfied risk management controls or detection methods (2/4). One
respondent indicated that cyber threat telemetry was being utilized for collecting data that is shared with
an Information Sharing Analysis Organization (ISAOs) (1/4). One also identified that it satisfied the
0 1 2 3 4
Other
Feedback for future product
development programs
Software updates to the current product
to ensure continued high-quality
operation
Yes No
N = 4
Respondents were
encouraged to
select all that

111
NIST requirements in the Framework for Improving Critical Infrastructure Cybersecurity (1/4) and one
that it helped to close a CAPA created by a previous AE or failure event (1/4).
Figure 37 Cyber Threat Monitoring Provides or helps to Satisfy
Q54 Check all of the requirements that cyber threat monitoring data satisfies for the device. Multiple
answers were permitted therefor, the numbers of selections exceeded the number of participants.

The respondents were asked if their company participated in an Information Sharing Analysis
Organization (ISAO). Most of the respondents confirmed that they did participate (3/4) and one
respondent indicated that they did not know (1/4). The names of the organizations in which they
participated included “NIST”, “HISAC” and Medtronic.

112
2.18. Data Collected from Participants not working with Targeted Device Types
The survey took the opportunity to collect feedback from 19 participants involved in the
development of “other” medical devices whose demographics varied widely (Table 31). This variability
made it difficult to interpret certain of the questions with which they were presented.
Table 31 Demographics of Non-Infusion Pump and Defibrillator Participants
Countries Company Size Participants Roles
12 - Global 7 - Large (>1,000
employees)
47% - Development, R&D or Engineering (9/19)
5 - US and Europe 42% - Regulatory Affairs (8/19)
1 - Unites States only 7 - Medium (201-1,000
employees)
32% - Other (6/19)
1 - Europe 21% - Quality Assurance / Risk Management (4/19)
1 - I do not know 5 - Small (1-200
employees)
11% - Postmarket Surveillance (2/19)

11% - Service Function (2/19)

11% - Product Security (2/19)
5% - Medical / Clinical Affairs (1/19)

Table 32 “Other” Roles of Non-Infusion Pump and Defibrillator Participants
no direct involvement. Conducted independent study about insulin pump.*
Sterilization
Commercial Development
Research
Technical Lead
This was a part of an earlier research project for children with ADD*
* These entries were adjusted to ensure anonymity of the respondents.
These “other” participants were asked about the types of data collected by their device(s). Half
identified that their device logged or LTM’d biometric data (50%, 9/18; 44%, 8/18 respectively), Many
also indicated that they logged (39%, 7/18) and LTM’d (33%, 6/18) device performance data, but few
indicated that their devices logged and LTM’d cyber threat data (6%, 1/18). A quarter did not perform
any of the above functions (28%, 5/18).

113
Figure 38 “Other” Cyber Threat Data Logged or LTM’d
Which of the following functions does the medical device perform? (Select all that apply)

Respondents were asked if they believed that the increased understanding of device performance
justified the additional cost to set up and maintain the telemetry capability. Half strongly agreed or agreed
that the additional cost was justified (strongly agree: 17%, 3/18; agree:33%, 6/18); several indicated that
they neither agreed nor disagreed (28%, 5/18). A few did not believe that costs were justified (22%, 4/18).
Figure 39 Does Increased Understanding of Device Performance Justify Cost?
Q8 Do you agree with the statement, "the increased understanding of the device’s performance in the
market justifies the additional cost to set up and maintain telemetric monitoring" ?

0
1
5
1
1
8
9
6
7
I don't know/cannot answer
Other
None of the above
LMT - cyber threat data
Log - cyber threat data
LMT - biometric data
Log - biometric data
LMT - performance data
Log - performance data
Respondents were
encouraged to select all
N = 18
0
4
5
6
3
Strongly Disagree
Disagree
Neither Agree nor Disagree
Agree
Strongly Agree
N = 18

114
This group was also asked about processes that should be considered to assist risk assessment.
Nearly all suggested that companies should consider the implementation of firmware cyber security
protection and continuous monitoring (94%, 17/18). Most indicated that postmarket software updates
(89%, 16/18), network cyber security protection and monitoring (83%, 15/18), data transmission
encryption (83%, 15/18), performance monitoring (67%, 12/18) and maintenance (61%, 11/18) should be
included. Fewer indicated that medical device fleet maintenance should be considered (39%, 7/18) and
one did not know (6%, 1/18).
Figure 40 Processes that Should Be Considered During the Risk Assessment
Q9 Which, if any, of the below processes should be considered during the risk assessment? (Select all that
apply)

1
0
7
11
12
15
15
16
17
0% 20% 40% 60% 80% 100%
I do not know
None of the above
Medical device fleet management
Device maintenance
Device performance monitoring
Device data transmission encryption
Data network cyber security protection…
Postmarket device software updates
Firmware cyber security protection and…
N = 18
Respondents were
encouraged to select all

115
Respondents were asked whether any of four suggested areas should be added to a postmarket
surveillance program for devices with telemetry capability. Most participants identified that the following
methods or requirements should be added in the order of preference: methods to identify, characterize,
and assess a cyber security vulnerability (78%, 14/18), requirement for device self-monitoring and
transmission of data (78%, 14/18), methods to analyze, detect, and assess threat sources (72%, 13/18) and
methods to communicate monitoring results (67%, 12/18). One participant did not believe that any of the
suggested areas should be added to the postmarket surveillance program (6%, 1/18) and one did not know
(6%, 1/18).
Figure 41 Additions to Postmarket Surveillance Programs for Telemetry Capable Devices
Q10 Which of the following areas should be added to a postmarket surveillance program for devices that
collect and transmit data? (Select all that apply)

1
1
12
13
14
14
0% 20% 40% 60% 80% 100%
I do not know
None of the above
Methods and frequency for communication
of monitoring results
Methods to analyze, detect, and assess
threat sources
Requirement for device self-monitoring and
transmission of data
Methods to identify, characterize, and assess
a cyber security vulnerability.
N = 18
Respondents were
encouraged to select
all that applied.

116
Chapter 5. Discussion
2.19. Methodological Considerations
5.1.1. Overview of Research
Rapid advances in radiofrequency technology, cloud-based data storage and miniaturized low-
cost sensors have given medical device developers the opportunity to interact with devices even when
they are implanted or out of reach. Using telemetry, these designers and users can now identify issues that
can trigger corrective actions before patients are harmed. Such visibility regarding product performance
cannot come fast enough, given the large numbers of adverse events that can come from the use of
medical devices. FDA’s Adverse Event database for the years 2006-2011 identified that 9,721,395
computer-related medical devices were subject to recalls associated with causes such as battery issues,
hardware failures and input/output errors. (Alemzadeh et al., 2013) It is therefore reasonable to ask
whether extending postmarket monitoring to exploit medical device telemetry would improve the
performance and safety of those devices and thus the safety of patients using them. To date, relatively
little has been written about how the medical device industry is using performance telemetric data to
improve device function. Here we have explored how this is done in one important subset of medical
device companies, those developing defibrillators and infusion pumps. Before examining the implications
of those results, it is worth taking a look at the delimitations and limitations that might affect the
interpretations and conclusions from this research.
5.1.2. Delimitations
The goal of the survey was to gather information about telemetry usage from participants in the
US medical device industry. The focus on US manufacturers was done to limit the scope of the study to
US regulations for this initial exploration into the telemetry usage. Nevertheless, some of these results
may generalize to other countries, as suggested by the literature review in chapter 2. It would, however,

117
be valuable to conduct a follow-up survey of global medical device manufacturers prior to drawing
detailed conclusions because many countries are in the process of upgrading their regulations to provide
clearer direction on how this technology should be used to keep medical devices safe and effective.
The study was further narrowed to consider only infusion pumps and defibrillators. These
products were chosen specifically because they were responsible for at least 35 US recalls and 392
fatalities between 2006-2011. (Alemzadeh, 2016) The significant safety, reliability and technological
challenges were felt to warrant their deeper evaluation. The collected data from the survey reflects
variations in the extent of telemetry usage of implanted and interventional devices as well as drug
delivery systems and therefore gave us a window into the spectrum of approaches across different device
types. By using three types of products, I hoped to gain preliminary insight into some of the differences in
implementation that might be overlooked if only one product was used. The variations in the use of
telemetry and in the views of developers suggest caution in generalizing the results to other types of
products, because telemetry is a tool that can have different implementations and challenges across
devices and applications. By understanding commonalities and differences in the implementation across
device types, it might be possible for future researchers to study in more detail the most interesting
observations.
The scope of the study was also delimited by its primary focus on telemetry performance
monitoring. My literature search established that performance monitoring via telemetry was possible but
patterns of use were unclear and specific regulatory direction was lacking. A secondary focus on cyber
threat monitoring as a particular element of performance monitoring was felt to be important because
much has been written about cyber threat concerns with medical devices. (Brubaker, 2020) (FDA, 2019b)
Further, cybersecurity has been a topic of recent FDA guidance documents that lay out regulatory
expectations, although these have not yet been formalized as regulations. (FDA, 2018a) Because this was
a subsidiary topic, questions regarding cyber threat monitoring were mostly placed toward the end of the
survey. Such a placement might be problematic if respondents were to drop out of the survey prematurely.

118
To understand if this would be an issue, participants were asked an early question about cyber threat
monitoring, to see if those same respondents continued to the end of the survey. Unfortunately, only 8
respondents working with infusion pumps or defibrillators indicated that their devices used telemetry for
cyber security purposes. Social science research underlines the importance of having a significant number
of views to draw meaningful conclusions that can be generalized to the whole industry. (Orcher, 2014)
The few responses that could be gathered were nonetheless interesting. The fact that so few devices had
this capability suggests a need to examine more fully the approaches used to ensure cybersecurity, which
seem still to be underdeveloped in at least these medical devices. This is discussed in more detail below.
The use of a specific framework also defines the scope of the survey, so is a de facto delimitation.
Frameworks are important to help assure a balanced and comprehensive approach to the questions
presented in the survey. (Oppenheimer, 2017) In this study, an implementation framework appeared to be
the best match to the goals of the research. An extended version of Fixsen’s Quality Implementation
Framework, commonly used for studying implementation, (Meyers et al., 2012) was valuable to ensure
that questions addressed exploratory as well as implementation stages. I expected that this framework
would help to focus on organizational strengths that encourage implementation as well as weaknesses that
cause delays. However, other aspects of telemetry, outside the scope of this framework, were
consequently not considered. Such elements included, for example, the effectiveness of the
implementation for any of the three products, the cost of implementation or privacy concerns associated
with monitoring of telemetry data.
In this study, recruitment was delimited to industry professionals from software, design, service,
clinical, regulatory and quality engineering areas who support product development or business sustaining
efforts, Within these groups, I sought individuals who were knowledgeable about telemetry usage or
postmarket surveillance, and would therefore be most likely to possess a relatively deep knowledge of the
product’s technical and development history. Responses to the initial demographic questions confirmed
that the sample included a broad distribution of individuals with different functional roles. Surprisingly,

119
though, individuals with roles involving information technology seemed underrepresented, accounting for
only 2 of the 85 respondents. These individuals have a particularly valuable set of insights, so it might be
useful in future research to assure that this group is better represented or even studied as a singular
population.
Other types of individuals who interact with the devices, such as patients or medical staff, were
excluded deliberately. These individuals were considered to be less familiar with the product development
process and full scope of telemetry usage. Because large parts of this study focused on implementation
aspects while the product was being designed, the restriction to industry-related personnel was seen to
avoid the inclusion of participants who could not answer most questions.
5.1.3. Limitations
Survey methods have several well-known limitations that can also affect validity if they are not
properly managed. In this survey, I was most concerned about three types of potential limitation: assuring
a suitable sample of participants; ensuring that the participants completed the survey; and assuring that
the survey structure and questions were suitable to collect good information related to the goals of the
research.
5.1.3.1. Obtaining Participant Enrollment
It is important to set boundaries around a research survey, but these boundaries can make it
difficult to assure sufficient recruitment of qualified individuals, a central requirement of a strong survey-
based research program. (Sadler et al., 2010) At the same time, work by Lawrence Orchard and others
suggest that a well-designed and executed study can reach saturation with as few as 30 participants if the
respondent pool is quite homogeneous. (Orcher, 2014) The reason for confidence in this relatively small
pool of respondents is that qualitative researchers can identify participants who are “information-rich,”
and can provide a high level of insight. This puts responsibility on the researcher to identify well-
qualified participants rather than large numbers of less qualified people to take the survey. Thus,

120
considerable effort was expended to assure that more than 60 respondents were part of the survey and at
least 40 had experience with telemetry for performance monitoring across the three device types. The
survey closed with 66 respondents from the targeted medical device families that met the inclusion
criteria. This respondent number appeared sufficient to provide a large number of responses that seemed
to reach “saturation” as described by Weller (Weller et al., 2018) when addressing most of the prominent
opportunities and challenges of performance monitoring at this exploratory stage of research.
The study relied on a survey that was distributed electronically. This approach allowed me to
quickly reach a larger number of knowledgeable participants. Surveys are not the only method available
to gain the input from participants; individual interviews or focus groups have often been used to poll
groups. One of the major benefits of surveys is that it allows respondents to provide their views and
experiences more efficiently at times that fit into their busy lives. These considerations have previously
been highlighted for studies of busy medical product professionals, who are also highly literate and
accustomed to computerized methods of interacting with others. (Magee et al., 2001)
However, electronic surveys have risks as well. A survey delivered by email is easily deleted or
lost in a long message list. To avoid losing the participation of these individuals, reminders were sent to
encourage potential participants. Further, as Magee (2001) and Sheehan (2001) have recommended, a
significant effort was made to engage potential participants individually using one-on-one
communications prior to disseminating the survey to cultivate their interest and commitment. This
technique proved to be successful based on the fact that the survey was open for 2 months and collected
responses from 85 individuals over that time. Additionally, the response rate was 75%, much higher than
typical 25% - 30% for email electronic surveys without follow-up emails and reinforcements. (Sheehan,
2001, Fincham, 2008)
5.1.3.2. Obtaining Survey Completion

121
Another limitation that can affect a survey is its potential length. Respondents who are presented
with a long survey might complete only a subset of the questions before “losing their breath” or
“dropping out”. (Galesic and Bosnjak, 2009) Survey fatigue resulting in high participant drop-out is a
limitation often highlighted by survey experts. (Pew Reserch, 2020, Galesic and Bosnjak, 2009) As
recommended by Nassar-McMillan (Nassar-McMillan et al., 2010), the survey questions were critiqued
by a focus group to ensure that the questions were appropriate and well-stated, in order to reduce dropouts
that occur because participants cannot answer questions. Particularly important in the effort to reduce
survey length was the use of “skip logic” which ensured an individual would only be presented with
appropriate questions, as recommended by the Crawford, in his journal article Applying Web-Based
Survey Design Standards. (Crawford et al., 2005) Crawford also noted that the way a question is written
and displayed must be scrutinized more carefully than when the human interviewer is available. The fact
that so few participants abandoned the survey prematurely (2/85) seems to be strong evidence that the
survey topic was important to the respondents and that the questions kept them engaged.
5.1.3.3. Survey Content
The way that questions are stated is not important only to assure respondent engagement. It is
central to assuring that the goals of the research are met. As questions are formulated, it is therefore
important that previous studies be analyzed carefully to guide the content and depth of the questions.
However, this area of research is new and has enjoyed little previous systematic analysis, as shown by the
exhaustive search that was conducted in Chapter 2.6.1 and that yielded only a few publications related to
the use of telemetry for product performance and cyber threat monitoring. From that previous work, it is
unclear whether the medical device industry had mostly failed to implement telemetry usage or had not
revealed publicly the extent of its use. The “Considering Results” section now permits me to discuss some
of these questions in more detail.

122
2.20. Considering Results
The objective of this survey was to explore how far the medical device industry has progressed in
the use of telemetric monitoring to assure the reliable operation of their devices. Relatively little could be
found in the literature to address this specific aspect of telemetry use. In contrast, many publications,
standards and guidances addressed the use of biometric telemetry and the expectations related to
cybersecurity. Thus, it was surprising that to find that the manufacturers of infusion pumps and the two
forms of defibrillators all used telemetry as extensively for device performance as well as biometric
monitoring but took advantage of it so little to assure cybersecurity.
5.1.4. Telemetry for Device Performance
In the past, the use of telemetry has been showcased as a tool to monitor patient health and
disease. (Rouse, 2005) The literature review demonstrated at least 81 examples in which telemetry was
used to monitor patient biometrics such as tremor frequency and intensity (Ripin and Chan, 2017), cardiac
resynchronization (Brown, 2017), compartmental forces in the body (Safaei, 2019) and head injuries.
(Nowinski, 2000) In these cases, telemetry provides a product feature from which the patients could
benefit, either by obtaining information directly from their devices or by allowing physicians to access the
data and use it to adjust their medical care.
One might think that telemetric methods would also provide an obvious opportunity to monitor
device condition in much the same way as it the does patient condition. Deviations in device performance,
communicated telemetrically, could signal that a repair or adjustment was needed to avoid device failure.
(Fiix Inc., 2019, Microsoft, 2018) Thus, it was surprising that literature relating to such use was much
harder to find. Compared to biometric monitoring, only two percent of the uses reviewed here addressed
the use of telemetry related to device performance. These 14 cases documented the use of telemetry for
assessing changes in electrode impedance in cochlear implants (Zeng, 2019), battery resistance in
pacemakers and implantable cardioverter-defibrillators (Ellenbogen, 2011) and wired or wireless

123
transmission problems for devices detecting heart rate or oxygen level. (Nyansa Inc., 2019) However, the
current survey suggests a much broader use of telemetry for performance monitoring. This finding makes
it clear that the medical device community, or at least the part making of infusion pumps and
defibrillators, has found a way to incorporate such telemetry into performance monitoring as well. One
might argue that these types of devices are particularly vulnerable and might therefore tend to use such
monitoring more than other types of devices. However, the fact that performance telemetry monitoring
use was also reported commonly by the subset of respondents who worked with other types of devices
suggests that the technology is being adopted quite widely.
Further. the current results suggest that the uses of performance-related telemetry are quite broad.
Why then is it so difficult to find examples in the literature? It maybe that such uses are viewed as a
proprietary activity that companies do not want to share publicly. They also may feel that some functions,
such as monitoring the state of battery charge, are so obvious or widely used that they are not worth to
mention in the literature. Alternatively, device designers may not see academic publication as a high
priority. Publication is often seen by academia as a gold standard of professional performance (Rawat and
Meena, 2014) but it is seldom a metric for advancement or financial reward in industry.
No regulations or guidance documents require that telemetry be used to monitor equipment
performance. However, CFR 21 part 820: Quality System Regulation (QSR), does require that devices are
designed to be reliable and have complaint monitoring to assure ongoing safe function. (FDA, 1996)
Device performance is also a subject of discussion in the FDA Guidance titled, “Design Control Guidance
for Medical Device Manufacturers”. (FDA, 1997) So clearly there is a benefit for implementing
performance telemetry monitoring apart from opportunities to publish in academic journals. It is possible
that companies using telemetry to market performance monitoring tools would view usage as part of the
product specifications or capability and could lead to demonstrations being showcased at industry
conferences or within trade journals. Quality and regulatory organizations might consider partnering with
these types of industry events to encourage the expanded use of this capability.

124
5.1.5. Cybersecurity as a Subset of Device Performance
Much has been written about cybersecurity weaknesses in medical devices, including the types of
devices examined in this study. For example, Haworth (Haworth, 2019) has described software flaws in
Becton Dickinson Alaris Gateway Workstations, used to run infusion pumps. The Department of
Homeland Security’s Industrial Control Systems-Cyber Emergency Response Team (ICS-CERT) has
warned about vulnerabilities in the GE Healthcare Aestiva and Aespire devices. (Donavan, 2019) In
response to such concerns, the FDA and NIST have released several guidances over the last 10 years
encouraging medical device manufacturers to strengthen the cybersecurity of their devices. The NIST
standard, “Framework for Improving Critical Infrastructure Cybersecurity”, also provides information on
the best practices to ensure corporate and hospital cybersecurity. (National Institute of Standards and
Technology, 2018) In more immediate areas of medical device quality and reliability, these expectations
appear in several technology-specific guidances and standards by the FDA and others. To guide
premarket considerations, FDA published the guidances, “Content of Premarket Submissions for
Management of Cybersecurity in Medical Devices” (FDA, 2014) and its revised version “Content of
Premarket Submissions for Management of Cybersecurity in Medical Devices”. (FDA, 2018a) The FDA
also formalized its expectations on postmarket cybersecurity in the guidance, “Postmarket Management
of Cybersecurity in Medical Devices”. (FDA, 2016) If those publications were not enough, the FDA has
also recognized several consensus standards that highlight the importance of ensuring cybersecurity and
strongly encourages manufacturers to participate actively in Information Sharing Analysis Organizations
(ISAOs). (FDA, 2016)
Given the significant number of guidances and standards, one might expect that medical device
manufacturers would be using telemetry to implement cyber threat monitoring, at least to the same extent
as device performance monitoring. However, the results suggest that this is clearly not the case. The
failure to implement cybersecurity protections may explain why a targeted literature search of indexed
newspaper articles, articles, dissertations, conference proceedings, books and reviews from 2016-2021

125
only provided few practical examples where telemetry was used to monitor cyber threats. As part of this
work, I reviewed 691 publications that contained references to medical devices using telemetry. Very few
of these discussed telemetry as tool for cyber threat monitoring. The three articles that had the greatest
relevance to the topic discussed the use of telemetry to monitor cyber threats of medical devices through
the use of adjacent devices that were connected to the medical devices or systems. Zhang, in 2013,
discussed transmission monitoring of cyber threats using a software to detect transmission anomalies.
This external device was able to warn patients and jam the suspicious transmission before the threatening
intrusion could damage the functionality of the medical device. (Zhang et al., 2013) Newaz (Newaz et al.,
2020) discussed an intrusion detection system and Rathore (Rathore et al., 2017) described three external
cyber threat detection and protection devices, Shield, IMDGuard and the Cloaker, that work with
implanted medical devices to ensure their cybersecurity. An occasional additional article made only
theoretical or minor references to this topic. (Abdulhammed et al., 2017, Almazyad et al., 2020)
The fact that only a few of the surveyed device manufacturers (14%) have incorporated cyber
threat monitoring into their telemetry programs seems problematic given the number of regulations and
standards on the topic and the consequences of device attacks for patients who depend on these life-
sustaining products. There may be at least two reasons why so little action appears to be taken in this area.
One interesting possibility is that the many guidances in the absence of specific regulatory requirements
has had the unintended effect of impeding cyber threat design changes to the product. One respondent
suggested that it was…
…difficult to gain buy in from upper management and key stake holders due to lack of
understanding of cyber security and related risk/mitigation models and perceived
rigidness of requirements from regulatory bodies in the implementation of these
processes. To unknowledgeable stake holders the fear of miss handling a cyber
security process from a regulatory perspective outweighed their perceived cyber
security risk.
Alternatively, many in industry may see cyber intrusion as a problem that does not warrant the
expenditure of time and money needed to put protections in place. Publicly acknowledged cyber-attacks

126
have rarely been reported for implanted medical devices. (Mukhopadhyay and Islam, 2017) If the small
number of reported cases reflects accurately the state of the industry, it may be that the low frequency of
cases has reduced the risk of the event and therefore made monitoring a low priority to device designers.
As another respondent suggests,
There has never been a cyber attack on a defibrillator. There have been reverse
engineered simulations done. It is easier to steal a programmer than reverse
engineering one. Currently the telemetry systems do not adjust functionality of
defibrillator.
In this study, I attempted to explore the implementation of cyber threat monitoring. However,
because its implementation was so rare, the number of respondents who could comment on its use was
too small to justify confident conclusions. What it did do, however, was provide a few interesting
observations that could open the door to further exploration in future with a targeted respondent group,
hopefully enriched with software and information technology experts.
5.1.6. Device Performance LTM Implementation
Insights into the implementation of telemetry reported here were guided by using Fixsen’s
Quality Implementation Framework (Fixsen et al., 2015) that divided the implementation sequence into
steps that could be examined individually. The discussion that follows considers implementation in
cascading phases of exploration, installation, implementation and business sustaining. Despite these
divisions, I acknowledge, as have others (Meyers et al., 2012), that these steps are not always done
independently or iteratively and can even blend into one another.
5.1.6.1. Importance of Guidances and Standards
As teams begin to explore a new method or technique, they often must seek advice from
colleagues and published materials to develop the new knowledge that they will need to plan the new
initiative. In this study, it seemed clear that many teams had deficits with respect to telemetry
implementation that required additional resources at the exploration stage. A resource ranked as more

127
important than many others was the availability of documentation, especially regulatory documentation
and advice. Thus, the early phase team would be interested in the content of several documents produced
by regulators and industry experts that attempt to instruct on various aspects of implementation.
(Humayed et al., 2017, Longras et al., 2020)
It is not unusual, however, for a new technology to precede regulations and even helpful “how-
to” or guidance documents by several years. The FDA has many technical advisory committees to help
them to modernize their scientific capabilities, but it remains difficult for them to keep up to date with the
newest of science and medicine. (Rettig et al., 1992) It is not surprising, then, that regulators are looking
to international standards to provide an authoritative reference for specialized technologies. The
production of such standards often depends on the participation of companies on the cutting edge of those
technologies. (Schmidt, 2017) The finding that international standards were ranked as most important,
compared to FDA guidance and NIST standards, is not surprising given that international standards are
becoming increasingly accepted by national regulatory agencies as “best practices” for many types of
technical activities. (SEBoK Editorial Board, 2017) Nevertheless, standards development is also a slow
and labor-intensive process. Thus, the developers of new technologies are only able to rely on past
standards that can be somewhat outdated when a field is developing quickly.
Some standards are also more detailed than others. In many cases, these documents will aim to be
as broad as possible to encompass activities in a wide range of applications. (Schmidt, 2017) However,
breadth can have its drawbacks. In this survey, the perceived value of regulatory documents at the
exploration phase seemed to diminish as the specificity of the documents increased. Some respondents
felt that they did not offer sufficient granularity and practical advice, as reflected in the comment below:
… The standards inferred, they suggested, and they spoke at high level
The same criticism might also be anticipated for FDA guidance documents. FDA has made a
deliberate effort to provide industry with guidance on what needs to be done without describing exactly

128
how to do it. In fact, all FDA guidance documents include within the first couple of pages a statement that
says, “You can use an alternative approach if it satisfies the requirements of the applicable statutes and
regulations.” (FDA, 2005a)
At the other extreme of familiarity were cybersecurity documents, identified as either unfamiliar
or poorly useful by most. At first glance this may seem unexpected, given the significant attention that
this topic has attracted for regulators and even the popular media. However, few of the respondents had
product portfolios that included devices with cyber monitoring capabilities, so they may have had no
reason to consult these guidances. Some unexpected feedback even suggested that the cybersecurity
guidances and standards may have contributed to a reluctance of manufacturers to engage in cybersecurity
interventions:
….difficulty making case based on regulatory documentation due to lack of
understanding by non-technical stake holders. all or nothing mindset with regards to
implementing cyber security practices with a leaning towards not implementing
practices.
The types and quality of deliverables developed during the exploration stage are foundational to
the subsequent implementation and can make or break the success of the implementation stage. (Fixsen et
al., 2015) Our survey results indicate that several deliverables, including requirements documents,
regulatory and risk assessments, as well as scoping and benefits documents, are being provided quite
consistently at the end of the exploration stage. Most of these documents are prescribed by the FDA in
their Design Control Guidance for Medical Device Manufacturers. (FDA, 1997) Our respondents also
confirmed in later questions that having risk management plans, well- defined requirements and a clear
scope were important factors in assuring the successful implementation of telemetric approaches. In this
case the FDA guidance has proven to be on target for directing medical device manufacturers to
approaches that can foster success with their implementation.
5.1.6.2. Major Impediments Encountered

129
Implementation of telemetry is a complex undertaking in which several streams of activities must
be completed before the new technology is included in routinely manufactured product. Companies
appeared to be grappling with the typical problems of any implementation project, including the
development of specifications and verification tests or the identification of contract testing labs. Some of
these challenges were apparent from comments such as these from two respondents:
Making sure that testing satisfied the many variables of FDA requirements
Testing labs were also not familiar with test requirements
The importance of well-skilled personnel is well known to be important in new projects. For
example, Fixsen’s model for implementation, described in chapter 2, calls out seven critical drivers that
can strengthen the implementation process (Fixsen et al., 2009), and three of those seven are directed at
staffing. They include ensuring adequate staffing levels; providing consulting support; and assuring
adequate staff training. Meyers reviewed several other implementation framework models and most of
them reiterated the critical importance of resources, training and ongoing technical assistance to assure a
quality implementation. (Meyers et al., 2012) Present results confirm that industry professionals also
recognize the importance of staffing, training and consulting support when teams expand their capabilities
by incorporating or expanding the use of telemetry in their products. Further, concerns were expressed
about the hiring and training of staff, as suggested by quotes such as the following:
The problems were that the team had little experience, the consultants had little
experience, the FDA had little experience and the standards did not clearly state what
was needed.
Not enough ppl
During implementation, processes are put into practice and are gradually refined to assure
technical reliability. (Iadanza et al., 2019) Getting to this point can take time. In the present study,
changes in timelines reflected the unexpected complexity of such implementations. Half of the

130
respondents indicated that the launch of their telemetry monitoring was delayed by 2 months to 2 years
compared to the initial project schedule. Most commonly, two reasons, a lack of resources and evolving
regulations, were identified as responsible for the delays. Facilitative administrative leadership and
support was the third area identified as an important factor for many. Although most participants
indicated that they did have active leadership, the expectations of that leadership could be difficult to
satisfy, as reflected in the comment, “project timelines were aggressive”. Aggressive timelines for
projects are not necessarily problematic. They can serve as “stretch goals” if they are accompanied by
sufficient personnel and resources to accomplish the needed tasks. However, poor technical understanding
or support of novel projects can reduce the motivation and performance of a team. (Gauthier and
Gauthier, 2020)
Another concern that seemed to emerge as implementation progressed was the need to educate
not only the staff but the regulators with whom the company was working. This need for additional
training was self-identified by the FDA as an area of concern, not helped by their high reviewer turnover
and the increased workload associated with increasing device complexity. In their report “Medical Device
Pre-Market Programs: An Overview of FDA Actions”, the FDA announced their intent to shift their
culture to one that makes greater use of external experts. (FDA, 2011) The survey respondents indicated
that the FDA is also using the companies themselves as educators, as illustrated by the comments of two
respondents:
…difficulty making case based on regulatory documentation due to lack of
understanding by non-technical stake holders.
First to market challenges meant we helped develop the regulatory requirements.
5.1.7. Device Performance Monitoring Expectations
Manufacturers of medical devices sold in the United States are expected to develop their products
according to the requirements title 21 of the Code of Federal Regulations section 820 whereas

131
manufacturers that sell devices in other countries typically must be compliant with the ISO 13485
standard. Both sets of requirements take a life cycle approach. Of the two, the ISO document makes
stronger statements about the use of feedback signals to monitor performance, in section 8.2 Monitoring
and measurement: (ISO, 2016)
The organization shall document procedures for the feedback process. This feedback
process shall include provisions to gather data from production as well as post-
production activities. (ISO, 2016).
In contrast, in 21 CFR part 820, which was written twenty years prior, says little about
performance feedback, and certainly says nothing about telemetered signals. Part 820 only refers to data
monitoring when it is associated with production processes. Instead, it is, at best, implied by a section on
complaint handling. It was however positive to see that when an AE or a device failure occurred, some
companies were using telemetry as their method to satisfy 21 CFR section 820.100 Corrective and
Preventative Action or in some cases are using it to satisfy a requirement for postmarket surveillance.
In 2018, the FDA announced their intention to move toward ISO 13485 in the future (FDA,
2018b) (Lewis, 2018) with the implementation expected to start in mid-2019. However, to date, no
updated announcements have been made by the FDA to clarify this transition. However, the move would
be useful. Global alignment of regulations is generally good for manufacturers because it reduces the
number of country-specific hurdles that need to be overcome. (ISO, 2015) In this particular case, it also
would help to bring attention to importance of post market device monitoring for high-risk devices.
Device telemetry will be implemented for different purposes depending on the focus of the
monitoring staff. Biometric data can be explicitly designed for the use of patients and their health care
professionals to better understand the health status of the patient. These same professionals may also use
certain types of performance data to prompt interventions such as battery changes or other user-
predefined maintenance activities. However, much of the medical device’s performance data is directed to
specialized service teams so that they can identify performance issues that would require a trained

132
engineer or company specialist to fix the device. Nonetheless, respondents in this survey mostly agreed
that manufacturers and medical staff have a joint responsibility to monitor the operational state of these
medical devices and thus the data that the telemetry provides. These findings may not be surprising.
Recent literature has also suggested that pacemakers and defibrillators are monitored by medical staff by
using their telemetry data. (Camara et al., 2020) However, the effectiveness of that monitoring is open to
question. Maines and colleagues (Maines et al., 2020) (Maines et al., 2020)have cast doubt on the
consistency with which medical facilities are performing these monitoring activities, which can be costly
depending on the required method and frequency of data monitoring.
Device manufacturers also monitor telemetered data associated with device performance,
especially if they are going to provide service and support for the products over their lifecycle. Stryker,
for example, monitors performance to detect issues early so that preventive maintenance that can be
conducted before the device fails. (Stryker, 2019) Similarly, other companies use it to conduct remote
diagnostics, predict maintenance needs and to develop upgrades to their products. (Quasim et al., 2019)
(Quasim et al., 2019)What does seem obvious is that the performance data collected by telemetry should
be monitored by someone, or the purpose of this added expensive feature will be lost. It was therefore
disappointing that one respondent, who worked with external defibrillators, reported that their telemetry
data was not being reviewed at all. Given the criticality of defibrillator performance in an emergency
situation, such neglect could be dangerous and even cost lives. It is unclear whether this statement is
really true or merely represents the respondent’s more limited exposure to the ways that telemetered data
could be used by different teams in that organization. Given the criticality of defibrillator performance in
an emergency situation, such neglect could be dangerous and even cost lives.
At the same time, substantial variability was seen in response to questions about how
performance data is presented to the user. This may indicate the flexibility of the tool or be evidence that
device performance monitoring is at this point an underdeveloped way to conduct post marketing
surveillance. Relatively few manufacturers are required by the FDA to monitor device performance after

133
commercialization. Further, when it is required, that surveillance is typically expected to last a maximum
of 36 months. (FDA, 2002b) (FDA, 2002b)Given that performance monitoring through telemetry is
nearly as common for devices as is biometric monitoring, it seems reasonable for regulators to recognize
the value of this source of performance information and to recommend or even require that data from
telemetric sources be monitored over the life of the device. This may be especially important because it
may be a way to detect better the incidence and cause of adverse events in the field. Typically, adverse
events are reported infrequently by health care professionals and patients. (Banerjee et al., 2019) When
such reporting does occur, an expensive investigation is needed to determine its root cause. Telemetric
information could shorten the investigation by highlighting or refuting any concerns about device
performance issues or cyber threats. Since this capability has proven to be a valuable addition to improve
health care, plans to ensure that not only are biometric telemetry but also the device performance and
cyber threat data monitored by medical staff. (Maines et al., 2020)
Given the multiplicity of ways that telemetry can be used, it is not surprising that most survey
respondents viewed it as helping to improve their company’s understanding of their products. What was
more surprising, however, was the fact that most respondents felt that regulators should require telemetry-
capable devices to monitor performance parameters for postmarket surveillance and risk management
activities. A requirement to monitor such data will impose new staffing costs and compliance
expectations. Further, a system yielding information about device failure that goes unmonitored until a
patient is injured will undoubtedly be litigated at some point as negligence. (Anonymous, 2014,
Anonymous, 2021) Give the potential value of the data for postmarket surveillance, however, it seems
reasonable to upgrade the regulatory expectations for medical device manufacturers in the postmarket
phase as best practices, if not requirements.

134
2.21. Conclusion and Future Direction
The medical device product industry has gone through an enormous change since 1976 when the
Medical Device Amendment was adopted. Now medical devices have not only the ability to diagnose and
improve health, but to monitor how outcomes are progressing through telemetric connections even when
devices are implanted or hard to reach. These advancements in monitoring have a great deal of potential
to improve patient care and ensure reliable device function. However, it seems that manufacturers are
leading regulators in their expectations for performance monitoring. Manufacturers from the products
included in this research have proven that they can monitor and learn from their device telemetry data
even without a specific directive to do so from regulators. Adoption of this capability, however, may not
be universal. Further, some aspects of this technology, such as its usefulness to detect cyber intrusions,
are still underdeveloped, despite the regulatory attention that this particular aspect has attracted.
It takes time, resources and technical capability to implement performance and cyber threat
monitoring. Companies need to build these critical elements into their implementation plans from the
exploration stage through business sustaining stages. Further, regulators could add value by explaining to
manufacturers what they view as responsible actions related to device performance monitoring. At least in
the areas of implantable defibrillators, infusion pumps and external defibrillators, industry appears
capable and willing to progress along a path to include this data in their risk management and post-market
surveillance programs. How far these practices extend to monitor the performance of other device types is
still unclear. The results here suggest that additional research should be conducted to see if other medical
device types are equally capable of using telemetry to conduct performance monitoring and whether such
an approach would be seen to have value considering the upfront cost of implementation.
One particular area of concern that is highlighted by this study is that of cybersecurity. The
literature is rife with warnings about potential, if not actual, safety threats posed by unprotected cyber
capable devices. (Longras et al., 2020) (FDA, 2017a) However, what is acceptable from a cybersecurity

135
and monitoring point of view still requires more definition. As this work shows, changes in cybersecurity
activities may not be initiated voluntarily. Instead, cybersecurity implementation may need a push
stronger than that currently given by guidance documents and standards whose recommendations are
currently unenforceable.

136
REFERENCES…..
94th U.S. Congress (1975). H.R. 11124: Medical Device Amendments. U.S. House of Representative Bill
Summary & Status. Library of Congress THOMAS
Abdulhammed, R., Faezipour, M. & Elleithy, K. Malicious behavior monitoring of embedded medical
devices. 2017 IEEE Long Island Systems, Applications and Technology Conference (LISAT), 5-
5 May 2017 2017. 1-6.
Alemzadeh, H.(2016) Data-driven resiliency assessment of medical cyber-physical systems. Doctorate of
Philosophy Electrical and Computer Engineering, Champaign, IL, University of Illinois at
Urbana-Champaign.
Alemzadeh, H., Iyer, R. K., Kalbarczyk, Z. & Raman, J.(2013) Analysis of Safety-Critical Computer
Failures in Medical Devices, IEEE Security & Privacy, 11(4), 14-26. Available at:
https://ieeexplore.ieee.org/abstract/document/6509886 (Accessed: 16 April 2020).
Alexander, B., Haseeb, S. & Baranchuk, A. (2019). Are implanted electronic devices hackable?, Trends
in Cardiovascular Medicine, 29(8), pp. 476-480.
Almazyad, I., Rao, A. & Rozenblit, J. A Framework for Secure Data Management for Medical Devices.
2020 Spring Simulation Conference (SpringSim), 18-21 May 2020 2020. 1-12.
Anonymous (2014). Failure to monitor, treat caused patient’s death, estate claimed. VerdictSearch:
VerdictSearch. Available at: https://verdictsearch.com/verdict/failure-to-monitor-treat-caused-
patients-death-estate-claimed/ (Accessed 11 May 2021).
Anonymous (2021). $4,000,000 Medical Malpractice Award When Telemetry Unit's Failed Response
Leads to Brain Injury. Available at: https://www.lipkinapter.com/results/medical-
malpractice/telemetry-unit%E2%80%99s-failed-response-leads-brain-injury-and-medical-
malpract (Accessed 11 May 2021).
Banerjee, S., Campbell, B., Rising, J., Coukell, A. & Sedrakyan, A.(2019) Long-term active surveillance
of implantable medical devices: an analysis of factors determining whether current registries are
adequate to expose safety and efficacy problems, BMJ Surgery, Interventions, & Health
Technologies, 1(1), e000011. Available at: http://sit.bmj.com/content/1/1/e000011.abstract
(Accessed: 16 April 2020).
Brown, J. R.(2017) Cardiac Resynchronization Therapy Device-Programmed and Device-Measured
Parameters as Predictors of Outcomes in Patients with Heart Failure. Doctorate of Philosophy
Philosophy, University of Minnesota.
Brubaker, K. (2020). Why Cybersecurity Awareness Is Important for Every Employee. BizLibrary.
Available at: https://www.bizlibrary.com/blog/compliance-safety/why-cybersecurity-is-
important/ (Accessed 24 April 2021).
Burns, A. J., Johnson, M. E. & Honeyman, P. (2016). A Brief Chronology of Medical Device Security,
Communications of the ACM, 59(10), pp. 66-72.

137
Callisch, D. (2019). New Healthcare IT Survey Reveals How Harnessing New Digital Technologies, IoT
Systems and Device Data Are Reshaping Industry Priorities. New York: PR Newswire. Available
at: https://www.prnewswire.com/news-releases/new-healthcare-it-survey-reveals-how-
harnessing-new-digital-technologies-iot-systems-and-device-data-are-reshaping-industry-
priorities-300792812.html (Accessed 16 February 2020).
Camara, C., Peris-Lopez, P., Fuentes, J. M. D. & Marchal, S. (2020). Access Control for Implantable
Medical Devices, IEEE Transactions on Emerging Topics in Computing, pp. 1-1.
Chapman, M. H. (1994). Rx: Just what the doctor ordered: International Standards for Medical Devices,
Northwestern Journal of International Law & Business, 14, pp. 566-601.
Chenok, D. J. (2012). Information Security and Privacy Advisory Board Letter to Acting Director of the
US Office of Management and Budget. Available at:
https://csrc.nist.gov/CSRC/media/Projects/ISPAB/documents/correspondence/ispab-ltr-to-
omb_med_device.pdf (Accessed 17 April 2020).
Committee on Fetus and Newborn (2003). Apnea, Sudden Infant Death Syndrome, and Home
Monitoring, American Academy of Pediatrics, 111, pp. 914-917.
Committee on the Public Health Effectiveness of the FDA 510(k) Clearance Process & Board on
Population Health and Public Health Practice (2011). Medical Devices and the Public's Health:
The FDA 510(k) Clearance Process at 35 Years, Washington D.C., National Academies Press.
Crawford, S., McCabe, S. E. & Pope, D. (2005). Applying Web-Based Survey Design Standards, Journal
of Prevention & Intervention in the Community, 29: 1-2, pp. 43-66.
DeMarco, C. T. (2011). 2-Medical Device Design Medical Device Design and Regulation. Meinholz, M.
T. edn. Available at: https://ebookcentral.proquest.com/lib/socal/detail.action?docID=3002641
(Accessed 18 April 2020)
Dolan, A. M. (2004). Risk Management of Medical Devices: Ensuring Safety and Efficacy through ISO
14971, ASQ World Conference on Quality and Improvement Proceedings, 58, pp. 69-72.
Donavan, F. (2019). Remote Attacker Could Manipulate GE Healthcare Medical Devices. xtelligent
Healthcare Media. Available at: https://hitinfrastructure.com/news/remote-attacker-could-
manipulate-ge-healthcare-medical-devices (Accessed 23 Mar 2021).
Ellenbogen, K. A. (2011). Clinical Cardiac Pacing, Defibrillation, and Resynchronization Therapy,
Philadelphia, Pa, Elsevier/Saunders.
European Commission (2016). Regulation (EU) 2016/679 of the European Parliament and of the Council
of 27 April 2016 on the protection of natural persons with regard to the processing of personal
data and on the free movement of such data, and repealing Directive 95/46/EC (General Data
Protection Regulation) Available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/PDF/?uri=CELEX:32016R0679 (Accessed 17 April 2020).
FDA (1996). Quality System Regulations, 21 CFR 820. Washington D.C.: FDA.
FDA (1997). Design Control Guidance for Medical Device Manufacturers. Available at:
https://www.fda.gov/media/116573/download (Accessed 17 April 2020).

138
FDA (1999). Guidance for Industry, FDA Reviewers and Compliance on Off-The-Shelf Software Use in
Medical Devices. Available at: https://www.fda.gov/media/71794/download (Accessed 17 April
2020).
FDA (2000). Software Related Recalls for Fiscal Years 1992-1998. Rockford, MD, FDA.
FDA (2002a). General Principles of Software Validation; Final Guidance for Industry and FDA Staff.
Available at: https://www.fda.gov/regulatory-information/search-fda-guidance-
documents/general-principles-software-validation (Accessed 17 April 2020).
FDA (2002b). Postmarket Surveillance, 21 CFR 822. Washington D.C.: FDA.
FDA (2005a). Guidance for Industry and FDA Staff Guidance for the Content of Premarket Submissions
for Software Contained in Medical Devices. Available at:
https://www.fda.gov/media/73065/download (Accessed 17 April 2020).
FDA (2005b). Guidance for Industry Cybersecurity for Networked Medical Devices Containing Off-the-
Shelf (OTS) Software. Available at: https://www.fda.gov/media/71794/download (Accessed 17
April 2020).
FDA (2007). Guidance for Industry and FDA Staff Recognition and Use of Consensus Standards.
Available at: https://wayback.archive-
it.org/7993/20170404004142/https://www.fda.gov/downloads/MedicalDevices/DeviceRegulation
andGuidance/GuidanceDocuments/ucm077295.pdf (Accessed 17 April 2020).
FDA (2011). Medical Device Pre-Market Programs: An Overview of FDA Actions. Available at:
https://www.fda.gov/media/81797/download (Accessed 30 April Access 2011).
FDA (2014). Content of Premarket Submission for Management of Cybersecurity in Medical Devices
Guidance for Industry and Food and Drug Administration Staff. Available at:
https://www.fda.gov/media/119933/download (Accessed 17 April 2020).
FDA (2015). Mobile Medical Applications Guidance for Industry and Food and Drug Administration
Staff. Available at: https://wayback.archive-
it.org/7993/20190422154548/https://www.fda.gov/downloads/MedicalDevices/DeviceRegulation
andGuidance/GuidanceDocuments/UCM263366.pdf (Accessed 18 April 2020).
FDA (2016). Postmarket Management of Cybersecurity in Medical Devices Guidance for Industry and
Food and Drug Administration Staff. Available at:
https://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocument
s/ucm482022.pdf (Accessed 18 April 2020).
FDA (2017a). Cybersecurity. Available at:
https://www.fda.gov/MedicalDevices/DigitalHealth/ucm373213.htm (Accessed 24 August
Access 2017a).
FDA 2017b. Federal Food, Drug, and Cosmetics Act. Title 21 Chapter 9. Washington, D.C: FDA.
FDA (2018a). Content of Premarket Submission for Management of Cybersecurity in Medical Devices
Draft Guidance for Industry and Food and Drug Administration Staff. Available at:
https://www.fda.gov/media/119933/download (Accessed 18 April 2020).

139
FDA (2018b). FDA Update Transition to ISO 13485:2016. Available at:
https://www.fda.gov/media/123488/download
FDA (2019a). Smiths Medical ASD, Inc. Recalls Medfusion® 4000 Syringe Pumps Due to Malfunctioning
Alarms and Potential Interruption of Therapy. Available at: https://www.fda.gov/medical-
devices/medical-device-recalls/smiths-medical-asd-inc-recalls-medfusionr-4000-syringe-pumps-
due-malfunctioning-alarms-and-potential (Accessed 22 December Access 2019a).
FDA (2019b). URGENT/11 Cybersecurity Vulnerabilities in a Widely-Used Third-Party Software
Component May Introduce Risks During Use of Certain Medical Devices: FDA Safety
Communication. Available at: https://www.fda.gov/medical-devices/safety-
communications/urgent11-cybersecurity-vulnerabilities-widely-used-third-party-software-
component-may-introduce (Accessed 18 April 2020).
Fiix Inc. (2019). Predictive maintenance Everything you need to know about predictive maintenance.
Fiix. Available at: https://www.fiixsoftware.com/maintenance-strategies/predictive-maintenance/
(Accessed 5 January 2020).
Fincham, J. E. (2008). Response Rates and Responsiveness for Surveys, Standards, and the Journal,
American Journal of Pharmaceutical Education, 72(2), pp. 1-3.
Fixsen, Blase, Metz & Dyke, D. K. A. M. V. 2015. Implementation Science. International Encyclopedia
of the Social and Behavioral Sciences. Amsterdam: Elsevier.
Fixsen, D. L., Blase, K. A., Naoom, S. F. & Wallace, F. (2009). Core Implementation Components,
Research on Social Work Practice, 19(5), pp. 531-540.
Flannery, E. J. (1991). The Safe Medical Devices Act of 1990: An Overview, Food Drug Cosmetic Law
Journal, 46(2), pp. 129-151.
Forman, J. 1992. Danger cited in Teflon jaw implants. Boston Globe, June 5, 1992, p.11. Available at:
http://libproxy.usc.edu/login?url=https://search.proquest.com/docview/294692382
Fu, K. & Blum, J.(2013) Inside Risks Controlling for Cybersecurity Risks of Medical Device Software,
Association for Computing Machinery. Communications of the ACM, 56(10), 35-37. Available at:
http://search.proquest.com/docview/1465592903/ (Accessed: 16 April 2020).
Galesic, M. & Bosnjak, M. (2009). Effects of Questionnaire Length on Participation and Indicators of
Response Quality in a Web Survey, Public opinion quarterly, 73(2), pp. 349-360.
GAO 1989. FDA’s Implementation of the Medical Device Reporting Regulation. Medical Devices.
PMED-89-10 ed. Washington, D.C.: General Accounting Office.
Gauthier, V. & Gauthier, P. (2020). The Agressive Leader. Montreal, QC: Insight Leadership. Available
at: https://insightlead.com/the-aggressive-leader/ (Accessed 31 Mar 2021).
Gonnelli, V., Satta, F., Frosini, F. & Iadanza, E. 2017. Evidence-based approach to medical equipment
maintenance monitoring. EMBEC & NBC 2017. 258-261 Springer, Singapore.

140
Goyen, M. (2020). Predictive maintenance in healthcare - If you can predict it, you can prevent it.
HealthManagement.org. Available at: https://healthmanagement.org/c/it/post/predictive-
maintenance-in-healthcare-if-you-can-predict-it-you-can-prevent-it (Accessed 5 January 2020).
Haworth (2019). Medical infusion pumps vulnerable to remote attacks. The Daily Swig. Available at:
https://portswigger.net/daily-swig/medical-infusion-pumps-vulnerable-to-remote-attacks
(Accessed 23 Mar 2021).
Hu, C., Ye, H., Jain, G. & Schmidt, C. (2018). Remaining useful life assessment of lithium-ion batteries
in implantable medical devices, Journal of Power Sources, 375, pp. 118-130.
Humayed, A., Lin, J., Li, F. & Luo, B. (2017). Cyber-Physical Systems Security—A Survey, IEEE
Internet of Things Journal, 4, pp. 1802-1831.
Iadanza, E., Gonnelli, V., Satta, F. & Gherardelli, M. (2019). Evidence-based medical equipment
management: a convenient implementation, Medical & biological engineering & computing, 57,
pp. 2215-2230.
ISO 2007. Medical Device - Risk Management – Application of Risk Management to Medical Devices
(ISO 14971-2007, Corrected version 2007-10-01). Geneva, Switzerland: ISO Copyright Office.
ISO 2015. A Guide to Good Practices Principles and Practices in Product Regulation and Market
Surveillance. Genève, Switzerland: ISO.
ISO 2016. Medical devices — Quality management systems — Requirements for regulatory purposes.
ISO 13485. Geneva, Switzerland: ISO Copyright Office.
ISO (2019). ISO - About US. ISO. Available at: https://www.iso.org/about-us.html (Accessed 19 April
2019).
Kaspersky (2019). What is WannaCry ransomware? : Kaspersky Lab. Available at:
https://www.kaspersky.com/resource-center/threats/ransomware-wannacry (Accessed 29
December 2019).
Kelly, S. (2019). FDA warns of rapid battery draining in some Medtronic pacemakers. MedTechDive.
Available at: https://www.medtechdive.com/news/fda-warns-of-rapid-battery-draining-in-some-
medtronic-pacemakers/554315/ (Accessed 9 September 2019).
Leavitt, N. (2010). Researchers Fight to Keep Implanted Medical Devices Safe from Hackers, Computer,
43(8), pp. 11-14.
Lewis, B. (2018). FDA Plans To Use ISO 13485 For Medical Device Regulation. ISO. Available at:
https://www.iso.org/news/ref2318.html (Accessed 24 April 2021).
Linberg, K. R. 2002. Apparatus And Method For Remote Troubleshooting, Maintenance And Upgrade of
Implantable Device Systems. U.S. patent application 6,442,433. August 27, 2002.
Longras, A., Oliveira, H. & Paiva, S. Security Vulnerabilities on Implantable Medical Devices. 2020
15th Iberian Conference on Information Systems and Technologies (CISTI), 24-27 June 2020
2020. 1-4.

141
Magee, C. G., Straight, R. L. & Schwartz, L. (2001). Conducting Web-Based Surveys: Keys to Success,
The Public manager (Potomac, Md.), 30(2), pp. 47-50.
Maines, M., Tomasi, G., Moggio, P., Peruzza, F., Catanzariti, D., Angheben, C., Simoncelli, M.,
Degiampietro, M., Piffer, L., Valsecchi, S. & Del Greco, M. (2020). Implementation of remote
follow-up of cardiac implantable electronic devices in clinical practice: organizational
implications and resource consumption, Journal of cardiovascular medicine (Hagerstown, Md.),
21, pp. 648-653.
Maisel, W. H., Sweeney, M. O., Stevenson, W. G., Ellison, K. E. & Epstein, L. M. (2001). Recalls and
Safety Alerts Involving Pacemakers and Implantable Cardioverter-Defibrillator Generators,
JAMA, 286(7), pp. 793-799.
Mann, S. (1995). An experiment in connectivity Look out through my glasses right now (or when last
tranmitted). Available at: http://wearcam.org/myview.html (Accessed 12 April 2019).
Marnfeldt, G. N., Parramon, J. & Gould, C. B. 2018. Signaling Error Conditions In An Implantable
Medical Device System Using Simple Charging Coil Telemetry. U.S. patent application
10,105,543. October 23,2018.
Meyers, D. C., Durlak, J. A. & Wandersma, A. (2012). The Quality Implementation Framework: a
Synthesis of Critical Steps in the Implementation Process, American journal of community
psychology, 50(3-4), pp. 462-480.
Microsoft (2018). Microsoft how field service is transforming the future. Microsoft. Available at:
https://download.microsoft.com/download/4/D/D/4DDD2092-9C22-412B-8E0E-
40BCC1D437A3/How_Field_Service_is_Transforming_the_Future_White_Paper.pdf (Accessed
2 August 2018).
Microsoft (2019). Medical technology company aims to improve healthcare with IoT solutions. Microsoft.
Available at: https://customers.microsoft.com/en-us/story/740795-stryker-azure-iot-edge-health-
united-states-en (Accessed 15 March 2020).
Mirriam Webster (2019). Definition of Internet of Things. Available at: https://www.merriam-
webster.com/dictionary/Internet%20of%20Things (Accessed 19 April 2019).
Mukhopadhyay, S. C. & Islam, T. (2017). Wearable Sensors : Applications, design and implementation,
Bristol, UK, IOP Publishing.
Nassar-McMillan, S. C., Wyer, M., Oliver-Hoyo, M. & Ryder-Burge, A. (2010). Using focus groups in
preliminary instrument development: expected and unexpected lessons learned, The Qualitative
Report, 15(6), pp. 1621-1634.
National Institute of Standards and Technology (2018). Framework for Improving Critical Infrastructure
Cybersecurity. Available at: https://nvlpubs.nist.gov/nistpubs/CSWP/NIST.CSWP.04162018.pdf
(Accessed 18 April 2020).
Newaz, A. I., Sikder, A. K., Babun, L. & Uluagac, A. S. HEKA: A Novel Intrusion Detection System for
Attacks to Personal Medical Devices. 2020 IEEE Conference on Communications and Network
Security (CNS), 29 June-1 July 2020 2020. 1-9.

142
Nowinski, C.(2000) Cognitive and Emotional Effects of One Season of Head Impact Exposure in High
School Contact Sport Athletes. Doctor of Philosophy School of Medicine, Boston University.
Nyansa Inc. 2019. Nyansa And GE Healthcare Introduce OnWatch NP Providing Deeper Insight Into
Clinical Networks And Devices: The first and only analytics solution to include patient waveform
data drops and critical device timing issues that have hampered networked clinical device
performance. PR newswire. Available at:
http://libproxy.usc.edu/login?url=https://www.proquest.com/wire-feeds/nyansa-gehealthcare-
introduce-onwatch-np/docview/2315058711/se-2?accountid=14749
O'Dowd, E. (2020). Predictive Maintenance, Sensors Critical to Health IoT Success. Xtelligent
HealthCare Media. Available at: https://hitinfrastructure.com/news/predictive-maintenance-
sensors-critical-to-health-iot-success (Accessed 5 January 2020).
Odesile, A.(2017) Mobile Agent based Intrusion Detection for Smart and Connected Medical Devices.
Masters of Science Computer Science & Software Engineering, Seattle, WA, University of
Washington.
Oppenheimer, D. S.(2017) Risk Management And Recalls: A Survey Of Medical Device Manufacturers.
Doctor of Regulatory Science School of Pharmacy, University of Southern California.
Orcher, L. (2014). Conducting Research Social and Behavioral Science Methods, New York, NY,
Routledge.
Pew Reserch (2020). Questionnaire design. Pew Reserch Center. Available at:
https://www.pewresearch.org/methods/u-s-survey-research/questionnaire-design/ (Accessed 14
Feb 2021).
Quasim, M. T., Khan, M. A., Abdullah, M., Meraj, M., Singh, S. P. & Johri, P. Internet of Things for
Smart Healthcare: A Hardware Perspective. 2019 First International Conference of Intelligent
Computing and Engineering (ICOICE), 15-16 Dec. 2019 2019. 1-5.
Radziwill, N. (2018). Lets Get Digital, Quality Progress, October 2018, pp. 24-29.
Rajan Prashant, V., Kramer Daniel, B. & Kesselheim Aaron, S.(2015) Medical Device Postapproval
Safety Monitoring Where does the United States Stand?, Circulation: Cardiovascular Quality
and Outcomes, 8(1), 124-131. Available at:
https://doi.org/10.1161/CIRCOUTCOMES.114.001460 (Accessed: 20 October 2019).
Rathore, H., Mohamed, A., Al-Ali, A., Du, X. & Guizani, M. A Review of Security Challenges, Attacks
and Resolutions for Wireless Medical Devices. 2017 13th International Wireless
Communications and Mobile Computing Conference (IWCMC), 26-30 June 2017 2017. 1495-
1501.
Rawat, S. & Meena, S. (2014). Publish or perish: Where are we heading?, Journal of research in medical
sciences, 19, pp. 87-89.
Rettig, R. A., Earley, L. E. & Merrill, R. A. (1992). Food and Drug Administration Advisory Committees,
Washington, D.C, National Academy Press.

143
Ripin, Z. M. & Chan, P. Y. 2017. Pathological Hand Tremor Measurement—Challenges and Advances.
2nd International Conference for Innovation in Biomedical Engineering and Life Sciences,
December 10-13 2017 Penang, Malaysia. Singapore: Springer Singapore, 3-8.
Romkey, J. (2017). Toast of the IoT: The 1990 interop Internet toaster, IEEE Consumer Electronics
Magazine, 6(1), pp. 116-119.
Rouse, M. (2005). Telemetry. TechTarget. Available at: https://whatis.techtarget.com/definition/telemetry
(Accessed 9 January 2020).
Rouse, M. (2019). internet of things (IoT). TechTarget. Available at:
https://internetofthingsagenda.techtarget.com/definition/Internet-of-Things-IoT (Accessed 19
April 2019).
Sadler, G. R., Lee, H.-C., Seung-Hwan Lim, R. & Fullerton, J. (2010). Recruiting hard-to-reach
population sub-groups via adaptations of the snowball sampling strategy, Nursing & Health
Sciences, 12(3), pp. 369-374.
Safaei, M.(2019) A Piezoelectric Instrumented Total Knee Replacement For Sensing And Energy
Harvesting. Doctor of Philosophy Engineering, Tennessee Technological University.
Schmidt, C. M. (2017). Best Practices for Technical Standard Creation. Bedford, MA: The MIRTE
Corporation. Available at: https://www.mitre.org/sites/default/files/publications/17-1332-best-
practices-for-technical-standard-creation.pdf (Accessed 28 Apr 2021).
Schwartz, S., Ross, A., Carmody, S., Chase, P., Coley, S., Connolly, J., Petrozzino, C. & Zuk, M. (2018).
The Evolving State of Medical Device Cybersecurity, Biomedical Instrumentation & Technology,
52(2), pp. 103-111.
SEBoK Editorial Board (2017). Guide to the Systems Engineering Body of Knowledge (SEBoK),
Hoboken, NJ, The Trustees of the Stevens Institute of Technology.
Shapiro, J. K. (2013). The Medical Device Amendments of 1976: The Statute That Went Awry. FDA Law
Blog. Available at: http://www.fdalawblog.net/2013/06/the-medical-device-amendments-of-
1976-the-statute-that-went-awry/ (Accessed 25 August 2018).
Sheehan, K. B. (2001). Using E‐mail to Survey Internet Users in the United States: Methodology and
Assessment, Journal of computer-mediated communication, 6(2), pp. 1-27.
Siwicki, B. (2019). Threat intelligence analyst documents, assesses 117 healthcare ransomware
incidents. HealthcareITNews. Available at: https://www.healthcareitnews.com/news/threat-
intelligence-analyst-documents-assesses-117-healthcare-ransomware-incidents (Accessed 21
December 2019).
Slaughter, S. A. (2014). A Profile of the Software Industry Emergence, Ascendance, Risks, and Rewards,
New York, NY, Business Expert Press.
Stryker (2019). Stryker aims to improve healthcare with IoT Solutions. Stryker. Available at:
https://www.stryker.com/us/en/about/news/features/smart-devices-deliver-actionable-
insights.html (Accessed 19 December 2019).

144
Study Group on Medical Devices Department of Health Education and Welfare 1970. Medical Devices:
A Legislative Plan. Indiana University Library: U.S. Department of Health, Education and
Welfare.
Sun, Y. 2012. The Incident of the Bjork-Shiley Heart Valve. BME 462 Biomedical Instrumentation. 1-5
University of Rhode Island.
Technopedia (2019). Definition - What does Internet of Things (IoT) mean? : Technopedia. Available at:
https://www.techopedia.com/definition/28247/internet-of-things-iot (Accessed 18 April 2019).
Thompson, D. L. 2000. World Wide Patient Location And Data Telemetry System For Implantable
Medical Devices. U.S. patent application 6,083,248. July 4, 2000.
Torrey, S. (2019). Armis Discovers Expanded Reach of URGENT/11 That Highlights Risk to Medical
Devices. Armis. Available at: https://www.armis.com/about/iot-security-news/armis-discovers-
expanded-reach-of-urgent11/ (Accessed 11 January 2020).
Untereker, D. F., Schmidt, C. l., Jain, G., Tamirisa, P. A., Hossick-Schott, J. & Viste, M. (2017). 8-Power
Sources and Capacitors for Pacemakers and Implantable Cardioverter-Defibrillators Clinical
Cardiac Pacing, Defibrillation, and Resynchronization Therapy. Available at:
Uzdavines, M. (2018). Dying For A Solution: The Regulation Of Medical Devices Falls Short In The 21st
Century Cures Act, Naevada Law Journal, 18, pp. 629-656.
van den Bekerom, P. (2019). 'RE: How does Phillips use medical device feedback data to monitor device
performance'. Email, 20 December. Correspondence to Wright, C.
Weaver, C. 2013. Patients Put at Risk By Computer Viruses. Wall Street Journal, June 14, 2013, p.A.1.
Available at: https://www.wsj.com/articles/SB10001424127887324188604578543162744943762
(Accessed 18 April 2020).
Weller, S. C., Vickers, B., Bernard, H. R., Blackburn, A. M., Borgatti, S., Gravlee, C. C. & Johnson, J.
C.(2018) Open-ended interview questions and saturation, PLOS ONE, 13(6), 1-18. Available at:
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0198606 (Accessed: 13 Feb
2021).
Yoder, M. R., Zhang, B., Kivi, G. P. & Sanden, R. A. 2019. Telemetry Overuse Reduction In A Medical
Device. U.S. patent application 16/273,241. June 6, 2019.
Zeng, F.-G. 2019. Cochlear Implants. In: Ball, J. S. D. M. J. (ed.) The Sage Encyclopedia of Human
Communication Sciences and Disorders. Thousand Oaks, CA: SAGE Publications, Inc.
Zhang, M., Raghunathan, A. & Jha, N. K. (2013). MedMon: Securing Medical Devices Through Wireless
Monitoring and Anomaly Detection, IEEE transactions on biomedical circuits and systems, 7(6),
pp. 871-881.

145
APPENDICIES

146
Appendix A: Focus Group Preparation Materials

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162
Appendix B: Survey Updates – Post Focus Group
The images displayed in this Appendix show the Qualtrics question numbers for organizational purposes.
These question numbers were removed prior to publishing the survey. Qualtrics also has the capability of
including “Piped Text” from one question into another. In this case the selected answer from Question 3
was incorporated into many of the following questions in the survey. This representation is displayed in
the below images as “${q://QID1/ChoiceGroup/SelectedChoicesTextEntry}”.

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181
Appendix C: Skip Logic Testing

182

Abstract (if available)

Abstract Most medical product regulations focus on premarket activities, to assure that innovative health care tools enter the market only after their risks and benefits have been assessed. However, those products must also be monitored after market entry to ensure that they remain safe and effective over time. A special set of risks and opportunities exist for electronic devices that can transmit data by telemetry. Such devices are vulnerable to mechanical problems, software errors or cyber-attack but these risks can often be monitored and mitigated using telemetered data to trigger corrective actions. This exploratory study assessed how device telemetry is currently being collected and monitored by defibrillator and infusion pump companies to ensure the safe and effective operation of their devices. A survey was designed based on a modified implementation framework and was disseminated to medical device industry professionals involved in different aspects of product development or field operations. The results from this survey indicate that many manufacturers are monitoring performance aspects of their products using telemetry data without specific requirements from regulators. However, adoption of this capability is not universal. Further, telemetered data appears to be rarely used to detect cyber intrusions, despite much regulatory attention to this vulnerability. Results suggest that regulators could add value by clarifying and perhaps expanding their expectations related to device telemetry usage related to risk management, cybersecurity and post-market surveillance.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Risk management and recalls: a survey of medical device manufacturers

PDF

Organizational communication of regulatory intelligence: a survey of the medical device industry

PDF

Design control for software medical devices: an industry survey of views and experiences

PDF

Regulatory programs to foster medical product development: user experience in the United States and Japan

PDF

Current practices in biocompatibility assessment of medical devices

PDF

Implementation of unique device identification in the medical device industry: a survey of the change management experience

PDF

Regulatory CMC strategies for gene and cell therapies during mergers and acquisitions: a survey of industry views

PDF

Use of electronic health record data for generating clinical evidence: a summary of medical device industry views

PDF

Efficient crowd-based visual learning for edge devices

PDF

Regulatory agreements for drug development collaborations: practices in the medical products industry

PDF

Examining the regulatory framework for drug compounding: industry views and experiences

PDF

Implementation of risk management in medical device companies: a survey analysis of current practices

PDF

Reprocessing of single-use medical devices: a survey investigation comparing the views of three unheard stakeholders

PDF

Regulatory team development in post-merger integration: a survey of views from medical product companies

PDF

Wearables in healthcare

PDF

Challenges in the implementation of Risk Evaluation Mitigation Strategies (REMS): a survey of industry views

PDF

Use of natural colors: experience and views in pharmaceutical and dietary supplement industries

PDF

Convergence of United States regulatory and reimbursement policies impacting patient access to humanitarian use devices (HUD)

PDF

The role of psychological safety in medical device quality

PDF

BRIM: A performance-based Bayesian model to identify use-error risk levels in medical devices

Asset Metadata

Creator Wright, Carolyn Marie (author)

Core Title Using telemetry to ensure safe and reliable medical device operation: experience with defibrillators and infusion pumps

School School of Pharmacy

Degree Doctor of Regulatory Science

Degree Program Regulatory Science

Degree Conferral Date 2021-08

Publication Date 08/04/2021

Defense Date 06/07/2021

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag AED,cyber threat monitoring,cybersecurity,data,design control,device performance,external defibrillator,feedback,implantable defibrillator,implementation,IoT,maintenance evolution,medical device,OAI-PMH Harvest,post market surveillance,process control,process monitoring,risk management,telemetry

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Richmond, Frances (committee chair), Bain, Susan (committee member), Kuo, Benson (committee member), Oppenheimer, Darin (committee member)

Creator Email carolynwright2000@gmail.com,wrightca@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC15676561

Unique identifier UC15676561

Legacy Identifier etd-WrightCaro-9990

Document Type Dissertation

Format application/pdf (imt)

Rights Wright, Carolyn Marie

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu