University of Southern California Dissertations and Theses

A survey analysis of transparency in three Asian regulatory agencies responsible for medical products

(USC Thesis Other)

A survey analysis of transparency in three Asian regulatory agencies responsible for medical products

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

A SURVEY ANALYSIS OF TRANSPARENCY IN THREE ASIAN
REGULATORY AGENCIES RESPONSIBLE FOR MEDICAL PRODUCTS

by

Martin Solberg

__________________________________________________________________

A Dissertation Presented to the
FACULTY OF THE USC SCHOOL OF PHARMACY
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF REGULATORY SCIENCE

May 2011

Copyright 2011 Martin Solberg
ii
DEDICATION
I dedicate this dissertation to my wonderful wife Karen Lynn Solberg who
stayed with me thru one graduate degree after another and now to this final academic
effort. And to my three children Erik, Kelsey and Kristen who were always
supportive and understanding of my desire to complete this work.
iii
ACKNOWLEDGEMENTS
I would like to thank all of those people who helped make this dissertation
possible.
First, I wish to thank my advisor, Dr. Frances Richmond for all her guidance,
encouragement, support and patience. Her expertise in the sciences, global regulatory
affairs and transparency issues related to the pharmaceutical industry has been a
great inspiration to me. And I am especially thankful for her unbelievable level of
positive energy and never ending support of her students. Also, I would like to thank
my committee members Dr. Pete Vanderveen, Dr. Dana Bell and Dr. Enrique
Cadenas for their very helpful insights, comments and suggestions. Also thanks to
Dr. Rickey Wilson for his professional and academic mentorship. Additionally, I
would like to acknowledge Dr. Kathy Rolle, my very patient and supportive doctoral
program manager along with Aditya Garg for his technical support. And finally
thanks to the University Of Southern California School Of Pharmacy Fellows who
provided early research support for this work: Dr. David Truong and Dr. Christiana
Pham.
iv
TABLE OF CONTENTS
Dedication ii
Acknowledgements iii
List of Tables v
List of Figures vii
Abstract ix
Chapter 1: Introduction 1
Chapter 2: Literature Review 13
Chapter 3: Methodology 63
Chapter 4: Results 69
Chapter 5: Discussion 92
Bibliography 117
Appendices 126
Appendix A: Drug Registration Transparency Survey 126
Appendix B: Focus Group 1 Participant Matrix 129
Appendix C: Focus Group 2 Participant Matrix 130
Appendix D: Survey Statistics 131
Appendix E: Cross Tabulation Data Analysis 141
v
LIST OF TABLES
Table 1: Transparency scores by dimension 90
Table D1: The drug registration process has a publicly accessible 131
comprehensive Internet information system that is easy to use
Table D2: There is an effective mechanism to discuss questions related to 131
submission requirements and issues directly with the agency
Table D3: There is a lead person identified and assigned for each 132
submission
Table D4: The lead person responsible for my dossier is accessible after 132
the registration has been submitted
Table D5: Drugs applications are easy to submit electronically 132
Table D6: Additional comments related to accessibility, please place here 133
Table D7: There are easy to understand written procedures on how to 134
register a drug
Table D8: There is an easy to use standard drug application format based 134
on the ICH CTD format
Table D9: There are clear and well defined requirements for the 134
manufacture of clinical and commercial drug products
Table D10: All fees required for submission of the drug application are 135
clearly defined
Table D11: Fees for submission of the drug application are standardized 135
by product type and therapeutic area
Table D12: Additional comments on clarity please place here 135
Table D13: The review committee reaches its decision on a drug 136
application via documented procedures for decision making
(e.g. a qualified majority or consensus)
vi
Table D14: The committee responsible for the drug application process 136
provides an official written report to the applicant on decisions
(e.g. accepted and rejected files) taken at key decision points
of the application process
Table D15: There is clear appeal process in place for applicants who have 136
their drug applications rejected
Table D16: The committee responsible for drug application review is also 137
responsible for the final decision making for approval/rejection
of the drug application
Table D17: Additional comments on accountability please place here 137
Table D18: Based on your professional experience please rank the four 138
countries below on an individual basis from an overall
transparency perspective from most to least transparent
Table D19: Please identify your years of regulatory experience and 138
currency of submission expertise in Asia
Table D20: Please select the size of your company 138
Table D21: Please describe the types of products in your company or 139
consultancy portfolio
Table D22: Where are you located? 139
Table D23: If your answer to the above is "other" please enter your 139
location here
Table D24: Additional comments please place here 140
Table D25: Would you be willing to participate in an additional survey 140
for doctoral research if the research is a fit for your area of
expertise?
Table E1: Cross-tabulation with experience of respondents 141
Table E2: Cross-tabulation with size of company 141
vii
LIST OF FIGURES
Figure 1: Schema for evaluating transparency, modified from 56
PricewaterhouseCoopers (Oliver, 2004)
Figure 2: The drug approval process in the US modified from Jenkins' 60
speech to congress
Figure 3: The drug registration process has a publicly accessible 76
comprehensive Internet information system that is easy to use
Figure 4: There is an effective mechanism to discuss questions related to 77
submission requirements and issues directly with the agency
Figure 5: There is a lead person identified and assigned for each 78
submission
Figure 6: The lead person responsible for my dossier is accessible after 78
the registration has been submitted
Figure 7: Drug applications are easy to submit electronically 79
Figure 8: There are easy to understand written procedures on how to 82
register a drug
Figure 9: There is an easy to use standard drug application format based 82
on the ICH CTD format
Figure 10: There are clear and well defined requirements for the 83
manufacture of clinical and commercial drug products
Figure 11: All fees required for submission of the drug application are 84
clearly defined
Figure 12: Fees for submission of the drug application are standardized 84
by product type and therapeutic area
Figure 13: The review committee reaches its decision on a drug application 86
via documented procedures for decision making (e.g. a qualified
majority or consensus)
viii
Figure 14: The committee responsible for the drug application process 86
provides an official written report to the applicant on decisions
(e.g. accepted and rejected files) taken at key decision points of
the application process
Figure 15: There is a clear appeal process in place for applicants who 87
have their drug applications rejected
Figure 16: The committee responsible for drug application review is also 88
responsible for the final decision making for approval/rejection
of the drug application
Figure 17: Based on your professional experience please rank the four 89
countries below on an individual basis from an overall
transparency perspective from most to least transparent
ix
ABSTRACT
This study evaluated drug registration transparency in three Pharmerging
markets in Asia. It took place in three stages. First it used focus-group methods to
evaluate a range of factors that are perceived by regulatory experts to be important in
defining and affecting transparency in regulatory agencies governing pharmaceutical
products. Second it created a drug registration transparency index in the form of a
survey instrument whose questions were guided by three key dimensions of
accessibility, clarity and accountability, This index was used to compare the
perceived transparency of certain regulatory agencies in South Korea, China and
India by surveying the views and experiences of regulatory professionals who have
specialized in these global registration processes. The US was considered to be most
transparent, South Korea was viewed as intermediate in transparency and China and
India were considered least transparent according to the response profiles of the
respondents. The survey instrument appeared to provide results consistent with the
way that transparency in these countries was rank-ordered by the same respondents.
Results suggested that the survey instrument was a useful tool with which to study
regulatory transparency. Although further study is needed to validate this approach,
the survey provides a first-of-its-kind index of drug-approval transparency based
upon the three key aspects of transparency, clarity, accessibility and accountability.
Further, the transparency index with minor changes could be adapted to study the
transparency of a broader range of countries and a broader range of product
categories including medical devices and in vitro diagnostics.
1
CHAPTER 1: INTRODUCTION
Research and Development (R&D) efforts in the 20
th
century created a global
pharmaceutical industry with an unprecedented ability to produce novel new medical
products for diverse health problems. The commercialization of these products,
notably drugs and devices, posed new health benefits and risks; the increasing
sophistication of these products also created the need for additional safety oversight
from regulatory agencies around the globe. As regulations became more onerous, it
became challenging to assure the timeliness and clarity of the pathway to be taken in
order to obtain regulatory approval. As part of this effort, the need for transparency
in regulatory processes and decision-making became subject to considerable
attention. Transparency is a general term that has a variety of different meanings
according to context. In the context of regulatory approvals and for the purpose of
this thesis, we have adopted the definition of Finkelstein that defines transparency by
stating, “We use transparency to describe those policies that are easily understood,
where information about the policy is available, where accountability is clear, and
where citizens know what role that they play in the implementation of the policy”
(Finkelstein 2000).
Transparency is a difficult state to ensure, even in the most developed
countries with a long history of economic strength and stability. In the US the
importance of increased transparency has been underlined by the formation of
Presidents Obama’s FDA Transparency Task Force in 2009. Its importance in the
EU is demonstrated by a new transparency policy that is now open for public
2
consultation, that is designed to provide more “openness, communication and
accountability, while respecting the protection of both personal data as well as
commercially confidential information” (EMA 2003).
In addition, attention to transparency has been a subject of focus
internationally. For example, the US and the EU are developing bilateral activities to
improve the transparency and effectiveness of the drug approval process. Beginning
in 2009 the FDA has placed staff in the EMA to improve the flow of information,
and discussions are underway about methods to harmonize the evaluation of
manufacturing operations. Together the US, the EU and Japan, the world’s three
largest economies, have combined their efforts on a global level to address issues of
drug-approval speed and transparency. Perhaps the most obvious manifestation of
this joint activity was the formation of the International Conference on
Harmonization (ICH). Since its inception in April 1990 the ICH has focused on
improving the harmonization of regulatory approvals, with an expressed intention of
shortening the time and reducing the resources needed to gain market approval of a
pharmaceutical product (Bale 2003).
The ICH was formed and to this point in time has been driven largely by
governmental and industry participants from the most highly industrialized countries.
Less developed regions of the globe typically have not participated to the same
extent in these kinds of activities. Rather, they maintained their focus on producing
domestic drugs and active pharmaceutical ingredients and excipients that were
ultimately incorporated elsewhere into products that were first introduced into the
3
markets of the US, Europe and Japan. Whether products are sold in less developed
countries depended in large measure on the size of the expected market and the
intensity of effort required in obtaining regulatory approval.
This thesis is directed at understanding transparency of drug approval
processes in three countries in Asia. Transparency in government generally, and
transparency in drug approvals particularly, has been more challenging to achieve in
the emerging economies of Asia. Even a decade ago, many less developed countries
had few regulations to challenge the introduction of such products into the
marketplace, and often these rules could be bent by appropriate relationship-building
or incentives. Less rigorous oversight was in some quarters justified because until
recently, many of these products already had a history of safe use in constituencies
with large sales. However, this situation had the potential to allow unsafe products
onto the market and did little to support the early development and availability of
high-quality, essential medicines in the Asia Pacific Region. Additionally high
margin blockbuster products in first tier countries had market price points that placed
them out of reach for the average consumer in the emerging markets until quite
recently (IMS 2010), so that relatively few people in those countries were likely to
use the products. Nevertheless, at the turn of the millennium, forces were already in
evidence that were to change dramatically the way that medical products, and for the
purpose of this thesis, pharmaceutical products specifically, were managed from a
regulatory and sales perspective. Newly emerging economies with increased
spending power began to constitute a new high-growth pharmaceutical market.
4
These new pharmaceutical emerging markets (Pharmerging Markets) were fueled by
rapid economic growth which provided an increase in disposable income, the
emergence of a new middle class, and an increase in out-of-pocket expenditure on
drugs

(IMS 2010).
In the last decade, pharmaceutical companies began to expand their global
regulatory and marketing departments in order to position themselves to enter
Pharmerging markets, particularly those in the Asia. Here they found significant
barriers to entry. Success in a foreign market is anchored to an understanding of
local market dynamics, political structures, pricing, reimbursement policies and most
importantly the highly complex drug registration process (Nangra 2009). As more
medical products became available in these emerging regions an increasing public
recognition of the issues of patient safety posed by the paucity of regulations has
moved to the forefront. Most Pharmerging markets moved quickly in the last decade
to remedy this gap by putting in place a complex framework of laws and associated
regulations to improve the safety of medicines (Nangra 2009; Netanel 2009). Some
of these regulations had an added role in that they served as non-tariff barriers to
protect a slowly developing indigenous pharmaceutical industry that was providing
jobs and tax revenues domestically (WHO 2009).
The increasing complexity of the registration process, the lack of
harmonization with standards in other countries, language barriers, limited resources
for communication with stakeholders and still-modest experience with regulatory
activities contributed to a lack of transparency in pharmaceutical registration
5
processes. The lack of transparency in such markets has been subject to numerous
articles in the trade press and talk in conventions dealing with regulatory
impediments. Yet the question arises: How do we characterize more systematically
the transparency of a regulatory system? Can metrics be developed that allow the
transparency of different regulatory systems to be compared, and can such metrics
help not only in assessing this important aspect of regulatory activities but also aid
regulatory agencies to identify weaknesses in their systems that might be easily
remedied to achieve greater transparency (if in fact, such greater transparency were
to be seen to be a desired objective)?
The use of metrics to benchmark and compare the state of certain social or
governmental activities from one constituency to another is not new. For example,
corruption metrics, developed to compare corruption from one country to another,
have been seminal in the evaluation of corruption globally over the last 15 years.
Since 1995, Transparency International (TI) has published an annual Corruption
Perceptions Index (CPI) ordering the countries of the world according to "the degree
to which corruption is perceived to exist among public officials and politicians" (TI
2008). This Index bases its conclusions by bringing together a series of 13 survey
instruments administered to business leaders and experts working in the target
countries. In parallel it also publishes research by looking at the experiences of
individuals who represent “the man on the street”. The Global Corruption Barometer
in 2009 assessed the views of over 73,000 people in 69 countries with respect to
corruption by questioning their experiences with the payment of bribes or other
6
inducements in their daily activities over the past year (GCB 2009). However, these
survey tools look generally at experiences across many aspects of business and
government interactions. The only attempt to measure the transparency of the drug
regulation process in emerging markets appears to be that conducted by the World
Health Organization in 2003, 2006 and 2008. However these studies and the
resulting indices maintain an emphasis on corrupt practices and the effect of
corruption on distribution, supply and registration, rather than the complexity and
transparency of the drug registration process (WHO 2009).
Statement of the Problem
Today few systematic, comparative instruments exist to measure, define or
compare the transparency of the drug registration process from country to country or
region to region (WHO 2009). According to the literature, measures under study
focus on avoiding corrupt practice rather than providing a path to improve drug
approval activities more generally. Therefore when a drug company is planning to
launch a new product into Pharmerging markets the regulatory strategy for
submission, timing, approval and market launch is at best a gamble. There is a need
for better information on the relative transparency of different Asian countries with
respect to different aspects of the drug registration process.
Purpose of the Study
This study evaluates drug registration transparency in Pharmerging markets
in Asia. It took place in three stages. First the study identified, using a focus group
methodology, a range of factors that are perceived by regulatory experts to be
7
important in defining and affecting transparency in regulatory agencies governing
pharmaceutical products. Second it created a drug registration transparency index
suitable to compare the transparency of regulatory agencies in different countries.
This index was constructed as a survey tool. Finally the study compared the
perceived transparency of certain regulatory agencies in Pharmerging Asian
countries by using the transparency index to survey the views and experiences of
regulatory professionals who have specialized in these global registration processes.
Significance of the Study
This study contributes to a small but important body of research concerned
with regulatory system transparency by providing a tool that has been tested,
modified and validated for the purpose of gaining insight into the current state and
challenges to transparency in different countries. This tool has three important
outcomes. First, it can enhance communication and foster dialogue by providing a
standardized tool with which individuals can study and compare information about
regulatory systems across countries and time periods.
Second, it can help to guide priority setting and logistical activities when
regulatory teams are planning a global product launch. A systematic tool to evaluate
transparency can help to pinpoint areas where problems with transparency create
critical delays in product introduction that could not have been otherwise anticipated
by an inexperienced regulatory team. It will also help to identify specific problems
likely to recur as hot button issues that require specific extra attention or enhanced
resource allocation when a product is introduced in a particular country. Such
8
information is critical when ranking countries for serial product launch according to
the risks introduced by regulatory complications.
Third, it can guide policy development and ultimately improve regulatory
environments. Regulatory agencies can use the outcomes to identify specific areas
where changes in policy or procedure would have a significant impact on
transparency. Results can also be used as part of benchmarking activities so that the
effects of interventions or changes with an effect on transparency can be gauged.
Delimitations, Limitations and Assumptions
In this study, we are delimiting our area of focus to the Asian markets that are
considered to be emerging market economies. Japan is not a focus of this thesis
except as a comparator for more general discussions, because its pharmaceutical
market (though not highly transparent) is well established.

Further, we do not
evaluate some of the other Pharmerging markets that drive the total market growth of
pharmaceuticals worldwide outside of the Asia, even though the survey tool is
sufficiently generic for such analyses at a later date. In addition, for this preliminary
test of the survey instrument, we limited the study to only a select group of Asian
Pharmerging Market countries, specifically, China, India, and South Korea.
Additionally this study of transparency does not have an emphasis on all
aspects of activity within the purview of medical product regulators but concentrates
on transparency associated with new drug approvals. Typically regulatory activities
are divided into pre- and post-approval phases, and this study predominately
considers the pre-approval phase. Other aspects not directly addressed here might,
9
for example, include activities related to the inspection of manufacturing facilities, or
activities associated with pharmacovigilence, even though these are important areas
in which regulatory agencies play a central role.
One challenging and potentially limiting aspect of this study was the ability
to sample a representative grouping of regulatory professionals with specific
experience in submitting drug applications in the target countries of China, India and
South Korea. The level of experience of these regulatory professionals included
managers and senior executives in drug companies as well as industry consultants.
Identifying experienced individuals to be surveyed and finding a sufficient number to
ensure a fair analysis was difficult. Also the willingness of such busy individuals to
participate this study impacted the sample size. Finally it was challenging to find
individuals who had the breadth of multiregional expertise in drug registration. Some
of those interviewed had a clear understanding of only one or two countries and
could only provide comparative insights into some of the countries under study. We
also made the assumption that the individuals who participated in the study would
respond to the questions honestly, but it is possible that one or more individuals
attempted to answer questions for which they have little experience and thus the
answers may not be reflective of general experience.
Another challenge was the ability to design a survey that captures the
essential features and weights them appropriately to provide a meaningful index.
Like most studies of this type, it was limited by the ability of the investigators to
design an appropriate tool and to interpret answers appropriately (McCracken 1988).
10
The investigators may have been biased by the input that comes from personal
experience and from the relatively small group of individuals who were selected for
the focus group. This potential bias was kept in mind when evaluating the ability of
the survey tool to provide meaningful insight, especially in early phases when
findings can be difficult to replicate or validate by using results from the literature.
Definitions
Transparency: Generally, implies openness, communication, and
accountability. It is a metaphorical extension of the meaning a transparent object is
one that can be seen through. Finkelstein states,
We use transparency to describe those policies that are easily understood,
where information about the policy is available, where accountability is clear,
and where citizens know what role that they play in the implementation of the
policy (Finkelstein 2000).

Pharmerging Markets: The key seven emerging pharmaceutical markets are
Brazil, Russia, India, China, South Korea, Mexico and Turkey. These 'Pharmerging'
economies are expected to account for 51 per cent of the industry's growth over the
next half decade.
Regulatory Agency: “A regulatory agency (also regulatory authority,
regulatory body or regulator) is a public or government agency responsible for
exercising autonomous authority over some area of human activity in a governmental
or supervisory capacity” (Webster 2010).
Competent Authority: Any person or organization that has the legally
delegated or invested authority, capacity, or power to perform a designated function.
11
Drug Approval Phases: The activities required to bring a molecular entity to
market are separated into two phases. Pre-approval is the first phase which includes
clinical and non-clinical trials designed to provide complete evidence of safety and
efficacy of the drug product. This evidence is then submitted to the relevant health
authorities as required by specific registration activities to gain approval to market
the drug. The pre-approval phase includes the activities often described as drug
registration which may also be described as the drug approval process. The post-
approval phase includes all activities related to the drug after it has market approval.
See Figure 2.
Drug Registration Process: A set of key activities and evaluations for
successful dossier submission that occur in the pre-approval phase. See Figure 2.
Pharmaceutical Drug: “Also referred to as medicine, medication or
medicament is any chemical substance intended for use in the medical diagnosis,
cure treatment, or prevention of disease” (Webster 2010).
Medical Device: “A product which is used for medical purposes in patients,
in diagnosis, therapy or surgery. If applied to the body, the effect of the medical
device is primarily physical, in contrast to pharmaceutical drugs” (Webster 2010).
Pharmacovigilence (PV): “The pharmacological science relating to the
detection, assessment, understanding and prevention of adverse effects, particularly
long term and short term side effects of medicines” (Webster 2010).
Corruption: “Transparency International defines corruption as the abuse of
entrusted power for private gain” (Webster 2010).
12
Drug Lag: A term used to describe the much slower introduction of new
drugs into a country or region country as compared to the US and EU.
Organization of the Study
In the chapters that follow, the nature of transparency will be explored in
three Pharmerging markets. Chapter 1 provides an overview of the problem and an
introduction to the research. Chapter 2 then reviews the current state of knowledge
in this field by studying the available literature relating to transparency in general
and to transparency in drug approval in particular. Chapter 3 outlines the methods
used to guide the construction of the survey instrument, and for a pilot deployment of
this survey to evaluate three Pharmerging Asian countries of interest. The goal of
this initial deployment was to evaluate perceived differences in China, India and
South Korea through the eyes of one group of end-users, drawn from the available
pool of mid-level to higher level regulatory professionals in pharmaceutical
companies and consultancy who have had at least two years of experience in product
registration in the Pharmerging markets under study. Chapter 4 analyses the results
of the focus groups and survey deployment, and chapter 5 then discusses the results
and their implications.
13
CHAPTER 2: LITERATURE REVIEW
Background
Transparency in government has been a key topic globally because it affects
the ability of stakeholders and the public to work efficiently in a system whose
conduct is both ethical and predictable. Much has been written in general about the
importance of transparency, and metrics have been developed to explore various
aspects of transparency at the level of the entire government. However, little has
been written specifically about the transparency of drug registration processes across
different international regulatory agencies. In this work, we are concerned with
differences in the transparency of the drug registration process in Pharmerging
markets. In the literature below, we first examine the economic factors that have
changed the global pharmaceutical landscape and have caused increased scrutiny of
regulatory systems governing pharmaceuticals. The concept of transparency is then
defined and examined in the context of pharmaceutical regulatory systems in three
highly developed constituencies and in five countries or regions that are considered
to constitute the key Pharmerging grouping of countries, including three countries
that will be further examined in the pilot study and two others, Turkey and Brazil,
which are provided as comparators. Finally, the literature review will examine what
is currently being done to characterize transparency from a theoretical and research
perspective, and will evaluate the current state of transparency research in the area of
drug regulation.
14
Pharmaceutical Expansion into Asia is driven by Economic Change
Activity in pharmaceutical markets worldwide is affected by the strategies
that companies pursue to sell their pharmaceutical products in different countries and
this in turn is affected by global economic climate for drug use and sales. IMS
Health in 2009 reports that the value of the global pharmaceutical market in 2010
will grow 4 - 6 percent on a constant-dollar basis, exceeding $825 billion United
States Dollars (USD), driven by near-term growth in the US market and continued
growth in the 7 Pharmerging markets (Brazil, Mexico, China, India, Russia, Turkey,
and South Korea). The IMS forecast further predicts global pharmaceutical sales to
grow at a 4 - 7 percent compounded annually through 2013. The total global
pharmaceutical market is expected to expand to 975+ billion USD by 2013.
Overall growth for the next several years is projected to be lower than seen in
the recent past due in part to recent economic challenges in key pharmaceutical
markets. Severe economic downturns in areas such as Russia, Mexico and South
Korea have led to dramatic market reductions for pharmaceuticals. However, the
projections clearly point to a growing expansion of sales into Pharmerging
economies that have large positive trade balances and improving standards of living
(Kermani 2009). A high correlation exists between the gross domestic product
(GDP) of a country (a basic measure of a country's overall economic output) and its
overall standard of living. A high standard of living is positively correlated with
increased disposable income which helps to fuel the growth in healthcare spending
(IMS 2008).
15
The sales mix in the strongest Pharmerging economies is also changing.
Demand for primary care products in Pharmerging markets in 2005 constituted 66
percent of total sales, or 49.6 billion USD. Primary care sales include, for example,
products such as antibiotics and antacids. However, trends suggest that the growth is
shifting from spending mainly on primary-care products to new specialty drugs.
Specialty products are a class of medications used to treat complex, chronic
conditions that include cancer, Crohn’s disease, growth hormone disorders,
hemophilia, hepatitis C, human immunodeficiency virus, infertility, and rheumatoid
arthritis, among others. Specialty pharmaceutical products dominate big
pharmaceutical research and development (R&D) pipelines, with global sales
expected to exceed 160 billion USD by 2013. They are often produced through
biotechnology and administered by injection, and/or require special patient
monitoring and handling which adds additional cost to drug prices and overall
healthcare spending by the payer. They also place more demands on the regulatory
agencies to assess and approve, because they may have a more complex risk-benefit
and safety profile.
In several respects, 2008 marked an important inflection point for the global
pharmaceutical market. For the first time, the seven largest markets contributed only
about half of the overall pharmaceutical growth, while the seven emerging markets
contributed nearly 25 percent of growth worldwide

(IMS 2010). China has the
largest healthcare spending budget of all emerging Asian economies, totaling 74.2
billion USD in 2006. China is currently the sixth-largest pharmaceutical market, will
16
become the third largest by 2011 and is projected to contribute 21 percent of overall
global growth through 2013. Korea and India are the next largest, with spending of
41.billion USD and 27.7 billion USD respectively (IMS 2010). The average
percentage of GDP spent on healthcare in the combined emergent markets was 5.9
percent or 18.1 billion USD in 2005, as compared to 9.2 percent, or 919.3 million
USD for the EU during the same period (IMS 2010).
The changing face of pharmaceutical markets is reflected not only by GDP
correlations to healthcare spending but also by changes in the language used to
describe differing groups of countries. The use of First World (also called First
Tier), Second World (Second Tier) and Third World (Third Tier) designations is now
seen to reflect an outdated model of the geopolitical world from the time of the cold
war (Sauvy 1952). Generally the term First World refers to so-called developed,
capitalist, industrial countries generally aligned with the United States after World
War II, with more or less common political and economic interests: North America,
Western Europe, Japan and Australia. The term Second World exists mainly by
default; it is very seldom used today. Third World was a term originally used to
distinguish nations that neither aligned with the West nor with the East during the
Cold War. Many were members of the Non-Aligned Movement. Today, however,
the term is used to denote nations with the smallest UN Human Development Index
(HDI) in the world, independent of their political status (Falk 2008).
Countries previously considered as part of the third world designation also
have been identified in academic circles by other descriptors: the Global South,
17
Developing Countries, Least Developed Countries and the Majority World (Falk
2008). Many Third World countries are located in Africa, Latin America, and Asia.
Most of the Pharmerging markets are comprised by Third World countries. They are
often but not in all cases nations that were colonized by another nation in the past.
The populace of Third World countries has in the past been generally very poor and
has been characterized by very high birth rates (Bloom 2007). In general the
countries have not been as industrialized or technologically advanced as the First
World. The majority of the countries in the world fit this classification. However, as
is evident from the economic indicators, this pattern is changing as the countries
exhibit increasing heterogeneity in their profiles. Thus countries are now being
grouped by different metrics, which primarily relate to manufacturing output and
trends in economic productivity (Bloom 2007). These changes have had a major
impact on the way that the pharmaceutical sector must now do business. It also
affects the regulatory systems and climate for the introduction of drugs into these
different markets.
Economic Indicators Typically Correlate with Maturity in Regulatory Systems
Traditional First World countries have had mature regulatory systems for a
long time and are almost always the first constituencies into which new drugs are
introduced. Although most regulatory professionals have a list of specific issues that
they dislike about each of these systems, they can generally navigate the drug-
approval processes with relative ease. Both the EU and the US have rigorous drug
review systems that are rich in procedures and feedback mechanisms. For example,
18
typical EU approval times for a new drug application range from 358 days for the
decentralized EMA procedure to 388 days for the centralized procedure (Business
Insight 2008). These times do not include potential clock stops and question delays.
In the US the FDA has target approval times that are called out by legislation in the
Prescription Drug User Fee Act (PDUFA) (FDA 2009). These are 6 months for fast-
track and 10 months for standard new drug applications. John Jenkins, Director of
the Office of New Drugs, reported to Congress in January 2010 that FDA is
reviewing roughly 85 percent of drug applications on time. Jenkins attributed the
agency's improving performance to a boost in hiring, by roughly 760 new full-time
positions over the last two years. Because of the increase in hiring, many new drug
reviewers have little experience. Jenkins said that he expects their performance to
improve with time. “Our focus is now shifting from recruiting to training so that new
staff can be fully productive, which can take one to three years depending on the
position," Jenkins said (Elsevier 2009). Although this record suggests that the FDA
still considers its regulatory system as a “work in progress”, the regulatory system
has both staffing capabilities and evaluative mechanisms in place that suggest a high
level of maturity. In mature regulatory systems, there is a relatively high level of
predictability in the way that submissions are handled and timelines are promised to
increase confidence in the process.
In contrast to the mature regulatory systems of the first tier countries, the
introduction of medical products in Asia is typically much slower, and in many
constituencies, much less predictable. Japan, one the wealthiest countries with a
19
long-established and mature regulatory system, is perhaps the benchmark system for
Asia, yet even there, concerns have been expressed about speed to market or “drug
lag”. An examination of the evolution of the system is perhaps instructive as a
predictor of what might happen in other Asian countries in the process of their
regulatory maturation.
“Drug lag” is a term used to describe the much slower introduction of new
drugs into the country under consideration compared to the US and EU. Japan is the
second largest single market for pharmaceuticals worldwide. However, over the past
two decades, significant regulatory barriers to new product approval led to an
extensive drug lag; for example, 120 products launched worldwide since 1985 were
unavailable in Japan as of 2009. In 2009, an OECD report revealed that on average,
Japanese new drug approvals (JNDAs) were taking 26 months in the review process,
compared to 18 months in the United States, the UK and Germany (Jones 2009).
One factor that has been identified as a significant contributor to drug lag is
the complexity and perceived opacity of the Japanese regulatory system (Thomas
2001). Transparency has always been a much discussed issue in the context of
Japanese regulatory submissions, or more generally in Asian regulatory submissions.
To some extent, the concerns that have been expressed may reflect problems inherent
when regulatory professionals from pharmaceutical companies are not fluent in the
language of the country in which they are operating. Transparency naturally suffers
if the individuals pursuing drug registration cannot read the rules that are
promulgated by a regulatory agency (Chalmers 2000). However, the concerns about
20
transparency in Asian regulatory agencies appear to go beyond language issues, in
ways that as yet have not been addressed to any great extent in the regulatory
literature.
As a response to the increasing drug and medical device lag, in the late 1990s
the Japanese Ministry of Health Labor and Welfare (MHLW) announced its
commitment to several measures designed to decrease drug lag and increase the
clarity of procedural activities (Sinha 2010). The earliest efforts to increase
efficiency and transparency fell short of expectations, but recently, more aggressive
measures have been implemented to enhance transparency, streamline import
procedures, and speed regulatory decisions. Key activities have also included
“cooperating with the United States to promote high transparency standards and
continuing transparent administration of Special Zones, including the national
application of measures” (MHLW 2007).
Drug Lag is Characteristic of Pharmerging Markets
Drug lag is even greater in Pharmerging markets, for reasons that have been
attributed to a variety of factors including market size, ease of doing business, trade
restrictions, corruption and product pricing constraints. Emerging pharmaceutical
markets like Mexico, Brazil, Turkey or China had access to 60-80 percent of global
blockbusters in 2000 while India had less access to less than 40 percent of those
products (Singer 2010). In 2007 Mexico’s performance continued to be robust, with
access to about 80 percent of the global basket of blockbuster drugs, while other
markets such as Turkey, China, Korea, Russia and Brazil showed healthy access
21
rates of about 70 percent and India lagged behind at 50 percent (IMS 2008).
According to IMS the reason that India has not gained access to a majority of block-
busters is the “lack of patent recognition and transparency of their legal systems”
(Karan 2007).
So What Is Transparency?
A central goal of this thesis centers on the transparency of regulatory
agencies, particularly regulatory agencies in Asian markets. The transparency of
emerging Asian markets and Pharmerging markets overall is much less studied than
that of the systems of First World countries. In order to explore the degree of
transparency in such countries, what is meant by transparency must first be explored
and characterized.
Transparency is defined by the Webster as “the quality or state of being
transparent”, in turn defined as “: free from pretense or deceit: frank b: easily
detected or seen through: obvious c: readily understood d: characterized by visibility
or accessibility of information especially concerning business practice” (Webster
2010). Transparency, as used in the humanities and in a social context more
generally, implies openness, communication, and accountability. It is a metaphorical
extension of the meaning that defines a "transparent" object as one that can be seen
through. Transparent procedures, for example, might be characterized by open
meetings, publicly available financial disclosure statements, freedom of information
legislation, documented budgetary review and audits.
22
Attempts to define transparency in government and the issues related to
governmental transparency have been with us throughout modern history. As early
as the 1800s, it was clear that a goal of the US government was to ensure
transparency, as reflected in the writings of Jefferson:
We might hope to see the finances of the Union as clear and intelligible as a
merchant's books, so that every member of Congress and every man of any
mind in the Union should be able to comprehend them, to investigate abuses,
and consequently to control them (Jefferson 1802)

A similar aspiration was most recently espoused by the current (at the time of
writing) President of the US. Barack Obama’s memorandum for the heads of
Executive Departments and Agencies states that:
Government should be transparent. Transparency promotes accountability
and provides information for citizens about what their Government is doing.
Executive departments and agencies should harness new technologies to put
information about their operations and decisions online and readily available
to the public. Executive departments and agencies should also solicit public
feedback to identify information of greatest use to the public (Obama 2009).

A more focused definition of transparency in government is that of Finkelstein, who
states,
We use transparency to describe those policies that are easily understood,
where information about the policy is available, where accountability is clear,
and where citizens know what role that they play in the implementation of the
policy (Finkelstein 2000).

The meaning of transparency can be clarified by considering not only what
transparency is but also what it is not. Transparency is not simply the absence of
corruption, even though transparency is often discussed in the context of evaluating
corruption. Furthermore, transparency is not just the presence of regulations—the
23
regulation itself may or may not be transparent. A regulation can increase
transparency by clarifying what is required to satisfy a law that is written very
generally, so that practitioners can understand operational aspects of decision
making. The mere activity of developing and deploying new regulations can be an
effective means of attaining transparency through public discussions and expert
consultations. However, the assumption that regulations always work to increase
transparency has been criticized. For example, Fenster describes transparency
theory flaws in his paper, “The Opacity of Transparency”.
Transparency theory flaws result from a simplistic model of linear
communication that assumes that information, once set free (put into
regulations) from the state that creates it, will produce an informed, engaged
public that will hold officials accountable. To the extent that this model fails
to describe accurately the state, government information, and the public, as
well as the communications process of which they are component parts, it
provides a flawed basis for open government laws (Fenster 2006)

To avoid the transparency theory flaw and find an effective route to
transparency in the regulatory setting we need to ask: what components must be put
in place by the respective governments to gain transparency? Potential elements to
achieve transparency should include making information available that is easy to use
and be understood by the public and is available in a timely manner. The information
should not require an “expert” to interpret its meaning (Fenster 2006).
When laws are enacted to support transparency the process for bringing that
law into effect must be transparent. Thus the method, value and findings of litigation
must be available for analysis and review. For example, to protect transparency in
government, every state in the United States has some variant of law mandating that
24
all government business be conducted in open meetings to which the public has
access. These are sometimes referred to as "Sunshine Laws," open government laws,
or, in California, the Brown Act. The Oklahoma Court's decision in Oklahoma
Association of Municipal Attorneys v. State (1978) gives a clear statement of why
open meetings are important:
If an informed citizenry is to meaningfully participate in government or at
least understand why government acts affecting their daily lives are taken, the
process of decision making as well as the end results must be communicated
openly (Oklahoma 1978)

Another perspective on transparency is discussed in some detail by Oliver,
who enters the discussion with this statement, “In some sense, transparency is an
oxymoron: How can we see something that by definition is invisible?” While he
acknowledges the difficulty of defining a term whose meaning may change
according to context and individual, he simplifies the concept by stating that
transparency can be viewed as having three elements: “an observer, something
available to be viewed, and a means or method for the observation”. As an initial
step in analyzing the transparency of an organization, this statement suggests that at
minimum, care must be taken to define the nature of the individual or group whose
perspective is being captured, the scope and nature of the activities under study, and
the ways in which the organization allows the activity to be understood (Oliver
2004). In the work that follows we will use this framework to define the scope and
methodology for the study.
25
Transparency in Public Policy is Multifaceted
Public policy and its relationship to transparency can be traced back at least
as far as the French revolution.
Jacobins believed that if you were not transparent, you had something to
hide. You were an enemy of virtue. They sent people to the guillotine
because they were opaque (Bosher 1988).

More recently, Gerring and Thacker have suggested that openness and
transparency can be “understood as the availability and accessibility of relevant
information about the functioning of the policy” (Gerring 2004). More specifically,
transparency has been described as a “characteristic of those policies that are easy to
understand, where information about the policy is available, where accountability is
clear, and where citizens know what role they play in policy implementation”
(Finkelstein 2000).
The challenge of defining transparency is its multifaceted nature. Thus,
different analysts focus on different aspects of transparency. For example, some
discussions focus on the role of decision-making or communication and
accountability as important elements. Transparency International suggests that
“holders of public office should be as open as possible about all decisions and
actions they take” (TI 2008). To maintain transparency policy-makers should give
reasons for their decisions and they should restrict information only when wider
public interest so demands (Curtin 2006). In the Memorandum on Transparency and
Open Government, the goal of transparency is to “promote accountability and
provide information for citizens about what their Government is doing” (Obama
26
2009). However, some aspects of transparency are not necessarily centered on
decision making and accountability. A much studied aspect of transparency is its
inevitable linkage to corruption. As noted earlier, access to information more
generally has been highlighted by others. However, it is difficult to find a systematic
dissection of the concept of transparency as it relates to the regulation of new
medical products. Thus a first step of this research will examine the concept of
transparency in the regulatory science arena, and explore what the concept means to
individuals who are engaged in activities related to medical product development and
commercialization.
Why Care About Transparency?
Transparency can be described as
creating a culture of candor, where we are really talking about the free flow
of information within an organization and between the organization and its
many stakeholders, including the public (Bennis 2008).

An organizational culture that fosters doubt and uncertainty can lead to loss of faith
and trust by the entire stakeholder base.
Thus, at the forefront of any leader’s mind should first be the establishment
and maintenance of the faith and trust of the people comprising the enterprise
that they serve, whether it is a department, an organization, a community or
nation (Walker 2008).

In the corporate setting, according to John White, Director of the Securities and
Exchange Commission relates:
Transparency is the bedrock principle of good disclosure. I think in its most
common usage, "transparent" refers to something that you can see through,
personally I also believe transparency is an important principle for the
27
regulators, as well as for the disclosure that companies provide in response to
our regulations (White 2007).

The need to facilitate trust through transparency is not just a matter of
establishing good working relationships. When evaluated from a global perspective,
risks associated with the lack of transparency in a country’s legal, regulatory and
governance structures can affect global investment and commerce. Two key indices
in common use today are the Transparency International index (TI) and Opacity
Index. The 15 year old biannually updated TI index ranks economies on the basis of
their perceived corrupt practices. The Opacity Index introduced in 2001 and attempts
to identify the causes and measures the costs and effects of transparency on
commerce. Another index that provides a particularly insightful look at corruption
internationally is the Global Corruption Barometer that measures corruption in
several dozen countries based on the feedback from surveys of the public more
generally (GCB 2009). By asking questions such as “What was the approximate
total amount of money paid overall in bribes by your household in the past 12
months?” and “How would you estimate this in terms of percentage of the household
income?”, a fascinating picture of commerce emerges that allows for direct
comparisons from one country to another. Unfortunately these indices do not
specifically analyze the pharmaceutical industry in their data sets. Thus,
pharmaceutical transparency is not discussed and the relevance of transparency as it
relates to drug registration is not included in this analysis. These indices instead
speak to transparency on a more general level and attempt to create a framework to
28
help companies make better portfolio and investment decisions regarding where to
develop markets, identify productive resources and find the best outsourcing partners
(Pearl 2009).
Western Regulatory Agencies May Provide Transparency Benchmarks
The US Food and Drug Administration
It is safe to say the US FDA is one of the most, if not the most, transparent
health agency in the world today (Permanand 2006). Such a statement does not
however mean that all of the activities between the agency and its stakeholders are
models of clarity. In the context of the drug registration process which is the subject
of this thesis, it only speaks to relatively well-defined nature and extensive
information available for the users of the approval pathways for different medical
products. Several examples of such activities are obviously useful in enhancing
transparency through the provision of information about policies, procedures and
decisions. The 1997 amendment to the US Freedom of Information (FOI) Act has
allowed federal agencies to post the most requested information from stakeholders
for public scrutiny without waiting for Freedom of Information (FOI) requests.
Clinical trial information and other types of information associated with the
development of new chemical entities are posted on the FDA website within 10
weeks. Extensive information about phase II and III clinical trials are posted on
www.clinicaltrials.gov and daily updates of new drug approvals are posted within
four days of approval for both New Drug Applications (NDAs) and Abbreviated new
Drug Applications (ANDAs). Historically much of the information on the FDA
29
websites was considered too technical for non-professionals, but consumer and
public interest groups in the US now routinely analyze such information and produce
complementary reports. The FDA website also provides information about drugs still
under review. For about one-third of new drug submissions, the FDA supplements its
available expertise by requesting consultation with a specifically configured federal
advisory committee. The discussions and analyses of these committees are held in
public and are streamed live on the web.
Despite the numerous efforts to create transparency in the FDA, the FDA
must, by the very nature of its work, keep some decision-making activities
confidential. For example, premature release of information about non-compliance
with regulations or intents with regard to drug approval decisions can result in large
shifts in stock prices that could be unwarranted. In some cases, too much
transparency could even be argued to be a bad thing. For example, new initiatives to
disclose payments to physicians on industry websites, an initiative to some extent
encouraged by the regulatory agencies and government through “Sunshine Bills” are
considered by some to have a dampening effect on the US clinical trials
environment. Sunshine legislation is intended to provide a greater focus on
transparency of payments and other benefits provided by industry to physicians,
hospitals and affiliated organizations. Legal actions against physicians via consent
decrees and State legislation to restrict payments to physicians suggests that public
disclosure of payments is the new norm (Healy 2008). As stated in a recent
pharmaceutical trade magazine,
30
the burgeoning ‘transparency’ campaign is likely to make research efforts
even more difficult, as physicians become increasingly fearful of being
stigmatized by public disclosure of payments from drug companies, even for
legitimate research activities (Wechsler 2010).

Nevertheless, in general, improvements in transparency are typically seen to
be beneficial by the US government and citizens.
The current federal administration, under President Barack Obama, often
speaks about eliminating the barriers between the American people and their
government, and about ways to maintain openness and accountability. To this end,
last year the administration announced the creation of the new Food and Drug
Administration Transparency Task Force. The Food and Drug Administration Task
Force is chartered to “gather recommendations for appropriate ways to make
information about FDA’s actions and decisions that is useful and understandable
more readily available to the public” (Obama 2009). The task force is “seeking
public opinion on how to make the agency’s information on activities and decision
making more transparent, useful, and understandable to the public, while
appropriately protecting confidential information” (FDA 2010).
The Task Force will be developing its recommendations for the Transparency
Initiative in three phases. The first phase, the web-based FDA Basics, went on line in
2009. The FDA Basics site is easy to use, informative and accessible from around the
globe, allowing anyone complete access to all available FDA information. The host
page covers a broad spectrum of topics from medical devices to blood products and
vaccines. It is searchable via the FDA Google-based search engine. The website has
31
biographical sketches of key FDA individuals, gives contact information for all FDA
employees and even provides video interviews with key agency officials. All FDA
guidance’s on drug registration activities can be accessed from this site. The site is
only available in English but a majority of content can be easily translated using
available Microsoft translator programs.
In the second phase the Task Force will make “recommendations to the
Commissioner regarding how to make information about agency activities more
transparent, useful, and understandable to the public, in a manner compatible with
the agency’s goal of protecting confidential information” (FDA 2010). For the final
phase of the initiative, the Task Force intends to make recommendations to the
Commissioner regarding FDA’s transparency to regulated industries. This
information, when available, will help to refine what is meant by transparency in the
context of a regulatory agency that has the oversight of medical products as its key
mandate.
Health Canada
Canada’s drug approval process falls under the responsibility of the Health
Products and Food Branch (HPFB), formerly known as the Health Protection
Branch, within the department known as Health Canada. The prescription drug
approval process is handled by the HPFB’s Therapeutic Products Directorate. This
directorate has been chastised for its relative lack of regulatory transparency. In fall
of 2004, for example, then Federal Health Minister Ujjal Dosanjh criticized Health
Canada’s lack of transparency and their policy of keeping secret the results of
32
clinical trials submitted by drug companies when they apply to have new
prescription drugs approved for marketing:
Questions have been raised as to whether Canadians are well served when the
results of all clinical trials are not publicly disclosed. As an advocate for
public health, it is difficult to me to defend such secrecy … I want to make it
clear that I have a bias on this issue in favor of disclosure (Dosanjh 2004).

In part, this criticism may reflect comparisons made to its US neighbor. In general,
much more information about the drug approval process is publicly available in the
US than in Canada. In the US, as described above, drug companies must present
details of their research and answer questions, often in a public forum, when they are
seeking FDA approvals. In Canada the same information is more often kept secret.
Health Canada sometimes convenes expert advisory committees to provide advice
about the approval of particular drugs (Dhalla 2008). However it posts only brief
summaries of the deliberations of these committees, which meet in private; these
summaries, on the internet, provide only a glimpse into what was discussed
(Silversides 2005). Further, unlike FDA advisory committees, criteria for
appointment to these committees are not available to the public. The definition and
treatment of “confidential third party information” are often highlighted in defense of
Health Canada’s track record of keeping information from the public (Ford 2009).
Valid reasons exist to protect manufacturing secrets and the identities of patients in
clinical trials. However, Health Canada “protects” far more information than this.
Important information from clinical trials and expert reviewers about the safety and
effectiveness of drugs is virtually impossible to obtain (Offerhaus 2005).
33
The level of secrecy maintained around Health Canada’s drug approval
process has been criticized by the media, consumer groups, medical and scientific
bodies (including Health Canada’s own Science Advisory Board), and at least one
Parliamentary standing committee. Health Canada, meanwhile, has publicly
endorsed the concept of greater transparency in the drug approval process —
especially in the past few years (Sim 2006).
The European Union and EMA
Background on the European Community
The EU is currently the source of about one-third of the new drugs brought
onto the world market each year (Broscheid 2005). The European Union (EU),
previously known as the European Community, originated in 1952 when six
countries- Belgium, France, the Federal Republic of Germany, Italy, Luxembourg,
and the Netherlands- created the European Coal and Steel Community (ECSC) by
pooling their coal and steel resources in a common market controlled by an
independent supranational authority. In 1958 the Rome Treaties set up the European
Economic Community (EEC), extending the common market to all economic sectors
in the member countries. The first European Economic Community (EEC) Directive
(a recommended policy used to integrate and consolidate procedures within the EU
member countries) was issued on the regulation of medicinal products in 1965
(Permanand 2006). Since then numerous other directives, regulations, and guidelines
have been issued by the Council of the European Union. An outcome of this planned
34
integration of policies is the European Medicines Agency (EMA) which is the
European agency charged with the evaluation of medicinal products.
From 1995 to 2009, the European Medicines Agency was known as
European Agency for the Evaluation of Medicinal Product (EMEA) but on 8
December 2009, they officially launched their new visual identity to EMA to reduce
confusion with the designation of EMEA as it relates to ‘Europe, Middle East and
Africa’. The EMA came to life not necessarily based on the aim of protecting the
health and safety of the public. As it was enunciated at its inception and remains true
today that the “rationale for forming the EMA has less to do with patients’ interests
and more with Community industrial policy goals” (Permanand 2006). Roughly
parallel to the US FDA, but without FDA-style centralization, the EMA was set up in
1995 with funding from the European Union and the pharmaceutical industry, as
well as indirect subsidy from member states, in an attempt to harmonize (but not
replace) the work of existing national medicine regulatory bodies. The hope was
expressed that this plan would not only reduce the 350 million USD annual cost drug
companies incurred by having to win “separate approvals from each member state
but also that it will eliminate the protectionist tendencies of states unwilling to
approve new drugs that might compete with those already produced by domestic
drug companies” (Broscheid 2005).
EMA operates as a decentralized scientific agency (as opposed to a
regulatory authority which remains at the member state level) of the European Union
and is responsible for the protection and promotion of human and animal health. For
35
products eligible for or requiring central approval, a pharmaceutical company
submits an application for a marketing authorization to the EMA. A single evaluation
is carried out, usually through the Committee for Medicinal Products for Human Use
(CHMP) or Committee for Medicinal Products for Veterinary Use (CVMP). The
majority of existing medicines throughout the European Union's member states
remain authorized nationally, but the majority of new and novel medicines are
authorized through the EMA. The Agency has a staff of about 500, and decentralizes
its scientific assessment of medicines by working through a network of about 3500
experts throughout the EU. The EMA draws on resources of National Competent
Authorities (NCAs) of EU Member states for its membership and support (Broscheid
2005).
Transparency within the EMA
In 2004 Martin Allchurch, the spokesman of EMA, in an interview with
Vermij published in Nature, summed up the agency expectations for themselves and
for the EU in by stating:
Transparency of non-product related issues should considerably increase. All
Regulatory Authorities in the EU should establish best practices in relation to
transparency of regulatory matters (Vermij 2004).

To accomplish this vision, the EMA is on a multi-year mission to increase
transparency. Drug registration transparency in particular, and transparency in
general, remains a priority for the EMA as emphasized in the 2004 Proceedings of
the “European Medicines Agency Road map to 2010: Preparing the Ground for the
Future”. The EMA road map to the future program aims to achieve the broadest
36
possible consensus on the EMA strategy across the entire member state landscape
(EMA 2010). Its focus as described in the EMA communication strategy document
states:
Transparency should be integrated in an overall Agency Communication
Strategy rather than applied case-by-case. Transparency and communication
to stakeholders and the general public should apply where there is a
perceived need and a clear rationale (EMA 2005).

To this end, the EMA, like FDA, has utilized a web based approach to transparency
improvement. All guidance's and directives are on a searchable web site that includes
information on key department and agency heads, including background and
qualifications. Time frames and requirements are well defined and in most cases
followed and face-to-face agency meetings are encouraged. Additionally, unlike the
FDA, the EMA is open to taking phone calls in addition to preset meetings to answer
questions and give guidance throughout the course of the registration process; in
most cases they stick to agreements from these informal meetings. EMA does differ
from FDA in relation to its use of advisory meetings. It appears to acknowledge the
greater use of such meetings in the US by saying that when considering
Transparency in relation to general issues on public access, reference is made
to the US situation where the FDA has open advisory meetings on general
issues (e.g. endpoints in clinical trials); to have the same at the level of the
EMA would be beneficial to the industry. Alternatively, publication of
meeting summaries or (partial) meeting transcripts could be a way forward. It
should be possible to apply the same or a similar level of transparency to the
FDA (EMA 2010).

37
The EU system does not furnish consumers with the same chance to gain information
via written requests, as is possible in the US through freedom of information. The
EMA describes as its expectation:
The EMA should take a middle approach by starting where industry has
already been collaboratively working, with the existing transparency policies
in a similar fashion as set up under the US FO (EMA 2005).

In the 2010 road map draft update the EMA is working to address the transparency
of its operations, extending the levels of transparency beyond the existing legal
requirements. As an emphasis for its 2010-2015 initiatives, all past initiatives are
being collected into one modern approach. The new draft transparency policy
describes for the first time in one comprehensive document the Agency's vision on
its level of openness towards its stakeholders (EMA 2010).
Pharmerging Markets Have Different Regulatory Challenges
A central focus of the current thesis is the comparative transparency of
regulatory systems in Pharmerging markets particularly those of Asia. In an attempt
to provide a global overview of key Pharmerging markets the next section of the
literature review provides a brief analysis of the market dynamics, health care
systems, health authority overview and transparency insight for Turkey and Latin
America, as examples of emerging countries outside of Asia, as well as China, India
and the Republic of Korea, that are subject to further study here. Turkey and Latin
America are useful comparators since they are similar in economic status and
regulatory maturity to the Asian countries in which we are interested. These
descriptions are necessarily brief, and will highlight some of the features of
38
regulatory transparency or opacity that will become important in the development of
a transparency index.
Transparency in Turkey
The Turkish Health Care System
Turkey, a middle-income country with 70 million inhabitants, is notable
because it is on a course toward full EU membership in the near future. The most
recent and internationally comparable estimates from the National Health Account
study indicate that Turkey spent, in 2005, 13.1 billion USD on healthcare. The share
of total health expenditures expressed as a percentage of GDP was estimated at 6.6
percent. Of the total healthcare spending 63 percent was paid for by public funds,
and 37 percent of this percentage came from social security organizations. Out-of-
pocket expenditures constituted 28 percent of total health expenditures (OECD
Turkey 2009). As of May 2009, 1300 active ingredients and 3500 products with
different forms were available on the Turkish market. The market is exclusively
served by either branded original products, or by branded generics. Unbranded
generics are absent from the Turkish market (OECD Turkey 2009).
The Turkish health care system is undergoing significant changes as it
prepares itself for EU membership and increases its focus on a “national
commitment to deliver healthcare in a fair and equitable manner to all citizens”
(Kanavos 2006). However, Turkey has a very long road ahead in providing a
countrywide foundation for a modern and effective healthcare system. One key issue
in providing a balanced healthcare system in Turkey is the significant gap between
39
urban and rural healthcare delivery systems where the rural systems lag far behind
the more modern and efficient urban setting (OECD Turkey 2009).
Transparency of the Turkish Drug Registration Process
Turkey may be a benchmark for drug registration transparency in the
pharmerging markets. Turkey has a well-defined set of laws, regulations and
guidance’s (OECD Turkey 2008). Its Ministry of Health continues to make
improvements in the transparency of the drug registration process. The provision of
information and clarification of procedures has had much attention in the last few
years as the EU preparations for entry into the EU increase. Marketing
authorization/approval details are clearly defined in the Licensing Regulation for
Pharmaceuticals Official Gazette No: 22218, 2.03.1995. Apart from certain
procedural differences between novel and generic products, the requirements for a
registration dossier for pharmaceutical products are standardized and all applicants
must comply with them (Turkey MOH 2003). The Appendix of the Licensing
Regulation for Pharmaceutical Products provides a full list of information and
documents that must be included in the dossier for the registration of medicinal
products and is available on the internet; however not all parts are in English at this
time. In order to assist applicants, the Turkish Ministry of Health has produced a
special checklist available on the web (Guideline of Evaluation of Authorization
Applications) where the items to be included in the registration dossier are listed, in
addition to WHO and FDA guidelines and recommendations (Turkey MOH 2003).
40
The decision-making process for product approvals also exhibits
characteristics that are important for transparency. If a license application is
rejected, the applicant is notified of the decision and is given the reason for the
rejection. The applicant is entitled to recourse to the decision by using the courts to
hear their rebuttal to the application outcome. One issue with the current system is
the absence of specific and binding deadlines for the conclusion of the processing a
marketing application. However in 2004 the Ministry of Health issued a notice “to
the Pharmaceutical Industry Manufacturers Association (IEIS) where they declared
that their intention was to complete the registration/marketing approval within 210
days for all products” (ECTurkey 2004).
The accountabilities inherent in the regulatory approval system are also
undergoing change and further definition. Historically the Ministry of Health
(MOH), General Directorate of Pharmaceuticals and Pharmacies (GDPP) was the
sole authority in charge of registration, marketing approval/authorization, pricing of
pharmaceuticals, legal classification and inspection. However, a recently announced
Health Transformation Program includes a component for the establishment of a new
National Drug Agency. The new agency resembles both the EMA and FDA. It will
be autonomous within the overall MoH structure and be funded by user fees and will
comply with all EU procedures.
Transparency in Latin America
Transparency issues related to pharmaceutical registration in Latin America
appear daunting for regulatory professionals not familiar with the inner workings of
41
the respective nations. Each country in Latin America has had its own approach to
registration and in some cases its own language requirements. For example,
Brazilians speak Portuguese, whereas Mexicans speak Spanish. The beginnings of
improvements in web access to the health authority guidances and expectations can
be seen in Brazil and Mexico. Each country now offers English and native language
web sites. However the English web sites are more limited in scope; updates relating
to legislative changes are slow and in some cases the updates reflect future intentions
for regulatory practice rather than current practice. These points notwithstanding, the
regulatory authorities in the four key Latin American markets (Argentina, Brazil,
Colombia and Mexico) are working more efficiently and in an increasingly
harmonized way. They have a relatively small backlog of compounds in the
approval queue and they currently face an average drug lag of 5-7 years or less for
generic and innovator drugs (IMS 2008). However, these countries still have barriers
to efficient registration. Lack of transparency during the review process, often a
function of limited resources, is perceived as the key problem in this region, although
recent investments in technology and personnel have improved review times and
transparency (Ghersi 2008). As a way of compensating for reduced efficiencies, the
countries often rely on a Certificate of Free Sale (CFS) to simplify their registration
activities. A Certificate of Free Sale, also known sometimes as a "Certificate for
Export" or "Certificate to Foreign Governments”, provides assurance to the drug
importing nation that the products listed on the Certificate are freely sold and
42
manufactured in the U.S with the implication that the same product will be both safe
and effective for human use elsewhere.
Transparency in China
The Chinese Healthcare System
According to IMS data, China’s pharmaceutical market, the world’s tenth
largest in 2006, is projected to climb to third by 2011. Currently, China has 20
percent of the world’s population but only 1.5 percent of the global drug market. As
of 2007, it had around 5,000 domestic pharmaceutical manufacturers and around
14,000 domestic pharmaceutical distributors. Foreign pharmaceutical companies are
responsible for only 10 to 20 percent of overall sales depending on the types of
medicines; sales by the top-tier Chinese companies are growing faster than those of
most western firms (OECD 2007). In contrast, the top 10 companies in most
developed countries control about half the market. Key negative factors related to
the penetration of global companies into this market include a lack of protection of
intellectual property and the lack of transparency of the drug approval process (Kong
2008).
To understand whether transparency plays a significant role affecting the
entry of global players into the Chinese pharmaceutical market, it must be
deconvolved from other factors, most notably economic factors that have historically
made the Chinese market less attractive than it is today. China has made recent
strides in improving their healthcare system, yet it is still ranked 144 out of 191
countries, and is even below Iraq in providing acceptable levels of medical treatment
43
to their populace according to the World Health Organization (WHO 2007). Since
the founding of the People's Republic, the goal of China’s health programs has been
to provide care to every member of the population and to make maximum use of
limited healthcare personnel, equipment, and financial resources (Yip 2004). After
the globally perceived mishandling of the 2003 SARS outbreak, China launched its
new cooperative medical system (NCMS). Unlike the old commune-based model
that dated back to the 1940s, NCMS is voluntary and operates at the county rather
than at the local or village level. Working with larger pools of citizens allows the
system to benefit from economies of scale.
The funding of NCMS is also different than its predecessor program. The old
system relied on worker contributions via welfare funds; NCMS now receives part of
its funding from government subsidies, with the central government helping poorer
county governments to pay for their share. Farmers covered under NCMS pay an
annual premium to enroll in the plan. To support this progressive program, the
Chinese central government increased its health budget by 87 percent between 2006
and 2007. Additionally in 2009 the government announced that it would inject
another 25 to 35 billion USD of funding (about 1-1.5 percent of the GDP) into health
care in order to provide universal basic health care (Anson 2006). The funding
commitments have allowed the NCMS to grow quickly. It is estimated that “by the
end of 2010, the NCMS will cover 86 percent of the rural population” (Yip 2009).
China’s community health centers integrate Western and traditional Chinese
medicine under “a six point framework of care: prevention, health education and
44
promotion, birth control, outpatient evaluation and management of common
illnesses, case management of chronic disease, and physical rehabilitation” (Hou
2009). Taken together, NCMS and community health centers represent a positive
starting point for healthcare improvement in China (Hou 2009).
Transparency Initiatives
In China, the regulatory agencies controlling health care products are more
decentralized than is typical elsewhere in Asia, due in part to the sheer land mass of
the country. Regulatory agencies at the provincial and even municipal level have
traditionally held, and in some cases continue to hold, roles in the marketing
approval of many products, notably those that have lower risk such as low risk
devices from domestic manufacturers, and over-the-counter drugs and herbals (Kong
2008). However, the entry of new molecular entities into the Chinese market is in
the hands of the State Food Drug and Administration (SFDA), which despite its
name is the federal body responsible for regulating the entry into Chinese
pharmaceutical markets. In March 1998, the Ministry of Health's Department of
Drug Administration merged with the State Pharmaceutical Administration of China
(SPAC) to become the State Drug Administration (SDA), with oversight over drug
manufacturing, trade, and registration. In 2003, the SDA was restructured to become
the State Food and Drug Administration (SFDA). This restructuring into a single
drug regulatory authority was an important step toward foreign access, because it
eliminated the conflicting standards that prevailed amongst provincial government
agencies. It also was transformational in improving the transparency of the drug
45
registration process. SFDA now oversees both the registration and advertising of all
Western and traditional Chinese medicines (Kong 2008).
The SFDA has been challenged to regulate the rapid growth of a combined
system of Western and traditional Chinese medicines, and has had to focus on
specific elements that have been judged to have most risk to public health.
Manufacturing is of central concern, driven by scandals related to drug
counterfeiting or substandard quality. SFDA appears to be using the US to
benchmark and improve their manufacturing operations across China as reflected in
their new rules for Good laboratory Practices (GLP) and Good Manufacturing
Practices (GMP). Beginning in 2004 the SFDA required all pharmaceutical
companies in China to obtain GMP certificates from SFDA before selling their drug
products in China. About 3000 of the companies met the deadline; companies in the
process of obtaining certification were allowed only to subcontract secondary
production to a certified company until they gained certification (SFDAGMP 2009).
SFDA has also launched a regulation on drug research and supervision management
aimed at enforcing GLP to investigative drugs, traditional Chinese medicine and
biotechnology products. The regulation aims to help China’s drug domestic
pharmaceutical suppliers to gain international recognition as a supplier of safe and
effective drugs. The new rules are also aimed at increasing consistency and
transparency in the processes by which drugs could be manufactured, thus
decreasing the likelihood of corrupt practices and negative public health outcomes
(SFDAGMP 2009).
46
Attention to the state of regulations and transparency in the drug approval
process has perhaps received less attention than quality systems improvements, but
improvements in regulations are gaining more attention of late. The SFDA
communicated the requirements for Special Approval of New Drug Registration (the
Requirements) in early 2009. These new measures now require that SFDA and local
food and drug administration bureaus provide drug applicants with comprehensive
information on the application process—including application procedures,
application charges, timelines, required application materials and lists of officials in
charge of the application and inspections. Additionally these details must be
publicly posted on the SFDA website and on the websites of the local bureaus. In
cases where there are multiple interested parties, as for drug registration, a new
requirement allows applicants to request a public hearing on findings, a system that
bears similarity to the advisory committee process of the FDA (SFDARegs 2009).
This new set of requirements is a step in a positive direction but transparency
improvement for drug registration is still a work in progress. It remains unclear as to
exactly what is expected for submissions in many cases and how long each step will
take. For example, the current status of Chinese clinical trials (at least late-phase
clinical trials that must be conducted in China for products to be registered in China)
are not available to the public, although this is a necessary reporting requirement for
companies in the US and EU (SFDARegs 2009).
47
Transparency in India
In the past 15 years a strong and growing Indian economy has helped to drive
the Indian pharmaceutical market to the top tier of the Pharmerging market sector.
India currently accounts for over 5 percent of the Africa/Asia/Australasia market and
is ranked 14th in the world pharmaceutical markets (BusinessInsight 2008). The
highly competitive market in India continues to be dominated by Indian
pharmaceutical companies, which are fragmented into over 20,000 registered
pharmaceutical, biotechnology and medical device companies (Tanzer 2005).
Currently, more than 400 APIs (Active Pharmaceutical Ingredients) and over 60,000
pharmaceutical formulations are available in India. Additionally 60 percent of the
APIs and 25 percent of patent medicines are exported (Kumra 2009). Currently,
India has nearly 100 pharmaceutical manufacturing facilities approved by the FDA.
This is the largest number of facilities located outside the US that are approved to
source pharmaceuticals for the US market. India ranks fourteenth in the world in
terms of the overall value of their medical products (Nangra 2009). According to the
McKinsey and Company, the Indian pharmaceutical market may reach 25 billion
USD in 2010 with growth at around 1.5-1.6 times that of the country’s gross
domestic product (Kumra 2009).
India's per capita healthcare expenditure is well below international norms
and compares poorly with the other Pharmerging markets (Brazil, Russian
Federation, China, Republic of Korea, Turkey and Mexico). Despite recent
improvements in the country's healthcare infrastructure, insufficient health centers
48
exist and health personnel remain in short supply (OECD India 2007). According to
WHO estimates, rural areas account for 21 percent of the country's pharmaceuticals
market. In 2006-07, the rural Indian pharmaceuticals market was valued at
approximately $1.4 billion. Remarkably, this grew at about 40 per cent in 2006-07
against 21 per cent in the previous year (Nangra 2009).
The transparency of the drug approval process is improving but protection of
intellectual property and infrastructure weakness are key concerns impacting the
continued growth of the pharmaceutical industry in India. All pharmaceuticals
imported into India must have prior approval from the Drug Controller General of
India (DCGI). The principal legislation for drug products in India is the Drugs and
Cosmetics Act of 1940 (DCA). Both state and national authorities share the work of
drug regulation. Depending on the type of pharmaceutical product, the product will
first go through a new drug approval process to qualify for drug registration. For
drugs that have never been marketed in India before, the DCGI typically requires
phase III trials to be performed in India to receive an import registration certificate.
Then the importing party uses the import registration certificate to obtain an import
license from the DCGI (DCGI 1997). The frame work for the Indian quality system
has its foundation in WHO Good manufacturing Practice (GMP) guidance.
Nevertheless as more western companies move into India and as export demands
increase, the US GMP and International Standard for Harmonization (ISO)
regulations are increasingly adopted to permit exportation (Srivastava 2009).
49
It would be a gross overstatement to say that India could set the benchmark
for transparency improvement in Asian pharmerging markets but it is clear that
improvements are being adopted and that they are modeled on US and EU
frameworks. For example, the DCGI has an easy-to-use English web site that gives
access to information such as current guidance’s, public meeting announcements,
registration fees and clinical trial requirements. The web site provides a clear and
detailed classification scheme for clinical trials. Category A includes clinical trials
whose protocols are approved by First World countries including US, UK,
Switzerland, Australia, Canada, Germany, South Africa, Japan and EU. Category B
includes all other applications. The rules require the applicant to submit details, such
as the regulatory status of the tested drug in other countries and the names of
countries where the drug is approved, along with the international package insert or
the place where Investigational New Drug (IND) application is filed. Applicants
have to report any Suspected or Unexpected Serious Adverse Reaction (SUSAR)
from other participating countries. This new transparency of the clinical trial process
is very important because 70 percent of the cost of development and registration of
new drugs stems from the conduct of clinical trials. This improved transparency and
the related improved emphasis on clinical trial safety reporting is playing a major
role in growing investment in pharmaceutical research and registration in India
(WHOIndia 2008).
50
Transparency in South Korea
In the IMS Market Prognosis report, South Korea is predicted to be one of the
fastest-growing markets among Asian countries, with projected growth of 14 percent
from 2006 to 2010. South Korea adopted Japan’s health insurance system as a model
and accomplished a rapid, successful introduction in just 15 years. Today every
South Korean citizen is entitled to free healthcare through universal insurance
coverage, but in many cases they end up paying for drugs out of their own pockets
because insurance premiums are not high enough to cover many of the
pharmaceutical products demanded by the public. The government is attempting to
address this problem, which should benefit both South Korean citizens and
pharmaceutical companies, but progress is slow. Overall health care expenditures, at
6.1 percent of GDP, are modest compared to other developed countries. However,
health expenditures overall are expected to rise considerably over the next decade,
both in total and per capita terms and as a percent of GDP (Sartorius 2009). South
Korea has improved and expanded their hospital system but their number of beds per
thousand remains fairly low, only slightly higher than China. Additionally the future
availability and quality of hospital care is expected to decline against the world
standard as South Korea attempts to cope with its aging population (Woo 2007).
The transparency of intellectual property and company confidentiality
regulations are a major barrier for foreign companies utilizing South Korea for drug
development or manufacturing. Lack of transparency of the drug registration process
is a frequent criticism levied against the KFDA (AsiaNews 2009). The drug
51
registration process is managed by the Korean Food and Drug Administration
(KFDA) which assumed the framework for its current structure in 1996 and its
current name in 1998. In 2009 the most significant set of changes to KFDA to date
were introduced as a “Massive Reorganization”. This major change involved a
reorganization that reduced both division and staffing numbers (AsiaNewsKFDA
2009). Today there are four divisions of the Pharmaceutical Safety Bureau
responsible for drug registration. KFDA maintains a web site that is available in
English but it is cumbersome to use, is not updated often and has only limited
guidance and details on drug registration. The web site is dominated by food topics
and importation rules for food products.
The KFDA is often criticized for a drug registration process that is not
transparent and has recently announced efforts to make the process simpler and more
comprehensible. To try and improve transparency of the registration process the
agency has prepared detailed manuals on Korean drug registration standards. There
are four manuals, divided by topic into safety, efficacy, product standards, and
testing methods. The agency also plans to introduce a new Common Technical
Dossier (CTD) standard in the near future (AsiaNews 2009). However, this new
format may not follow the ICH CTD format that is accepted in other countries with
mature regulatory systems. This decision would only add complexity to the
registration process. Attempts at improving transparency are a part of the campaign
by the KFDA Commissioner, Mr. Yun Yeo Pyo, to improve public trust in the
KFDA which was lost in 2008 due to a scandal related to melamine contamination of
52
Chinese foods on the Korean market. Mr. Yun hopes “to improve the agency's image
in its regulation of drugs, medical devices, and food, under the broad slogan, peace
of mind for the public, vitality for industry” (KFDA 2009).
How Do We Compare Countries at Different Stages of Regulatory Maturity?
Although every country has its own particular approaches to improving
transparency, comparisons of economically mature and Pharmerging markets suggest
that the foci of their activities vary. In countries with mature economies and
regulatory systems, efforts to increase transparency are more nuanced. They are
primarily designed to ensure that stakeholders have superior access to information
and have trust in the process, if not the details, of decision-making. In contrast, most
Pharmerging markets are putting regulations in place, often for the first time. They
must deal with the underlying challenge of replacing a system in which corruption
and rule-bending is more prevalent, and in which rules are new and often
unenforceable because of resource restrictions. In these latter countries, attempts to
introduce new drugs can currently be plagued by risks of violating laws of both the
local country and the country of origin. For example a US country that is
introducing a drug overseas must comply with US laws aimed at curbing corruption
internationally, such as the Foreign Corrupt Practices Act (Laufman 2009). Payment
of a bribe in a foreign country would thus constitute violation not only of the law in
the country in which the infraction occurred but also in the US. Thus, any metric of
transparency must take into account the different levels of maturity of different
countries. At one pole are relatively underdeveloped countries that have few rules at
53
all. At the other are countries in which the rulebook has been written for some time
and the efforts at transparency are directed toward improving rather than creating a
useful system of regulations.
Few studies of transparency in the past have focused on transparency for
regulation of medical products in a comparative way that would involve metrics.
Further, a paucity of information exists concerning the experiences of regulatory
professionals with the drug registration process in emerging markets. Many of the
publications on transparency initiatives and experiences are not related to emerging
markets but instead focus on economically mature markets such as Canada, Japan
and the US. The very small amount of literature on drug approval processes in Asia
typically dwells on a specific aspect of the submission process such as clinical trial
design (Zaran 2007).
To explore transparency in the drug approvals process, it is useful to look at
previous ways in which transparency of government activities have been studied
more generally. For such information, it seems most useful to look to the World
Health Organization whose role in the study of comparative regulatory transparency
has been particularly important (WHO 2006).The WHO early framework for
assessing transparency in emerging markets was based on a survey of key
individuals involved at many different levels of the drug supply chain from
development through distribution. The WHO studies in 2003, 2006 and 2008
provided a survey instrument called a “transparency index” that was valuable as a
potential starting point for more targeted assessments of the transparency process.
54
However, the focus of the transparency index was largely directed at assessing
transparencies related to corruption only and did not maintain a singular focus on the
drug approval processes.
A more voluminous literature exists on corruption in general. One useful
source of information is the set of corruption indices published by the World Bank
and Transparency International, that rate and rank economies on the basis of their
perceived corrupt practices (TransparencyInternational 2008). Although the
prevalence of corrupt practices is often considered to be a reflection of a poorly
transparent system, it is difficult to correlate the two directly. For example, Japan
has often been described as relatively non-transparent as compared to other nations
yet its corruption index ranks relatively well, at 18
th
in 2008
(TransparencyInternational 2006).
Framing the Study of Regulatory Transparency
The transparency and corruption indices that now exist are not sufficiently
granular to serve as good tools to compare the specific transparency of drug approval
systems in different countries. However, they do provide an interesting model to
guide the construction of a more specific and appropriate tool for this purpose. A
central objective of the current study is to develop a more focused instrument which
could be used to gain insight into the state of transparency in different regulatory
agencies. The structure of the instrument could be modeled on the previous indices,
but the approach, content and deployment of the tool must be more specific. As
noted above, an index of drug-approval transparency would have to capture
55
transparency differences in countries that vary widely in their maturity levels and
types of initiatives. A questionnaire would have to be flexible enough to
characterize the full range of countries to which it is applied.
The theoretical basis for the study of transparency in applications such as the
one outlined here is immature. One approach to the evaluation of transparency, for
example, was advanced by PricewaterhouseCoopers (Oliver 2004). They proposed a
model that has three tiers of evaluation, as shown in figure 1. From the bottom up,
the tiers identify the need to adhere to standards or policies that are defined 1) at the
level of the public at large, 2) the sector under study, and 3) the organization itself.
However, this model seems designed to address only one dimension of transparency,
related to the clarity and compliance with policies and procedures. This approach
seems predicated on studying a system whose policies and standards at these levels
have been defined and are already quite mature, such as the financial industry. In the
area of regulatory transparency, standards are not yet well-characterized. Further
this approach does not consider some of the other dimensions of transparency such
as decision-making transparency.
A second framework that is equally concise and perhaps more useful for the
work proposed here is that advanced by Oliver (2004) who suggested that analyses
of transparency might be framed by considering three elements. First, the evaluation
must establish the perspective of the analysis by specifying the “observer”; as Oliver
states, “Who’s watching what, when, where…and why?” Second, it must identify
“something available to be observed”. This could be interpreted as a requirement to
56
identify the scope and nature of the activities to which the examination is applied.
Third, it should characterize the “means or method for the observation” used to
evaluate transparency (Oliver 2004).

Figure 1: Schema for evaluating transparency, modified from
PricewaterhouseCoopers (Oliver, 2004)

Specifying the Observer
Specifying the observer is important both from the point of view of
identifying the sampling method to be used in the experimental design and the
relevance of the analysis to one or more beneficiaries of the research. Several
audiences may be interested in the transparency of agencies concerned with drug
approvals. One group with a particular interest in the transparency of the FDA and
similar authorities elsewhere would be governmental stakeholders concerned with
benchmarking efficacy and performance of this important government department, in
57
order to improve its operations or reduce problems that might result in expensive
remediation of bad decisions. A second group is that comprised by industry
stakeholders. This diverse group might be stratified to include regulatory affairs
practitioners who submit applications to the various health authorities and work with
them on a regular basis; individuals who are seeking information about some aspect
of the health authority function but are more peripheral to direct interactions, such as
individuals in research or quality functions without direct interaction with the
agency; and executive managers who are less concerned with process than outcome,
particularly when the decisions that are made affect the timeline or success of market
performance. Finally, a range of other stakeholders would include students who are
trying to learn about the policies and practices of the health authority; lawyers who
are engaged in litigation concerning the interpretation of regulatory decisions or
actions; and citizens who are interested in the protections and performance of the
health authorities more generally. It is quite possible that any one of these groupings
could be further sub stratified. For example, a survey of perceptions regarding
transparency in different Asian countries might yield different results if the sampled
group were to be composed of individuals who had worked extensively in Asia,
compared to another group whose activities were normally confined to the US and
had only brief experience with the Asian regulatory authority. Thus specifying the
observer is critical in ensuring that the bias inherent in the captured perspective is
appreciated, and might be controlled by appropriate triangulation.
58
Specifying “What is being observed”
In this study, we have a stated intention to examine transparency in the drug
approval process, also called the drug registration process. A framework that
specifies the scope and nature of elements encompassed by the term “drug
registration process” should therefore be defined. For the purposes of the present
study, these elements can be identified as a set of key activities and evaluations for
successful dossier submission. Although each country and to some extent each
region has a somewhat unique approach to the drug approval process there are
dossier commonalities (Temple 2002). The building blocks for a registration dossier
can be characterized as a set of gates to assure that the arrival of a new drug to
market is a controlled process. This control is intended to assure that the health
authority responsible for approval of the drug has adequate oversight to assure that
its approval criteria are achieved consistently (Ratanawijitrasin 2002).
To structure thinking about the drug approval process, the submission
activities and deliverables for an NCE involve a multistep pathway, including
interactions with the regulatory agency of interest at different stages of drug
development. The scope of the approval process can be defined narrowly or widely.
Narrowly, it would involve only the submission process itself, including direct
interactions concerned with the submission and approval process (that might be
arbitrarily defined, for example, as the period from 6 months before submission to
the date of approval) or more widely to include interactions earlier in the
development process (when, for example, animal testing, clinical trials and
59
manufacturing methods require clarifying information from the regulator). For the
purposes of this study, I will define the scope quite widely, as the period prior to
product approval from the point of first interaction or contact with the regulatory
agency, usually starting with information-seeking behavior from its literature,
website or personnel. Typically this period will include an early epoch when
information from the agency is sought, an intermediate epoch when the submission is
being written and submitted, and a terminal epoch when regulatory personnel ask
questions about the dossier that is submitted, ending with the approval or rejection of
marketing permission (Figure 2). The dossier that is submitted may vary in its
format and content. Request for the registration of a new chemical entity is often
submitted using an application in either a harmonized format known as the common
technical document (CTD) format, or a format specified by that country uniquely.
However, generic drugs may also be submitted using the CTD or specialized
formats. Sometimes the dossier for registration is preceded by applications seeking
permission to conduct clinical trials. For discussions of the particular requirements
of these submission routes, the reader is recommended to refer to the various region
health authority web sites as a starting point for defining requirements for
registration activities.
60

Figure 2: The drug approval process in the US modified from Jenkins' speech to
congress

As part of the decision-making about “what should be observed”, we can also
define dimensions of transparency that would be important to consider. In the
literature review above, at least four dimensions were discussed frequently, including
access to information; clarity of policies and procedures; openness of decision
making; and accountability for actions. These overlap with, but are not identical to,
IMF guidelines for governmental transparency, that identify four key dimensions:
clarity of roles and responsibilities; public availability of information; open budget
preparation, execution and reporting; and assurances of integrity. Oliver identifies
61
four alternative, more specific tactical dimensions that he regards as important for
organizational transparency: leadership commitment; formalized processes; training
programs; and communication with stakeholders (Oliver 2004). Although these
dimensions have subtle differences, it is clear that transparency is multidimensional
and these dimensions must be considered before launching any type of research.
Governments often use a basic set of vehicles to enhance transparency. These
may include ways to foster communication, such as website development,
scheduling of public meetings, and policy and guidance promulgation. It may include
ways to enhance accountability, including anti-corruption measures or increased
scrutiny of personnel performance. One option for this thesis would be concentrate
on the vehicles to accomplish one dimension of transparency, such as accountability.
The previous transparency indices focused on this dimension. For example the WHO
2006 study of transparency of four Asian countries focused on the need to develop a
“national ethical framework for implementing a strategy of promoting good
governance in medicines regulation and procurement” (WHO 2006). The WHO
assessment emphasized the requirement to build a code of conduct and improve the
national ethical framework of government officials and health professionals.
Alternatively another option of the thesis would be to examine the role of one
vehicle, such as the role of the internet, to enhance transparency.
Specifying the Means/Method for Observation
Evaluation of transparency could take many forms, according to the question
that is being asked. In this study, I am interested in the drug regulatory system, and
62
within this system the transparency of the drug approval process in different
countries. Thus the thesis will delimit the scope of the activities to be studied (the
drug approval process) and the number of countries that are initially examined, but
will to look as broadly as possible at the many different ways that governments work
to enhance transparency in this process. Many different methods are available to
provide insight into this question, including in-depth interviews, document and
artifact analysis, focus group discussions, and surveys. The decision to use two
primary methods, focus groups and survey methods was chosen because the methods
lend themselves well to the problem under investigation, and allow a relatively
broader exploration of many dimensions of the problem while still giving the kind of
semi-quantitative results that would permit more effective comparisons across
countries. The use of the survey instrument will be tested in a pilot exploratory
study on three Pharmerging countries, and directed at one specific subgroup of
observers, composed of regulatory professionals in the US who regularly interact
with at least two of the three target countries as they attempt to gain product
approval. By using initial focus groups to guide the construction of the survey, I
hope to reduce the extent of bias that might develop if the survey was to be
constructed by a single person, and to increase the likelihood that the survey will
capture most of the salient dimensions needed to be able to characterize transparency
more fully.
63
CHAPTER 3: METHODOLOGY
The methods to study drug registration transparency in Pharmerging markets
used a dual approach to create and then deploy an index for transparency. First, two
expert focus groups were formed to assist in the design of the survey tool used to
develop the transparency index. The expert groups represented a cross-section of
regulatory professionals with strong regulatory expertise and knowledge of
international regulatory activities. The first focus group was used to explore the
issues and dimensions that are felt to be important to the “experts in the field”. This
group was provided with no background information about the findings of the
literature research outlined in chapter 2, and was given no access to any preliminary
transparency index that may have been created prior to the focus group. The goal of
this focus group was to identify whether regulatory experts in this relatively
unstructured forum could identify through brainstorming any new elements or
dimensions of drug registration transparency that were not captured by the literature
review. These ideas were then fed into the framework of the key dimensions and
attributes and were addressed by constructing specific relevant questions in the
survey tool. This new tool was created by using the WHO study of Transparency in
2003 as a beginning template. A second focus group was then convened to critique
the survey instrument. Results of this focus group were used to modify the survey
instrument prior to deployment.
64
Second, the drug registration transparency index was finalized based upon the
data collected in focus group discussions. The modified instrument was then
distributed to key informants familiar with registration processes in Asia.
Focus Group 1
Focus group 1 was comprised of 8 regulatory professionals and was
convened in November 2009 in a neutral setting that allowed for discussions
amongst all participants in an open and informal manner. The moderator attempted
to use a style like that recommended by McCracken, to create “a permissive and
nurturing environment that encourages different perceptions and points of view,
without pressuring participants to vote, plan or reach consensus” (McCracken 1988).
The discussions were conducted in two stages due to logistical
considerations. The first stage of the focus group discussion was conducted in
Beijing, China and the second phase was conducted in Inchon, South Korea, as part
of an educational delegation to Asia. All but one of the participants worked and
lived in the Southwestern US at the time of the focus group; the last individual
resided in the Midwest. A detailed description of the individuals and their
backgrounds are listed in Appendix B. The individuals have been listed
anonymously by using a coded set of initials for each individual. A “dual
moderator” approach was used for the focus group. The focus group met face-to face
for a combined total of one hour and half hours of discussion. Both sessions were
led by an academic regulatory professional with extensive global regulatory
experience in the Pharmerging markets of Asia. I served as a secondary moderator;
65
in this focus group I attempted to remain silent except to probe and clarify responses
from the panel. The focus group was not recorded but notes were taken by both
moderators throughout the session. Both sets of notes were then combined and
compared for consistency and clarity of content. To set the stage for discussion I
provided a brief overview of the thesis and the concept of transparency as it relates to
drug registration. I also provided clarification of the plan for this index to focus
solely on drug registration transparency and to exclude discussions on corruption or
other factors as they might relate to drug registration. Because this focus group
occurred early in the research process, its outputs were later used to guide the
literature review in chapter 2.
Survey Development
The initial survey instrument was built with reference to the WHO
framework from the 2003 study on corruption and transparency in Pharmerging
pharmaceutical markets in Asia. The initial survey was produced in a manner that
allowed it to be sent to individuals targeted by selection using a snowball technique,
in which regulatory experts with strong experience in Asia were asked to recommend
individuals who they knew to be active in submitting drug registrations in Asia.
Specific questions were developed according to input from the first focus group and
from the literature review and analysis. The survey was constructed using single
answer, Likert, matrix-table and text-response options. The index was then
developed more fully into an internet-ready format using the Qualtrics electronic
66
survey tool. This program allowed both quantitative and qualitative questions, and
questions with open and closed formats.
Focus Group 2
Focus group 2 was held in May 2010 on the University of Southern
California Irvine Campus in a state-of-the-art media room which allowed
videotaping of the focus group and remote call-in capability. The draft survey
instrument was provided electronically to the participants in advance of the meeting
and was also distributed on paper during lunch. The call-in capability was used by
one participant located in the southeastern United States. A “dual moderator”
approach was used, in which I served as the primary moderator and the secondary
moderator served principally to clarify and record proceedings. The focus group of 9
regulatory professionals met for two hours; this time included about 30 minutes for a
light lunch and beverages. A detailed description of the individuals and their
backgrounds is provided in Appendix C. This description was developed from a
questionnaire provided to each participant who was then listed anonymously by
using a coded set of initials for each individual.
The focus group commenced with an introduction of the participants. It was
followed a short presentation to give the background to the problem and a brief
summary of the research activities conducted to date. After a brief discussion of the
limitations and delimitation of the study I asked the group if there were other general
areas to consider before beginning specific discussions of the survey questions,
which then followed. The outputs of this activity are described in chapter 4.
67
Survey Delivery
The survey tool was modified according to the advice of the focus group. It
was then reformatted and electronically distributed to 10 administrative reviewers to
validate that the electronic version was distributed without corruption and that
response data was captured correctly for analysis. After this validation step the
survey was distributed to the participants. Most individuals were solicited by phone
or email in advance to identify their willingness to receive the survey, and to clarify
that they met the criteria for inclusion into the surveyed population. These criteria
included 1) employment in a multinational pharmaceutical company or consultancy,
2) at least 2 years of employment in global regulatory affairs, and 3) participation in
at least one registration activity in one or more of the three countries under study.
Individuals meeting these criteria and willing to participate were mailed the survey
by internet unless they requested a paper survey. No remuneration was employed to
encourage participation. Follow up was carried out using two staged reminders when
a survey was not returned within a preset time frame, and a thank you message was
sent after each survey was returned. A report on the findings of the survey was
promised to participants when the study was completed.
Survey Analysis
Data provided from the survey was analyzed to establish the baseline
responses of the participants. Questions yielding continuous data were graphed and
analyzed statistically when appropriate, and questions with open-ended responses
were examined for their information content. The answers to the questions were
68
further scrutinized to determine if some questions appeared to yield little or no data.
Finally, we determined the extent to which a summary question, asked of the survey
participants, to rank the three countries overall in order of perceived transparency,
had responses that correlated with the ranking established by the accumulated scores
on the preceding, more detailed questions.
69
CHAPTER 4: RESULTS
Focus Group 1 Outputs
Discussions of the initial focus group considered the nature and key
indicators of transparency in a drug registration system in a largely unstructured way.
Nevertheless, responses could be clustered around three topics:
1. The value of a survey for the purpose of gauging transparency: The
comments of the group were positive about the potential value of an instrument to
measure transparency. Amongst the areas where participants saw particular value
was in using the outputs of such a survey to navigate the different registration
systems more easily and to make decisions about the time and resources that might
be needed to secure product registrations in countries with non-transparent systems.
This could allow better decision-making about which countries to enter first. It was
also seen as useful for regulatory agencies and governments interested in ways to
improve their registration processes as a “system owner”. A well-designed survey
might help the system owner to identify areas where improvements are most needed
to enhance transparency.
2. The extent to which transparency was a useful attribute of a regulatory
system: The group discussed the possibility that not all governments “may want to
have a more transparent process as some use an opaque process as a barrier to trade”.
The group shared their thinking on the potential boundaries of transparency and the
potential for problems if boundaries moved too far in either an open or opaque
direction. One member of the group commented that “a fully transparent system
70
could lead to loss of Intellectual Property rights while too little transparency could
impact patient safety and availability of new drugs reaching the market in a timely
manner”. Examples shared by the group of ‘too much transparency’ included an
anecdotal report of Canada’s stake holder meetings in the early 2000s that were
criticized for potentially sharing too much information; these meetings were
ultimately abandoned. On the other hand, participants seemed to reach consensus
that some countries showed great caution about increasing transparency, especially
in regard to clinical trial results, and Japan was used as an example.
3. The specific attributes associated with a transparent system: The
discussion focused on the attributes of a transparent system. When the moderators
evaluated this discussion, the key outputs from the discussion were clustered around
the “transparency concepts” of accessibility, clarity and accountability. Accessibility
was considered in some detail. It was defined by the group as the level of access and
ease of access to relevant components for drug registration submissions. The group
noted that different types of “vehicles” exist to provide access including the internet
and agency meetings and that each can be an aid to accessibility. However, the
group felt that no one vehicle alone is sufficient to provide adequate transparency for
the drug submission process. A second key concept identified by the group was
clarity as related to drug registration transparency. Clarity was considered a key
concept because a primary focus of a regulatory department is the ability to estimate
what is really required for successful submission of a drug on a schedule and at a
cost that is practical. A third key concept was accountability. The group
71
acknowledged the need for clear accountability of the regulators responsible for key
activities and decisions related to submission interactions and approval of drug
applications. They noted that accountability was related to corruption, but that the
absence of corruption was not enough to assure a transparent system. These outputs
of focus group 1 were used to guide the literature review presented in chapter 2.
Focus Group 2 Outputs
Focus group 2 was primarily interviewed to assist the investigator to design
an effective survey and deploy it in an effective manner. The 90 minute period of
the focus group was divided into three phases. In the first phase, that took about 30
minutes, broad questions regarding the overall survey structure and deployment were
discussed. In the second 45 minute phase, the strengths and weaknesses of
individual questions were considered. In the last 15 minute phase, there was again
general discussion and wrap-up, followed by informal conversations.
Phase 1: The overall survey approach
The focus of initial discussion was the potential difficulty of finding enough
qualified participants with experience in each of the three target countries to
complete the survey. Participants questioned how much experience might be needed
to assure a qualified survey participant. After discussion, a recommendation was
made to accept participants who had experience in just one of the target countries
and to add a question to the survey that identifies specifically the countries in which
the participant has drug registration experience at the time that they take the survey.
Additionally a snowball approach to building the survey participant pool was viewed
72
to be the most effective way to increase the numbers of interested and qualified
survey respondents.
The participants then raised the issue of “using a transparent country as
control against the three countries under study to provide a control point to help
establish a frame of reference on transparency.” Various countries for use as a
“control” were discussed including Japan, the EU and the United Sates. Japan was
viewed to have certain advantages because it was “more in the region than the
Western countries” but the group also recognized limitations because Japan is not
generally considered to have a highly transparent registration system. After further
discussion consensus was reached that the US might be the best choice as a control
because it was considered to have most transparent Western drug registration
process. Further it was noted that most US and even foreign regulatory managers
will have had experience with submissions to the US FDA. A commitment was
made by the moderator to include the US as a comparator on each country-specific
question in the survey.
The moderator was asked about other areas of potential consideration. The
value of including all medical products, including medical devices, or to include
more countries, was discussed. The group supported a narrowed approach to the
area of pharmaceutical registration as a first stage in this exercise to ensure a more
homogeneous sample. However the moderator expressed the hope that the survey
tool would be sufficiently generic to be used, with certain modifications, for studies
beyond the scope of the specific study under consideration, including, for example,
73
the medical device approval system, and that the scope of countries under study in
future could easily be expanded to make this analysis truly global.
The group also suggested that “filters be added to help select out the certain
respondents, for example number of years of experience, country experience, clinical
experience only versus the full spectrum of submissions expertise and therapeutic
areas of expertise”. After discussion the group reached consensus on the value of
adding the following filters: years of experience, company size, company portfolio
expertise and the participant’s geographic location. The moderator agreed to add
these questions to the survey.
Survey language was discussed and the value of distributing the survey in
multiple languages was considered in depth. There were pros and cons identified by
the group to an approach of using English as the preferred single language. The
group expressed the belief that distributing the survey in English was adequate to
meet the needs of this preliminary study but should be borne in mind when
interpreting the results.
Phase 2: Discussion of specific questions
The survey participants were led through a more structured, question by
question, discussion of the three specific transparency concept areas: Accessibility,
Clarity and Accountability. At this stage changes that would be made later to the
questionnaire were specifically identified.
74
Accessibility
The discussion on the Accessibility question-set was concentrated on simple
wording changes intended to make the questions more specific. The questions were
changed to reflect this input from the group. Discussion also focused on the value of
adding an additional question related to the presence or absence of a lead person or
project manager who serves as a primary point of contact for the registration process,
as is typical for the FDA process. The question added to the survey to include this
input is: “There is a lead person identified and assigned for each submission”.
Clarity
The group discussion on the Clarity question-set centered on
recommendations for wording changes or changes in the structure of two questions
to a Yes or No format from a1-6 Likert ranking format. A question regarding fees for
registration as well as other services, including bribes, was discussed at length; the
group suggested that the question on application fees be rewritten to include
reference to the full scope of the fees required for applications, as now shown in
Appendix, question 10.
Accountability
The discussion on the Accountability question-set suggested only minor
wording changes on three of the four questions; the fourth question was accepted as
written.
75
Phase 3: Final Discussion Points
The final discussion was an in-depth consideration of methods for weighting
the questions if the transparency level was to be “scored”. Most felt that a scoring
system in which each question was given an equal weight was acceptable for this
first deployment but as the survey is expanded in future the weighting should be
reassessed. All questions that remained in the survey after the focus group were felt
by the participants to add value to the instrument. However, participants
recommended that experience gained after the deployment of the survey would be
valuable and suggested that this experience should be used to eliminate some
questions and rework others.
Survey Results
The survey tool was modified according to the advice of the focus group.
The final survey, as shown in Appendix A, was then distributed to the key
participant list. Analysis of survey results showed differences in the relative
perceptions of transparency in different countries across all three dimensions of
accessibility, clarity and accountability. In all three dimensions, the US was
typically viewed by the respondents as most transparent, South Korea was judged to
be intermediate in transparency, and China and India were viewed to be least
transparent.
Accessibility
Accessibility in the context of this survey was considered to depend on ready
access to information and resources regarding the registration process. With regard
76
to access to information on the internet, polled individuals generally agreed that the
US has a comprehensive internet information system that is easy to use; in
comparison, most did not consider the internet systems of China, India and South
Korea to be easy to use. Less than 20% of respondents indicated that the internet
information system was comprehensive and easy to use for any of the identified
Asian countries (Figure 3).

Figure 3: The drug registration process has a publicly accessible comprehensive
Internet information system that is easy to use

Respondents also agreed that the US had an effective mechanism to discuss
questions and issues with the agency, but results were mixed for Asian countries.
For South Korea a relatively wide range of views were reflected by similar numbers
of respondents who agreed with the statement, there is an effective mechanism to
discuss questions related to submission requirements and issues directly with the
agency, compared to the number who disagreed with this statement. In contrast,
77
respondents typically expressed the view that China and India had a much less
effective mechanism to discuss questions and concerns with regulators (Figure 4).

Figure 4: There is an effective mechanism to discuss questions related to submission
requirements and issues directly with the agency

Respondents also typically agreed that in the US and Korea, a lead person was
identified, assigned and accessible for each submission. For China, experience was
mixed, and for India most respondents did not believe that there was a lead person
identified and accessible at the regulatory agency (Figure 5).
78

Figure 5: There is a lead person identified and assigned for each submission

Having access to a lead person who can support post submission activities is
critical to keeping the submission process moving forward in a timely manner. The
accessibility of the lead person assigned to a dossier was ranked highest in the US.
Of the Asian countries, South Korea was judged to be most accessible. India and
China were ranked lower than the USA and Korea (Figure 6).

Figure 6: The lead person responsible for my dossier is accessible after the
registration has been submitted
79
The ability to provide submissions electronically, viewed by respondents as
relatively easy in the US, was viewed to be more challenging in the Asian countries.
South Korea was viewed to be somewhat more accessible with regard to electronic
submissions than India and China. For China and India, only a few respondents
viewed drug applications as easy to submit electronically (Figure 7).

Figure 7: Drug applications are easy to submit electronically

Additional comments related to accessibility were provided by three
participants. One respondent commented that it was very important to have in-
country representation to support the submission process and that it can be difficult
to navigate submission processes from a non Asian location.
You need LOCAL staff on the ground in India and China, it's insane to try to
submit from the US or Europe. S Korea - local presence also helps. US - not
as easy as FDA might think, very bureaucratic and dysfunctional. For the last
question, submission in China and India is not via electronic submission. For
80
South Korea, they are looking into CTD format, but think not implemented
eCTD. Regulatory burden continues to be greater in China.

Another respondent also noted the need for in-country support. The respondent
spoke to the complexity and long process times required to gain drug approval in
China today, and provided information on the specific offices of the Indian
government responsible for drug and device registrations.
Drug registration in China is still complex and takes approximately two years
for new drugs. In addition, if a product is approved elsewhere, the SFDA will
most likely still require the foreign manufacturer to conduct at least some
clinical testing in China before the drug is approved in China. Central Drug
Standards Control Organization (CDSCO) is the Indian regulator of
pharmaceutical/drug registrations and some medical devices. CDSCO is also
commonly referred to as the Drug Controller General India (DCGI). All
pharmaceuticals imported into India must have prior approval from the
DCGI. There is no registration system designed for medical devices as a
class, but some medical devices now require product approval as "drugs"
before being marketed. Note: For certain questions I selected "Unable to
Judge" for China, India and S. Korea since product registrations in these
countries were contracted to regional regulatory consultants for direct drug
filings.

A final response below describes transparency concerns related to local language and
the lack of a dedicated lead person throughout the process. This individual
commented on electronic submissions for China and India and described the process
as still paper-based.
In China and South Korea, publicly accessible information is in local
language. In China, India and South Korea, there is no lead person identified
per submission. Each submission undergoes a process of screening, and upon
satisfactory screening, the relevant part of the dossiers are then handled to the
relevant divisions for their evaluation. Electronic submission is not available
for China and India; they are paper based submissions. Electronic submission
in Korea is allowed but the language of communication and format is in local
language and local format respectively, although documents can be in
English.
81
Clarity
The clarity of the regulatory process was explored by a number of questions
pertaining to procedural and logistical features of each regulatory system.
Respondents generally expressed reservations about the ease with which procedures
on drug registration could be understood in Asia compared to the US. Responses
regarding clarity reflected a view that South Korea was somewhat more transparent
than China and India.
For the US, respondents felt that procedures were easy to understand; all but
one agreed or strongly agreed with the statement, there are easy to understand
written procedures on how to register a drug, with a single individual indicating that
he/she neither agreed nor disagreed with the statement. For the other nations no
respondent strongly agreed that procedures were easy to understand. Nevertheless
there were differences in the pattern of responses. Slightly more than half of the
respondents felt that South Korea had drug registration procedures that were easy to
understand, while nearly an equal number felt that the procedures were not easy to
understand. For China and India only a few respondents felt that the procedures
were easy to understand (Figure 8).
Respondents presented with the statement, there is an easy to use standard
drug application format based on the ICH CTD format, generally agreed that this
feature was characteristic of the US system. However, there were mixed responses
with regard to South Korea. About one-third of respondents agreed that there is an
easy to use drug application format, while about one-half of respondents disagreed or
82
strongly disagreed with this statement. In contrast, for China, less than 10% of the
respondents agreed with the statement. Typically respondents also disagreed with the
statement with regard to India (Figure 9).

Figure 8: There are easy to understand written procedures on how to register a drug

Figure 9: There is an easy to use standard drug application format based on the ICH
CTD format
83
Most respondents also appeared to have the view that clear and well-defined
requirements existed for the manufacture of clinical and commercial drug products
for the US. For the Asian countries, most respondents expressed the view that
requirements for the manufacture of clinical and commercial drug products are not
clear and well-defined, but a minority of respondents had the opposite view (Figure
10).

Figure 10: There are clear and well defined requirements for the manufacture of
clinical and commercial drug products

However, on one attribute, respondents expressed more similar views for all
of the studied countries. Most respondents agreed with the statement, all fees
required for submission of the drug application are clearly defined for all of the
countries considered in this survey. The US scored highest and South Korea scored
marginally higher than China and India (Figure 11).
When asked about agreement with the statement, Fees for submission of the
drug application are standardized by product type and therapeutic area, most
84
respondents expressed agreement with respect to the US and South Korea, but
responses were mixed for China and India, and many respondents were unable to
judge this feature of transparency for China and India as is evident by the smaller
number of responses in the bins in Figure 12 and the raw data in Appendix D.

Figure 11: All fees required for submission of the drug application are clearly
defined

Figure 12: Fees for submission of the drug application are standardized by product
type and therapeutic area
85
A specific comment related to clarity was provided by one respondent, who
noted the prevalence of the ICH/CTD format.
The drug application format is unique to each country. However, dossier in
ICH CTD format is acceptable in China, India and South Korea.

Accountability
Questions on accountability probed views on whether respondents felt that
individuals at the regulatory agency under study had documented procedures for
decision-making that were followed and then open to challenge. With regard to the
statement, the review committee reaches its decision on a drug application via
documented procedures for decision making (e.g. a qualified majority or consensus),
respondents expressed strong agreement with respect to the US. In contrast, this
same question evoked mixed responses when Asian countries were considered.
South Korea was viewed as having the most transparent accountability by this
criterion, with only about 10% of respondents expressing disagreement. For China
and India, more respondents disagreed rather than agreed with this statement (Figure
13).
When asked whether the committee responsible for the drug application
process provides an official written report to the applicant on decisions (e.g.
accepted and rejected files) taken at key decision points of the application process,
respondents typically opined that this was a true statement with regard to the US and
South Korea . For China and India, the responses were mixed, but most respondents
86
also agreed that official written reports were provided at key decision points (Figure
14).

Figure 13: The review committee reaches its decision on a drug application via
documented procedures for decision making (e.g. a qualified majority or consensus)

Figure 14: The committee responsible for the drug application process provides an
official written report to the applicant on decisions (e.g. accepted and rejected files)
taken at key decision points of the application process
87
In contrast, when asked whether there is clear appeal process in place for
applicants who have their drug applications rejected, agreement was only typical
when the US was under consideration. Responses were mixed for South Korea, with
more respondents agreeing than disagreeing. More respondents disagreed rather than
agreed when considering this statement with respect to China and India (Figure 15).

Figure 15: There is a clear appeal process in place for applicants who have their
drug applications rejected

When asked whether the committee responsible for drug application
review is also responsible for the final decision making for approval/rejection of the
drug application, most agreed with this statement for the US and South Korea.
However, respondents showed a diversity of views with regard to China and India
(Figure 16).
88
Figure 16: The committee responsible for drug application review is also
responsible for the final decision making for approval/rejection of the drug
application

Additional narrative comments related to accountability were provided by a
few participants. One commented on the fact that review occurs in FDA across the
different levels within the agency.
In the USA, some applications are reviewed at higher level than the review
Division. The review is done by the lead reviewer and final decision on our
file is made by the lead reviewer and upper manager level. It is not a
committee.

Another comment suggested that even though the US registration process is public it
is not simple.
The registration process is clear and open to public but still requires
experiences in this area to understand and apply it smoothly. The Committee
responsible for drug application process provides an official written report to
the applicant at the end once a decision is taken.

A final comment noted that written feedback is provided on the application by the
reviewers enabling the applicant to respond to issues and concerns.
89
During the application process, questions regarding the dossier, raised by the
various evaluators, are provided to the applicant so that the applicant can
provide a reply to the questions raised.

A summary question asked respondents to rank the four countries according
to their overall perceived transparency. Respondents identified the US as the most
transparent of the countries under survey. Of the Asian countries, South Korea
ranked highest in its perceived transparency. China had 9 scores as least transparent
and 8 as less transparent. India was identified as the least transparent country with 8
scoring it as less transparent and 9 scoring it as least transparent (Figure 17).

Figure 17: Based on your professional experience please rank the four countries
below on an individual basis from an overall transparency perspective from most to
least transparent

Analysis of the responses to questions to construct a transparency score was
carried out both dimension-by-dimension and as a cumulative score on all of the
multiple-choice and yes/no questions as shown in Table 1. Cumulative scores and
90
dimension- specific scores showed the US as most transparent, scoring 1.8 on a five-
point scale in which 1 was most transparent and 5 was least transparent. Korea
scored 2.8 overall and India and China scored 3.3 and 3.5 respectively. These same
rankings held when comparisons were made for specific dimensions of accessibility,
clarity and accountability.

Table 1: Transparency scores by dimension
Dimension
China
India
South Korea
US

Accessibility
3.6
3.5
3
1.8

Clarity
3.3
3
2.8
1.7

Accountability
3.5
3.2
2.7
1.9

Overall rank
3.5
3.3
2.8
1.8

Profiles of Respondents
The majority of respondents were highly experienced regulatory
professionals. About half of the respondents (48%) had 10 or more years of
regulatory experience, and a further 19% had 6 to 10 years of experience. Most
respondents (65%) were from companies of 500 or more employees. The large
majority (83%) of the respondents was employed by pharmaceutical and biologics
companies, and about 70% were located in the US. Respondents from outside the US
were located in Europe (4), China (3), South Korea (2), Singapore (2) and Canada
(1).
91
A preliminary examination was conducted to evaluate if the profiles of the
respondents affected their views on transparency. Two examples of data sets
subjected to cross tabulation analyses are shown in Appendix E. Because the
numbers are relatively small the dataset is often skewed, it is difficult to make
inferences about the relationships between respondent profile and answers to specific
questions. Two examples of relationship that seemed most apparent are presented
below. For the statement, There is an effective mechanism to discuss questions
related to submissions requirements and issues for South Korea, the modal value of
responses for individuals with 10 years of more of experience was agree whereas
that for individuals with 5 years of experience or less was strongly disagree.
However, the approximated P value for this correlation using a Chi Square test was
still only 0.13 with only 16 degrees of freedom. A second example to explore is
whether differences between large companies and small companies exist with regard
to the statement there are clear and well defined requirements for the manufacture of
clinical and commercial drug products, in South Korea. The modal value of
responses for individuals from companies of 500 or more people was agree whereas
that for individuals from small companies of less than 50 people was split between
agree and disagree. Nevertheless results were not statistically significant; the P value
using a Chi Square test was 0.23 with 8 degrees of freedom. For most other
questions the profile of respondents did not seem to vary in any highly correlated
way with the responses to various questions.
92
CHAPTER 5: DISCUSSION
Regulatory transparency is believed to be an important attribute toward
which governments and their associated agencies strive. In this study we attempted
to examine the views of regulatory professionals with regard to the transparency of
one aspect of this complex system, transparency of the new drug approval process.
The results showed differences in the rankings of three countries in Asia with respect
to the US and to a lesser degree with respect to each other, at least from the point of
view of the selected sample of regulatory experts located principally in the US. In
order to facilitate this analysis, we were also faced with the challenge of developing
a tool with which we could gauge drug approval transparency. The study offered a
first opportunity to use this survey tool and to attempt an early-stage validation of its
usefulness by comparing cumulative scores on the responses to the survey questions
as a group to an overall ranking of transparency from the same respondents.
Differences in Transparency across Countries
The ability to understand the drug submission process in any constituency has
an important impact on the timelines and costs of introducing a product into that
constituency, and on the ease with which a regulatory agency responsible for product
review will be able to work with those applicants. Increased transparency of the
submissions process thus can have advantages to both companies and regulatory
agencies as they negotiate to gain marketing approval. The present study suggests
that the Asian systems under study still have some way to go before their drug
submission procedures are considered to be transparent. Further, it suggests that
93
differences exist between Asian countries. Why these differences exist is to some
extent illustrated by the specific answers to the questions posed in the survey.
Survey results indicated that the country used as a “control”, the USA, was
most transparent in all of the questions of the survey. The fact that the US scored so
well is perhaps not surprising. The US was picked as a benchmark both on the
advice of the focus group of experts advising on the survey, and from a review of the
literature in which the US is often used as an example of the most transparent of
regulatory agencies, at least from the point of view of the drug submission process
(Frank 2003). The strong scores on this survey might suggest the US has little more
to do to assure a transparent system. However, this perception may be a limitation
introduced by the nature of the questions posed in the survey. A more critical
analysis might be obtained if other questions or other methods of gauging responses
were to be used. This was a concern that was raised by one quoted respondent who
strongly stated the belief that the US was not such a transparent system, even though
the survey responses suggested that the US would get high marks for transparency.
Furthermore, the US FDA itself seems to acknowledge that further work on
transparency is needed as demonstrated by the new Obama transparency task force
initiatives (Obama 2009).
To some extent, it is reasonable to expect that a relatively short and high-
level survey such as the one used here might not be able to capture a full picture of
transparency in a range of countries where practices range from very transparent to
very opaque. Such a tool may be unable to provide fine degrees of transparency
94
characterization for countries with the highest high levels of transparency or the
lowest levels of transparency. This challenge is akin the problems of range and
linearity in an analytical testing procedure, where the analytic methodology is most
effective for the midrange of values for which the method is intended but becomes
less useful when very high levels or only trace levels of a constituent are present.
Nonetheless, the results of the survey clearly supported the belief initially expressed
by the focus group that the level of transparency in the US was high and that it would
be an appropriate benchmark against which to compare other regulatory systems.
South Korea occupied a midrange position in overall ranking based both on
cumulative scores to survey questions and on the final survey question that solicited
a judgment about overall ranking of the four surveyed countries. The finding that
South Korea was considered more transparent than the other two Asian countries in a
number of attributes, including the effectiveness of its communications with end-
users and the clarity of procedural and decision-making activities, may reflect the
relative maturity of the South Korean system with respect to those of China and
India. The South Korean Food and Drug Administration has a longer history as a
fully developed regulatory entity and is continuing to evolve and reorganize as was
discussed in the earlier chapters. Further, much recent activity in the South Korean
FDA has been directed at improving the transparency of the registration process. As
part of this process, the agency has prepared detailed manuals on Korean drug
registration standards and the agency also plans to introduce a new Common
Technical Dossier (CTD) standard in the near future (AsiaNews 2009). Outside of
95
the narrow scope of the drug approval process, several more general policies and
initiatives can be recognized. These include: the formation of a “Corruption
Reporting Center” under the Anti-Corruption Council that is a 24 hour hotline for
reports of corruption; the centralization of regulations and enforcement under a
single ministry in some key areas of focus, such as the environment; simplifying
language to increase the ability of the public to understand policies and practices;
and emphasizing the use of international standards (Kim, Kim & Yang 2002).
The results for China and India were mixed but generally suggested that
transparency initiatives still had some way to go. However, by using a survey in
which questions addressed different key attributes of transparency, it was possible to
identify areas of strength and weakness. For example, one area of strength appeared
to be in the area of clarity with respect to fees associated with the submission process
for both China and India. The stronger scores on this attribute may be surprising and
encouraging, since, only a few years ago, a major concern about both of these
regulatory systems was the need for additional “non-authorized” and unpublished
payments as bribes to regulatory officials (Shengland 2010). However, in 2007, the
Chinese regulatory system underwent a major crackdown in which the head of the
State Food and Drug Administration was executed for taking bribes totaling more
than $850,000 USD to approve a drug that later caused at least 10 deaths (China
Daily 2007) . A number of other measures were then taken to reduce the likelihood
that corruption could damage the integrity of the drug approval process. These
measures included the establishment of a “drug watchdog agency” in China tasked
96
with hunting down corrupt practices and bring the perpetrators to justice. That these
measures were effective and taken seriously is reflected in the 2010 arrests for taking
bribes of five officials employed in the drug registration and certification
departments of the National Institute for the Control of Pharmaceutical and
Biological Products (Shenglan 2010). Thus the scores on questions related to fees
could be interpreted as consistent with more recent reports that the SFDA “has
started to realize it can no longer be non-transparent in its activities” (Hulihan, as
quoted by Bortiz, 2008), and that the “extra” payments over and above published
fees is no longer to be tolerated.
In India, similar attention to reducing corruption in the regulatory agency has
been in evidence. In early 2000s corruption was considered to be a significant
problem, and payments for services of regulators to facilitate drug approvals posed a
problem. For example, investigation by the Karnataka Lokayukta, a judiciary with
authority over governmental administrative corruption, identified that companies
were bribing officials.
It was discovered that companies that paid bribes were allowed to circumvent
drugs standards and those that refused to pay were harassed…It was claimed
at the meeting that each drugs inspector was required to give around Rs20,
000 (around $460) every six months to the Drugs controller, who then passed
it on to the Minister of Health (Cameron 2006).

Since that time, India has been expending considerable effort to modernize its
regulatory systems and attitudes.
From an era of bureaucratic hurdles, delayed approval times and lenient
attitude towards misconducts, it is rapidly moving to an era of fast-track
approvals, active inspection of sites and mandatory registration of Indian
97
trials in national trial registry in an attempt to meet global regulatory
expectations of stringency and transparency (Bhatt & Brahmachari 2010).

However, we cannot dismiss the possibility that the strong scores reflect the fact that
the question itself was not sufficiently specific to address bribery and corruption
explicitly, and that the respondents did not think that the payment of bribes would be
included in the question. This might be an area in which subsequent surveys could be
clarified and improved.
The results of the survey also identified several areas of weakness in the
transparency of Chinese and Indian Regulatory approval processes. For example,
India and China typically had low scores on questions relating to the appeals process
available to applicants after a registration submission had been rejected. In part
these lower scores may reflect the fact that the regulatory systems of China and India
are still undergoing significant transition as the countries take on a more prominent
role in global economic affairs. In China, drug registration procedures are still often
described as being early in development, complex and lacking in transparency (Jyton
2010). For many years China was concerned with the internal development of
regulatory systems and had a major focus on Traditional Chinese Medicines. These
today still represent the majority form of available therapeutic products on the
Chinese market (OECD China 2007). A strong orientation toward ensuring drug
quality, a typical goal of more mature western markets, has been slower to develop
in China but steady progress is being made. SFDA has now adopted a more
harmonized framework for their approach to regulatory science but they still have
98
much to do if they are to gain global credibility as a compliant and transparent source
of drugs for the global marketplace. As systems become more established, the
appeals process and basis for decision-making may become more explicitly defined
and practiced.
India, like China, is in a relatively early phase of development. Its regulatory
systems have just started to focus on the requirements of GMPs for their 60,000 plus
API manufacturing sites. India has also been much in the news for problems with
GCP compliance (Edwards 2008). However, new drug approval systems for
traditional and fast-tracked products have been introduced, methods for exporting
samples have been simplified and pre-review processes have been tightened (Bhatt &
Brahmachari 2010). It will be interesting to see whether these initiatives translate
into a clearer and more accessible drug registration system particularly in the areas of
accountability.
The relative ranking of the three countries may also reflect the resources
available to the respective regulatory agencies to do their jobs. Transparency
initiatives are often expensive and require a large amount of infrastructure. For
example the three-phase Obama transparency task force will utilize the time and
energy of potentially thousands of FDA staff to manage the internet resources, face
to face conferences and reporting requirements that were outlined by the president
(Obama 2009). In comparison, Asian countries have a much lower level of support
available for bureaucratic activities. For example, in South Korea, the entire project
review team for the therapeutic proteins division in the KFDA office of biologics is
99
one person, but in the US FDA this same office has more than 100 people. It seems
reasonable to anticipate that the scale of resources available to support transparency
activities and task forces will be much lower for Pharmerging nations than for the
US and the EU.
Three Dimensions of Transparency
In the present study, we used a triad of dimensions, accessibility, clarity and
accountability, to characterize regulatory transparency. These three dimensions
seemed to incorporate most of the features identified in the literature as typical
measures of transparency, as outlined in chapter 2. Whether other dimensions exist
and would add more insight to the estimation of transparency is difficult at present to
judge. The dimensions seem to parallel the areas defined by the OECD , though the
dimensions are defined in different terms. The OECD report of 2001 states that
regulatory transparency is “the capacity of regulated entities to express views on,
identify, and understand their obligations under the rule of law”. One aspect that is
identified by this definition and is not well covered by this survey is the ability of the
individual or company to “express views on” the law. However, this aspect deals
primarily with the development of regulation and in this research; we have focused
primarily on the way in which existing law is managed and communicated. Thus,
for the purposes of the present analysis, the three concepts of accessibility, clarity
and accountability appeared to be useful in dissecting out and comparing some of the
elements that make this part of the regulatory system transparent.
100
It was interesting to find that the evaluations provided by respondents on the
three different dimensions of accessibility, clarity and accountability seemed to
reinforce one another. The results presented here are preliminary given the relatively
small sample size in this early-stage investigation. Nevertheless, scores suggesting
that the US was most transparent and that South Korea was intermediate in
transparency compared to India and China were congruent across all three
dimensions, and were similar to the overall rankings of transparency. These results
suggest good construct validity, where construct validity is defined as “the degree to
which a measure relates to other variables as expected within a system of theoretical
relationships” (Babbie 2007).
An interesting divergence between countries was whether the review
committee reaches its decision on a drug application via documented procedures for
decision making (e.g. a qualified majority or consensus). Perhaps this is due to the
complexity of the decision-making process in each region, so that a respondent
familiar with one or two countries could not recognize that the decision-making
process was equally transparent but just different. On the other hand, the decision-
making process may in fact be less transparent than other dimensions of the study.
Drug submissions are very complex, and decisions are often taken by different
groups of individuals across multiple levels of review, in part to spread the risk of
making a bad decision. Because bad decisions can result in negative consequences
for patient health, no one department may want to be responsible for the potentially
controversial result. Transparency in this area may therefore be difficult to achieve.
101
Considerations of the Methods
An important question when using any new research tool is whether or not
that method can capture the essence of the issue under study systematically and
effectively. Survey methods are often characterized as having both strengths and
weaknesses (Trochim 2006). The strengths compared to qualitative methods include
the ability to reach more than a small number of respondents, the ability of
respondents to answer questions anonymously, and the ability to standardize and
even quantitate the data that is derived. The potential weaknesses are related at least
in part to whether the questions composing the survey are appropriate, whether
methods used to develop a respondent pool are adequate, whether honest and
sufficient responses can be assured, and whether data from the chosen sample can be
generalized across other constituencies. The degree to which these concerns may
limit the validity of the data in this study is important to consider (Trochim 2006).
1) Appropriateness of the approach
In the present study, a relatively small number of questions were chosen to
compose the survey. Such limits on survey length were felt to be important to ensure
that the survey was completed by the targeted respondents, but does restrict the
variety of questions that can be asked in order to probe the views of the respondents
in any detail. This approach puts more emphasis on selecting appropriate questions.
The use of two separate focus groups of experienced regulatory professionals to
provide review and input prior to the launch of the survey appeared to be an
excellent way to enhance confidence in the face validity of the survey and to reduce
102
the risk of bias due to the preconceived notions of the survey designer. There is not
much evidence that such an approach is commonly used elsewhere. Typically in
mixed method research, the order of this exercise is reversed; there, surveys are often
used to collect large amounts of information from a general group of respondents,
and then followed further exploration of the data by stratified focus groups (Patton
2002; Hess-Biber & Leavty 2010).
As part of the effort to develop metrics, questions in this survey were scored
and summed to provide a “grade” relating to the maturity of regulatory transparency.
Such a grading system must be approached with caution because it is unclear
whether all features assessed by individual questions are equally important and
should be given equal weighting (Trochim 2006). Further it is not clear whether the
approach to scoring the yes/no questions by giving the yes answer a score of 2 and
the no answer a score of 4 is the best method to incorporate such questions into the
mix of multiple choice questions with scores between 1 and 5. However, it was
encouraging to note that the scores on different dimensions of the survey were
congruent with each other and with the overall ranking of transparency. By itself,
consonance between the different dimensions of clarity, accessibility and
accountability is not necessarily required for the survey to be valid. For example, it
is possible to imagine a scenario in which a regulatory system has a clear exposition
of the drug approval process, and thus relatively high clarity, but a closeted and
unpredictable decision-making process, and thus a relatively low score on
accountability. Then the two dimensions would not be consonant with one another.
103
Nevertheless, it seems from the current data that strength or weakness in one aspect
of transparency to a large extent predicts strength and weakness in the other two
dimensions.
In this study the survey was delimited to a relatively narrow topic. Instead of
developing a survey that dealt with all aspects of activity within the purview of
medical product regulators, I chose to study only the views of the respondents with
respect to new drug approvals. However, even with this narrowed focus, the study
still addressed a rather broad scope of transparency-related activities, from electronic
and staff communications, to decision-making and procedural clarity. If the survey
had been extended to include regulatory rulemaking, commercial and clinical
activities it may have become too unfocussed and potentially unmanageable. What
this means is that other interesting topics related to transparency, such as the
transparency of inspectional processes associated with manufacturing and clinical
trial facilities, were not included in the scope of this first work. There is a great deal
of activity and controversy around the transparency of manufacturing (GMP) and
clinical (GCP) audits that deserve an independent study of transparency in the future
(Bhatt & Brahmachari 2010).
It is difficult to weigh the importance of the various questions or even
dimensions with respect to transparency. We do not know, for example, whether it is
more important to have an easily accessible and navigable website, or ready
accessibility to the project manager or reviewers for the regulatory dossier. Thus
care should be taken not to see this survey as an index of the relative importance of
104
the elements but rather as a relative ranking of the countries with respect to a
collection of benchmarks with arbitrary weightings. Further, transparency is only
one aspect of the success of regulatory filings. A system can be transparent and yet
difficult and expensive to navigate. For example, the US system is typically judged
to be quite transparent, yet time delays in approval, extended PDUFA dates or
missed PDUFA dates are becoming the norm and signal a less-than-optimal
operational system (Hay 2009). All too often, and much to the surprise of the
regulatory team who has numerous face-to-face meetings, teleconferences, and
written correspondences, the end of the submission process can result in a delay or
denial of marketing approval. Thus the decision itself can be opaque even in the most
transparent of regulatory systems (Carpenter 2004).
2) Sampling of respondents
The stated goal of this study was to evaluate transparency differences through
the eyes of one group of mid-level to higher level regulatory professionals in
pharmaceutical companies and consultancy that have had at least two years of
experience in product registration in the Pharmerging markets under study. It was
recognized at the outset that research might be limited by the challenge of finding
regulatory professionals who have worked in international regulatory affairs with a
multicounty Asian focus and enough depth of experience to complete the survey. In
the present early-stage study, a snowball technique was used to identify specialists to
participate in the focus groups and to complete the survey. This is not the most
unbiased method to gather input and final data for analysis, but was perhaps the only
105
way that sufficient expertise could be secured, given the specialized nature of the
survey and the fact that few regulatory professionals might respond to a survey
unless approached personally to ensure their interest and support. The 64 individuals
who met these criteria might actually represent a large part of the pool of individuals
with such experience, but the data on numbers of regulatory professionals with this
type of experience has never been identified. The challenge of finding qualified
respondents is underlined by the fact that, despite the careful selection of individuals
with Asian experience, a significant proportion of respondents answered “unable to
judge” to many of the questions, particularly those with respect to India. These
results may reflect the fact that the pharmaceutical markets of Pharmerging countries
have only recently been seen to be large enough to warrant registration of the drugs
for sale, given the challenges of the regulatory processes (Hou 2009). Thus,
relatively few regulatory specialists in US companies appear to have extensive
experience in the newer markets of China and India. At the same time, the concern
that individuals might try to answer questions for which they did not have experience
was to some degree allayed by the large number of responses in the unable to judge
category. The results also seem to support the view that the participants were honest
in their responses, as also reflected by the frank comments provided in the narrative
section of the survey and in the focus group setting.
3) Respondent response rate and composition
A major concern that is often recognized in literature on survey limitations is
the relatively low response rate of individuals to surveys, particularly those that are
106
administered electronically. For example a recent survey by the journal,
Pharmaceutical Technology, was sent to 34,247 people and resulted in 353
responses, which equates to an approximately 1% response rate (Hoffman 2010).
Such a low response rate is below the stated expectations for response rates in
textbooks, that have often been identified to vary between 20 and 30% (Kaplowitz,
Haddock & Levine 2004) but is more in line with the results of surveys in the
regulatory sector, where response rates of less than 10% are often typical (FJR
Richmond, personal communication). Thus, efforts made in this study to directly
contact the targeted respondents and interest them in the topic of the survey seem to
have been effective in increasing the response rate, and thus hopefully the
representativeness of the sample. The facts that 44 of 64 surveys were returned,
resulting in a 70% response rate, and 40 completed the survey to its end, was a
higher response rate than is generally considered acceptable for survey methods of
this kind, especially for the busy professionals in this field.
Given the high response rate, we have little reason to believe that bias was
introduced by respondents with particular views. However, it remains to be seen
whether the results are externally valid, particularly if the sample is extended to
regulatory professionals in Asian companies. It would be interesting, for example, to
identify whether Chinese regulatory professionals view China as more transparent
because of their greater familiarity with the Chinese agencies. At the outset of this
study, we were concerned that the choice of US-based regulatory professionals as the
primary respondents might affect the results. Even from the comments it is clear that
107
much of the regulatory filing work is carried out locally in the targeted country by
staff members whose home is located in that country, and they may have different
views about transparency. In future studies the opportunity to have a sample
population all based in the countries under study may provide and interesting
comparator to the results of first study undertaken in this thesis.
The decision to delimit the focus of this thesis to only a few of the primary
Asian markets also had implications for the composition and possibly even the
successful response rate of the survey. Typically, no matter how experienced a
regulatory professional, there is a low likelihood that the professional will have had
experience in many developing countries. The more countries that are represented in
the survey and the smaller and less developed those countries are as markets for drug
products, the more difficult it will be to obtain survey respondents with the requisite
experience to comment usefully on transparency questions. In order to answer
questions about regulatory transparency, it is important not to have only a passing
acquaintance with the regulatory system under study but to have interacted with the
system sufficiently to test its usability. A focus on only a few of the Asian markets
that are considered to be emerging market economies was believed to be helpful in
assuring that respondents had enough experience to respond to most questions. Even
then, however, the relatively large numbers of “cannot judge” responses with regard
to the regulatory system of India in particular raises flags about the
representativeness of the data presented here for that country.
108
Because respondent numbers are relatively small and data were often not
distributed normally, a detailed analysis of the responses of different subgroups of
respondents is probably of limited value. Further the answers that were received to
many of the questions appeared not to be normally distributed so that use of non-
parametric statistics such as Chi-square tests would be most appropriate. Such tests
are not easy to use for small datasets. However, some observations provide
interesting insights that might lead to more detailed analysis or specific exploration
by later studies. For example, it appeared from cross-tabulation that increasing
regulatory experience of the participant may be correlated with increasing perception
of regulatory transparency. This might be expected if greater experience translates
into easier navigation of the complex regulatory system, and an overall increase in
the level of comfort with that system. The results presented here are a first step in
understanding the relative rankings of countries with regard to regulatory
transparency, but they leave unanswered questions about what other elements might
affect the views of respondents that might limit the external validity of the findings
beyond the particular group of survey recipients under study. A significant question
that this survey cannot answer is whether the language capabilities of the respondent
affect assessments of transparency in countries where a different language is spoken
and often used as the official language on the website. The comments of the survey
respondents did not speak to this question specifically. However, comments that in-
country staff or consultants were needed for success in the Asian countries might be
in part due to the need to work in the language of the country to which submissions
109
are made. Is this need to have local regulatory professionals in country related in
some measureable way to language issues that are best left to the local regulatory
professionals to navigate? It would be interesting to know if regulatory professionals
skilled in Chinese found the Chinese system more transparent, for example.
Additionally the remarks of focus groups suggesting a survey in multiple languages
might imply individuals with different language capabilities would perceive more
transparency in the country where their language skills fit. Perhaps the impact of
language capabilities could be a key area for investigation for a future study, limited
to regulatory professionals located in the target Asian countries. And if language is
an important but as yet unstudied aspect of transparency is familiarity with local
customs and regulations another aspect to transparency? Kim and colleagues (2003)
have stated with regard to transparency in South Korea that:
Typically, foreigners have difficulties in understanding the culture and
precedents in the domestic economy, and thus they will have the most
difficulty in finding information or making their opinions felt by regulatory
agency.

These are interesting questions that must be taken up by surveys or other research
methods with a different structure and focus.
Does experience play a role in perceived transparency? The data in this
survey might hint that perception of transparency increases with experience. This
may be due to learning the systems by trial and error until what was once complex
and opaque now seems more transparent. It could also suggest that experience allows
the individual to combine many other resources to bring to bear on the challenge,
110
that perhaps are not even recognized explicitly, such as the gradual development of
personal networks and acquaintance with local experts. Additionally larger
companies scored transparency somewhat more positively than small companies.
This may be due to many factors. However, it is safe to assume that the larger
companies have greater resources and often have dedicated international experts who
provide institutional knowledge to their peers on an as-needed basis. Such additional
sources to assist the regulatory professional may be confounding variables that could
make a regulatory system seem more transparent because it seems easier to navigate
than would be true without such additional help.
What else could be learned if the survey were expanded to other countries
also perceived as challenging regulatory environments by experienced professionals?
Possibly countries such as China and India when compared to Thailand or Samoa
may seem more mature and transparent than they do when compared to South Korea
and the US. The ceiling effect that was suggested to characterize this thesis with
regard to US scores may be accompanied by a floor effect, which would limit the
discrimination of transparency rankings at the bottom of the scale as well. Of
interest also would be the effect of adding a very opaque country to the mix of
surveyed countries. One might expect that this would not shift relative rankings but
might shift absolute scores of countries with intermediate rankings upward, if a
poorly transparent country rather than a highly transparent country were to be used
as a benchmark.
111
In retrospect, was it a good decision to use a survey as opposed to other
qualitative tools for this work? The ability to poll a larger number of individuals and
deal quantitatively with the data was considered to be very helpful in understanding
the perceptions of a key group of stakeholders in an anonymous and standardized
way. This advantage is particularly important when dealing with regulatory
processes that are often done only infrequently so that a bad singular experience can
greatly affect the views of an individual respondent. For example at the outset of
this research, my personal experience with the submission of regulatory dossiers in
South Korea led me to anticipate that South Korea would be seen to be less
transparent than the other countries and reinforced the danger of basing conclusions
on only a few inputs. This can also be seen in the individual responses to the
questions of transparency regarding South Korea where there was often a diversity of
views. If the study relied on interviews, it might have been difficult to avoid over
interpretation of the responses of only a few individuals or of individuals who were
particularly outspoken or articulate.
Nevertheless, surveys cannot provide everything that one might like to know
about the perceptions of the polled individuals. For example they do not provide in-
depth information about elements or processes specific to the dimensions that were
queried. More in-depth study might allow a greater understanding of particular hot-
button issues whose improvement could potentially make a large difference in the
transparency of that dimension. For example, it might be seen that a website was
considered nontransparent because its links were commonly broken or it was not
112
clear where to find information, but that the information that was provided was in
fact quite good. Thus the fix for this problem might be more circumscribed and less
costly than would be possible to gauge from the survey here. Follow-up interviews
to this survey would be very useful in finding specific areas where remedial
measures would greatly improve transparency.
Conclusions and Future Directions
The output of the focus groups and the subsequent survey results derived
from the survey tool used in this study appear to support the proposition that the
index developed here is a useful vehicle with which to study regulatory transparency.
Although further study is needed to validate this approach, the survey provides a
first-of-its-kind index of transparency based upon the three key aspects of
transparency, clarity, accessibility and accountability. This first deployment of the
survey tool is only a beginning. It is hoped that this transparency tool is sufficiently
generic that it can be used across any region to characterize drug approval
transparency. This type of standardized instrument is important to enhance
communication by providing a method to assess the relative transparency of
regulatory systems across countries and time periods. Such systematic comparisons
are very difficult to obtain from anecdotal and qualitative approaches.
The present results provide an interesting picture of the state of transparency
in three Pharmerging markets, South Korea, China and India, as seen through the
eyes of regulatory professionals primarily located in the US and interacting with
those regulatory systems to obtain new pharmaceutical approvals. Demonstrable and
113
measurable differences in the level of transparency seem indicated both by the
overall rankings of countries assigned by the respondents and by the transparency
index scores. This snapshot provides a benchmark against which changes in
transparency can be identified over time, as these important markets mature. From
this basic characterization, it will be possible to see whether one or more of these
countries change markedly in the level of transparency as they embrace global
commerce or whether less transparency is seen as a way of encouraging protectionist
trade policies. Thus it might not be surprising to find that some countries elect to
enhance transparency in some dimensions that they feel would be benefit their
particular circumstances without making other changes that might threaten certain
aspects of a desired objectives such as indirect tariffs to trade based on opacity. As
stated by Tilton (1997), “various regulatory barriers as well as unofficial regulations
from industry groups hinder market access for imports, thus raising their price level”
(Tilton 1997).
In many situations, a correlation has been assumed between corruption and a
lack of transparency. It is true that the ranking of countries seen in this research
seems to parallel their rankings in another well-known survey to study corruption.
However, it is important to emphasize that transparency is not related directly to
corruption, but rather that the opportunity for corruption may be enhanced in poorly
transparent systems. No evidence was obtained in either the survey results or the
anecdotal comments reported here that respondents had been faced with clear
114
situations of corruption, although it was not an element that was explored directly in
this work.
The results presented here assign transparency “grades” to the various
countries under study. There still remain questions about what these transparency
scores really mean. Does a lower transparency score mean that the regulatory system
is easy to use? If so, you might conclude that obtaining drug registrations in the US
would be an easy and risk-free task. If you ask this question to regulatory
professionals working with the US process the answer would be a strong no! The
new transparency initiatives that have been launched as a second stage process by the
US FDA after a decade or more of improvements to websites and other vehicles of
communication seem directed at the reporting of regulatory decisions or policies and
the provision of other information surrounding specific approvals or inspections
rather than the improvement of transparency with respect to the processes
themselves. This type of transparency has modest implications in the accessibility
and accountability dimensions, but would not probably change much the rank or
cumulative scores of the US in this survey.
It seems important to differentiate transparency from ease of use or
efficiency. It is possible for different countries to have different review systems that
in one country may be complicated with many requirements but are clear to
understand, and in another has fewer specific requirements but is opaque in its
application. This study would propose instead that indicators for transparency are
reflected not by ease of use but by accessibility, clarity in its procedural requirements
115
and accountability from the point of view of the decisions made by the organization
under study.
One aspect that also will become clear over time is whether individual
countries have an appetite to improve identified transparency weaknesses. Hopefully
the results of this study can stimulate further positive dialogue on this subject with
key health authorities, so that they can recognize areas of potential strength and
weakness. This may allow more informed decisions about where to concentrate
valuable resources to obtain the best return if transparency is of concern.
When companies launch global registration campaigns for new drug
registration in many countries, it is important to understand which countries will be
immediate and later targets for entry. This decision depends on many features of the
target country. One factor that should be considered as part of the launch strategy is
the transparency of the regulatory system. In a transparent system the odds are
increased that no unanticipated surprises will set back or complicate the registration
process. Nevertheless such decisions rely on a multifaceted framework of
information that goes well beyond considerations of regulatory transparency; they
will necessarily include many other factors, such as the size of the market , the
pricing of the product and the costs of doing business in that constituency. In terms
of priority setting, the application of a transparency tool can help to highlight
countries that might be sensible targets for early product registration when resources
are limited and time-to-approval is important. Also it will help regulatory
professionals to communicate and manage expectations when setting milestones for
116
global market development by anticipating potential delays and approval challenges
on a country-specific basis. Until now there has been no tool available to aid in
making these types of decisions. Corporations have had to rely heavily on in-
country specialists and consultants to predict these potential pitfalls in the
registration process. Having a clear arms-length ranking of transparency in different
countries relative to one another will provide great benefit when planning complex
serial product launches.
117
BIBLIOGRAPHY
Anson, S. (2006). Health Care in Rural China: Lessons from HeBei Province. China
Review International, 13. DOI: 10.1353/cri.2007.0054

AsiaNews. (2009) Korea Plans to Speed Drug Registration. Asia Medical
eNewsletter. Retrieved from
http://www.pacificbridgemedical.com/newsletter/article.php?id=400

AsiaNewsKFDA. (2009) Korea to Reorganize KFDA. Asai Medical News eLetter.
Retrieved from
http://www.pacificbridgemedical.com/newsletter/article.php?id=401

Babbie, E. (2007). The Practice of Social Research. Belmont Ca: Thomson.

Bale, A. (2003). Background and History of the ICH. Retrieved from
http://stats.ich.org/5_Closing_Plenary/Harvey_Bale_files/
frame.htm#slide0001.htm

Bennis, G., O'Toole (2008). Transparency: How Leaders Create a Culture of
Transparency. San Francisco, Ca: Jossey-Bass.

Bhatt, A. & Brahmachari, B. (2010). Changing Clinical Research of Regulatory
Scenarios in India. Pharmabiz. Retrieved from
http://www.pharmabiz.com/article/detnews.asp?
articleid=53094§ionid=50

Bloom, K. (2007). The Urban Revolution. Finance and Development, 44. Retrieved
from http://www.imf.org/external/pubs/ft/fandd/2007/09/bloom.htm

Bosher, A. (1988). The French Revolution. New York, NY: W.W. Norton.

Broscheid, A. (2005). Towards a European FDA? The Review of European
Pharmaceuticals Authorization. Max Planck Institute for the Study of
Societies. Retrieved from
http://aei.pitt.edu/3181/01/BroscheidFeick_EUSA05.pdf

Business Insight (2008). Drug Approval Trends at the FDA and EMEA. Retrieved
from http://www.globalbusinessinsights.com/content/rbhc0200p.htm

Cameron, S. (2006). Global Corruption Report 2006. Ann Arbor, Michigan: Pluto
Press.

118
Carpenter, D. (2004). The Political Economy of FDA Drug Review: Processing,
Politics and Lessons for Policy. Health Affairs, DOI: 10.1377/hlthaff.23.1.52

Chalmers, A. (2000). International Pharmaceutical Registration.Washington, DC:
Interpharm/CRC.

China Daily (2007). Former SFDA Chief Executed for Corruption.China Daily.
Retrieved from http://www.chinadaily.com.cn/china/
2007-07/10/content_5424937.htm

Curtin, D. (2006). Does Transparency Strengthen Legitimacy? Information Policy,
11. Retrieved from http://portal.acm.org/citation.cfm?id=1412570#abstract

DCGI (1997). Drug Importation Requirements. Retrieved from
http://cdsco.nic.in/html/importdrugs.htm

Dhalla, L. (2008). Moving from Opacity to Transparency in Pharmaceutical Policy.
CMAJ, 178. DOI: 10.1503/cmaj.070799.

Dosanjh, U. (2004). Canadian Medical Association's Annual General Meeting.
Toronto. CMAJ, 171. DOI: 10.1503/cmaj.1041402

EC Turkey (2004). TBR Proceedings Concerning Turkish Practices Affecting Trade
in Pharmaceutical Products, European Commission. Retrieved from
http://trade.ec.europa.eu/doclib/html/119478.htm.

Edwards, B. (2008). How Should You Safely Outsource Pharmacovigilence to an
Indian Contract Research Organization? Indian Journal of Pharmacology,
40. Retrieved from http://www.ijp-online.com/article.asp?issn=0253-
7613;year=2008;volume=40;issue=7;spage=24;epage=27;aulast=Edwards;ty
pe=0

Elsevier (2009) PDUFA Performance In The FDAAA Era. Pharma Approvals
Monthly. Retrieved from
http://sis.windhover.com/buy/abstract.php?id=06150010002.

EMA (2003). New EMA Transparency Policy Measures. Retrieved from
http://www.ema.europa.eu/docs/en_GB/document_library/
Other/2009/10/WC500005269.pdf

EMA (2005). Road Map to the Future. London. Retrieved from
http://www.emea.europa.eu/docs/en_GB/document_library/
Annual_report/2009/12/WC500016660.pdf
119
EMA (2010). The European Medicines Agency Road Map to 2010. Retrieved from
http://www.ema.europa.eu/docs/en_GB/document_library/
Report/2009/10/WC500004903.pdf

Falk, S., Rajagopal, S. (2008). International Law and the Third World: Reshaping
Justice. New York, NY: Routledge Group.

Fenster, M. (2006). The Opacity of Transparency. [Abstract]. Iowa Law Review, 91.
Retrieved from http://works.bepress.com/mark_fenster/4/

Finkelstein (2000). Transparency in Public Policy : Great Britain and the United
States. New York, N.Y.: Houndsmills.

Ford, A. (2009). The Push to Prescribe: Women and Canadian Drug Policy.
Toronto, Canada: Women's Press.

Frank, S. (2003). A New Model for European Medical Device Regulation; a
Comparative Legal Analysis in the EU and USA. Groningen, Netherlands:
Europa Law Publishing.

GCP (2009). Global Corruption Barometer. Retrieved from
http://www.transparency.org/policy_research/surveys_indices/gcb/2009

Gerring, T. (2004). Political Institutions and Corruption: The Role of. Unitarism and
Parliamentarism. British Journal of Political Science, 34:2:295-330.

Ghersi, P. (2008). En route to International Clinical Trial Transparency. The Lancet
372, Issue 9649, Pages 1531-1532.

Gupta, R. (1997). DCGI Drug Registration Procedure. Retrieved from
http://www.perfect-consultants.com/pdf%20files/DCGI%20
India%20registration%20procedure.pdf

Hay, M. (2009). FDA PDUFA Goals Missed. BioBusiness Briefs. Retrieved from
http://www.sagientresearch.com/pt/News/NatureReviews.pdf

Healy, M. (2008). A National Medical Register: Balancing Public Transparency and
Professional Privacy. Med J Aust 188, pages 247-249.

Hess-Biber, S., & Leavy, P. (2010). The Practice of Qualitative Research.
Washington, DC: Sage Press.

120
Hoffman, M. (2010). New Data Provide Insight into Pharma-Industry Professional
Daily Lives. Pharmatech, 34. Retrieved from
http://pharmtech.findpharma.com/pharmtech/IT/
Survey-Says/ArticleStandard/Article/detail/685646

Hou (2009). The Chinese Primary Care System: Its Evolution, Challenges and Legal
Aspects of Reform. CUREJ - College Undergraduate Research Electronic
Journal. Referenced from http://repository.upenn.edu/curej/96

IMS (2008) Pharmerging Markets: Opportunities for Growth and Change.
Retrieved from
http://www.pmlive.com/training_and_advice/reports_and_guides/
IMS_Reports/IMS_Report_on_Pharmerging_markets__opportunities_for
_growth_and_change

IMS (2010). Pharmerging Shake-Up: New Imperatives in a Redefined World.
Retrieved from
http://www.imshealth.com/deployedfiles/imshealth/Global/Content/
StaticFile/Pharma_Shake-up_Imperatives_3_10.pdf

Jones (2009). Health Care Reform in Japan: Controlling Costs, Improving Quality
and Ensuring Equity. [Abstract]. Retrieved from
http://ideas.repec.org/p/oec/ecoaaa/739-en.html#abstract

Jyton, M. (2010). Registration for Drugs in China SFDA. Jyton Consulting.
Retrieved from http://www.jtmedical.com/article/articleContentAction.do?
list=rs&articleId=13

Kanavos, U., Costa (2006). Pharmaceutical Reimbursement Policy in Turkey.
Retrieved from http://www.suvak.org/kitap-2ingilizce.pdf

Kaplowitz, M, Hadlock, T & Levine, R. (2004). A comparison of web and mail
survey response rates. Public Opinion Quarterly 68(1):94-101.

Karan, M., Pandhi, P. (2007). The Drug Lag: New Drug Introductions in India in
Comparison to United States. Retrieved from
http://www.ncbi.nlm.nih.gov/pubmed/12240842

Kermani, F. (2009) Black Gold and Little White Pills. Retrieved from
http://www.scrip100.com/emerging_markets

KFDA (2009). Major Policy KFDA. Retrieved from http://eng.kfda.go.kr/index.php

121
Kim, J, Kim, T, & Yang, J. (2002). Regulatory Transparency: What we learned in
Korea. Retrieved from
http://unpan1.un.org/intradoc/groups/public/documents/
APCITY/UNPAN014177.pdf

Kong, L. (2008). Drug Registration in China: A Comprehensive Overview of
Procedures, New York, N.Y.: Urch Publishing Ltd.

Kumra, M., Pasricha, C. (2009). India Pharma 2015. Retrieved from
http://www.mckinsey.com/locations/india/mckinseyonindia/pdf/
India_Pharma_2015.pdf

Laufman, D. (2009). Managing Risk and Compliance Under the Foreign Corrupt
Practices Act. Pharmaceutical Outsourcing. Retrieved from
http://pharmoutsourcing.com/ViewArticle.aspx?ContentID=3096

McCracken, G. (1988). The Long Interview. Guelph, Canada: Sage University Press.

MHLW (2007). New Vision for the Pharmaceutical Industry. Retrieved from
http://www.mhlw.go.jp/bunya/iryou/shinkou/dl/01.pdf

Nangra, D. (2009). Indian pharmaceutical sector research and analysis. Retrieved
from http://www.stockmarketsreview.com/pricetargets/
indian_pharmaceutical_sector_research_and_analysis_october_2009_200910
07_1000001/

Netanel, N. (2009). The Development Agenda : Global Intellectual Property and
Developing Countries. New York, NY: Oxford University Press.

Obama, B. (2009). Memorandum for the Heads of Executive Departments and
Agencies W. House, Office of the Press Secretary. Retrieved from
http://www.whitehouse.gov/the_press_office/Memorandum-for-the-Heads-
of-Executive-Departments-and-Agencies-3-20-09/

OECD (2001). Regulatory Transparenct: What we have learned in Korea. Retrieved
from http://www.google.com/search?hl=en&source=hp&
q=OECD+2001+Flagship+Report+on+Regulatory+Quality++PUMA+REG+
%282001&btnG=Google+Search&aq=f&aqi=&aql=&oq=&gs_rfai=

OECD China (2007). China Health Technical Paper. Retrieved from
http://www.oecd.org/country/0,3377,
en_33873108_36016481_1_1_1_1_1,00.html

122
OECD India (2007). Economic Survey of India 2007. Retrieved from
http://www.oecd.org/document/5/0,3343,
en_2649_33733_39428741_1_1_1_1,00.html

OECD Turkey (2009). Economic Outlook for Turkey 2009. Retrieved from
http://www.oecd.org/about/0,3347,
en_33873108_39418537_1_1_1_1_1,00.html

OECD Turkey (2008). Health Technical Papers: Turkey. Retrieved from
http://www.oecd.org/about/
0,3347,en_33873108_33873854_1_1_1_1_1,00.html

Offerhaus, L. (2005). The Pharmaceutical Industry in Discredit: Mercury or
Aesculapius? The International Journal of Risk and Safety in Medicine, 17.
Retrieved from http://iospress.metapress.com/content/dup81vlq54t0q637/?
p=5b26621295424090ad7384c4e1b16986&pi=3

Oklahoma, S. C. o. (1978). Oklahoma Assn. of Mun. Attys. v. State. Retrieved from
http://www.google.com/webhp?sourceid=navclient&ie=UTF-
8#sclient=psy&hl=en&rlz=1R2ACGW_en&site=webhp&q=Oklahoma%2C+
S.+C.+o.+(1978).+OKLAHOMA+ASS%27N+OF+MUN.+ATTYS.+v.+ST
ATE.+transparency+&btnG=Google+Search&rlz=1R2ACGW_en&aq=&aqi
=&aql=&oq=Oklahoma%2C+S.+C.+o.+(1978).+OKLAHOMA+ASS%27N+
OF+MUN.+ATTYS.+v.+STATE.+transparency+&gs_rfai=&psj=1&fp=956a
b502b015b9a1

Oliver, R. (2004). What is Transparency? New York, NY: McGraw-Hill.

Patton, M. (2002). Qualitative Research and Evaluation Methods 3rd Ed.: Thousand
Oaks, CA: Sage.

Pearl, M. (2009) Global Strategies: Opacity Index: One Tool to Measure Risk.
Retrieved from
http://www.weforum.org/pdf/GCR09/GCR20092010fullreport.pdf

Permanand, G. (2006). EU Pharmaceutical Registration: The Politics of Policy
Making. New York, NY: University Press.

Ratanawijitrasin, S. (2002). Effective Drug Registration: A Multicountry Study.
Retrieved from http://apps.who.int/medicinedocs/en/cl/CL5.18/clmd,50.html

Sartorius, N. (2009). The Right to Health: A Viewpoint. Mental Health Family Med
,7. Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2777599/
123
Sauvy, A. (1952). Théorie générale de la population. Paris, France: Presses
Universitaires de France.

SFDA GMP (2009). GMP's. Retrieved from http://eng.sfda.gov.cn/eng/

SFDA Regs (2009). Regulations on Administrative Protection for Pharmaceuticals.
http://eng.sfda.gov.cn/cmsweb/webportal/W45649089/A55413921.html

Shaoo, A. (2008). Drug Approval Trends at the FDA and EMEA. BusinessInsight.
Retrieved from
http://www.globalbusinessinsights.com/content/rbhc0200t.pdf

Shengland, X.(2010). Drug Inspectors Arrested for Bribery, Global Times China.
Retrieved from http://china.globaltimes.cn/chinanews/2010-04/523440.htm

Silversides, A. (2005). Transparency and the drug approval process at Health
Canada.Retrieved from http://www.whp-apsf.ca/pdf/transparency.pdf

Sim, C., Gülmezoglu, Evans, Pang (2006). Clinical trial registration: transparency is
the watchword. Lancet 367(9523): 1631-1633.

Sinha, S. (2010). Japan Works to Shorten Drug Lag, Boost Trials of New Drugs.
JNCI Journal of the National Cancer Institute 102(3):148-151.

Srivastava, A. (2009). Background Analysis of the Indian Pharma Market. Retrieved
from http://www.scribd.com/doc/36908960/
Background-Analysis-of-the-Indian-Pharmaceutical-Sector

Tanzer, A. (2005). Pill factory to the world. Forbes.com. Retrieved from
http://www.forbes.com/global/2001/1210/026.html

Temple, R. (2002). Policy developments in regulatory approval. Statistics in
Medicine , 21. DOI: 10.1002/sim.1298

Thomas, L. (2001). The Japanese Pharmaceutical Industry: The New Drug Lag and
the Failure of Industrial Policy.New York, NY: Edward Elgar Publishing.

TI (2006). 2006 Transparency International Corruption Perceptions Index.
Retrieved from http://www.infoplease.com/ipa/A0781359.html

TI (2008). Transparency Index Methodology. Retrieved from
http://www.transparency.org/regional_pages/americas/crinis%20

124
Tilton, M. (1997). Japanese Deregulation: What you should know - Why Regulatory
Reform won't open Japan's Markets to Imports. Retrieved from
http://www.nmjc.org/jiap/dereg/papers/deregcon/tilton.html

Trochim, M (2006). The Qualitative-Quantitative Debate. Research Methods
Knowledge Base. Retrieved from
http://www.socialresearchmethods.net/kb/qualdeb.php

Turkey MOH (2003). Licensing Regulation for Pharmaceutical Products. Retrieved
from
http://www.saglik.gov.tr/extras/birimler/iegm/yt_ruhsatlandirma_eng.pdf

Vermij, P. (2004). Expanded EU Presents Promise and Challenges. Nature, 3. DOI:
10.1038/nrd1403 Walker, K. (2008). Transparency: The Clear Path to
Leadership Credibility. Retrieved from
http://www.linkageinc.com/thinking/linkageleader/Documents/Karen_Walke
r_%20Barbara_Pagano_TRANSPARENCY_Is_the_Clear_Path_to_Leadersh
ip_Credibility_0105.pdf

Websters (2010). Websters on Line Dictionary. Retrieved from
http://www.websters-online-dictionary.org/

Wechsler, J. (2010). Resolving Conflicting Interests. Pharmaceutical
Executive.Retrieved from
http://chromatographyonline.findanalytichem.com/lcgc/
article/articleDetail.jsp?id=566068&sk=&date=&pageID=2

White, J. (2007). Where's The Analysis? S. a. E. Commission. San Francisco.
Retrieved from
http://www.sec.gov/news/speech/speecharchive/2007speech.shtml

WHO (2006). Measuring Transparency in Medicines Registration, Selection and
Procurement: Four Country Assessment Studies. Retrieved from
http://www.who.int/medicines/areas/policy/goodgovernance/
Transparency4CountryStudy.pdf

WHO (2009). Measuring Transparency to Improve Good Governance in the Public
Pharmaceutical Sector. Retrieved from
http://www.emro.who.int/dsaf/dsa968.pdf

WHO China (2007). China's Health Sector Development. Retrieved from
http://www.wpro.who.int/china/sites/hsd/

125
WHO India (2008). Clinical Trials in India: Ethical Concerns. Bulletin of the World
Health Organization. Retrieved from
http://www.who.int/bulletin/volumes/86/8/08-010808/en/index.html

Woo, H., Jo, Park, Kim, Lee (2007). Morbidity and Related Factors Among Elderly
People in South Korea: Results from the Ansan Geriatric (AGE) Cohort
Study. BMC Public Health, 7. DOI: 10.1186/1471-2458-7-10

Yip, H. (2004). The Chinese Healthcare System at Cross Roads, 27. DOI:
10.1377/hlthaff.27.2.460

Yip, W. (2009). Purchasing Health Care in China: Experiences, Opportunities and
challenges, 20.DOI:10.1016/j.chieco.2009.08.003

Zaran, D. (2007). Issues in the Registration of Clinical Trials. JAMA 27. Retrieved
from http://jama.ama-assn.org/cgi/content/abstract/297/19/2112

126
APPENDIX A: DRUG REGISTRATION TRANSPARENCY SURVEY
Access
1. The drug registration process has a publicly accessible comprehensive
Internet information system that is easy to use.
2. There is an effective mechanism to discuss questions related to submission
requirements and issues directly with the agency.
3. There is a lead person identified and assigned for each submission.
4. The lead person responsible for my dossier is accessible after the registration
submission.
5. Drugs applications are easy to submit electronically.

Clarity
1. There are easy to understand written procedures on how to register a drug.
2. There is an easy to use standard drug application format based on the ICH
CTD format.
3. There are clear and well defined requirements for the manufacture of clinical
and commercial drug products.
4. All fees required for submission of the drug application are clearly defined.
5. Fees for submission of the drug application are standardized by product
type/therapeutic area.
127
Accountability
1. The review committee reaches its decision on a drug application via
documented procedures for decision making (e.g. a qualified majority or
consensus).
2. The committee responsible for the drug application process provides an
official written report to the applicant on decisions (e.g. accepted and rejected
files) taken at key decision points of the application process.
3. There is clear appeal process in place for applicants who have their drug
applications rejected.
4. The committee responsible for drug application review is also responsible for
the final decision making for approval/rejection of the drug application.

Ranking
Based on your professional experience please rank the four countries below
on an individual basis from an overall transparency perspective from most to least
transparent.

Experience Base
Please identify your years of regulatory experience and currency of
submission expertise in Asia.
128
Company Size
Please select the size of your company.

Product Types
Please describe the types of products in your company or consultancy
portfolio.

Location
Where are you located?

Future Survey Participation
Would you be willing to participate in an additional survey for doctoral
research if the research is a fit for your area of expertise?
129
APPENDIX B: FOCUS GROUP 1 PARTICIPANT MATRIX

# Years in
Industry

# Years in
Reg. Affairs

Geographic Areas of
Expertise

Area of
Emphasis

Current
Position

1 (CK/FJR)
10
6

Asia Pacific,
LatAm

Clinical
Academic

2 (AM)
30
30

EU,
Middle East

Strategy
Business Dev.

Senior VP, RA

3 (DM)
7
1
China, US

CMC/
Toxicology/
Commercial

Manager,
RA

4 (BL)
6
2
Global
Quality

Manager,
QA

5 (MJ)
22
15
US, LatAm, EU
All
Consultant

6 (TS)
8
3
US, India
Quality

Senior
Manager

7 (SB)
25
8
US
Quality/Reg
VP, QA/RA

8 (GD)
0
3
US
Quality/Bus
Manager

130
APPENDIX C: FOCUS GROUP 2 PARTICIPANT MATRIX

# Years in
Industry

# Years in Intnl.
Reg. Affairs

Geographic Areas
of Expertise

Area of
Emphasis

Current
Position

10(EP)
6
Asia Pacific, LatAm
Clinical
Academic

30(MLS)
30
EU, Middle East

Strategy Business
Dev.

Senior VP RA

7(VQ)
1
China, US

CMC/Toxicology/
Commercial

Manager RA

20(AM)
16

US, Canada, Japan,
EU, Asia

CMC/Clinical/
Preclin

Senior Manager
RA

23(DB)
14

EMEA, LatAm, Japan,
Aus, NZ, Canada

CMC
Consultant

15(DM)
3
Global
CMC

Senior Manager
RA

30(MH)
25
Global
All
Consultant

17(MM)
11
Global
All
Senior Director

10(WW)
2
China US
CMC
Dir RA

35(DS)
15
Global
All
Consultant

131
APPENDIX D: SURVEY STATISTICS
In the tables below, N= the number of respondents with an opinion of the
question. The shaded zone shows those who were unable to judge and not included
in cumulative evaluations.

Table D1: The drug registration process has a publicly accessible comprehensive
Internet information system that is easy to use

Question
Strongly
Agree Agree
Neither Agree
nor Disagree Disagree
Strongly
Disagree
Unable to
Judge N* Mean(SD)
China 0 4 5 10 11 8 30 3.9 (1.0)
India 1 4 6 6 6 14 23 3.5 (1.2)
South
Korea
0 5 6 9 7 11 27 3.7 (1.1)
USA 14 20 0 0 0 3 34 1.6 (0.2)

Table D2: There is an effective mechanism to discuss questions related to
submission requirements and issues directly with the agency

Question
Strongly
Agree Agree
Neither Agree
or Disagree Disagree
Strongly
Disagree
Unable to
Judge N
Mean
(SD)
China 0 3 6 10 8 9 27 3.9 (1.0)
India 0 3 8 6 3 15 20 3.5 (0.9)
South
Korea
2 11 6 2 6 9 27 3.0 (1.7)
USA 13 17 2 0 1 3 33 1.8 (0.8)
132
Table D3: There is a lead person identified and assigned for each submission

Question
Yes No Unable to Judge N Mean
China 9 10 18 19 1.5 (0.3)
India 2 7 27 9 1.8 (0.2)
South Korea 14 4 19 18 1.2 (0.2)
USA 30 1 5 31 1.0 (0.0)

Table D4: The lead person responsible for my dossier is accessible after the
registration has been submitted

Question
Yes
No

Unable to
judge
N
Mean

China 9 9 19 18 1.5 (0.3)
India 4 4 28 8 1.5 (0.3)
South Korea 15 3 19 18 1.2 (0.1)
USA 27 2 7 29 1.1 (0.1)

Table D5: Drugs applications are easy to submit electronically

Question
Strongly
Agree Agree
Neither Agree
nor Disagree Disagree
Strongly
Disagree
Unable to
Judge N Mean
China 0 1 1 10 11 14 23 4.3 (0.6)
India 0 2 1 10 5 18 18 4.0 (0.8)
South Korea 0 6 4 7 5 15 22 3.5 (1.3)
USA 15 11 4 0 0 6 30 1.6 (0.7)
133
Table D6: Additional comments related to accessibility, please place here
Text Response

You need LOCAL staff on the ground in India and China, it's insane to try to
submit from the US or Europe. S Korea - local presence also helps. US - not as
easy as FDA might think, very bureaucratic and dysfunctional.
For the last question, submission in China and India is not via electronic
submission. For South Korea, they are looking into CTD format, but think not
implemented eCTD.
1. Drug registration in China is still complex and takes approximately two years for
new drugs. In addition, if a product is approved elsewhere, the SFDA will most
likely still require the foreign manufacturer to conduct at least some clinical testing
in China before the drug is approved in China.
2. Central Drug Standards Control Organization (CDSCO) is the Indian regulator
of pharmaceutical/drug registrations and some medical devices. CDSCO is also
commonly referred to as the Drug Controller General India (DCGI). All
pharmaceuticals imported into India must have prior approval from the DCGI.
There is no registration system designed for medical devices as a class, but some
medical devices now require product approval as "drugs" before being marketed.
Note: For certain questions I selected "Unable to Judge" for China, India and S.
Korea since product registrations in these countries were contracted to regional
regulatory consultants for direct drug filings.
1. In China and South Korea, publicly accessible information is in local language.
2. In China, india and South Korea, there is no lead person identified per
submission. Each submission undergoes a process of screening, and upon
satisfactory screening, the relevant part of the dossiers are then handled to the
relevant divisions for their evaluation. 3. Electronic submission is not available for
China and India; they are paper based submission. Electronic submission in Korea
is allowed but the language of communication and format is in local language and
local format respectively, although documents can be in English.
FDA is most accessible. South Korea is accessible through a local RA contact. I
have little experience with China and India.
The Chinese SFDA has an English-language website that significantly aids
understanding of their regulatory requirements. India's DCG (I) Office strongly
encourages manufacturers to use their website for information on requirements.
The South Korean KFDA website has very little device information. FDA's
website is well-established and highly informative, but the large volume of content
requires expert navigation.
134
Table D7: There are easy to understand written procedures on how to register a drug

Question

Strongly
Agree
Agree

Neither Agree
nor Disagree
Disagree

Strongly
Disagree

Unable to
Judge
N
Mean

China 0 4 10 6 5 9 25 3.5 (1.0)
India 0 4 5 6 2 16 17 3.4 (1.0)
South Korea 0 10 6 5 3 10 24 3.0 (1.1)
USA 9 19 1 0 0 4 29 1.7 (0.5)

Table D8: There is an easy to use standard drug application format based on the ICH
CTD format

Question

Strongly
Agree
Agree

Neither Agree
nor Disagree
Disagree

Strongly
Disagree

Unable to
Judge
N
Mean

China 0 2 2 14 5 11 23 4.0 (0.8)
India 1 2 4 4 4 17 15 3.5 (1.2)
South
Korea
1 8 1 7 3 14 20 3.2 (1.3)
USA 14 12 0 0 0 6 26 1.5 (0.5)

Table D9: There are clear and well defined requirements for the manufacture of
clinical and commercial drug products

Question

Strongly
Agree
Agree

Neither Agree
nor Disagree
Disagree

Strongly
Disagree

Unable to
Judge
N
Mean

China 0 7 3 9 4 10 23 3.4 (1.1)
India 0 6 4 5 1 16 16 3.1 (1.0)
South Korea 2 8 7 4 2 10 23 2.8 (1.1)
USA 14 13 0 0 0 6 27 1.5 (0.5)
135
Table D10: All fees required for submission of the drug application are clearly
defined

Question

Strongly
Agree
Agree

Neither
Agree nor
Disagree
Disagree

Strongly
Disagree

Unable to
Judge
N
Mean

China 1 12 1 4 3 12 21 2.8 (1.2)
India 1 9 4 0 1 17 15 2.4 (0.9)
South Korea 2 12 4 2 1 11 21 2.4 (1.0)
USA 14 14 0 1 0 3 29 1.6(0.7)

Table D11: Fees for submission of the drug application are standardized by product
type and therapeutic area

Question
Yes
No
Unable to Judge
N
Mean (SD)

China 7 3 34 10 1.3 (0.5)
India 4 3 33 7 1.4 (0.5)
South Korea 8 2 34 10 1.2 (0.4)
USA 19 6 33 25 1.2 (0.4)

Table D12: Additional comments on clarity please place here

Text Response

I am unable to judge because I work on medical device submissions only.
1. The drug application format is unique to each country. However, dossier in ICH
CTD format is acceptable in China, India and South Korea.
The last question is not clear to me. FDA fees are standardized but not by
therapeutic area.
136
Table D13: The review committee reaches its decision on a drug application via
documented procedures for decision making (e.g. a qualified majority or consensus)

Question

Strongly
Agree
Agree

Neither Agree
nor Disagree
Disagree

Strongly
Disagree

Unable to
Judge
N
Mean (SD)

China 0 3 4 5 2 18 14 3.4 (1.0)
India 0 2 3 4 1 21 10 3.4 (1.0)
South Korea 1 6 7 2 1 15 17 2.8(1.0)
USA 8 13 2 0 0 8 23 1.7 (0.6)

Table D14: The committee responsible for the drug application process provides an
official written report to the applicant on decisions (e.g. accepted and rejected files)
taken at key decision points of the application process

Question

Strongly
Agree
Agree

Neither Agree
nor Disagree
Disagree

Strongly
Disagree

Unable to
Judge
N
Mean (SD)

China 0 7 4 2 3 16 16 3.1 (1.2)
India 1 6 2 2 1 19 12 2.7 (1.2)
South Korea 1 12 2 2 1 14 18 2.4 (1.0)
USA 11 11 2 0 0 6 24 1.6 (0.6)

Table D15: There is clear appeal process in place for applicants who have their drug
applications rejected

Question

Strongly
Agree
Agree

Neither Agree
nor Disagree
Disagree

Strongly
Disagree

Unable to
Judge
N
Mean (SD)

China 0 2 5 6 5 14 18 3.8 (1.0)
India 0 2 4 4 3 18 13 3.6 (1.0)
South
Korea
0 8 3 2 3 16 16 3.0 (1.2)
USA 10 13 2 0 1 5 26 1.8 (0.9)
137
Table D16: The committee responsible for drug application review is also
responsible for the final decision making for approval/rejection of the drug
application

Question

Strongly
Agree
Agree

Neither Agree
nor Disagree
Disagree

Strongly
Disagree

Unable to
Judge
N
Mean (SD)

China 0 2 1 7 2 20 12 3.8 (1.0)
India 0 4 1 3 1 22 9 3.1 (1.2)
South Korea 1 8 1 4 1 17 15 2.7 (1.2)
USA 5 11 3 5 0 6 24 2.3 (1.0)

Table D17: Additional comments on accountability please place here

Text Response

In the USA, some applications are reviewed at higher level than the review
Division.
The review is done by the lead reviewer and final decision on our file is made by
the lead reviewer and upper manager level. It is not a committee.
The registration process is clear and open to public but still requires experiences in
this area to understand and apply it smoothly.
1. The Committee responsible for drug application process provides an official
written report to the applicant at the end once a decision is taken. During the
application process, questions regarding the dossier, raised by the various
evaluators, are provided to the applicant so that the applicant can provide a reply to
the questions raised.
Experience is primarily in the medical device area. Accordingly, comments on
drugs relate to general aspects of country-specific regulations and practices.
138
Table D18: Based on your professional experience please rank the four countries
below on an individual basis from an overall transparency perspective from most to
least transparent

Question

Most
Transparent

More
Transparent
Transparent

Less
Transparent

Least
Transparent

Unable to
Judge
N

Mean
(SD)

China 0 1 4 8 10 10 23 4.2 (0.9)
India 0 0 6 5 6 16 17 4.0 (0.9)
South
Korea
0 5 10 7 1 10 23 3.2 (0.8)
USA 24 5 1 0 0 3 30 1.2 (0.5)

Table D19: Please identify your years of regulatory experience and currency of
submission expertise in Asia

Answer
Response
%

1 to 5 years of regulatory experience 8 23.5%
6 to 10 years of regulatory experience 6 17.6%
10 or more years of regulatory experience 16 47.1%
1 or more submissions in the past 36 months 3 8.8%
1 or more submissions beyond 36 months 1 2.9%
Total 34 100.0%

Table D20: Please select the size of your company

Answer
Response
%

Less than 50 employees 8 23.5%
50 to 500 employees 3 8.8%
More than 500 employees 23 67.6%
Total 34 100.0%
139
Table D21: Please describe the types of products in your company or consultancy
portfolio

Answer
Response
%

Medical device 5 14.7%
Pharmaceutical 15 44.1%
Biologics/biopharma 11 32.4%
Medical Device and Pharmaceutical 5 14.7%
Medical Device and Biologics/bio-pharmaceutical 4 11.8%
Pharmaceutical and Biologics/bio-pharmaceutical 3 8.8%
Medical Device, Pharmaceutical and Biologics/
bio-pharmaceutical
15 44.1%

Table D22: Where are you located?

Answer
Response
%

China 3 8.8%
India 0 0.0%
South Korea 2 5.9%
USA 23 67.6%
EU 4 11.8%
Japan 0 0.0%
Other 3 8.8%

Table D23: If your answer to the above is "other" please enter your location here

Text Response

Singapore
Canada
Singapore
140
Table D24: Additional comments please place here

Text Response

Years of experience is more than 10; global regulatory affairs group includes staff
in Asia that is actively making medical device and some pharmaceutical
submissions in all Asian markets including China and Japan.

Table D25: Would you be willing to participate in an additional survey for doctoral
research if the research is a fit for your area of expertise?

#
Answer
Response
%

1 Yes 32 94.1%
2 No 2 5.9%
Total 34 100.0%
141
APPENDIX E: CROSS TABULATION DATA ANALYSIS
Cross-tabulations to show the relationships between the experience of the
respondents (E1) or size of company (E2) for specific questions with regard to South
Korea. These cross-tabulations represent the strongest relationships seen in this
relatively small dataset.

Table E1: Cross-tabulation with experience of respondents

Table E2: Cross-tabulation with size of company

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

“Regulatory” due diligence: a survey investigation of best practices in the medical products industry

PDF

Examining the regulatory framework for drug compounding: industry views and experiences

PDF

Regulatory agreements for drug development collaborations: practices in the medical products industry

PDF

Promotion of regulated products using social media: an industry view

PDF

Challenges in the implementation of Risk Evaluation Mitigation Strategies (REMS): a survey of industry views

PDF

The impact of FDA Patient-Focused Drug Development (PFDD) on US drug development strategies: a survey of views from pharmaceutical product companies

PDF

Implementation of risk management in medical device companies: a survey analysis of current practices

PDF

Effect of GDUFA legislation on the development and approval of generic drugs: a survey of industry views and experiences

PDF

A survey of US industry views on implementation of decentralized clinical trials

PDF

Regulatory team development in post-merger integration: a survey of views from medical product companies

PDF

Regulatory CMC strategies for gene and cell therapies during mergers and acquisitions: a survey of industry views

PDF

Regulatory programs to foster medical product development: user experience in the United States and Japan

PDF

Challenges to implementation of alternative methods to animal testing for drug safety assessment in North America

PDF

Implementation of tobacco regulatory science competencies in the tobacco centers of regulatory science (TCORS): stakeholder views

PDF

Continuity management in biobank operations: a survey of biobank professionals

PDF

Regulating cosmeceuticals in the United States: a cosmetic industry view

PDF

The role of universities in the commercialization of medical products: a survey of industry views

PDF

FDA influence on advisory committees through documentation: a content analysis and survey of industry views

PDF

Evaluation of FDA-sponsor formal meetings on the development of cell and gene therapies: a survey of industry views

PDF

Computerized simulation in clinical trials: a survey analysis of industry progress

Asset Metadata

Creator Solberg, Martin A. (author)

Core Title A survey analysis of transparency in three Asian regulatory agencies responsible for medical products

School School of Pharmacy

Degree Doctor of Regulatory Science

Degree Program Regulatory Science

Publication Date 01/10/2011

Defense Date 11/18/2010

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag Asia,drug,medical,OAI-PMH Harvest,pharmaceutical,Pharmerging,registration,transparency

Place Name China (countries), India (countries), South Korea (countries)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Cadenas, Enrique (committee chair), Richmond, Frances (committee chair), Bell, Dana (committee member), Vanderveen, Pete (committee member)

Creator Email solberg@usc.edu,solbergmartin@yahoo.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-m3608

Unique identifier UC1402968

Identifier etd-solberg-4008 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-606676 (legacy record id),usctheses-m3608 (legacy record id)

Legacy Identifier etd-solberg-4008-1.pdf

Dmrecord 606676

Document Type Dissertation

Rights Solberg, Martin A.

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Repository Name Libraries, University of Southern California

Repository Location Los Angeles, California

Repository Email cisadmin@lib.usc.edu