University of Southern California Dissertations and Theses

Goal setting and performance in federal agencies

(USC Thesis Other)

Goal setting and performance in federal agencies

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

Goal Setting and Performance in Federal Agencies

by

Heejin Cho

A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
Public Policy and Management

May 2021

Copyright 2021 Heejin Cho

ii

Acknowledgments

I am deeply grateful to my advisor Dr. William Resh for his invaluable guidance and support.
Especially, I thank that he never gave up on me even when I was almost giving up on myself. It
was because of his encouragement that I could finish this journey. I am also indebted to my
committee members, Dr. Peter Roberson and Dr. Shui-Yan Tang. I took both of their classes, and
not only are they wonderful teachers inside the classroom, but they have been great mentors who
kindly provided intellectual support.

I would like to thank my family and friends for the love and support they provided throughout
this long journey.

This is and possibly be the biggest accomplishment of my life. But I consider everything a loss
compared to the surpassing greatness of knowing Christ Jesus my Lord, for whose sake I am
ready to lose all things. So, may I never boast except in the cross of our Lord Jesus Christ,
through which the world has been crucified to me, and I to the world. Glory to God in the
highest.

iii
Table of Contents

Acknowledgements …………………………………………………………………………...…. ii
List of Tables …………………………………………………………………………………….. v
List of Figures …………………………………………………………………………………… vi
Abstract ………………………………………………………………………………………… vii
Chapter 1 Introduction .................................................................................................................... 1
1.1 Background ....................................................................................................................... 2
1.2 Chapter Outline ................................................................................................................11
Chapter 2 Revisiting James Q. Wilson’s Agency Typology ......................................................... 14
2.1 Introduction ..................................................................................................................... 14
2.2 Outcome Observability and Wilson’s Agency Typology ................................................ 17
2.3 Data and Measures .......................................................................................................... 21
2.4 Results ............................................................................................................................. 31
2.5 Conclusion and Discussion ............................................................................................. 41
Chapter 3 Presidential Appointees and Federal Agency Performance .......................................... 50
3.1 Introduction ..................................................................................................................... 50
3.2 Literature Review............................................................................................................ 53
3.3 Data and Measures .......................................................................................................... 63
3.4 Results ............................................................................................................................. 70
3.5 Conclusion and Discussion ............................................................................................. 77
Chapter 4 Full Range of Leadership Theory, Organizational Performance, and Wilson’s Agency
Typology ....................................................................................................................................... 79
4.1 Introduction ..................................................................................................................... 79
4.2 Literature Review............................................................................................................ 83
4.3 Data and Measures .......................................................................................................... 92
4.4 Results ........................................................................................................................... 100
4.5 Conclusion and Discussion ........................................................................................... 109
iv
Chapter 5 Concluding Remarks ................................................................................................... 111
5.1 Summary and Contributions .......................................................................................... 111
5.2 Limitations and Future Research Agenda ......................................................................113
List of References ........................................................................................................................115

v

List of Tables

Table 2-1. Number of PAR Goals Used in the Analysis, by Agency-Year ................... 22
Table 2-2. Examples of Agency Goals Coded by Goal Observability .......................... 29
Table 2-3. Agency Goal Observability .......................................................................... 32
Table 2-4. Goal Observability by Agency Type ............................................................ 35
Table 2-5. 2007 GAO Survey of Public Managers ....................................................... 39
Table 3-1. Descriptive Statistics ................................................................................... 68
Table 3-2. Politicization and Agency Performance (in Odds Ratio) ............................. 71
Table 3-3. PAS Vacancy and Agency Performance (in Odds Ratio) ............................. 75
Table 4-1. The Full-Range of Leadership Model .......................................................... 83
Table 4-2. Principle Component Factor Analysis ......................................................... 96
Table 4-3. Descriptive Statistics ................................................................................... 98
Table 4-4. Baseline Leadership Models (in Odds Ratio) ............................................ 101
Table 4-5. Leadership Models with Agency Interaction (in Odds Ratio) ................... 103

vi

List of Figures

Figure 1-1. Summary of Wilson's Agency Typology ...................................................... 8
Figure 2-1. Within Group Variation for Agency Goal Observability ............................ 33
Figure 2-2. Superordinate Coded Agency Types by GAO Survey Means.................... 41
Figure 3-1. Politicization and Agency Performance ..................................................... 73
Figure 3-2. PAS Vacancy and Agency Performance ..................................................... 76
Figure 4-1. Transactional Leadership ......................................................................... 104
Figure 4-2. Transformational Leadership ................................................................... 105
Figure 4-3. Full-Model: Transactional Leadership ..................................................... 107
Figure 4-4. Full Model: Transformational Leadership ............................................... 108

vii

Abstract

This dissertation is motivated by James Q. Wilson’s seminal book, Bureaucracy: What
Government Agencies Do And Why They Do It. Although Wilson explicitly mentions that the
propositions made in the book are not a theory but rather observations that need further empirical
validation, studies in public administration and political science cite his propositions without any
further validation. In order to examine whether Wilson’s propositions are applicable beyond his
time and observations, this study establish Wilson’s famous agency typology based one empirical
data.
It uses a novel data set based on approximately 21,000 performance measures from 28
agencies across a 10-year time span with each goal coded by its outcome observability, or the
extent to which the achievement of a given goal can be attributable largely to the agency efforts.
As a result of the analysis of average goal observability, 28 federal agencies are placed on the
continuum of agency outcome observability scores and grouped into three categories (low,
medium, and high).
Next, the study hypothesizes that performance in federal agencies varies as a function of
agency leadership --- (1) who runs the agency (political appointees and/or careerists); and (2)
how managers motivate employees (transformational and/or transactional leadership). The
impacts of leadership on performance are modeled through the moderating mechanism of
outcome observability in order to account for the inherent differences in agency tasks and the
environment. The findings convey that Wilson’s propositions continue to hold true. First,
viii
performance in the agencies with low outcome observability (Coping/Procedural agencies) is
positively associated with an increased influence of political appointees, while performance in
the agencies with high outcome observability (Production/Craft agencies) is negatively
associated with it. Second, transformational leadership is positively associated with performance
in all agency types, but transactional leadership is negatively associated with performance except
in the Production/Craft agencies.
In the end, the project contributes to the literature by introducing the outcome
observability as a construct. The suggested construct allows new insights to the questions in
public administration and political science.

1
Chapter 1
Introduction
Organizational goals specify human action plans as guidelines, constraints, sources of
legitimacy, standards of performance evaluation, and motivation (Bedeian, 1984, p. 108). Studies
find that clear, specific, challenging, yet achievable goals increase individual commitment to goal
accomplishments, which in turn leads to increased performance (Locke et al., 1994; Locke &
Latham, 2002; Slocum et al., 2002). At the organizational level, choices on defining organizational
goals are attempts to construct “courses of action” that reflect the “value premises” of an
organization’s purposiveness (Simon, 1997). The course of action chosen by a given
organizational leader is formulated by motives based on organizational, resource, political, and
environmental constraints.
In public administration literature, the benefits of clear goals on performance are well-
documented. Clear goals are found to be associated with improved organizational performance
(Chun & Rainey, 2005a), increased employee job satisfaction (Chun & Rainey, 2005b; Jung,
2014b), increased employees’ prosocial motivation (Jung & Rainey, 2011; Moynihan & Pandey,
2005), and decreased employee turnover intention (Chun & Rainey, 2005a; Jung, 2014a).
However, ambiguity, or “the state of having many ways of thinking about the same
circumstances or phenomena” (Feldman, 1989), is a common feature of goals in public
organizations. Public organizations are notorious for setting goals that are “multiple, conflicting,
and vague (Wildavsky, 1979).” Indeed, Rainey (2012, p. 232) noted that the claims about public
organizations having “particularly vague, hard-to-measure, multiple, and conflicting goals are so
2
nearly universal among scholars that any person interested in the study of American bureaucracy
should be familiar with them.”
Still, performance management initiatives in the federal government share a common
belief that strategic planning and goal setting are critical to successful government management.
Hence, given the popularity of performance management among politicians and agencies, it is
important to examine whether and how the goal theory applies to public organizations, particularly
given the increasingly politicized environment of public sector organizational environments.

1.1 Background
Moynihan and Pandey (2005) characterized the current management trends in public
organizations as “an era of government by performance management” that prioritizes efficiency
and management effectiveness. The number of academic publications on performance
management reflects high levels of interest in the topic. Using topic modeling techniques, Walker,
Chandra, Zhang, and Witteloostuijin (2019) examined 3796 articles in Public Administration
Review (PAR) and Public Administration Times (PA Times) and identified the 35 most studied
topics in public administration. Performance management was ranked third in PAR and the fifth in
PA Times, and the topic’s popularity is steadily increasing over time.
The federal government has experimented with various performance management
approaches intended to improve policy outcomes and management effectiveness. In the 1960s, the
widespread practice of Planning, Programming, and Budgeting (PPB) as a strategic planning tool
in the private sector prompted its adoption to federal agencies. Under the Johnson administration,
PPB was implemented in order to hold each federal agency accountable to the President for the
3
distribution of goods and services to the American people (Greenhouse, 1966, p. 272). PPB was
the first federal reform that emphasized clear organizational objectives and anticipated outcomes,
although public managers hardly had a chance to participate in the goal-setting process. Because
PPB aimed to carry out presidential goals related to the Great Society, the Bureau of Budget (BOB)
led the PPB process and enforced a top-down implementation (Randall, 1982).
In the 1970s, the Nixon administration advocated Management by Objectives (MBO), also
known as Management by Planning, to be integrated into agency management. As the name
suggests, goal setting and strategic planning are at the core of MBO. Compared to the PPB, which
relied heavily on the decisions of top executive leadership for setting organizational goals and
strategies, MBO allowed for bottom-up participation from programs (Dirsmith et al., 1980).
Neither Nixon nor the OMB leadership was involved in setting agency goals. Instead, agency
managers and their subordinates collaboratively set specific goals, targets, timetables, and
strategies within the allocated resources (Drucker, 1976).
In the 1980s, MBO was replaced by Zero-based Budgeting (ZBB) under the Carter
administration. Although PPB and MBO attempted to establish a link between performance and
budget, their actual implementation was focused on goal setting and not budget. ZBB, on the other
hand, was centered on the agency budget. Under the guidance of OMB, agencies examined
objectives, operations, and costs for their programs and ranked them by order of importance
(Schick, 1978). This approach contradicts the traditional incremental budgeting process in which
budget money is adjusted at a margin (General Accounting Office, 1979). Through the review
process, only the programs of importance and proven records of performance are provided with
funding. Thus, determining organizational goals and priorities was critical for ZBB.

4
In the 1990s, the Clinton administration declared to create a government that “works
better, costs less, and gets results Americans care about” (National Performance Review, 1993).
The National Performance Review (NPR), which was later renamed as the National Partnership
for Reinvention, sought to adopt the best management practices of the private sector to the federal
government. Headed by Vice President Al Gore, the NPR required drastic changes in federal
government operations to promote the economy, efficiency, and effectiveness through downsizing,
deregulation, and outsourcing (Lapenta et al., 2012). Because Clinton had a divided government
for most of his administration, his performance reforms were presented as a politically neutral tool.
Although OMB took the lead in implementing the NPR, the reinvention models allowed bottom-
up participation of the agencies to set organizational goals and strategies (Milakovich, 2006).
President Bush mocked the Clinton administration’s legacy on NPR and e-government
initiatives, but the Bush administration’s President's Management Agenda (PMA) resembles the
core principles of NPR (Milakovich, 2006; Newcomer, 2007). Like the NPR, PMA also
emphasized downsizing government to maximize efficiency and results. However, because the
Bush administration had more congressional support than the Clinton administration, it allowed
the President to infuse his political priorities into the federal performance reform (Milakovich,
2006). The Performance Assessment Rating Tool (PART), a scoreboard generated to monitor
program performance, gave authority to OMB to set performance expectations and evaluation
standards. Thus, it is not surprising that some studies find political bias in the PART assessment
(S. Lavertu & Moynihan, 2013; Stéphane Lavertu et al., 2013).
The Obama administration discontinued the PART and created a new, non-statutory
position of Chief Performance Officer (CPO) inside the OMB to oversee government-wide
performance progress. The Obama OMB underscored that agencies concentrate on achieving high-
5
priority performance goals and cross-agency goals (Kamensky, 2011). Notably, the Obama
administration's performance management put an emphasis on evaluating the impact of
presidential initiatives, such as the American Recovery and Reinvestment Act (ARRA),
Obamacare, and response to the financial crisis (Kamensky, 2011). Consequently, compared to the
previous administration’s focus on increasing efficiency through effective management, the
Obama administration’s performance management paid more attention to program evaluation
(Joyce, 2011; Kroll & Moynihan, 2018).
In addition to the presidential performance management reforms, Congress passed the
Government Performance and Results Act (GPRA) of 1993 to reinforce performance management
in the federal government. Compared to the presidential management initiatives, the GPRA has
advantages in terms of longevity and bipartisan support (Kroll & Moynihan, 2020). The Act
requires agency executives to (1) formulate 3- to 5-year strategic frameworks and annual
performance goals with numerical targets; (2) establish strategies to accomplish the goals; (3)
oversee implementation of the strategies; and (4) generate annual reports, known as the
Performance and Accountability Reports (PAR). Prior to the enactment of the GPRA, performance
information in federal agencies was collected at the program level, and there was no central
strategic planning at the agency level. Through the collection and monitoring of agency-wide
performance data, the GPRA seeks to “help Federal managers improve service delivery” and to
“improve the internal management of the Federal Government” (GPRA, 1993).
Moreover, the GRPA seeks to improve “congressional decision making by providing more
objective information on achieving statutory objectives, and on the relative effectiveness and
efficiency of Federal programs and spending” (GPRA, 1993). Agencies submit PAR and
supplemental performance information to Congress as a part of their budget justification
6
document. By linking annual agency performance with budget requests, the Act expects
congressional committees to carefully examine agency performance and hold them accountable
through the budget decisions.
Critics of the goal-driven performance management initiatives point out that the rational
strategic planning approach overlooks that public organizations are not always rational, deliberate,
and goal-seeking. Goal setting is an integral part of rational strategic planning, based on the
assumption that goals are specific, measurable, purposeful, meaningful, and reflect outcomes
(Martz, 2013, p. 387). However, this assumption contradicts the reality of public organizations
where multiple stakeholders with a variety of preferences involve in the agency goal setting.
Established research proposes that many goals in public organizations can be purposely ambiguous
in order to accommodate agencies’ political needs to satisfy various stakeholders with competing
interests (Allison, 1980; Dahl & Lindblom, 1953; Lowi & Lowi, 1969; Wilson, 1989). For
example, Aberbach (2002, p. 17) posits that the “[o]versight of agency performance plans and their
actual performance will not occur in a political vacuum, considering that the executive and
legislative branches often disagree on what, why, and how agencies set and achieve priorities.
Roberts (2000, p. 301) argues that the goal theory approach to performance management
embedded in the GPRA is applicable in public organizations only when the following
circumstances are met: an agency mission and goals are narrowly defined; there is broad and deep
agreement on goals; goals are specific enough to guide strategy development; only a few
influential stakeholders are involved; participants in strategic planning are relatively homogenous
and share a consensus on values. In reality, even though many federal agencies do not meet the
criteria, uniform performance measurement and management standards are enforced on all
agencies.
7
Early evaluation of the GPRA addressed the problems with applying “one-size-fits-all”
requirements to all federal agencies (Long & Franklin, 2004; Radin, 1998, 2000, 2003; Roberts,
2000). Complying with the GPRA requirements can be relatively easy in the service delivery
agencies where agency activities impact the quality of public services directly. It is more
challenging to apply the uniform results-based performance standards to the agencies that deal
with “wicked” problems (DeGroff et al., 2010). When agencies have abstract, long-term goals, the
causal link between policy intervention and performance outcome is weak because of exogenous
variables (National Research Council, 1999; Frederickson, 2006).
To highlight the need for differentiating performance management strategies to
accommodate various agency missions and constraints, scholars often refer to James Q. Wilson’s
famous agency categorization. Wilson’s categorization of agencies is introduced in his seminal
book, Bureaucracy: What Government Agencies Do and How They Do It, where he cogently
acknowledges the inherent diversity of public organizations (namely U.S. federal agencies) in
terms of their mission, resources, levels of political support, and stakeholder interests in agency
activities. His categorization marks the departure from the notion that there is one right
management approach applicable to all government agencies.
To elaborate on the key differences in political and organizational constraints of the
agencies, Wilson developed an agency typology that classifies public organizations into four
categories --- production, procedural, craft, and coping agencies ---- based on the observability of
agency outcomes and outputs as Figure 1-1 illustrates. Simply put, observability of outcomes refers
to the degree to which an outsider can attribute the changes in community and society to agency
activities, and output observability refers to the degree to which managers can monitor operator
activities. In terms of performance measurement, outcome observability is associated with
8
organizational performance, whereas output observability is associated with individual employee
performance. In production organizations, both agency-level and operator-level performance
outcomes are readily observable. In procedural organizations, operator output is visible, but the
agency-level outcome is not. In craft organizations, the operator output is not observable while the
agency outcome is observable. Lastly, in coping organizations, neither outcome nor output is
observable. Subsequent chapters will elaborate on the examples and characteristics of each type in
full detail.
Carpenter (2020, p. 83) praises Wilson’s categorization for introducing to “scholars and
students of bureaucracy how to think about agencies and the human actors within them as being
of different types, as having different styles, as doing fundamentally separable different things.”
For this reason, Wilson’s agency categorization has been used widely as a framework to
understand fundamental differences in agencies. To name a few, Wilson’s typology was referenced
in the context of Congressional control of agency budget and resources (Ting, 2001), the
Figure 1-1. Summary of Wilson's Agency Typology

Output Observable

Procedural
Organizations
Production
Organizations

Outcome
Unobservable
Outcome
Observable
Coping
Organizations
Craft
Organizations

Output Unobservable

Source: Wilson (1989: pg. 158-171)

9
performance of the federal environment agencies (Folwer, 2013), federal employee turnover
intention (Ali, 2018), inducing credible commitment of employees (Tang et al., 1996), employee
incentives (Dixit, 2012), monitoring costs and of moral hazards in the principal-agent relationship
(G. Miller & Whitford, 2007), and the role expectations and motivations of public safety volunteers
(Musso et al., 2019). Although Wilson’s typology was developed based on his observations of the
American federal government, his typology has been applied to government agencies outside of
the United States as well. For instance, it was applied to explain Total Quality Management in
Taiwan (Hsieh et al., 2002), the productivity of social policy in New Zealand (Tavich, 2017),
performance measurement of the New Zealand Public Service (Lonti & Gregory, 2007), and the
typology’s theoretical implication to Australian public administration (Gregory, 1995).
However, Wilson’s typology is not without criticism. It is criticized because of the
ambiguities in the definition of observability (Trommel et al., 2012, p. 123). Wilson did not provide
clear guidelines to determine observability. Sometimes, even within the same agency, some tasks
can be observable while other tasks are unobservable. Hospitals and other health agencies can be
classified as coping organizations if one focuses on their efforts to maximize the life expectancy
of patients. However, if one focuses on the procedures that doctors perform, the same agencies can
be classified as procedural agencies (Kervasdoué, 2008). Then, on what basis should we determine
the observability of agency outcomes and outputs? Most scholars, including Wilson, have relied
simply on common sense. For example, Wilson (1989, p.160) classifies the Internal Revenue
Service (IRS) as a production agency because the agency can “observe the activities of its clerks
and auditors” as well as “the amount of money collected in taxes as a result of their efforts.” Other
than one short sentence in the book, Wilson makes no further justification to explain why IRS is a
production agency. No additional research from other scholars was established to validate whether
10
IRS can actually observe their outputs and outcomes. Still, scholars take Wilson’s classification of
IRS as a production agency without questioning (e.g., Bertelli, 2007, Thompson and Fulla (2001).
To my knowledge, only two studies used somewhat systematic measurement
methodologies to determine agency classification. For example, Gueorguieva et al. (2008) selected
eight federal programs and reviewed their organizational mission and performance indicators for
PART and GPRA to determine agency typology. Fowler (2013) conducted an online survey of
state environment agency employees to create two separate observability indexes for agency
output and outcome. He classified the state agencies into Wilson’s agency categories by comparing
the average response scores of six questions intended to measure employee perception of agency
outcome and output observability.
Although Gueorguieva et al.’s (2008) and Fowler's (2013) works show advancements in
the measurement approach for Wilson’s agency typology, their methodologies have limitations to
establish reliability and validity. Gueorguieva et al.'s (2008) work was based on convenient
sampling. They chose eight agencies of their convenience and reviewed only a few performance
indicators for each agency to determine agency typology. On the other hand, Fowler's (2013)
classification methodology was based on employee perception, which may not reflect how the
performance goals for employees and agencies are set and managed. Also, because the indexes for
outcome and output observability are based on z-scores, the agency classification can change even
with a small change in data.
This dissertation takes unprecedented efforts to validate Wilson’s agency typology and
his famous propositions with empirical data. By collecting more than 20,000 performance goals in
28 agencies across ten fiscal years, this dissertation provides a more valid and reliable
measurement tool for outcome observability. To my knowledge, this is the first work that
11
quantifies outcome observability across federal agencies using a reliable classification standard. I
also test Wilson’s propositions on effective leadership for each agency type to demonstrate how
the incorporation of agency typology based on outcome observability can add new insights to the
research questions that many existing studies have already answered. A summary of the chapters
to follow is provided below.

1.2 Chapter Outline
Chapter 2: Revisiting James Q. Wilson’s Agency Typology. This chapter provides the first
systematic measurement of agency outcome observability. It places 28 federal agencies on a
continuum of outcome observability scale. Wilson’s agency typology suggests that agencies fall
predominantly between two dichotomous concepts (either the outcomes they pursue are perfectly
observable or relatively unobservable in terms of how they are attributed solely to agency actions),
but I find that many agencies fall in between the two extremes. In addition to Wilson’s
Coping/Procedural and Production/Craft agencies, empirical data shows that there is a group of
agencies whose observability score is in the middle. I label them Mixed/Undetermined, as
Wilson’s typology did not explicitly categorize their existence. Results from within-agency,
between-year analysis show that the proposed outcome observability typology is resistant to time-
variant influences, although it allows for enough flexibility for changes in rare cases with an
introduction of a new strategic plan. Known-groups and predictive validity tests, however,
demonstrate the validity of Wilson’s typology as well as my outcome observability measure, the
latter of which are covered in Chapters 3 and 4.

12
Chapter 3: Appointee Leadership and Federal Agency Performance. The third chapter
examines Wilson’s proposition about the effectiveness of appointee and careerist leadership.
While the literature suggests that increased politicization of the agency through the presidential
appointees has detrimental effects on agency performance, Wilson suggests that political
appointees can effectively manage agencies with low outcome observability (Coping and
Procedural agencies). For the agencies with high outcome observability (Production and Craft
agencies), Wilson agrees that careerist leadership would be more effective. To test Wilson’s
propositions, I combine the federal performance goals dataset with other data that represent the
influence of political appointees at top executive leadership (PAS) and mid- to low-level leadership
positions (SES and Schedule C). The findings fall in line with Wilson’s expectations. In
Coping/Procedural agencies, the likelihood of performance goal achievement increased with an
increased level of appointee penetration and vice versa for Production/Craft agencies.

Chapter 4: Full Range of Leadership and Wilson’s Agency Typology. The fourth chapter tests
Wilson’s propositions on effective leadership styles. Wilson assumes that agency outcome
observability affects incentives, relationship to managers, and motivation of operators. In
Production and Craft agencies, Wilson recommends the use of an extrinsic motivation system. On
the other hand, in Coping and Procedural agencies where performance is difficult to be determined,
Wilson suggests improving employee commitment by inspiring intrinsic motivations. Combining
Wilson’s proposition with the Full Range of Leadership (FRL) theory, I find that both transactional
and transformational leadership are effective in improving agency performance. Furthermore, in
line with Wilson’s propositions, transactional leadership highlighting the importance of tangible
rewards for good performance is found to be more effective in Production/Craft agencies. In
13
contrast, transformational leadership that stimulates intrinsic motivation, such as intellectual
stimulation, individualized consideration, inspirational motivation, and idealized influence, is
more effective in Coping/Procedural agencies.

Conclusion. The final chapter provides a summary of my findings as well as contribution and
future directions in research.
14
Chapter 2
Revisiting James Q. Wilson’s Agency Typology
1

2.1 Introduction
According to the Section 200 of the Office of Management and Budget (OMB)’s Circular
No. A-11 that introduces the federal performance framework, outcome measures demonstrate
“progress against achieving the intended result of a program” and “changes in conditions that the
government is trying to influence (2017, p. 18).” In other words, outcome-oriented performance
measures describe the impact of government activities on societal-level phenomena.
However, establishing a causal inference between agency activities and performance
outcomes is challenging in public organizations. Public managers reported difficulties in
“distinguishing between the results produced by the federal program and results caused by external
factors or nonfederal actors (Government Accountability Office, 2004)” when they set
performance goals to comply with the Government Performance and Results Act (GPRA). For
example, the Center for Substance Abuse Treatment (CSAT)’s guide to GPRA client outcome
measures provides examples of outcome measures for Substance Abuse and Mental Health Service
Administration (SAMSHA): “substance use, criminal activity, mental and physical health, family
and living conditions, education/ employment status and social connectedness (2014, p. 3).”
Although the goals are inarguably important to society, it is questionable how much of SAMSHA’s

1
This chapter is a modified version of a paper co-authored with Dr. William Resh.
15
activities would contribute to the goal achievement since many exogenous factors outside of the
agency control interfere with the outcomes.
Jennings and Haist (2004, p.186) insist that despite the ostensible difficulties linking
agency activities to performance outcomes, “the current wave of reform in the performance
measurement system is to focus on outcomes as a means of accountability.” GPRA requires
agencies to use various types of performance indicators
2
, but the greatest emphasis is placed on
pursuing outcome measures (GAO, 2004).
To address the causal link between the agency activities and their outcomes, Wilson (1989)
introduced the concept of outcome observability. Wilson defines outcomes as “results of agency
work” that reflect “how, if at all, the world changes (p. 158)”. In his terms, observability refers to
the degree to which an outsider can associate the achievement of agency outcomes to agency
efforts. If the achievement of outcome goals is attributable directly to agency activities, then the
outcome is observable. Observable outcomes can be captured in quantifiable terms such as dollars
saved, the number of people who directly receive a given entitlement, or the average speed of
claims processing. Conversely, if the achievement of the agency outcomes depends on external
factors other than the agency activities, the outcome is considered to be unobservable. Examples
of unobservable outcomes are increased national security, strengthened economy, stimulating
technological innovation, or creating market parity.

2
Outputs measures indicate the quantity of products or services agencies provide over
time; efficiency measures are usually represented in a ratio of inputs to its outputs or outcomes
(i.e. time and cost spent per unit of outcome); customer service measures indicate the level of
satisfaction of those who interact with agency; quality measures indicate the quality of the
agency products and services; and outcome measures indicate whether the agency is achieving
intended results (Government Accountability Office, 2004; Office of Management and Budget,
2017).
16
It should be noted that Wilson’s definition of outcome is different from conventionally
used definition. For example, according to the OMB (2017), output measures indicate the quantity
of products or services agencies provide over time, and outcome measures indicate whether the
agency is achieving intended results. The definition of output measures used in OMB document
aligns with Wilson’s terminology for high outcome observability, and OMB’s definition of
outcome measures aligns with Wilson’s terminology for low outcome observability. Wilson
defines output as the degree to which individual operator activities are visible to their supervisors.
To illustrate how the degree of agency outcome observability shapes agency environment
and bureaucratic behavior, scholars often use Wilson’s agency typology as a useful analytic
framework. Wilson’s typology describes various public organizations as inherently “production,”
“craft,” “procedural,” or “coping” organizations based on a basic two-dimensional premise of
observability of individual-level outputs and agency-level outcomes (H. G. Frederickson et al.,
2018; Melnick, 2012). Carpenter (2020) writes on Wilson’s talent and penchant for categorization,
“science depends massively, perhaps entirely, upon categorization. Whether verbal or
mathematical, what we call ‘theory’ is often nothing other than the mobilization of categories. (p.
2)” Hence, it is important to the science of public administration that Wilson’s oft-employed
typology is revisited as systematically as possible. Continual refinement is critical to any science.
For external validity to be established in Wilson’s theory, specifically, it is necessary to understand
when and how his classifications (and the propositions derived from it) are generalizable across
contexts (e.g., political time, country, policy domain) or upon which factors updates may be
required (Carpenter, 2020, p. 3).
However, although Wilson’s agency typology is referenced widely in public
17
administration literature, there has been no systematic validation of his framework with empirical
data. With little other requirements than the faith of the reader in the validity of their
determinations, scholars often arbitrarily assigned agencies into one or the other category. Thus, it
is important to validate Wilson’s typology with empirical data. One of the potential advantages of
the quantitative measurement of Wilson’s typology is that it is less susceptible to subjective
interpretations of Wilson’s theory. Even Wilson suggested the need for testing his typology by
stating, “My classification is a crude effort to sort out some important differences. It is hardly a
theory and many agencies do not fit its categories. Use with caution (1989, p. 159).”
Using over 20,000 federal performance goals over the years between 2000 and 2011, this
paper provides a quantitative measurement of Wilson’s typology based on the observability of
agency outcomes. As the first systematic measurement approach to Wilson’s typology, this study
contributes to the literature by providing modest modifications to Wilson’s typology based on
empirical data and introducing outcome observability as a useful tool to examine questions in
public administration.

2.2 Outcome Observability and Wilson’s Agency Typology
James Q. Wilson famously argued that managerial challenges in government agencies
could differ in two main respects: (1) whether the individual level outputs of agency operators can
be observed and (2) whether the societal changes as a result of agency-level activities can be
observed. According to Wilson, observability of outputs refers to the degree to which the day-to-
day activities of individual operators can be monitored, while observability of outcomes refers to
the degree to which an outside observer can evaluate the societal results of agency work. As with
18
the agency outcomes, the individual-level activities within the agency may not be measurable nor
adequately linked to agency-level outcomes (Olsen, 2015), especially when the operator activities
are esoteric or take place out of managers’ sight.
Because Wilson’s typology is, indeed, so common to even fledgling students of public
administration and bureaucratic politics, it is unnecessary to belabor the categories too much here.
Nonetheless, it is important to review them briefly and to provide definitions and examples of how
these categories have been operationalized in the scholarship. Wilson’s primary focus was on the
U.S. federal bureaucracy. Wilson (1989) divided public organizations into four types according to
the observability of agency outcomes and outputs.
First, in production organizations, agency outputs (operator-level activities) and outcomes
(agency-level activities) are both easily observable. These organizations tend to focus on the
efficiency of product and service delivery. Wilson’s examples of production organizations are the
Social Security Administration (SSA), the Internal Revenue Service (IRS), and the U.S. Postal
Service (USPS). Wilson argues that management is relatively easy in production organizations
since individual employee’s contributions to the organization’s output and, collectively, the
agency’s direct attribution to a given outcome can be measured. Scholars noted that the goal
theory-based performance management initiatives are most relevant in production organizations
(Lee et al., 2010; Lynn, 1999; Radin, 1998; Roberts, 2000). Compared to other types of
organizations, clear goals in production organizations enable managers to evaluate the
performance of operators and organizations against the clearly established evaluation standards.
While Wilson argues that the observability of outputs and outcomes simplifies managerial
dilemmas in production agencies, his argument is not entirely deterministic. He allows that
19
managers of production agencies can be guilty of goal displacement, whereby Gresham’s Law
predominates, i.e., measurable outcomes tend to crowd out unmeasurable outcomes (Wilson, 1989,
p. 161). Hence, the higher-order, mission-oriented objectives of the agency might be lost or
undermined in lieu of more easily measurable outcomes (Bohte & Meier, 2000).
Second, in craft organizations, the operator-level process is difficult to observe, but
agency-level outcomes are relatively easier to capture. Wilson suggests that the tasks in craft
organization require professional expertise and autonomy of operators. According to Wilson,
investigative agencies, such as the Federal Bureau of Investigation (FBI), are stereotypical craft
organizations. The agency’s work process is unobservable not only to the public but also within
the organization to some extent (e.g., an agent’s process of establishing informational contacts in
undercover investigations), but the agency outcome is measurable, for instance, in terms of the
number of investigations, conviction rates, or criminal sentences (B. Miller & Curry, 2018).
Another example of craft organization is the military during wartime. It is difficult to monitor what
soldiers do on the battlefield, but it is clear to know who won the war aftermath. In craft
organizations, staffs are decentralized and work independently outside of the managers’ direct
supervision. It is difficult to implement the Standard Operating Procedure (SOP) because operators
deal with situations in which a generalized solution does not work. Instead, operators rely on
professional norms to guide their actions. As a result, managers exercise little control over the
operators, and the organizational hierarchy is weak.
Third, in procedural organizations, managers can monitor the activities of operators, but
it is difficult to figure out if the operator's efforts are directly reflected in the goal achievement.
The situation drives anxious managers to monitor employee actions too closely. Managers develop
20
the SOP and enforce strict adherence to the procedures prescribed in the document in the daily
work of operators. The problem arises when the organization becomes overly reliant on the rules
and procedures. Indeed, the stated goals of the organization may become procedure- or
compliance-oriented under such conditions (e.g., counting the number of sanctions an agency
enforces, rather than measuring the outcome the sanctions are intended to induce). According to
Wilson, enforcement agencies, such as the Wage and Hour Division (WHD) and the Occupational
Safety and Health Administration (OSHA) under the Department of Labor, are classic examples
of procedural organizations. The investigators and inspectors work in accordance with the SOP
that lays out how to evaluate records, but it is questionable whether passing all items on the
inspection checklist would guarantee a safe and healthy workplace.
Lastly, in coping organizations, performance is too ambiguous to determinably observe
both at individual operator-level and aggregated agency-level. It is strongly advised that coping
organizations do not adopt the same performance management standards that production agencies
do (Lynn, 1999; Radin, 1998; Roberts, 2000). Goals in coping organizations are ambiguous
because people have a different definition of success, and it is difficult to implement standardized
procedures as various external factors increase the level of uncertainty in the process. For instance,
it is highly complicated to determine a foreign country’s attitude toward the U.S. as a direct
function of the tasks that diplomats perform --- how does one measure the “soft power” of
diplomatic work, in and of itself? In the first place, there is no consensus on what stance the
American government should take on various foreign policy issues. Although Wilson is pessimistic
that effective management is nearly impossible in coping organizations, he admits that some things
that managers do make changes. For instance, using the New York City Police Department and
Washington D. C. Public Schools as examples, Maranto and Wolf (2013) argue that performance
21
in coping agencies improves when political and bureaucratic entrepreneurs bring innovations to
the organization.

2.3 Data and Measures
A. Data Collection and Data Structure
The primary source of data is Performance Accountability Reports (PARs) from federal
agencies. The Government Performance and Results Acts of 1993 (GPRA) requires all federal
agencies to submit their long-term strategic plans, set annual performance plans with specific
targets, and report the result in their PARs. Sometimes, agencies publish such reports under
different names (i.e., Citizen’s Report, Agency Financial Report, etc.), but they all contain the
agency’s performance information in relation to its budget and financial plan. For the agencies that
provide brief highlights of annual performance in their PARs, detailed performance indicators and
achievement status were extracted from the Congressional Budget Justification documents.
Hupe (2010, p. 123-124) notes that Wilson’s agency typology is based on Wilson’s
observations when he wrote the book in 1989, and it does not reflect the changes brought by the
New Public Management. This paper tests the generalizability of Wilson’s typology beyond 1989
by incorporating a sample of more than 20,000 performance goals from 28 agencies across ten
fiscal years spanning the George W. Bush and Barack H. Obama administrations (FY2002 – FY
2011). Table 2-1 provides the number of goals used in the analysis by agency and year.

22
Table 2-1. Number of PAR Goals Used in the Analysis, by Agency-Year
Agency 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Total
BBG . . 32 21 27 27 26 26 25 29 213
CPSC . . 107 106 104 75 88 116 123 96 815
DOC 114 172 160 142 127 127 116 115 113 84 1,270
DOD 18 55 61 57 51 73 95 65 74 549
DOE 258 198 261 247 203 203 218 336 362 190 2,476
DOI 308 311 176 190 193 . 205 225 53 119 1,780
DOJ 117 91 28 33 37 33 32 32 30 27 460
DOL 109 . 114 94 85 86 85 17 59 77 726
DOT 40 . 31 35 34 34 36 36 34 34 314
ED 90 149 125 61 61 55 75 81 81 35 813
EEOC 39 32 24 24 24 10 9 9 9 8 188
EPA 71 64 79 83 80 88 91 110 125 67 858
FAA 10 12 31 32 30 30 29 30 31 29 264
FERC 57 70 79 91 113 98 91 91 16 17 723
FTC . 14 14 14 14 46 46 45 41 41 275
GSA . . . 78 76 . 95 70 63 26 408
HHS 46 18 29 31 40 42 41 13 49 51 360
HUD 182 193 128 116 118 150 119 111 . . 1,117
NASA 103 125 296 210 207 127 149 179 195 266 1,857
ODS 220 172 211 233 101 105 87 70 58 59 1,316
OPM 0 239 . 153 58 67 27 26 24 24 618
SBA 17 122 . 76 84 . 90 64 85 40 578
SEC . . . . . . . 110 86 56 252
SSA 14 17 17 42 39 38 26 25 27 33 278
USAID . . 61 71 103 54 37 50 55 86 517
USDA 34 40 31 39 38 35 32 35 38 40 362
USDT 227 206 246 97 107 106 114 122 118 . 1,343
VA 22 25 21 23 22 22 25 26 23 23 232
Total 2078 2288 2356 2403 2182 1709 2062 2265 1988 1631 20962

Although the GPRA was enacted in 1993, its full implementation did not take place until
FY 1999. From FY 1994 to 1996, five selected agencies participated in the pilot program. It was
by the end of FY 1997 that all federal agencies were required to submit strategic plans with their
FY 1999 budget (GAO, 1997). Most of the early strategic plans failed to comply with the GPRA
23
as they lack clear performance indicators and targets. Thus, the analysis of performance goals
begins from FY 2002, when the reports became consistent and structured according to the GPRA
standards. The analysis ends with FY 2011, when many agencies changed the reporting structure
for PARs after the GPRA Modernization Act (GPRAMA) of 2010. The GPRAMA emphasizes
long-term planning and long-range goals. Immediately after the Act’s enactment, several agencies
were piloted set 2-5 year long-term performance targets in lieu of annual targets. The changes in
the reporting structure prevent coders from determining whether the goal is achieved until the end
of the strategic planning cycle, which usually lasts for three to five years. Hence, to keep the
consistency of data, the analysis excludes performance goals published after 2011 when the
reporting structure changed
3
.
Information about strategic objectives, annual performance targets, and goal attainment
was extracted from the PARs. Strategic objectives refer to the highest level of goals that state the
broad, long-term goals of the agency. Depending on the size and number of programs of the agency,
agencies usually have about 3-7 strategic objectives. Under the strategic goal, agencies set annual
goals with specific indicators and targets, which we use as the unit of analysis for this paper. They
represent the contextual goals that the agency operationalizes in order to reach the strategic goal
(Wilson, 1989). Most of the annual goals have numerical targets since GPRA requires agencies to
focus on measurable outputs and outcomes rather than inputs. However, some goals do not have
numeric values attached to the target because they simply measure whether the agency had

3
In case of the Department of Housing and Urban Development (HUD), the agency changed the
reporting structure in FY 2010. Instead of using annual performance measures, the agency
measured two-year progress on Priority Goals. Because no annual target was provided, HUD’s
goals from 2010 and 2011 were not included in the dataset.
24
completed a required activity or not. For example, some agency goals measure whether the agency
received an unqualified audit opinion or updated IT system. In these cases, even though the goals
do not have quantifiable targets, we can still determine whether the agency had met the goal or
failed it.
To take the Department of Commerce (DOC) as an example, it set three strategic
objectives in 2008:
1. Maximize U.S. competitiveness and enable economic growth for American industries,
workers, and consumers
2. Promote U.S. innovation and industrial competitiveness
3. Promote environmental stewardship

The DOC set a total of 116 annual performance goals in 2008. Fifty-seven annual
performance indicators were established under the first strategic goal, thirty indicators under the
second, and twenty-nine indicators under the third strategic goal. Sample indicators include:
1. Number of jobs created/retained over three-year totals
2. Patent applications filed electronically
3. Winter storm warnings – Accuracy (%)

The following criteria were used to exclude the goals from the dataset. First, goals without
specific performance targets were removed. This explains why the dataset does not include FY
2002 and FY 2003 goals for some agencies. Even though the GPRA requires that agencies set
performance goals with clear targets, some agencies, especially during FY 2002 and FY 2003,
provided only a vague description of what agency inputs were invested into achieving the goals.
25
Second, discontinued goals that no longer report performance results were removed. Third, some
qualitative goals that do not indicate evaluation standards were removed. Last, there is a data
availability issue. While some agencies publish all of their historical PARs, other agencies do not
keep a record of their PARs, especially if the report was published prior to 2010. Attempts to
acquire data were made through the Freedom of Information Act (FOIA) requests, but still, not all
reports were obtainable.

B. Advantages of PAR
There is no universal consensus on how way to measure “objective” performance that is
free of bias, and PAR is subject to such biases as addressed in the Introduction chapter. However,
when compared to other commonly used datasets, data from PARs have certain advantages.
The Bush administration’s Program Assessment Rating Tool (PART) enabled researchers
to access government-wide performance data on 1) program purpose and design, 2) strategic
planning, 3) program management, and 4) program results. The PART is designed to hold programs
accountable by connecting program performance to budget. The OMB staffs evaluated program
performance based on a 5-point scale, ranging from ineffective, adequate, moderately effective, or
effective. The PART is criticized because it evaluates agency performance by the uniform
standards established by the OMB. Although it is devised as a “neutral” instrument, its actual
implementation was considered to be a political tool that imposes presidential priorities on
agencies (Dull, 2006; Heinrich, 2012; Moynihan, 2013). Due to the inevitable political bias of the
OMB, PART scores of the liberal agencies are consistently lower than those of conservative
agencies (Gallo & Lewis, 2012b). Complying with the PART reporting requirements is more
26
burdensome for managers in liberal agencies than in conservative agencies (Stéphane Lavertu et
al., 2013), yet their reported use of PART performance information is lower (Stéphane Lavertu &
Moynihan, 2013). Furthermore, because the PART was discontinued by the Obama
administration, PART data does not allow researchers to examine agency performance over the
years across different political settings.
Another commonly used dataset to measure federal agency performance is the Federal
Employee Viewpoint Survey (FEVS). FEVS is popular because data is easily downloadable on the
Office of Personnel Management (OPM) website
4
. The large sample size is an advantage of this
data source as the average number of respondents for each wave exceeds 200,000 federal
employees. Moreover, since the survey is conducted every year (every two years until 2010),
longitudinal data can be created. However, several limitations lie with the data when used to
measure agency performance. First, measuring agency performance based on the self-administered
survey from the internal organizational stakeholders invites bias as they tend to overestimate their
performance (Andrews et al., 2006, 2010; Meier & O’Toole, 2013). Many studies assess agency
performance based on the aggregated response scores for the following question: “How would you
rate the overall quality of work done by your work group?”
5
(Caillier, 2012; Fernandez &
Moldogaziev, 2013; Pitts, 2009; Somers, 2018; Whitford et al., 2010). The question asks
employees about their perception of work group performance. Because the evaluation of
workgroup performance is subjective, the data contains respondents bias. Second, in terms of
wording, work group is not an official unit of the agency, and its scope is subjective to individual

4
https://www.opm.gov/fevs/
5
This question was presented as item number 61 in 2002; number 10 in 2004, 2006 and 2008;
and number 28 in 2010 and subsequent years.
27
interpretation. Depending on the interpretation of the respondent, work group may refer to a
program, division, or even cross-divisional team. Third, the question asks about the overall quality
of work, but performance is a broader concept. Quality is only one of the important aspects of
performance. Public organizations face trade-offs of different values (Resh & Pitts, 2013; Wenger
et al., 2008), and quality can be compromised at the price of other values, such as timeliness,
efficiency, productivity, and cost-savings. Fourth, because answers to this question are often
aggregated to calculate the performance score of the division or the entire agency, the danger of
ecological fallacy is embedded; even if all workgroups are actually performing well, coordination
of different workgroups may result in poor agency performance. Moreover, when a researcher
extracts other variables from the FEVS to see their relationship to the workgroup performance
question, the results are subject to the common source bias (Resh et al., 2019).
PARs are “archival” data of agency performance, which reduces the levels of biases that
“perceptual” data like the FEVS contains (Song & Meier, 2018). Also, PARs provide a
comprehensive picture of the agency's “effectiveness” in a consistent manner across
administrations. Goals included in the PARs are developed primarily by agencies. Inputs of
employees are reflected in setting the GPRA goals, so they better represent the organizational
priorities and constraints than externally imposed performance measures and targets (Kroll &
Moynihan, 2020). As a result, performance information from GPRA data is associated with more
purposeful use by public officials than data from PART (Kroll & Moynihan, 2020).
Moreover, the GPRA goals better reflect the policy preferences of multiple stakeholders
as well. Because agencies are required by law to consult with OMB, Congress, and various
stakeholders to set their performance goals, the PARs implicitly recognizes that agencies are
28
directly accountable not only to the president but also to various constituencies, including
Congress, interest groups, and the public (Hammond et al., 1996; Lee & Hong, 2011; Lee &
Whitford, 2013; Whitford, 2005).

C. Determining Goal Observability
It should be noted that this article focuses only on the observability of outcomes and not
outputs because agency performance, as expressed in the GPRA goals, is a reflection of aggregated
agency-level activities. Achievement of the GPRA goals often requires inter-departmental and
even inter-agency collaborative efforts. Thus, the observability of individual contribution toward
goal achievement is difficult to measure with the given data.
Since the purpose of this paper is to classify agencies into Wilson’s outcome observability
spectrum, the main variable of interest is the observability of agency outcome, as indicated in their
annual performance goals. After extracting the performance measures from the PARs, each goal
was coded qualitatively to determine goal observability. If the attainment of a goal is solely
attributed to the agency activities, then the outcome is determined to be observable. In other words,
is the attainment of a certain goal directly attributable to that agency’s efforts alone? If the answer
is yes, the goal is considered observable and got coded as 1. On the other hand, if external factors
such as changes in demography, global economy, or political landscape could plausibly influence
the attainment of the goal, the outcome is considered unobservable and was coded as 0. Table 2-2
provides some examples across agencies.

29

Agency Year Strategic Objective Agency Goal Observability
EPA 2011 Taking Action of Climate
Change and Improving Air
Quality
By 2015, reduce emissions of nitrogen
oxides (NO2) to 14.7 million tons per
year compared to the 2009 level of 19.4
million tons emitted

Unobservable (0)

The achievement of
the goal depends on
many exogenous
factors other than
the agency
activities.
LAB 2010 Prepare Workers for Good
Jobs and Ensure Fair
Compensation
Improve employment outcomes for
veterans who receive One-Stop Career
Center services: Percent of veteran
participants employed in the first quarter
after exit
HHS 2009 Public Health Promotion and
Protection, Disease
Prevention, and Emergency
Preparedness
Reduce the number of suicide deaths
USDA 2008 Improve the Nation’s
Nutrition and Health
Increase Food Stamp payment accuracy
rate

Observable (1)

Performance goal
achievement
depends largely on
agency activities.
TRS 2006 Management and
Organizational Excellence
Number of Completed Audit Products
SSA 2007 To protect the integrity of
Social Security programs
through superior
Stewardship
SSA hearings case production per work
year

Table 2-2. Examples of Agency Goals Coded by Goal Observability
30
Three coders coded 97 random goals using the criteria described above. Intercoder
reliability was determined using Cohen’s Kappa, which is calculated using the following equation:
K=
𝐏 (𝐚 )−𝐏 (𝐞 )
𝟏 −𝒑 (𝒆 )
[2.1]
where P(a) is the percent agreement between raters, and P(e) is the expected percent agreement
that would occur randomly. To calculate P(a), the number of goals that all three coders agreed on
was summed. Out of 97 goals, the coders agreed that 47 goals were observable, while 26 goals
were unobservable. Since the coders agreed on 73 out of 97 goals, the numerator P(a) was
calculated at .753. To calculate the denominator P(e), the probability that each coder would
randomly decide whether a goal is observable was first calculated. The first coder determined that
67.01% of the goals in the sample were observable, while the second and third coder both coded
56.70% observable. Then the probabilities were multiplied to calculate the random agreement
score for observable goals. The same procedure was repeated to calculate the random agreement
score for unobservable goals. By combining the two scores, P(e) is calculated to be 0.277. As a
result, Cohen's Kappa was calculated at .658, thereby achieving a "substantial" strength of
agreement (Gwet, 2014; Landis & Koch, 1977). The benchmarks for Kappa (the inter-rater
reliability statistics most often used) are outlined in Appendix 2-1.
31
2.4 Results
A. Between-Group Analysis
Table 2-3 displays the mean goal outcome observability scores for each of the 28 agency
samples derived from 298 annual PAR reports and 20,962 goals therein. A higher percentage of
average goal observability indicates that achievement of performance goals is largely dependent
on agency efforts. The overall results are consistent with some basic expectations associated with
Wilson’s typology. On average, about half of the sample goals are determined to be observable
(55.37%). However, when looking at individual agencies, it is found that the average observability
scores are uneven across the agencies. On one extreme, the United States Agency for International
Development (USAID) scores the lowest on the scale, with only 0.77% of their goals being
observable. On the other extreme, the National Aeronautics and Space Administration (NASA)
scores the highest, with 90.74% of the goals being observable.

32
Table 2-3. Agency Goal Observability
Agency

Unobservable

Observable

Total

Average Goal
Observability
BBG 119 94 213 44.13%
CPSC 92 723 815 88.71%
DOC 891 379 1,270 29.84%
DOD 312 237 549 43.17%
DOE 512 1,964 2,476 79.32%
DOI 719 1,061 1,780 59.61%
DOJ 379 81 460 17.61%
DOL 517 209 726 28.79%
DOS 1,029 287 1,316 21.81%
DOT 298 16 314 5.10%
ED 652 161 813 19.80%
EEOC 23 165 188 87.77%
EPA 760 98 858 11.42%
FAA 52 212 264 80.30%
FERC 69 654 723 90.46%
FTC 104 171 275 62.18%
GSA 271 137 408 33.58%
HHS 198 162 360 45.00%
HUD 386 731 1,117 65.44%
NASA 172 1,685 1,857 90.74%
OPM 224 394 618 63.75%
SBA 205 373 578 64.53%
SEC 72 180 252 71.43%
SSA 40 238 278 85.61%
USAID 513 4 517 0.77%
USDA 131 231 362 63.81%
USDT 512 831 1,343 61.88%
VA 103

129

232

55.60%
Total 9,355

11,607

20,962

55.37%

Information in Table 2-3 is graphically displayed in Figure 2-1, accompanied by a 95%
confidence interval (CI). CI allows the assessment of whether a given between-sample difference
is statistically significant at conventional alpha levels. In general, agencies with a fewer number
of goals tend to have wide CI as the effects of outliers grow.
33
Figure 2-1. Within Group Variation for Agency Goal Observability

The empirical measurement corroborates that outcome observability is not a dichotomous
concept. While Wilson’s typology assumes that agency outcomes are either observable or
unobservable, data verifies that outcome observability is on a continuum. On the left end of the
diagram are agencies with the least observable outcomes, including the Environmental Protection
Agency (EPA), Agency for International Development (AID), Department of Transportation
(DOT), and the Department of Education (Ed). The finding falls in line with Wilson’s expectations
that agencies tasked with wicked social problems would be Coping or Procedural organizations.
On the right end of the diagram are agencies with highly observable outcomes (i.e., Production or
Craft organizations), such as the Social Security Administration (SSA) and National Aeronautics
and Space Administration (NASA). It is noteworthy that several commissions with explicit
34
enforcement responsibilities were found to have high outcome observability. The finding is in
contradiction with Wilson’s expectation because he assumed that federal enforcement agencies
would be procedural agencies.
Figure 2-1 illustrates that a group of agencies falls in between the two poles of
observability and unobservability. The middle-range group (referred to as “Mixed/ Undetermined”
hereafter) tends to include comparatively large departments (e.g., Department of Defense and
Department of Health and Human Services). Also included in the middle range are agencies with
varied missions or professional orientations (e.g., Department of Interior (INT), Department of
Veterans Affairs (V A)). The average observability score of these agencies implies that
Mixed/Undetermined agencies seek to achieve observable outcomes while pursuing unobservable
outcomes, supporting that public organizations pursue multiple and often conflicting goals.
Findings of Gueorguieva et al. (2009) implied the existence of Mixed/Undetermined category at
the program level. They suggested that Community Oriented Policing Services (COPS),
Compliance and Enforcement Program of the Federal Election Commission, and Medicare can be
classified into more than one category in Wilson’s typology.
To establish clear substantive and statistical differences between organizations on the basis
of observability, agencies are classified into three groups according to the following standards:
Craft/Production (observability > .65), Coping/Procedural (observability < .35), and Mixed/
Undetermined (.35 ≤ observability ≤ .65). Table 2-4 summarizes the average goal observability
scores for the three groups. Additionally, a one-way ANOV A test was conducted to compare the
differences in the average observability scores. The between-group analysis suggests a statistically
significant difference among the three groups as the p-value is less than 0.05 [F (2, 209590) =
3967.10, p=0.00]. A Tukey posthoc test was conducted for a pairwise comparison of the mean
35
observability scores. The results reveal that any of the three groups were distinguishable when
compared to another group. Details of the ANOV A test are included in Appendix 2-2 and Appendix
2-3.

B. Within-Group, Between-Year Analysis
One of the criticisms of Wilson’s typology is that it is static and overgeneralized (Robert
Maranto & Wolf, 2013). However, Kervasdoué (2008) suggests that agency typology can change
over time. Based on his observations, the British National Health Service shifted from
Coping/Procedural agency to Production/Craft agency as they emphasize observable outcomes
such as more on the length of waiting lists. Likewise, Kervasdoué finds that psychiatric hospitals
in France and U.S used to be Coping/Procedural organizations with highly ambiguous outcomes.
Because social perception of mental illness was negative, the hospitals had abstract goals.
However, as social conception and technological tools have changed over time, psychiatric
hospitals transformed into Production/Craft organizations that highlight the visibility of doctors’
activities for patient care.
Table 2-4. Goal Observability by Agency Type

Mean of Goal
Observability
Score
Std. Deviation Frequency
Production/Craft .205 .404 6682
Mixed/Undetermined .594 .491 7427
Coping/Procedural .849 .358 6853
Total .553 .497 20962

36
Because the analysis implemented 10-year longitudinal data, we test whether agency
observability scores change significantly over time. When new strategic plans are developed,
agencies realign their goals to the new political and organizational priorities presented in the
strategic plan. As a result, agencies make substantial changes to their performance goals every 4
to 7 year period when a new strategic plan is introduced. Since the analysis typically spans more
than seven years, even if 3- to 7-year strategic plans retain the exact same annual goals each year
(most do not), most of the agencies included in the analysis had changed a substantial amount of
their goals at least once during the time period. Even when a given agency in our analysis has less
than seven years of observations, the strategic plan has typically changed within the timespan of
data collection.
. The year-by-year observability scores must be understood with caution. Meaningful
conclusions can be drawn from within agency year-by-year analysis only when agencies have
enough number of performance goals across the years. However, as shown in Table 2-1, the
number of observations within each agency is not even across the years. For some agency-year,
the sample size is too small. In such cases, variation in the percentage of observability by year
does not provide enough statistical power.
Overall, it is found that across agencies, there are some variations across years in average
goal observability, but those variations are mostly contained in the same agency type. Appendix
2-4 includes graphs of year-to-year analysis for each agency. Agencies like USAID, Ed, EPA,
CPSC, and NASA fall into the same agency category throughout the 10-year time period, thus
demonstrating the measurement’s resistance to time-variant influence. Some agencies change their
agency categories over time, but many of them have too few observations per year to draw
statistically meaningful conclusions. As an exception, it is interesting that DOC shifts between the
37
Mixed/Undetermined type and the Production/Craft category. The results show that Wilson’s
agency typology is stable but not entirely static.

C. Known Groups Validity
To validate the empirical measurement to Wilson’s typology presented above, an
additional measure that quantifies the public managers’ perceived level of goal observability was
obtained to determine known-group validity. Known group validity refers to an instrument’s
capacity to distinguish between groups that it should, in theory, be able to distinguish between
(Davidson, 2014). In this case, if the classification of agencies based on observability scores is a
valid measurement tool, survey responses from the managers regarding their perception of goal
observability in Coping/Procedural agencies should be statistically different from their responses
in Production/Craft agencies.
Data comes from the Government Accountability Office (GAO)’s Federal Managers
Survey on Performance and Management Issues (GAO-08-1036SP, July 2008). The survey was
conducted between October 2007 and January 2008. Out of 107,326 mid-level and upper-level
civilian managers and supervisors in the 29 executive branch agencies covered by the Chief
Financial Officers Acts of 1990 (CFO), GAO extracted a stratified random probability sample of
4,412 persons. The response rate ranged from 55 percent to 84 percent per agency. The final sample
size used in this analysis includes 2399 responses from 22 agencies.
Question #9 in the survey asks public managers to determine the factors that hindered
measurement or usage of performance information in the program(s)/ operation(s)/project(s) that
they are involved with. One of the 13 sub-questions, question 9g, asks respondents to indicate their
38
perceived level of difficulties in “distinguishing between the results produced by the program and
results caused by other factors.” Responses were recorded on a 5-point Likert scale (“to a very
great extent”, “to a great extent”, “to a moderate extent”, “to a small extent”, and “to no extent”).
200 responses that indicated “No basis to judge/Not applicable” were excluded.
Since the measurement of average observability score depends on whether the attainment
of a given goal is solely attributable to the agency activities, this survey question measures the
same concept. However, it should be noted that the question asks respondents’ experience with the
program(s)/ operation(s)/ project(s) that they were involved with. We can reasonably expect that
the perceived program-level goal observability may be different from agency-level goal
observability, depending on the internal agency structure.
Table 2-5 shows the aggregated mean scores of the survey responses by the agency. Given
the wording and scale of the question, higher mean scores in the survey correspond to higher levels
of outcome observability. Consistent with the outcome observability measurement, the average
response scores for USAID, Ed, EPA, and DOT are low. The average response score for SSA and
NASA, which score high on the outcome observability scale, is high. Graphical representation of
the information is presented in Appendix 2-5.
The 95% confidence interval for most agencies overlap due to the small sample size. The
95% confidence intervals that accompany each sample mean to allow us to informally assess
whether a given between-sample difference is statistically significant at conventional alpha levels.
We say “informally” because although analysts sometimes make decisions about whether two
estimates are statistically different by inspecting whether their respective confidence intervals
overlap, this “overlap” method is more conservative than the standard method. That is, it rejects a
null hypothesis of no difference less frequently than the standard approach to hypothesis testing,
39
which uses a test statistic to make a determination of statistical significance (see, e.g., Cumming
2009). Consequently, if two sample means’ confidence intervals overlap, it could still be the case
that they are statistically different from each other according to the standard approach.

Table 2-5. 2007 GAO Survey of Public Managers

Number of
Respondents
Mean Score
Standard
Error
95% Confidence Interval
DOC 101 3.4 0.12 3.17 3.63
DOD 113 3.3 0.1 3.1 3.51
DOE 84 3.27 0.13 3.01 3.54
DOI 107 3.23 0.12 2.99 3.48
DOJ 86 3.41 0.13 3.15 3.67
DOL 108 3.19 0.13 2.94 3.45
DOS 87 3.03 0.13 2.77 3.3
DOT 94 2.77 0.13 2.51 3.02
ED 80 2.87 0.14 2.59 3.15
EPA 89 2.81 0.13 2.56 3.06
FAA 90 3.22 0.13 2.97 3.47
GSA 99 3.26 0.12 3.02 3.5
HHS 184 2.98 0.09 2.81 3.15
HUD 81 3.14 0.14 2.87 3.42
NASA 85 3.61 0.13 3.36 3.87
OPM 86 3.28 0.14 3.01 3.55
SBA 95 2.98 0.12 2.73 3.22
SSA 121 3.81 0.08 3.64 3.98
USAID 65 2.73 0.14 2.44 3.01
USDA 221 3.28 0.08 3.13 3.44
USDT 217 3.01 0.08 2.85 3.17
VA 106 3.24 0.11 3.01 3.46
Total 2,399 3.19 0.02 3.14 3.24

40
Figure 2-2 collapses the 22 sample agencies into their three superordinate groups as
determined by the observability coding and plots three pooled means. Now that each of these
pooled means is drawing on a larger number of subjects, their respective confidence intervals have
narrowed. A simple inspection of Figure 2-2 indicates that the pooled Coping/Procedural mean is
statistically distinguishable from the pooled Craft/Production mean. Additionally, the
Mixed/Undetermined pooled mean is substantively distinguishable from the pooled
Coping/Procedural means and our pooled Craft/Production means. A one-way ANOV A test was
conducted to examine whether the differences are statistically different. The results presented in
Appendix 2-6 reveal that [F(2, 2396) = 14.99, p=0.00], suggesting that variance in the three groups
is statistically distinguishable. However, post hoc pairwise Tukey tests imply that
Coping/Procedural and Mixed/Undetermined groups are statistically distinguishable only at 90%
level instead of the conventional 95% level (See Appendix 2-7 for details).
On balance, these results provide solid evidence for our measurement approach. The survey
responses suggest that managers in Coping/Procedural agencies generally face more challenges
than Production/Craft agencies in establishing a causal inference between program activities and
performance results. Its ability to distinguish between these groups using manager perceptions
suggests that the measurement based on GPRA goals is a valid instrument.

41
Figure 2-2. Superordinate Coded Agency Types by GAO Survey Means

2.5 Conclusion and Discussion
James Q. Wilson’s agency typology is a product of inductive reasoning based on his
observations of federal agencies. He used anecdotal accounts to explain key differences among the
four agency types he proposed. The inductive approach is useful to comprehend his core argument
that the observability of outcomes and outputs play a significant role in agency’s political and
organizational environment. However, in terms of generalizability of the typology beyond the time
frame and examples Wilson brought up in the book, more work remained to refine his “crude effort
(1989, p.159).”
This chapter is the first attempt to systematically provide empirical validation of the outcome
observability aspect of Wilson’s typology. The merit of the work derives largely from expansive
data collection efforts and qualitative coding for more than 20,000 federal agency performance
42
goals over a 10-year time period. As a result of rigorous data works, 28 federal agencies are placed
on a continuous scale of outcome observability. The outcome observability scores provided in this
chapter can be used by scholars to explore important questions regarding organizational and
political differences among the agencies.
The findings verify that the average outcome observability score for some agencies Wilson
used as an archetype of agency typology (e.g., SSA for “production” agency) aligns with his
expectations, while other examples he used (e.g., federal enforcement agencies as “procedural”
agency) did not. Moreover, we verify that the existence of the Mixed/Undetermined category.
Wilson expected that “many agencies do not fit its [the typology’s] categories,” but he did not
provide explanations on how operators, managers, executives, and political stakeholders in the
Mixed/Undetermined agencies would behave. We recommend that future works provide
quantitative and qualitative accounts of how the Mixed/Undetermined agencies operate.
For those who agree with Carpenter (2020), who finds it “nonsensical” to compare agencies
on a singular dimension using quantitative measurements because of the questions regarding the
measure’s reliability and validity, multiple tests were conducted to alleviate the problem. First,
based on Cohen’s Kappa score, the measure demonstrates a substantial level of agreement among
the coders, thus demonstrating the interrater reliability of the coding scheme used to determine
goal observability. In terms of validity, we achieved criterion validity by conducting a known-
group validity test. Using the observability measure as a reference point, we find that the proposed
measure is able to differentiate the survey responses from federal agency managers in three agency
groups. Moreover, the year-by-year analysis was conducted as a test-retest reliability check.
Although small sample sizes limit the statistical explanatory power in some agencies, overall
results suggest reliability of the measure over time.
43
Nonetheless, several limitations warrant caution in interpreting the results. First, the coding of
agency performance goals to determine outcome observability involves the coders' subjective
judgment. Even though we achieved substantial intercoder agreement, qualitative coding invites
inconsistencies and biases. Second, because data was not available across all agency-years. Third,
this paper assumes that all GPRA performance goals are equally important. Only after the
enactment of the GPRA Modernization Act (GPRAMA) of 2010 did the agencies identify agency
priority goals (APG). Because the data used in this paper only includes goals before the GPRAMA,
it is possible that the agency is classified as Production/Craft based on the average observability
score, even when its most important goals are unobservable. Future works that extend data
collection efforts can add more validity to presented measurement by applying weights to agency
APGs.
44
Appendix 2-1. Landis and Koch Kappa’s Benchmark Scale
Kappa Statistic Strength of Agreement
<0.0 Poor
0.0 to 0.2 Slight
0.21 to 0.4 Fair
0.41 to 0.6 Moderate
0.61 to 0.8 Substantial
0.81 to 1.0 Almost Perfect

Appendix 2-2. Comparison of Variance for Three Agency Types: One-Way ANOV A
Sum of Squares df Mean Square F Prob>F
Between
Groups
1422.455 2 711.227 3967.10 0.0000
Within
Groups
3757.561 20959 0.179 3757.561

Appendix 2-3. Tukey Post Hoc Test
Contrast
Standard
Error
T P>|t|
95% Confidence
Interval
Coping/Procedural vs.
Mixed/Undetermined
-.389 .007 -54.49 -0.00 -.406 -.372
Production/Craft vs.
Mixed Undetermined
.255 .007 35.97 -0.00 .238 .272
Production/Craft vs.
Coping/Procedural
.644 .007 88.48 -0.00 .627 .661

45
Appendix 2-4. Year-to-Year Analysis of Agency Outcome Observability

BBG

CPSC

DOC

DOD

DOE

DOI

46
DOJ

DOL

DOS

DOT

ED

EEOC

EPA

FAA

47
FERC

FTC

GSA

HHS

HUD

NASA

OPM

SBA

48
SEC

SSA

USAID

USDA

USDT

VA

49
Appendix 2-5. GAO Survey of Public Managers

Appendix 2-6. One-Way ANOV A based on GAO Survey
cccc Sum of Squares df Mean Square F Prob>F
Between
Groups
43.77 2 21.89 14.99 0.0000
Within
Groups
3499.25 2396 1.46

Appendix 2-7. Tukey Post Hoc Test of GAO Survey Results
Contrast Standard
Error
T P>|t| 95% Confidence
Interval
Coping/Procedural vs.
Mixed/Undetermined
-.12 .05 -2.15 0.08 -.25 .01
Production/Craft vs.
Mixed Undetermined
.29 .07 4.12 0.00 .13 .46
Production/Craft vs.
Coping/Procedural
.41 .08 5.47 0.00 .23 .59

50
Chapter 3
Presidential Appointees and Federal Agency Performance
6

3.1 Introduction
On December 21, 2020, the House Select Subcommittee on the Coronavirus Crisis issued
subpoenas to investigate the interference of political appointees in the Centers for Disease Control
and Prevention (CDC) 's response to Coronavirus. Amid the COVID-19 crisis, political appointees
at the Health and Human Services (HHS) and CDC are accused of modifying the Morbidity and
Mortality Weekly Reports (MMWR) and other scientific reports. The appointees are alleged to
have changed, delayed, and edited the reports in retrospect in fear of undermining public opinion
of the president (Select Subcommittee on the Coronavirus Crisis, 2020). One of the appointees,
the HHS spokesperson Michael Caputo, called a group of career scientists at CDC as a "resistance
unit" and accused them of "sedition" when the scientists tried to publish scientific reports based
on their research.
This incident shows how blindly loyal political appointees can be to the president who
appointed them. Even though what the HHS and CDC appointees did was contrary to the public
interests, they worked for the interests of the president. Studies show that is what presidents want
and expect of his appointees. Following the Jacksonian tradition of the spoils system, the
presidents of the United States have brought in their loyal patronage to the federal agencies. Since

6
This chapter is a modified version of a paper co-authored with Dr. William Resh.
51
Nixon appointed like-minded experts to federal agencies to "take on Congress and to take over
bureaucracy" (Nathan, 1983, p. 42), all modern presidents have aggressively used appointment
power as a means to advance their policy preferences. In December 2020, the United States
Government Policy and Supporting Positions, commonly referred to as the Plum Book, identified
a total of 7078 positions subjected to be filled in with political appointees.
7
Compared to the
approximately 2.2 million civilian workers in the federal bureaucracy, not including the military
and the contracted labor force, it is up to question how much these individual appointments actually
matter. However, the numbers compare almost 1-to-1 with the number of career members in the
Senior Executive Service, which is consisted of approximately 7200 personnel.
Under the American separation of powers system, presidents compete with Congress for
control over the bureaucracy. A central principle that drives the presidential use of political
appointees is, as an old adage says, "personnel is policy.” Presidents prefer those who are loyal
and responsive ("responsive competence") to professional bureaucrats who are insulated from
presidential control ("neutral competence") (Moe, 1985). West (2005, p. 147) points out that “it
may not be realistic to expect that civil servants can be nonpartisan and still satisfy the president’s
need for responsiveness.” For example, President Trump claimed to appoint the “very best people,”
but his definition of the “best people” is those who are responsive to his direction (Lewis &
Richardson, 2021). By placing trustworthy allies in the key positions, presidents seek to impose
centralized control over the bureaucracy so that their agenda is prioritized over the policy
preferences of the career bureaucrats, interest groups, and Congress.
It is not only the presidents who need strong support from the agencies, but the agencies

7
To exclude the Senior Executive Service “General”, the number of positions is 4,568.
52
also need the support of the president. In an interview with Aviation Week, Jim Bridenstine, who
served as the Administrator of the National Aeronautics and Space Administration (NASA) under
the Trump administration, expressed his intent to resign when then President-elect Biden would
take office. The reason is, he explains, "what you need is somebody who has a close relationship
with the president of the United States. You need somebody who is trusted by the administration….
including the OMB [Office of Management and Budget], the National Space Council and the
National Security Council, and I think that I would not be the right person for that in a new
administration" (Klotz, 2020). Bridenstine believed that NASA made good progress in the past
few years, and for the continued success of the agency, "[y]ou have to have those relationships.
Whoever the president is, they have to have somebody they know and trust and somebody the
administration trusts. That person is not going to be me"(Klotz, 2020).
Bridenstine's interview highlights the importance of having a close relationship with the
president for the agency's success. Does this mean that having more political appointees increases
the chance of agency success? According to James Q. Wilson, the answer depends on agency
outcome observability. Wilson argues that careerists will better manage agencies with highly
observable outcomes than appointees, while agencies with low outcome observability will benefit
from the leadership of political appointees. His proposition is partly in conflict with the literature
on appointee politics. Many studies affirm the detrimental effects of appointee leadership on
agency performance (Gallo & Lewis, 2012; Gilmour & Lewis, 2006; National Commission on the
Public Service, 2003).
The conflict between Wilson’s famous proposition and the politicization literature
provides motivation for this study. By accommodating agency typology established with more than
20,000 Government Performance and Result Act (GPRA) goals, this paper questions if Wilson’s
53
agency types modify the effectiveness of presidential appointments on the goal achievement. The
answers to the question provided in this article affirm Wilson’s expectations. Moreover,
discussions regarding how different levels of political appointments impact performance in
different agency types will contribute to our understanding of organizational performance and
appointee politics in the modern era of the administrative presidency.

3.2 Literature Review
A. Appointment Power of the President
The U.S. has a long history of the patronage system, which allows the winner of the
election to bring in supporters to the civil service. However, as policy issues grew complex, the
emergence of a government workforce that specializes in specific policy issues was inevitable.
Proponents of the politics-administration dichotomy, such as Woodrow Wilson (1887), insisted on
creating a civil service system based on merit to pursue a "science of administration." According
to Woodrow Wilson, "[a]dministrative questions are not political questions. Although politics sets
the tasks for administration, it should not be suffered to manipulate its offices (p. 210)."
The growth of professional bureaucracy in the 20
th
century posed a question about its
policymaking power. When politicians who need policy support from expert bureaucrats delegate
some of their policymaking authority to the bureaucracy, whose policy interests should the
bureaucrats pursue? Under the separation of powers system, this involves bargaining between the
executive and legislative branches. Neither presidents nor Congress is likely to defer readily to the
bureaucrats in bringing definition to the congressional delegation. To gain control over the
54
bureaucracy, the president will counter or complement the existing power structure of a given
agency with an array of strategic tools, such as budgeting, agency reorganization, executive orders,
and appointments, to more precisely define and prioritize the agency goals (Dickinson, 2005;
Durant & Resh, 2010; Lewis, 2008; Moe, 1989; Moynihan & Roberts, 2010). In other words, the
relative focus of organizational goals is potentially narrowed by the president's efforts toward
executive cohesiveness and coordination (Lewis, 2010; Rosenbloom, 1983).
The most prominent tool that presidents use to exert their influence on the bureaucracy is
the appointment power (Bertelli & Feldmann, 2007; Lewis, 2008; Resh, 2015). By appointing
those who have proven loyalty, ideological congruence, and support for the president’s policy
prerogatives to the civil service, presidents can control agency goals and course of actions to
achieve those goals (Moe, 1993). The appointment power of the president derives from Article II,
Section 2 of the Constitution:
"the President shall nominate, and by and with the Advice and Consent of the Senate, shall
appoint Ambassadors, other public Ministers and Consuls, Judges of the supreme Court,
and all other Officers of the United States, whose Appointments are not herein otherwise
provided for, and which shall be established by Law; but the Congress may by Law vest
the Appointment of such inferior Officers, as they think proper, in the President alone, in
the Courts of Law, or in the Heads of Departments” (U.S. Const. art II, § 2).
As written in the Constitution, some positions require "the Advice and Consent" of
Congress. So-called Presidential Appointments with Senate Confirmation (PAS) work closely with
the president to advance his policies. PAS are high-profile positions that include ambassadors, the
Cabinet-level secretaries and their deputies, and independent agencies' heads. In 2020, only 1118
positions out of 7078 positions subject to the noncompetitive appointment (15.8%) were PAS.
Successful appointee placement is ostensibly a legislative delegation because appointees are
answerable to the president only by the Senate's consent (i.e., confirmation) (Bertelli & Grose,
55
2011). To prevent the adverse selection and all other ex-post dilemmas of principal-agent
relationships, the Senate carefully examines nominees before it delegates powers to the
presidential appointee. Although presidential nominees are rarely rejected, interbranch
competition causes frequent delays in the Senate confirmation process to hinder a nominee's
confirmation and bargain with the president (Dull et al., 2012; Madonna & Ostrander, 2017; Shipan
& Shannon, 2003).
To avoid an onerous confirmation process, presidents seek to circumvent the Senate and
increase their autonomy in their personnel decisions. Because they have unilateral appointment
power over the mid-and low-level positions, presidents seek to increase their control over the
bureaucracy through positions that do not require Senate confirmation. In general, presidents
exercise their unilateral appointment power over the Senior Executive Service (SES) and Schedule
C (SC) positions. Although these positions comprise a substantially higher percentage of all
presidential appointees than PAS, the salience of these positions remains low. The appointments
are often considered “invisible” as they receive little to no attention from the press or scholars
(Lewis & Waterman, 2013).
The SES are high-level management positions classified as General Schedule grades 15
or above. People in SES are senior-level agency executives in managerial, supervisory, and policy
positions who work just below the top presidential appointees and supervise career bureaucrats.
SES coordinates the agency work to be aligned with political directions. While most of the SES
are career bureaucrats, some positions are open only to noncareer appointments (NA). There are
724 NA positions in 2020. The total number of NA cannot exceed 10% of the SES positions and
25% of the individual agency's SES position allocation. Also included in the SES appointees are
Limited Term and Limited Emergency appointment (TA), who serve for a short period of time (1.5
56
to 3 years) in the agency for a special project and leave.
Schedule C appointments (SC), which consist the largest category of appointees, are low-
to mid-level managers. There are 1566 SC positions listed in the 2020 Plum Book. They support
NA and other SES as "executive assistant," "special assistant," or "policy advisor" (Cohen, 1998).
SCs are rewarded to young people who work in the campaign. These invisible appointees have
little formal authority, but they have informal authority to influence the policy outcomes. In
addition, there are the Presidential Appointments without Senate Confirmation (PA) positions,
used mostly to fill up the White House staff positions. In 2020, there are 354 PA positions.
Studies find that presidents “layer” different types of appointees within the agency. As
shown above, a majority of presidential appointments are lower-executive, middle management,
and line staff positions that are unilaterally appointed by the president. Presidents enjoy the
flexibility of making appointments to these positions. Examining appointees of George W. Bush
and Barack H. Obama, Waterman and Ouyang (2020) found that it is unilateral appointments
through which presidents promote loyalty of the federal bureaucracy. Competency is less
considered for these positions as public attention is rarely drawn to them. For this reason, Cohen
(1998) criticized that politicization puts the government at risk of running by amateurs. This
finding is consistent with previous research that suggests the triumph of loyalty over competence
during the GWB presidency (Moynihan & Roberts, 2010).
On the contrary, the top-level leadership positions in most of the executive branch (and
many independent) agencies are filled subject to Senate confirmation of a president’s nominees
(PAS). PAS positions have the most authority in agencies, but the appointees may not be the
most loyal persons to the president. In line with Hollibaugh's (2015) findings that presidents face
57
trade-offs between ideology and agency performance when making PAS appointments,
Waterman and Ouyang (2020) found that presidents appoint the most competent candidates to
the PAS at the price of loyalty in order to win the Senate confirmation process.
Thus, in order to fully understand how appointees influence agency performance, both
levels of presidential appointees (PAS and the “invisible” appointments) need to be investigated.

B. Politicization and Agency Performance
Appointees "monitor bureaucratic activity and communicate the president’s vision to the
press and agency employees, clients, and stakeholders (Lewis, 2010, p. 7)” to run the agencies.
Given the influence of their positions over strategic planning and decisions on resource allocation,
it is expected that what appointees do in the agencies matters to agency performance. Previous
research on the effectiveness of political appointees in the management of government agencies
found that politicization is associated with diminishing management capacity (Light, 1995;
Mackenzie, 2002; National Commission on the Public Service, 2003; Peters, Kettl, & Milward,
1996) . Studies that used the Program Assessment Rating Tool (PART) scores found that programs
run by the appointees score lower on PART than the programs run by careerists (Gallo & Lewis,
2012a; Gilmour & Lewis, 2006; Lewis, 2007). Also, studies find that politicized agencies are less
responsive. They are slow in their responses to information requests from the members of Congress
and the public (Lowande, 2018; Wood & Lewis, 2017).
To explain why political appointees are less effective than careerists, scholars proposed
the following reasons. First, the average tenure of appointees is much shorter than career
58
bureaucrats. Heclo (1977) found that political appointees serve for two years on average. Because
the appointees come and go, Heclo called the American federal government a "government of
strangers." The average tenure of the appointees has not changed much from Heclo’s time.
Examining 2,000 PAS positions between 1989 and 2009, Dull, Roberts, Keeney, and Choi (2012)
found that the average tenure of PAS positions is about 2.8 years. The short tenure of the appointees
hinders their learning of the program and institutional knowledge. By the time they learn how the
agency operates, they leave the agency. Also, the brief length of their tenure limits appointee
perspectives to short-term goals rather than long-term strategic management (Dull, 2009). On the
other hand, careerists have the advantages of established networks and institutional knowledge to
implement agency goals.
Second, agency performance declines with more appointees because they change
organizational dynamics and employee behaviors. Many career bureaucrats enter the public service
with the hope to influence policy outcomes (Downs, 1964). When the president and career
bureaucrats do not share the same policy preferences, presidents place more appointees to control
the bureaucracy (Hollibaugh, 2014; Lewis, 2008). The increased political pressure limits the level
of policy discretion for career bureaucrats, so they lose incentives to invest in policy expertise
(Richardson, 2019). Trust and intellectual capital available in the organization (Resh, 2015) and
the innovative attitudes (Lapuente & Suzuki, 2020) decline with increased agency control. In turn,
increased agency control through politicization compels careerists to exit the agency. Bertelli and
Lewis (2013) found that careerists who perceive that appointees exert more influence on agency
policies than senior executives tend to express their intent to exit the agency within a year.
Richardson's (2019) example of the Environmental Protection Agency (EPA) illustrates that
Administrator Scott Pruitt's leadership, whose policy ideology differed vastly from the employees’,
59
dampened employee morale and instigated employee exodus. When professionals leave, their
policy expertise and connections are lost, thus contributing to diminishing agency performance.
Third, there is a selection problem in appointee positions. Because they are selected non-
competitively, there are no formal qualification standards for the appointee positions (Cohen,
1998). Studies show that political appointees, on average, have higher levels of education, more
work experience in private and nonprofit organizations, and a more varied work history than career
bureaucrats (Lewis, 2007). Although they have a good background, political appointees are
considered less effective in managing public organizations because they do not have previous
experience in the policy area or public organizations (Gilmour & Lewis, 2006). Furthermore,
Lewis (2009) explains that it requires different skill sets to run the campaign and run government
programs. Career professionals have strength because they have more direct agency experience
and longer tenure (Lewis, 2008), and they have subject matter expertise, institutional knowledge,
and relationships with internal stakeholders necessary to facilitate the policy work.
Nevertheless, another body of research provides findings that refute the adverse effects of
appointees on agency performance. First, not all appointments are made with the same standards.
Presidents distinguish qualified candidates with expertise and relevant experience from those who
need to be placed simply because of their political loyalty. Appointees with few credentials are
placed in positions that require little specific expertise. Hollibaugh, Horton, and Lewis (2014)
found that the appointments based on political reasons are placed in the agencies that do not pursue
the president’s high-priority policy agenda. If presidents make strategic appointment decisions that
take appointee qualifications into account, they can layer different types of appointees within an
agency to sustain or to improve agency performance. Krause, Lewis, and Douglas (2006) found
evidence from the state governments that it is critical to find the optimal balance between the
60
appointees and careerists at different levels of an organizational hierarchy in order to reduce biases
of each group and to maximize agency competence.
Moreover, political appointees bring their resources and skillsets that careerists generally
do not have. For example, outsiders bring new ideas and innovation to the bureaucracy (Nathan,
2000). Also, one apparent advantage that the appointees have over careerists is their political
resources. Appointees have a strong connection to external political stakeholders to garner support
for their strategic planning and policymaking efforts. Maranto's work (2005) contends (and even
the work evidencing the negatives of political leadership acknowledges) that "an important
component of agency leadership is political work such as strategic planning, policymaking, and
building relationships with key stakeholders." In other words, having political appointees in place
(as a delegation function of both the president and Congress) should positively impact the ability
of agencies to define and achieve a multiplicity of goals clearly. Thus, while leaders of federal
agencies may have to make a series of tradeoffs, the political nature of their positions ostensibly
offers a democratic dimension to leadership as emissaries of the sole elected office in the United
States executive branch. Indeed, the delegation function of appointments implies at least a
modicum of capacity to secure external (i.e., legislative) support that will contribute to the agency's
success.

C. Wilson's Agency Typology and Leadership
Then, under what conditions do political appointees help or hurt agency performance? One
way to answer the question is to consider the diversity of agency tasks and the environment.
Discussing the challenges of choosing agency leadership, James Q. Wilson (1989, p.200) points
out how business executives and other outside observers often "lament the fact that the "best
61
people” are not in charge in government." Wilson responds that this statement is only half-true; he
admits that many outsider experts improved the performance of government agencies, but there
were also cases when the outsiders brought more harm to the agency than good. The difference,
according to Wilson, is contingent upon agency outcome observability. Wilson made a proposition
that career leadership is more effective than appointees when agency outcomes are observable.
Conversely, Wilson contends that appointed leadership is more effective than careerist leadership
when agency outcomes are unobservable.
Production organizations, in which outputs and outcomes are visible to external
stakeholders, have relatively narrow goals focused on the efficient provision of quality services
and products. Because careerists know how internal operations work, careerists are better suited
to lead the agency, according to Wilson. Likewise, careerists are more qualified to run the craft
agencies than appointees because the inherent nature of the work at craft agencies requires highly
specialized knowledge. Employees tend to rely on their professional norms and the standard
operating procedures (SOPs) more than direction from management. To control the employees,
agency leaders need to understand the jargon, symbols, and culture of the professionals. Wilson
expects that agency outcome is more likely to be achieved when the agency is run by people who
share professional identity prevailing in the organization.
In coping agencies where ambiguities of agency mission and goals prevent the assessment
of agency outcomes, it is helpful to have many appointees in leadership positions to acquire
external support. In production and craft organizations, as long as the agencies produce “good”
performance in the eyes of the stakeholders, bureaucrats enjoy more autonomy. However, coping
and procedural agencies face challenges in proving their contribution to making visible
improvements in society. Moreover, goals and means to achieve the goals in coping and procedural
62
organizations are highly controversial. Without political support, the ambiguities of the tasks and
mission put Coping/Procedural agencies in jeopardy of constant external pressure. Appointees who
have connections can communicate goals and challenges to the external stakeholders and induce
their acceptance of agency goals and performance. A deft example is provided by Corrêa
d’Almeida and Klingner (2008), whose case study of effective leadership focuses on political
appointee James Witt (Director of the Federal Emergency Management Agency (FEMA)) during
the Clinton administration. FEMA, according to Corrêa d’Almeida and Klinger (2008), is a coping
agency and had a long history of poor performance. However, its performance improved under
Witt as he was able to secure the agency’s autonomy with his political skills.
Following Wilson's precept that careerists may be more effective managers in agencies
with more observable outcomes, we expect that politicization will mollify the impact of
observability on performance in Production and Craft type agencies and increase performance in
Coping and Procedural agencies. Hence, within agency types, we predict differential performance
outcomes as a function of politicization. In addition, we separate the effects based on the type of
political appointments by dividing the hypothesis into 1) SES and SC appointments and 2) PAS
appointments. Based on Wilson’s propositions, we propose the following hypotheses:

1. Politicization through SC and NA Appointments
Hypothesis 1.1: Increased influence of SC and NA appointees in Production/Craft agencies will
be associated with diminished performance.
Hypothesis 1.2: Increased influence of SC and NA appointees in Coping/Procedural agencies will
63
be associated with increased performance.
2. P AS Appointments
Hypothesis 2.1: Increased influence of P AS appointees in Production/Craft agencies will be
associated with diminished performance.
Hypothesis 2.2: Increased influence of P AS appointees in Coping/Procedural agencies will be
associated with increased performance.

3.3 Data and Measures
A. Dependent Variable
The dependent variable is agency performance. The GPRA goals are drawn from the
Performance and Accountability Reports (PARs), which are described in detail in Chapter 1.
Agency performance is measured as the likelihood of individual goals being achieved.

B. Independent Variables
Politicization. Politicization refers to the "increasing number and penetration of
appointees" within agencies (Lewis, 2007, p. 2). Specifically, politicization is used to measure
the penetration of presidential appointees in low and mid-level management positions, thus
excluding PAS. Data is drawn from the Office of Personnel Management's Central Personnel
Data File (CPDF) records. Guided by the previous studies (e.g., Dull et al., 2009; Lewis, 2008,
64
Resh, 2015), this paper expresses politicization as a ratio of political appointees in SC. and SES
to the career SES. In other words, the total number of SC, NA, and TA was divided by the total
number of career SES in each agency, as expressed:
𝑃𝑜 𝑙𝑖𝑡𝑖𝑐𝑖𝑧𝑎𝑡𝑖𝑜𝑛 =
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 (𝑆𝑐 ℎ𝑒𝑑𝑢𝑙𝑒 𝐶 +𝑁𝑜𝑛𝐶𝑎𝑟𝑒𝑒𝑟 𝑆𝐸𝑆 +𝐿𝑖𝑚𝑖𝑡𝑒𝑑 𝑇𝑒𝑟𝑚 𝑆𝐸𝑆 )
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝐶𝑎𝑟𝑒𝑒𝑟 𝑆𝐸𝑆 [3.1]

PAS Appointee Vacancy. In addition to the lengthy Senate confirmation, various factors,
such as frequent position turnover (Chang, Lewis, and McCarty, 2001) and the president’s strategic
choice to sustain vacancies (Hollibaugh, 2015; Kinane, 2019; Resh et al., 2020), make it inevitable
that some PAS positions remain vacant. The Federal Vacancies Reform Act of 1998 (Vacancies
Act), which defines the procedures for filling appointee vacancies, allows 210 days for the
president to unilaterally fill in the vacant positions with acting officers, with the exception of the
incoming president who gets 300 days. If the first or second presidential nominee for the position
is rejected by the Senate or withdrawn, the acting officer gets additional 210 days until the position
is filled.
When there is no vacancy in the agency PAS positions (which is not a realistic
assumption), the agency’s top leadership is led entirely by political appointees, thus limiting the
influence of careerists in key policy decisions and agency operations. On the other extreme, if all
appointee positions are vacant, acting officials would temporarily hold the office. However, acting
officials who serve in the appointees' positions do not hold the full authority of the office (Dull &
Roberts, 2009). Unless their attempts to perform a “function or duty of a vacant office” were
specified in the Vacancies Act, the decisions of the acting officials have “no force or effect.” When
65
the appointee positions are vacant, agencies delegate much of the decision making authorities to
careerists --- “if a duty [of the acting official] is delegable, one can say that it may be performed
by other official and therefore is not a duty that may be performed “only” by the officer in the
vacant office (Congressional Research Service, 2020, p. 7).” Moreover, agency-specific statutes
can further restrict the authority of acting officers. Thus, with increased appointee vacancies,
careerists exercise a great amount of discretion.
A detailed longitudinal roster of PAS positions and the officials who occupy them was
acquired through formal requests to the Office of Personnel Management and Government
Accountability Office. Additional data was obtained through the Senate nominations database
8
,
Congressional Research Service reports, news coverage accessed by LexisNexis, and online
sources such as the GAO's Federal Vacancies Act Resources website
9
. The dataset used in this
paper stands as the most comprehensive account of appointee vacancies in the United States.
In addition to the vacant positions filled by acting officials, some of the PAS positions
were excluded from the dataset: U.S. attorney and U.S. marshal positions in the Department of
Justice; Foreign Service and diplomatic positions in the Department of State; officer corps
positions in the civilian uniformed services of the National Oceanic and Atmospheric
Administration in the Department of Commerce, and of the Public Health Service in the
Department of Health and Human Services; and the officer corps in the military services.
To account for the differential power of a given appointee position and the varying number
of appointees at a given level across and within organizations, we derive a Vacancy Index that

8
Available at Thomas.gov
9
https://www.gao.gov/legal/other-legal-work/federal-vacancies-reform-act
66
standardizes vacancies expressed as follows:
𝑉𝐼
𝑑 = ln (∑
𝑣 𝑘 𝑑 𝑛 𝑘 𝑑 𝑁 𝑘 𝑘 =1
∗ ln(𝐸𝑆
𝑘 𝑑 )) [3.2]

where 𝑉𝐼
𝑑 is vacancy index at agency d (d= 1, …, 21), 𝑣 𝑘 𝑑 is total vacancy days at level (k) in a
given agency d, 𝑛 𝑘 𝑑 is the number of appointees at level (k) in a given agency d, ES is Executive
Schedule (ES) pay, and k = 1, …, Nk (= the number of levels).

C. Control Variables
The following variables are included in the analysis to control the effects of variables that
influence agency performance. The first group of variables is related to the agency’s political
environment: agency independence, president-agency ideological distance, and presidential
attention. The second group of variables is related to the agency’s organizational characteristics:
agency size; percent of employees who are classified as clerical, professional, or technical;
organizational age; and prioritization ambiguity, which measures "the level of interpretive leeway
in deciding on priorities among multiple goals" (Chun & Rainey, 2005, pg. 4). For more
information, see descriptive statistics in Table 3-1.

D. Moderating Variables
Agency typology based on outcome observability is included as a moderating variable.
Chapter 2 provided a detailed account of the measurement methodology and results. Based on the
categories made in Chapter 1, we use three agency categories as moderating variables:
67
Production/Craft (for the agencies with average observability score less than .35),
Mixed/Undetermined (for the agencies with average observability score between .35 and .65), and
Coping/Procedural (for the agencies with average observability score greater than .65).

68
Table 3-1. Descriptive Statistics
Variable Description Obs Mean
Std.
Dev.
Min Max
Level 1 Variables
Goal Achievement 1 (met); 0 (unmet) 17,256 0.67 0.47 0.00 1.00
Goal Observability
1 (observable);
0 (unobservable)
17,256 0.50 0.50 0.00 1.00
Level 2 Variables
Vacancy ln (∑
𝑣 𝑘 𝑑 𝑛 𝑘 𝑑 𝑁 𝑘 𝑘 =1
∗ ln(𝐸𝑆
𝑘 𝑑 )) 17,256 10.17 1.06 6.60 11.54
Politicization
(Sched C + NCSES + Ltd
Term)/Career SES
17,256 0.54 0.50 0.00 2.46
Prioritization Ambiguity
As derived by Chun and
Rainey (2005)
17,256 0.60 0.91 -1.07 3.50
Agency Independence As derived by Selin (2015) 17,256 -0.24 0.27 -0.69 0.62
President-Agency
Ideological Distance
As derived by Chen and
Johnson (2015) agency
ideology scores and DW-
NOM presidential scores
17,256 0.12 0.06 0.00 0.30
Presidential Attention As derived by Lewis (2011) 17,256 2.95 0.93 0.00 5.16
Agency Size
Natural log of agency’s FTEs
during FY
17,256 10.14 1.09 7.71 12.67
Percent Clerical
% of FTEs who have
“clerical” classification by
OPM system
17,256 0.07 0.06 0.01 0.25
Percent Professional
% of FTEs who have a
“professional” classification
by OPM system
17,256 0.31 0.15 0.04 0.68
Percent Technical
% of FTEs who have a
“technical” classification by
OPM system
17,256 0.12 0.09 0.02 0.45
Organizational Age
Natural log of the agency's
age.
17,256 4.23 0.75 2.56 5.40
Wilson Agency Types
Craft/Production
Average goal observability
equal to or smaller than .35
6,656
Coping/Procedural
Average goal observability
equal to or greater than .65
3,924
Mixed/Undetermined
Average goal observability
between .35 and .65
6,676

69
E. Modeling
Because the dependent variable is binary, I use logit regression to measures the chance of
goal achievement. In particular, I use multilevel logit regression modeling. Multilevel modeling is
appropriate when hierarchies exist in the data structure. When observations are clustered within
units, error terms among observations that belong to the same unit are inevitably correlated with
each other, thus violating the assumption of independence (Raudenbush et al., 2002). Because the
conventional regression model fails to capture the correlation among individual-level
observations, it inflates standard errors, which can cause Type 1 error (Steenbergen & Jones, 2002).
Multilevel modeling is useful as it adjusts standard errors to account for clusters. It allows
researchers to determine how higher-level clusters affect lower-level observations (S. Park & Lake,
2005; Raudenbush et al., 2002). In this paper, because individual performance goals are nested
within the agency, the goals share institutional characteristics of their respective agencies. The
assumption of independence is violated due to the shared institutional environment. Therefore, it
is necessary to employ multilevel modeling.
Level 1 (individual goals) equation includes goal observability and performance. Level 2
(agency) equation includes the main variable of interest and control variables, along with agency-
level intercept and error term. In addition, I add the year as a dummy variable. In the random part
of the equation, the agency is used as a cluster variable. Because goal observability differs across
agencies, goal observability is included as a random coefficient to allow each agency to have a
distinct slope.
Level 1 (Goal Level):
𝑃𝑒𝑟𝑓 𝑖𝑗
= 𝛽 0𝑗 + 𝛽 1𝑗 𝑂𝑏𝑠 𝑖𝑗
+ 𝑟 𝑖𝑗
[3.3]
70
Level 2 (Agency Level):
𝛽 0𝑗 = 𝛾 00
+ 𝛾 01
∗ 𝐴𝑝𝑝𝑜𝑖𝑛𝑡𝑒𝑒 𝑗 + 𝑢 0𝑗 [3.4]
𝛽 1𝑗 = 𝛾 10
+ 𝑢 1𝑗 [3.5]
Where 𝑃𝑒𝑟𝑓 𝑖𝑗
is the goal performance, 𝛾 00
is the average intercept, 𝛾 10
is the average slope for
each agency’s goal observability, 𝑂𝑏𝑠 𝑖𝑗
is goal observability, 𝐴𝑝𝑝𝑜𝑖𝑛𝑡𝑒𝑒 𝑗 is either politicization
or PAS Vacancy, 𝑢 1𝑗 is the agency slope for observability, 𝑟 𝑖𝑗
is unique goal effect, and 𝑢 0𝑗 is
unique agency effect.

3.4 Results
The effects of increasing politicization on goal achievement are shown in Table 3-2. The
results are expressed in the odds ratio. Model 1 does not consider agency types, while Model 2
includes agency typologies as moderating variables. Because agency type was used as an
interaction term, the Mixed/Undetermined agencies serve as a baseline. The coefficients of the
Coping/Procedural and Production/Craft agencies need to be interpreted in comparison to the
baseline group. The intraclass correlation coefficient (ICC) for Model 1 is 0.228, indicating that
22.8% of the variance is explained by agency effects. The ICC increases to .36 when agency types
are included, as in Model 2.

71
Table 3-2. Politicization and Agency Performance (in Odds Ratio)
Model 1 Model 2
Politicization 1.615** 0.532*
(0.289) (0.138)
Coping/Procedural

0.160*

(0.115)
Production/Craft

1.001

(0.874)
Coping/Procedural * Politicization

8.633***

(3.142)
Production/Craft * Politicization

0.722

(1.190)
Goal Observability 1.189 1.196+
(0.125) (0.125)
Agency Independence 1.778 0.717
(1.166) (0.644)
Ideological Distance 0.560+ 0.853
(0.190) (0.303)
Prioritization Ambiguity 0.760*** 0.745***
(0.0318) (0.0321)
Presidential Attention 1.020 0.999
(0.0478) (0.0478)
Organizational Age 1.473 1.400
(0.414) (0.506)
Agency Size 0.907 0.766
(0.135) (0.133)
Percent Professional 26.25*** 44.89**
(21.54) (56.45)
Percent Technical 1.209 1.088
(1.200) (1.117)
Percent Clerical 24.61+ 43.88+
(42.57) (88.32)
Constant 0.219 2.318
(0.345) (4.757)
Level 2 Variance
Goal Observability 0.185 0.171
(0.075) (0.066)
Constant 0.968 1.858
(0.325) (0.734)
N Observation 17255 17255
N Groups 21 21
Chi-Square 143.75 179.94
Log Likelihood -9869.648 -9846.203
ICC 0.228 0.36
Exponentiated coefficients; Standard errors in parentheses
+ p<0.1; * p<0.05; ** p<0.01
Note: the year dummy variable is not included in the table.

72
According to Model 1, politicization increases the log odds of goal achievement. It
contradicts findings from the previous studies that used the PART measure (e. g. Gallo & Lewis,
2012; Gilmour & Lewis, 2006). However, when the politicization variable is interacted with the
agency type variable as in Model 2, we see that its effect on goal performance is contingent on
agency type. The log odds ratio of politicization is .532, which indicates a declining log odds of
goal achievement with increased politicization in the Mixed/Undetermined agencies. The
coefficient for the Production/Craft agency type is not statistically distinguishable from the
Mixed/Undetermined, thus suggesting a diminishing performance with increasing levels of
politicization. However, in Coping/Procedural agencies, one unit increase in politicization
improves the log odds of performance by 8.63 when compared to the log odds in the
Mixed/Undetermined category.
To assist interpretation of the results, Figure 3-1 provides a graphical representation of
Model 2. The likelihood of goal achievement in Coping/Procedural agencies steadily grows with
the increase in politicization. The chance of goal achievement when politicization is at its
maximum value at 2.46 is statistically higher than its minimum value at 0, as indicated by their
95% confidence intervals. For Mixed/Undetermined and Production/Craft agencies, the odds of
goal achievement show a downward trend as the level of politicization increases. However, for
Production/Craft agencies, the confidence interval for the goal achievement gets too wide as the
level of politicization increases.
73
Figure 3-1. Politicization and Agency Performance

It is notable that the percentage of professional employees is significant in both models.
One unit increase of professionalism increases the log odds of goal achievement by 26.25 in Model
1. When the agency typology is accommodated, professionalism increases the log odds of goal
achievement by 44.89, as shown in Model 2. Because SES and SC appointments are made
primarily based on loyalty rather than competence (Hollibaugh et al., 2014; Waterman & Ouyang,
2020), the findings indicate that the support of professional staff who possess policy expertise and
institutional knowledge is critical to the achievement of agency goals.
Table 3-3 presents the association between PAS Vacancy and the chance of goal
achievement. As with Table 3-2, all coefficients are expressed in the odds ratio. The ICC for Model
74
3 is .187 and .111 for Model 4. Model 4 includes agency interaction, with Mixed/Undetermined
group being the baseline, while Model 3 does not include any agency types.
The regression results represented in Model 3 denote that PAS Vacancy has no discernable
effect on the likelihood of goal achievement. However, when Model 4 illustrates the breakdown
of the effects of PAS Vacancy by agency types, it becomes clear that PAS Vacancy is associated
with goal performance. In Model 4, the log odds ratio coefficient for the PAS Vacancy variable is
0.858 and significant at a 95% significance level, suggesting the negative relationship between the
chance of goal achievement and PAS Vacancy. Compared to the Mixed/Undetermined agency
category, one unit of PAS Vacancy increases the log odds of goal achievement by 1.34 in
Production/Craft agencies.

75
Table 3-3. PAS Vacancy and Agency Performance (in Odds Ratio)
Model 3 Model 4
PAS Vacancy 0.954 0.858*
(0.0386) (0.0512)
Coping/Procedural

1.058

(0.925)
Production/Craft

0.222+

(0.180)
Coping/Procedural * PAS Vacancy

0.965

(0.0786)
Production/Craft * PAS Vacancy

1.337***

(0.102)
Goal Observability 1.190 1.222+
(0.126) (0.139)
Agency Independence 1.257 0.705
(0.743) (0.266)
Ideological Distance 0.550+ 0.488*
(0.187) (0.168)
Prioritization Ambiguity 0.766*** 0.765***
(0.0319) (0.0316)
Presidential Attention 1.032 1.041
(0.0482) (0.0479)
Organizational Age 1.423 1.606*
(0.365) (0.329)
Agency Size 0.868 0.876
(0.120) (0.0984)
Percent Professional 15.30*** 3.376+
(11.26) (2.136)
Percent Technical 1.386 1.024
(1.357) (0.997)
Percent Clerical 41.84* 269.5***
(70.18) (427.7)
Constant 0.825 1.609
(1.207) (1.797)
Level 2 Variance

Goal Observability 0.187 0.243
(.755) (.101)
Constant 0.755 0.412
(.241) (.129)
N Observation 17255 17255
N Groups 21 21
Chi-Square 140.68 190.91
Log Likelihood -9872.633 -9852.61
ICC 0.187 0.111

Exponentiated coefficients; Standard errors in parentheses
+ p<0.1; * p<0.05; ** p<0.01
Note: the year dummy variable is not included in the table.
76
The graphical illustration of Model 4 is presented in Figure 3-2. The only graph with an
upward trend is for Production/Craft agencies. Graphs for Mixed/Undetermined and
Coping/Procedural agencies show a downward trend. The results correspond to the hypothesis that
predicted positive performance for increased PAS Vacancy in the agencies with high outcome
observability and negative performance for decreased PAS Vacancy in agencies with low outcome
observability. It is interesting that performance declines with increased PAS Vacancy in the
Mixed/Undetermined category because performance decreased with increased politicization in
those agencies.

Figure 3-2. PAS Vacancy and Agency Performance

77
It is noteworthy that the Priority Ambiguity variable is significant at a 95% significance
level across all four models. This finding underscores the importance of goal clarity in public
organizations. When agencies set multiple strategic objectives and performance measures, it is
challenging to determine agency resources and staffing decisions (Chun & Rainey, 2005b).

3.5 Conclusion and Discussion
This article examined the relationship between the increasing influence of presidential
appointees and agency performance. This article provides an in-depth analysis of the topic by
incorporating three agency types based on outcome observability (Production/Craft,
Mixed/Undetermined, and Coping/Procedural). Without considering agency typology, findings
suggest that politicization improves goal performance and PAS vacancy does not affect
performance, which contradicts the findings from existing literature. However, once agency
typology was added to the analysis, the results are adjusted to meet Wilson’s expectations. A
decrease in politicization and increased PAS Vacancy are associated with diminished performance
in Production/Craft agencies. Conversely, increased politicization and a decrease in PAS Vacancy
are associated with improved performance in Coping/Procedural agencies.
Interestingly, increased politicization hurts performance in Mixed/Undetermined
agencies, but at the same time, PAS Vacancy does not help their performance. Given that agencies
that fall into the Mixed/Undetermined agencies are generally large agencies with multiple goals
and varied missions, they are expected to face high goal ambiguities. Because PAS appointees can
define and prioritize organizational goals according to the agencies’ political stakeholders, the
Mixed/Undermined agencies can benefit from the increased influence of PAS appointees in the top
78
executive leadership. However, the findings suggest that it may take a different set of skills to
deliver multiple and conflicting goals. An increase in NA and SC appointees relative to career
bureaucrats may decrease performance as they lack the skills and knowledge necessary to handle
the challenges.
Previous studies suggested that presidents consider factors such as competency of the
candidates, the ideological distance between the president and agency, and the importance of
agency policy agenda to the president when making strategic appointment decisions (Hollibaugh,
2015; Hollibaugh et al., 2014; Lewis, 2009). The findings of this study hint that presidents should
also consider agency types to account for different agency tasks and constraints. (Lewis, 2009).
One limitation of the study lies in the potential bias in the data. Because political
appointees, especially PAS appointees, are deeply involved in setting the GPRA goals, there is a
possibility of an endogenous relationship between GPRA goal achievement and appointee
influence. In other words, if GPRA goals are designed to reflect the preferences of political
appointees, the chance of goal achievement may increase with the increasing influence of political
appointees. We recommend that future studies adopt the agency typology to PART data and test
Wilson’s proposition. PART evaluation is conducted by the OMB staffs, which introduces the
political bias of the administration (Gilmour & Lewis, 2006). Comparing results from PART and
GPRA data will provide interesting accounts of how biases embedded in different dataset predicts
performance with politicization.

79
Chapter 4
Full Range of Leadership Theory, Organizational Performance, and
Wilson’s Agency Typology

4.1 Introduction
One of the numerous contributions that James Q. Wilson’s book Bureaucracy (1989) made
to the public administration literature is recognizing the differences in government agencies. In the
preface of the original edition, Wilson declares that the purpose of his book is to “persuade the
reader that bureaucracy is not the simple, uniform phenomenon it is sometimes made out to be
(p.xvii).” Throughout the book, Wilson not only discusses the differences among various
government agencies but he also addresses the differences among the executives, managers, and
operators in terms of their roles, interests, constraints, and opportunities. Wilson emphasizes that
organizations matter, as well as how people inside of them behave. In discussing the role of public
managers, he admits that inducing operator compliance in government is strenuous work. Wilson
highlights that agencies must differ with respect to managing compliance based on the
observability of outcomes and outputs.
In production agencies, where both outputs and outcomes are readily observable, it is
feasible to define what good performance is. For example, the performance of mail carriers
working for the USPS can be evaluated by the number of mails delivered every day. Managers can
set clear performance goals for operators and monitor, track, and rank employee performance. To
encourage employees to reach high-performance goals, managers are encouraged to use a rewards
80
system in production agencies. Managers who cultivate a culture of performance by providing
extrinsic rewards face fewer challenges. However, Bertelli (2007) studied the Internal Revenue
Service (IRS), which Wilson claims to be a production agency, and found that using a contingent
rewards system can increase stress levels and turnover intention of employees when managers
overly emphasize meeting the performance goals.
In craft organizations, outcomes are observable while outputs are not. Employees are
professionals whose job requires high levels of skills, expertise, and discretion. According to
Wilson, professionals are those who “receive some significant portion of their incentives from
organized groups of fellow practitioners located outside the agency. The behavior of a professional
in a bureaucracy is not wholly determined by incentives controlled by the agency (1989, pg. 60).”
They are relatively decentralized and procedurally self-regulating. In other words, the behavior of
professional employees is largely dictated by the interests of their own professional groups.
Managers in craft organizations have little direct control over operators’ daily routines, but they
can ensure the quality of work by emphasizing professionalism. However, while professional
ethics prevent employees from shirking and enhance performance (Miller & Whitford 2016),
professionalism can become an obstacle to management. Employees in craft organizations may
not act in accordance with agency norms and culture when it is in conflict with their reference
group. Especially when the agency is composed of several professional groups, managers face the
challenge of coordinating different professional goals and norms of the employees. Thus, to
prevent professional balkanization, managers should constantly remind employees of a shared
mission to ensure their commitment to performance and induce collaboration among different
groups (Durant, 2000).
Managers in procedural organizations can observe employee activities but cannot measure
81
their work outcomes, so they try to control the work process to enhance productivity. Due to the
anxiety that employee work outputs are not visible. Managers in procedural organizations tend to
prioritize compliance with rules, procedures, and standard operating procedures (SOP). They use
continuous surveillance to ensure that employees follow the rules and procedures, and as a result,
employee morale suffers. Studies have found that red tapes are associated with increased turnover
intention (Brewer and Walker 2010), decreased job satisfaction (Moynihan and Pandey, 2007), and
decreased public service motivation (Giauque, Ritz, Varone, & Anderfuhren-bidget, 2012).
Organizational performance declines with increased employee perception of red tapes (Brewer &
Walker 2010; Campbell, 2019; Kaufmann, Taggart & Bozeman 2000; Jacobsen & Jacobsen 2020;
Van loon, 2017). Therefore, Wilson recommends that managers resist their inclination to
micromanage and allow for employee autonomy and creativity to lessen the emphasis on rules and
procedures. In addition, Wilson suggests that managers in procedural agencies should actively
support employee development. If managers are assured that employees possess the right skills
and knowledge to perform the job without the help of the SOPs, it reduced their need for
micromanagement.
In coping organizations, Wilson argues, “effective management is almost impossible.”
Neither employee activities nor agency outcome can be observed. Managers have little means to
control what and how employees are operating, nor can they determine the success of employee
work outcomes. If managers implement management practices that are successful in production
organizations, the tension between managers and employees escalates. Therefore, trust in
management is critical in coping agencies.
Wilson never used the word leadership in his book to describe management strategies.
Indeed, he used the word only five times in the book in order to describe a position (i.e., “careerist
82
leadership” on pg. 201). Still, his propositions on how managers need to approach employee
motivation overlap with the findings from public leadership literature. Leadership and
management are distinct concepts, yet there is a substantive overlap between them (Bass, 2010).
In comparing management and leadership, Kotter (1990) argues that management focuses on
planning and budgeting, organization and staffing, and controlling and problem-solving. On the
other hand, leadership focuses on establishing direction, aligning people, and inspiring followers.
According to Lunenburg (2011), the key difference between managers and leaders is that managers
focus on tasks, whereas leaders focus on people. Northouse (2018) defines leadership as “a process
whereby an individual influences a group of individuals to achieve a common goal” (p.3).
In this vein, because his propositions on managerial strategies appropriate for each agency
category are focused on motivating operators to achieve organizational goals, Wilson’s discussion,
in essence, is about leadership. In particular, Wilson’s propositions resemble the core ideas of
transformational leadership and transactional leadership. Transactional leadership accomplishes
organizational goals by providing contingent rewards to high-performing employees, while
transformational leadership encourages employees by inspiring their intrinsic motivation.
This paper examines the link between the full range of leadership (transformational and
transactional) and organizational performance. The existing literature on the topic provides mixed
results (See section 4.2 for further explanations). Therefore, this article contributes to the literature
by providing solid evidence from large-scale longitudinal performance data of federal government
agencies. In doing so, it tests the validity of James Q. Wilson’s propositions and suggests
leadership fit for each of the three agency typologies devised based on outcome observability.

83
4.2 Literature Review
A. Full Range of Leadership Theory
The full range of leadership theory (FRL, hereafter) (Bass, 1985; Burns, 1978) suggests
three distinct approaches to leadership: laissez-faire, transactional, and transformational. Bass
(1996) argues that laissez-faire is the least effective style of leadership, while transformational
leadership is the most effective.

Table 4-1. The Full-Range of Leadership Model
Laissez Faire Transactional Leadership Transformational Leadership
Non-leadership
• Management by Exception
(MBE)
- Passive MBE
- Active MBE
• Contingent Rewards
• Individual Consideration
• Intellectual Stimulation
• Inspirational Motivation
• Idealized Influence

Laissez-Faire. The laissez-faire approach, which translates to hands-off, represents the
absence of leadership. People assigned to leadership positions do not take responsibility. They
avoid making decisions, which causes delays in operation. When serious conflicts that require the
intervention of the leadership arise, laissez-faire leaders avoid the situation. They are not interested
in the growth of followers nor the organization. In other words, laissez-faire leadership is
characterized by ignorance and negligence of the management. Literature provides evidence that
laissez-faire is associated with destructive effects, such as increased employee stress, decreased
employee motivation, and poor organizational outcomes (Buch et al., 2015; Judge & Piccolo,
2004; Skogstad et al., 2007; Yammarino et al., 1993). Recent literature suggests a positive aspect
of this non-leadership behavior as it promotes empowerment, autonomy, and innovation of
84
followers (Ryan & Tipu, 2013; Yang, 2015), yet they clarify that positive outcomes are achieved
only when the non-involvement is a strategic choice. Otherwise, laissez-faire is a risky choice
because followers may perceive management as passive or apathetic about follower development
(Wong & Giessner, 2016).

Transactional Leadership. Transactional leadership is characterized by a social exchange
between leaders and followers. According to the transactional leadership model, the relationship
between leaders and followers is based on a contractual one. At the core, transactional leaders
intrigue the self-interests of followers through pay raise, promotions, recognition, job security, and
other extrinsic motivation tools in order to induce high performance
There are three components of transactional leadership. Management by Exception (MBE)
is a strategy that leadership uses to intervene in follower operation. Passive MBE is similar to
laissez-faire. Leaders wait until problems arise. The leader’s role is to punish the followers who
deviate from rules and procedures. Because leadership interaction with followers is limited to
disciplinary actions, there is fear associated with the leadership. On the other hand, active MBE
emphasizes preventive actions. Leaders continuously monitor follower performance and provide
feedback. Lastly, contingent rewards induce compliance through incentives that are appealing to
followers. When followers meet or exceed performance expectations, they are rewarded with
tangible benefits; when their performance falls under the standards, followers get sanctions (Bass
and Riggio, 2006).

85
Transformational Leadership. Transformational leadership brings organizational change
through the employees who are committed to the vision and values of the organization (Wright et
al., 2012; Wright & Pandey, 2010). The key assumption of transformational leadership is that
leaders can place a strong influence on followers’ beliefs, assumptions, and behavior. As a result
of their interaction with transformational leadership, followers transcend their self-interests for
larger organizational and societal interests (Moynihan, Pandey, et al., 2012). Transformational
leadership emphasizes the leader’s role to reinforce the organizational vision and stimulate positive
changes to create lasting impacts (Bass, 1985; Burns, 1978).
Transformational leadership is composed of four behavioral dimensions. First, individual
consideration consists of promoting followers’ growth through coaching and mentoring.
Transformational leaders are good listeners as they invest time in understanding each follower’s
unique situation. Through close observations and communication, they identify each individual’s
talents, needs, and goals. Leaders with high social intelligence are successful because their skills
to empathize with followers allow them to build an interactive relationship that promotes mutual
trust.
The second dimension is intellectual stimulation. Transformational leaders refuse the status
quo. They challenge old assumptions and ask followers to see the problem from different
perspectives. Instead of relying on rules and traditions, transformational leaders invite followers
to suggest innovative solutions. Kotter (1990) argues that “the essential function of leadership is
to produce adaptive or useful change.” Leaders promote horizontal communication where ideas
are exchanged. Good ideas are selected regardless of complex organizational hierarchy and rules.
The open environment allows followers to enjoy autonomy and active participation in the decision-
making process.
86
The third dimension is inspirational motivation. Transformational leaders are highly
skilled at articulating clear and compelling visions, and they are also skilled at communicating the
visions to followers (Jensen et al., 2018). They present the picture of an ideal future that followers
can attain if they work in accordance with the leadership’s vision. Transformational leaders
persuade the importance of followers’ contribution to goal achievement, and they help followers
internalize the organizational values. Thus, while transactional leadership motivates followers with
a rewards system that fulfills followers’ self-interests, transformational leadership enables
followers to transcend their immediate interests for the organizational mission and collective goals.
This aspect of transformational leadership is important in public organizations where followers
must strive beyond their self-interests to serve the public (Wright & Pandey, 2010). Not only do
organizational leaders hire people who are motivated to help others, but they also provide
appropriate training to motivate people to do so. In this vein, research finds that inspirational
motivation is strongly associated with fostering public service motivation of the employees (Jensen
& Bro, 2018; Krogsgaard, Thomsen & Andresen, 2014; Park & Rainey, 2008; Wright et al., 2011).
The last dimension of transformational leadership is the idealized influence.
Transformational leaders are trusted and respected by their followers. They demonstrate qualities
that followers want to emulate. They are trusted and respected by their followers. In his 1985 book,
Bass described idealized influence as the extraordinary quality of leadership that attracts followers.
Early studies in leadership focused on the leader’s innate charisma to change followers’ beliefs
and behavior. For example, Weber (1947) explains that charismatic authority originates from the
“virtue of which he [the leader] is set apart from other men and treated as endowed with
supernatural, superhuman, or at least specifically exceptional powers or qualities (pg. 358).” So-
called “great man theory” of leadership highlights that followers idolize leaders who are perceived
87
to be confident and powerful. As a result, followers become active implementors of the collective
mission set forth by the charismatic leader (Tucker, 1968).
Bass expanded the definition of idealized influence in the subsequent book to emphasize
the leaders' ethical and trustworthy behavior. Followers may be attracted to the charisma of strong
leadership, but in the long-run, leaders who demonstrate moral values garner trust and respect.
Employees take “guerrilla government actions” to resist the leadership when the leadership
pursues a course of actions that the employees perceive as unethical (DeHart-Davis 2007; O’Leary
2019). To explain microfoundations of the guerilla government movement, Hollibaugh, Miles, and
Newsfounder (2020) found that employees’ personal ethics have the most decisive influence on
compliance. In other words, leaders must demonstrate their commitment to ethical management to
establish leadership authority and to prevent employee shirking.

B. Full Range of Leadership Theory and Organizational Performance
Despite the proliferation of studies on leadership in public administration literature,
surprisingly little has been discovered about the direct link between leadership practices and
organizational performance. Many studies found an indirect link between transformational or
transactional leadership and the factors contributing to increasing organizational performance. For
example, transformational leadership is found to be associated with low employee turnover
intention (Caillier, 2014; Park & Rainey, 2008), increased inter-organizational collaboration
(Campbell 2018), increased job satisfaction (Park and Rainey 2008), increased public service
motivation (Jensen & Bro, 2017; Park & Rainey, 2008; Vandenabeele, 2014; Wright et al., 2012),
increased extra-role behavior (Caillier, 2014, 2016), increased performance information use
88
(Moynihan, Pandey, et al., 2012), and reduced employee perception of red tapes (Moynihan,
Wright, et al., 2012). Relatively less is studied about the effects of transactional leadership on
antecedents of organizational performance, but studies found transactional leadership’s association
with increased employee creativity (Min et al., 2016), trust in leadership (Asencio & Sun, 2020),
and employee turnover (Caillier, 2018).
When studies address the association of FRL and organizational performance, their focus
is usually on transformational leadership. For example, Bellle (2014) conducted a field experiment
with nurses at a public hospital in Italy to find that the participants exposed to transformational
leadership outperformed the control group. A qualitative study conducted by Andersen, Bjørnholt,
Bro, and Holm-Petersen (2017) found that transformational leadership is positively associated with
professionals’ evaluation of work quality. Sun and Henderson (2017) found that transformational
leadership improves performance through the mediating effects of purposeful performance
information use and stakeholder engagement. Examining U.S. Army platoons, Bass, Avoilio, Jung,
and Berson (2003) found that transformational leadership and contingent rewards are positively
associated with unit performance.
A relationship between transactional leadership and performance has not been thoroughly
investigated because of the practical barriers to implement transactional leadership in public
organizations. Public managers have limited authority to provide monetary rewards, nor can they
expedite promotion for high-performing employees (Wilson, 1989). The use of pecuniary rewards
in public organizations can be dangerous as it crowds out intrinsic motivation (Weibel et al., 2010).
Nevertheless, Andersen and Pallesen (2008) hint that financial incentives can increase employee
performance if they accept performance evaluation criteria. Also, there are more aspects to
transactional leadership than monetary rewards. To compensate for the limited monetary rewards,
89
managers can employ non-pecuniary rewards such as verbal praise to induce high performance
(Jensen et al., 2016).
Based on the literature that attests to the effectiveness of transformational and transactional
leadership, I hypothesize the followings:
Hypothesis 1A. Employee perceived transactional leadership improves organizational
performance.
Hypothesis 1B. Employee perceived transformational leadership improves organizational
performance.
I used employee perceived leadership as explanatory variables because Lee and Carpenter
(2018) found that leaders tend to overrate their performance in relation-oriented leadership when
compared to how followers evaluate them. In other words, even when leaders consider themselves
as practicing effective leadership, employees may disagree. Similarly, Jacobsen and Bøgh
Andersen (2015) found a gap between leader-intended leadership and employee-perceived
leadership. Employee motivation and commitments are dependent upon their perception of
leadership behavior. Employee-perceived leadership is significantly related to organizational
performance, while leader-perceived leadership is not.

Augmentation Theory. In practice, leaders can implement both transactional and
transformational leadership simultaneously. Bass and Avolio (1990) suggest that successful leaders
master the skills of both transformational and transactional leadership. The augmentation theory
argues that transformational leadership builds upon transactional leadership. Transactional
leadership, according to Bass and Avolio (1990), is the foundation that enables transformational
90
leadership to be effective; without first establishing transactional leadership, transformational
leadership alone is ineffective (Bass, 1998). Waldman, Bass, and Yammarino (1990) assert that
combined effects of transformational and transactional leadership lead to higher performance
because transformational leadership reinforces positive effects of contingent rewards.
In the context of public organization, only a few studies examined transformational and
transactional leadership together, and they produced mixed results. Oberfield (2014) found that
implementing both leadership types increases cooperation, satisfaction, and work quality in federal
agencies, though the positive effects of transformational leadership outweigh transactional
leadership. Studying a large IT team inside the government of Indonesia, Kindarto, Zhu, and
Gardner (2020) found that transformational leadership is positively associated with organizational
performance while transactional leadership is not a significant predictor of organizational
performance. Moreover, Nielsen, Boyne, Holten, Jacobsen, and Andersen (2019) found that
transformational leadership and contingent rewards undermine each other. Their findings suggest
that the simultaneous use of transformational leadership and positive verbal recognition produces
positive effects on employee motivation, but an increased emphasis on contingent rewards crowds
out positive effects of transformational leadership. The mixed findings on the effectiveness of
augmentation theory in public organizations require further investigation.
Based on the augmentation theory, I hypothesize the following:
Hypothesis 2. The positive effects of transactional and transformational leadership amplify when
they are implemented together (augmentation theory).

91

Wilson’s Agency Typology. The fit between the leader and the organizational environment is
critical in determining organizational performance (Hanbury et al., 2004). As reviewed in the
introduction, Wilson proposed appropriate management strategies for each of his four agency
categories. Wilson’s propositions are in line with Bass and Avolio (1990), who insist that
organizational environment plays a key role in determining the effectiveness of transformational
and transactional leadership. Transactional leadership is appropriate in a stable work environment
where managers can predict future performance outcomes. On the contrary, transformational
leadership is more effective in a turbulent work environment to manage organizations to
uncertainties and rapid changes.
Wilson emphasizes using an effective incentive system to motivate employees in the
agencies with observable outcomes. Therefore, transactional leadership is expected to be effective
in the Production/Craft agencies. Also, because Wilson urges the importance of setting clear
performance goals and communicating them with operators, transformational leadership,
especially inspirational motivation, is expected to be effective in the Production/Craft agencies.
Hypothesis 3. Both transactional and transformational leadership are associated with increased
performance in Production/Craft agencies.

According to Wilson, managers in Coping/Procedural agencies have few means to
motivate employees through extrinsic rewards. Instead, managers should build a trustful
relationship with employees, demonstrate their sincere interests in employee growth, promote a
sense of mission, and allow operators’ creativity and autonomy to become successful. Thus, the
92
implementation of transformation leadership is expected to be effective in the Coping/Procedural
agencies. On the other hand, since Wilson explicitly mentioned that a contingent reward system
could be detrimental to the Coping/Procedural agencies, transactional leadership is expected to
hurt performance.
Hypothesis 4A. Transformational leadership is associated with increased performance in
Coping/Craft agencies.
Hypothesis 4B. Transactional leadership is associated with decreased performance in
Coping/Craft agencies.
In addition to the four agency categories that Wilson devised, Chapter 2 found another
agency category, the Mixed/Undetermined type. Because no proposition about appropriate
leadership type was made about this category, I hypothesize the following:
Hypothesis 5. Organizational performance in the Mixed/Undetermined agencies changes as a
function of transformational leadership and transactional leadership.

4.3 Data and Measures
A. Dependent Variable
The dependent variable of the study is organizational performance. It is measured in terms
of the likelihood of goal achievement; it is a dichotomous variable that measures whether a given
goal is met (1) or unmet (0) the target that the agency had set in the previous year. The data is
drawn mainly from agency Performance and Accountability Reports. Information about the data
source and the variable is explained in detail in Chapter 2.
93
B. Independent Variable
Data Source and Sampling. Independent variables represent transformational and
transactional leadership evaluated by non-supervisor employees. Because I am interested in
employee’s evaluation of their leadership, I use self-reported survey data drawn from the Federal
Employee Viewpoint Survey (FEVS), formerly known as the Federal Human Capital Survey
(FHCS), until 2008. The FHCS was conducted biennially, while the FEVS was conducted on an
annual basis. The dataset includes four waves of the survey: 2006, 2008, 2010, and 2011
10
.
Responses were collected from a self-administrated web survey. The survey population
includes full-time, permanent employees. In 2006, more than 220,000 employees in 43 agencies
were surveyed, and the governmentwide response rate was about 57% (OPM, 2007). In 2008,
more than 210,000 employees in 45 agencies were surveyed, and the response rate was about 51%
(OPM, 2009). In 2010, more than 250,000 employees in 45 agencies were surveyed, with a
response rate of 52% (OPM, 2011). In 2011, a total of 266,376 employees in 45 agencies
participated in the survey, resulting in a response rate of 49.3% (OPM, 2012). FEVS asks a
question about the respondent’s supervisory status. Because this study is inspired by Wilson's
(1989) proposition’s about how to manage operators, only non-supervisory employees, which
compose 70.81% of the respondents, were included. As a result, the final sample is based on
590,899 responses
11
across four waves.
Since OPM publishes only the “Complete” cases where respondents answer a substantial

10
Prior to 2006, respondents’ supervisory status was not reported accurately for several
agencies.
11
The final sample includes 134,105 responses from 2006; 130,874 from 2008; 163,154 from
2010; 162,766 from 2011
94
number of survey items, the dataset includes only a small percentage of missing data. One
disadvantage of excluding responses with missing data is that it reduces valuable information from
nonresponse and “no basis to judge” answers (Goldenkoff, 2015). To remedy biases from sample
selection and nonresponse, I applied full-sample survey weights as provided in the dataset.
Responses are based on a 5-point Likert scale, ranging from 1 (“strongly disagree) to 5
(“strongly agree”). Since the survey respondents did not reveal their names or identification
numbers, it is impossible to create individual-level panel data. Longitudinal data was generated by
aggregating individual responses to the agency level.
Measuring Full-Range of Leadership. The FEVS provides valuable information to
researchers as it provides insights into the federal government employees. Leadership is one of the
most researched topics using FEVS data (Fernandez et al., 2015). Studies used various items in
FEVS to construct transformational and transactional leadership, but this study builds upon the
measurement approach used by Trottier, Van Wart, and Wang (2008) and Oberfield (2014), with a
few modifications to better reflect Wilson’s propositions.
Transactional leadership was measured using the following six items: “In my work unit,
steps are taken to deal with a poor performer who cannot or will not improve” (passive
management by exception); “Employees are rewarded for providing high quality products and
services to customers” (active management by exception); “Pay raises depend on how well
employees perform their jobs” (contingent reward); “Promotions in my work unit are based on
merit” (contingent reward); “Awards in my work unit depend on how well employees perform
their job” (contingent reward); and “In my work unit, differences in performance are recognized
in a meaningful way” (passive management by exception, active management by exception, and
contingent reward).
95
Transformational leadership was measured using the following seven items: “I feel
encouraged to come up with new and better ways of doing things” (intellectual stimulation);
“Employees have a feeling of personal empowerment with respect to work processes”
(individualized consideration); “Supervisors/team leaders in my work unit provide employees with
the opportunities to demonstrate their leadership skills” (individualized consideration); “In my
organization, leaders generate high levels of motivation and commitment in the workforce”
(inspirational motivation); “I know how my work relates to the agency’s goals and priorities”
(inspirational motivation); “My organization’s leaders maintain high standards of honesty and
integrity.” (idealized influence); “I have trust and confidence in my supervisor” (idealized
influence).
Because the measures address the definition of transformational and transactional
leadership, they have high face validity. The Cronbach’s alpha coefficient is 0.89 for transactional
leadership and 0.91 for transformational leadership. High Cronbach’s alpha coefficients ensure
that the measures have strong internal reliability.
I conducted principal-component factor analysis with Promax rotation to calculate agency-
year scores for transactional and transformational leadership scores. As Oberfield (2014) points
out, since transactional and transformational leadership are not distinct concepts, it is expected that
there is a high correlation between the two factors. This argument coincides with the augmentation
theory that transactional and transformational leadership are distinct yet not mutually exclusive.
Thus, I used Promax rotation, a kind of oblique rotation, which is proper to use when factors are
highly correlated (Abdi, 2003). The factor analysis results demonstrate that questions are split into
two factors as expected. Table 4-2 provides detailed information about factor scores and
eigenvalues.
96
Table 4-2. Principal Component Factor Analysis

Factor1 Factor2 Uniqueness

Transformationa
l Leadership
I feel encouraged to come up with new
and better ways of doing things.
0.73 0.33 0.35
Employees have a feeling of personal
empowerment with respect to work
processes.
0.80 0.09 0.36
Supervisors/team leaders in my work unit
provide employees with the opportunities
to demonstrate their leadership skills.
0.75 0.31 0.34
In my organization, leaders generate high
levels of motivation and commitment in
the workforce.
0.80 0.14 0.35
I know how my work relates to the
agency’s goals and priorities.
0.51 0.44 0.54
My organization’s leaders maintain high
standards of honesty and integrity.
0.76 0.18 0.39
I have trust and confidence in my
supervisor.
0.73 0.29 0.38
Transactional
Leadership
In my work unit, steps are taken to deal
with a poor performer who cannot or will
not improve.
0.69 -0.35 0.40
Employees are rewarded for providing
high quality products and services to
customers.
0.83 -0.11 0.30
Awards in my work unit depend on how
well employees perform their jobs.
0.81 -0.28 0.26
In my work unit, differences in
performance are recognized in a
meaningful way.
0.83 -0.28 0.24
Pay raises depend on how well employees
perform their jobs.
0.73 -0.39 0.31
Promotions in my work unit are based on
merit.
0.81 -0.20 0.31

Average factor loadings 0.76 0.68

Reliability (Cronbach's Alpha) 0.89 0.91

Eigenvalue 7.44 1.03

97
C. Moderating Variable
Agency typology based on Wilson’s definition of goal outcome observability, as
established in Chapter 2, is used to examine whether the effects of leadership vary by agency type.
A total of 24 agencies are included in the analysis. Coping/Procedural agencies have low average
goal observability scores (smaller than 0.35). Production/Craft agencies have high average goal
observability scores (greater than 0.65). Lastly, agencies whose goal observability score falls in
the middle range (between 0.35 and 0.65) are classified as “Mixed/Undetermined” type as Wilson’s
typology did not assume that there would be a middle ground.

D. Control Variables
In order to control variables that affect goal performance, the following variables are
included in the analysis. Goal observability, determined by whether the goal outcome is directly
attributable to agency efforts or to the external environment, is included as a Level 1 variable.
Level 2 variables represent organizational characteristics related to agency human resources and
political environment: natural logarithm of the number of full-time equivalents (FTEs) employees;
the percentage of employees who were identified as clerical, professional, and technical in OPM’s
FedScope; organizational age; and priority ambiguity; agency politicization, presidential attention
to agency; and agency independence measure that Selin (2015) developed. Lastly, the year variable
was added as dummies to capture the influence of time. Descriptive statistics for the variables used
in the model are presented in Table 4-3.

98
Table 4-3. Descriptive Statistics
Variable Description Obs Mean
Std.
Dev.
Min Max
Level 1 Variables
Goal Achievement 1 (met); 0 (unmet) 6954 0.67 0.47 0 1
Goal Observability
1 (observable);
0 (unobservable)
6954 0.51 0.50 0 1
Level 2 Variables
Transactional Leadership Aggregated agency-year
average of FEVS survey
response score
6954 0.08 0.16 -0.06 0.49
Transformational
Leadership
Aggregated agency-year
average of FEVS survey
response score
6954 0.04 0.17 -0.68 0.43
Politicization
(Sched C + NCSES + Ltd
Term)/Career SES
6954 0.46 0.44 0 2.10
Prioritization Ambiguity
As derived by Chun and
Rainey (2005)
6954 -0.14 0.76 -1.41 2.61
Agency Independence As derived by Selin (2015) 6954 -0.11 0.47 -0.69 1.72
Presidential Attention As derived by Lewis (2011) 6954 2.66 1.03 0.00 4.92
Agency Size
Natural log of agency’s FTEs
during FY
6954 9.94 1.19 6.97 12.67
Percent Clerical
% of FTEs who have
“clerical” classification by
OPM system
6954 0.06 0.06 0.01 0.25
Percent Professional
% of FTEs who have a
“professional” classification
by OPM system
6954 0.33 0.18 0.04 0.69
Percent Technical
% of FTEs who have a
“technical” classification by
OPM system
6954 0.11 0.09 0.02 0.45
Organizational Age
Natural log of the agency's
age.
6954 4.15 0.69 1.95 5.40
Wilson Agency Types
Production/Craft
Average goal observability
equal to or smaller than .35
1965
Coping/Procedural
Average goal observability
equal to or greater than .65
2471
Mixed/Undetermined
Average goal observability
between .35 and .65
2518

99
E. Modeling
The unit of analysis is individual GPRA goals, and it is highly unlikely that the achievement
of each goal is independent of one another. Given that the organizational and institutional
arrangements of the agency affect performance levels, it is reasonable to suspect that the
achievement of goals within the same agency is not independent. Thus, as explained in Chapter 3,
I use a multilevel logistic regression model.
Goal observability and performance are included as level 1 (individual level) variables, and
the remaining independent and control variables are included as level 2 (agency level) variables.
Year dummy variables are included to control for common trends caused by time. Because goals
are nested within agencies, the agency variable is included in the random effects. Also, since the
outcome observability varies to a great extent by agency, the observability variable was included
as a random coefficient.

Level 1 (Goal Level):
𝑃𝑒𝑟𝑓 𝑖𝑗
= 𝛽 0𝑗 + 𝛽 1𝑗 𝑂𝑏𝑠 𝑖𝑗
+ 𝑟 𝑖𝑗
[4.1]
Level 2 (Agency Level):
𝛽 0𝑗 = 𝛾 00
+ 𝛾 01
∗ 𝐿𝑒𝑎𝑑𝑒𝑟𝑠 ℎ𝑖𝑝
𝑗 + 𝑢 0𝑗 [4.2]
𝛽 1𝑗 = 𝛾 10
+ 𝑢 1𝑗 [4.3]

100
Where 𝑃𝑒𝑟𝑓 𝑖𝑗
is the goal performance, 𝛾 00
is the average intercept, 𝛾 10
is the average slope for
each agency’s goal observability, 𝑂𝑏𝑠 𝑖𝑗
is goal observability, 𝐿𝑒𝑎𝑑𝑒𝑟𝑠 ℎ𝑖𝑝
𝑗 is leadership
(transformational and/or transactional), 𝑢 1𝑗 is the agency slope for observability, 𝑟 𝑖𝑗
is unique
goal effect, and 𝑢 0𝑗 is unique agency effect.

4.4 Results
Table 4-4 presents regression outputs for the baseline models without agency typology
interactions. Because I use the logistic regression, coefficients for the fixed part of the equation
are expressed in the odds ratio. The intraclass correlation coefficient (ICC) indicates the degree of
common environment that observations share (Park & Lake, 2005). The ICCs ranged from .287 to
.33 across the three models, suggesting substantial variance in performance results due to the
grouping variable, which is agency.
Model 1 indicates that the transactional leadership variable is not statistically significant at
the conventional 95% significance level, thus suggesting that it is not associated with the odds of
goal achievement. Model 2 suggests a positive association between transformational leadership
and goal achievement. One unit increase in transformational leadership increases the log odds of
goal achievement by 6.67. Model 3 includes both types of leadership to test the augmentation
theory.

101
Table 4-4. Baseline Leadership Models (in Odds Ratio)
Model 1 Model 2 Model 3
Transactional Leadership 1.366 0.102**
(0.768) (0.0891)
Transformational Leadership 6.671** 52.09***

(4.236) (52.80)
Goal Observability 1.428** 1.420** 1.420**
(0.178) (0.176) (0.173)
Agency Size 0.821 0.985 0.963
(0.243) (0.261) (0.260)
Percent Professional 78.53** 22.50* 19.26*
(111.3) (29.12) (25.24)
Percent Technical 5.759 1.896 0.594
(17.50) (5.413) (1.757)
Percent Clerical 0.0166 0.0576 0.190
(0.0459) (0.158) (0.539)
Politicization 2.790* 2.739* 2.941*
(1.308) (1.256) (1.377)
Prioritization Ambiguity 0.603*** 0.606*** 0.603***
(0.0412) (0.0410) (0.0410)
Presidential Attention 1.194* 1.239** 1.216*
(0.0921) (0.0940) (0.0929)
Organizational Age 1.486 1.207 1.262
(0.557) (0.416) (0.450)
Agency Independence 1.376 1.783 1.706
(0.808) (0.965) (0.947)

Level 2 Variance

Goal Observability 1.250+ 1.250+ 1.236+
(0.151) (0.151) (0.142)
Constant 5.092* 3.770* 4.155*

(3.593) (1.998) (2.415)
N Observation 6954 6954 6954
N Groups 24 24 24
Chi-Square 89.96 96.15 101.44
Log Likelihood -3940.0665 -3935.8859 -3932.4252
ICC .330 .287 .302

102
The results indicate that transactional and transformational leadership place opposite
effects on performance. While transactional leadership is associated with decreasing log odds of
goal achievement, transformational leadership is associated with increasing log odds of goal
achievement. It is worth noting that transactional leadership, whose coefficient is not statistically
significant in Model 1, becomes a significant predictor of goal performance with the inclusion of
the transformational leadership variable. Furthermore, the positive effect of transformational
leadership is amplified with the inclusion of transactional leadership. In Model 3, one unit increase
in transformational leadership increases the log odds of goal achievement by 52.09 times, holding
other variables constant. For public managers, the results in Table 4-4 signifies the importance of
implementing both types of leadership. To maximize performance, the managers need to
implement high levels of transformational leadership while exercising low levels of transactional
leadership.
In order to examine whether the effects of FRL vary by agency outcome observability,
agency type interaction was added to the baseline models. The results are presented in Table 4-5.
Again, coefficients for the fixed part of the equation are expressed in the odds ratio. It should be
noted that coefficients that involve agency typology interaction are expressed in comparison to the
Mixed/Undetermined category. Overall, ICCs slightly decreased from the baseline models.

103
Table 4-5. Leadership Models with Agency Interaction (in Odds Ratio)
Model 4 Model 5 Model 6
Coping/Procedural 0.353 0.327+ 0.230*
(0.242) (0.200) (0.154)
Production/Craft 1.974 2.107 1.129
(1.386) (1.255) (0.743)
Transactional Leadership 0.126* 0.00219***
(0.117) (0.00293)
Coping/Procedural*Transactional Leadership 44.95** 131.9**
(60.63) (221.2)
Production/Craft*Transactional Leadership 530.5*** 8954.1***
(841.8) (18760.9)
Transformational Leadership 1.717 266.1***
(1.360) (361.8)
Coping/Procedural*Transformational Leadership 18.88* 0.414
(24.33) (0.726)
Production/Craft*Transformational Leadership 11.13* 0.0396+
(13.70) (0.0694)
Goal Observability 1.349* 1.395** 1.417**
(0.160) (0.171) (0.159)
Agency Size 0.759 0.987 0.880
(0.226) (0.248) (0.235)
Percent Professional 5.443 3.827 1.933
(8.592) (5.330) (2.829)
Percent Technical 1.541 0.0973 0.0832
(5.369) (0.306) (0.280)
Percent Clerical 0.0120 0.0483 0.106
(0.0388) (0.142) (0.335)
Politicization 3.531* 3.770** 3.722*
(1.939) (1.924) (1.947)
Prioritization Ambiguity 0.582*** 0.573*** 0.587***
(0.0403) (0.0396) (0.0422)
Presidential Attention 1.156+ 1.184* 1.190*
(0.0910) (0.0908) (0.0929)
Organizational Age 2.239* 1.648 1.849
(0.858) (0.566) (0.691)
Agency Independence 1.022 1.162 1.280
(0.606) (0.577) (0.686)

Level 2 Variance

Goal Observability 1.209+ 1.226* 1.164*
(0.122) (0.116) (0.0862)
Constant 4.059+ 3.078* 3.813*
(3.024) (1.402) (2.093)
N Observation 6954 6954 6954
N Groups 24 24 24
Chi-Square 103.63 109.49 128.11
Log Likelihood -3926.4941 -3925.3115 -3912.1646
ICC .299 .255 .289
104
First, Model 4 represents the relationship between transactional leadership and
performance, moderated by agency typology. To assist the interpretation of the odds ratio, Figure
4-1 provides a graphical illustration of the results. Transactional leadership is found to have a
negative association with performance in the Mixed/Undetermined type, whereas it is positively
associated with the odds of goal achievement in both of the original Wilsonian agency types. In
particular, one unit increase in transactional leadership is associated with higher log odds of goal
achievement in Production/Craft agencies than in Coping/Procedural agencies as Wilson
suggested; the log odds coefficient for Production/Craft agencies is 530.5, and it is 44.95 in
Coping/Procedural agencies. Figure 4-1 demonstrates that the slope of the graph for
Production/Craft type is steeper than Coping/Procedural type.

Figure 4-1. Transactional Leadership

105
Model 5 conveys the association between transformational leadership and agency
performance. The effects of transformational leadership are not statistically significant for the
Mixed/Undetermined group. In Figure 4-2, which provides a graphical illustration of the results in
Table 4-5, the graph for Mixed/Undetermined is nearly flat. For the other types of agencies,
transformational leadership is positively associated with the odds of goal achievement. Consistent
with Wilson’s proposition, the positive effect of transformational leadership is more dramatic in
Coping/Procedural agencies than in Production/Craft, as shown by the steeper slope of the graph
in Figure 4-2.

Figure 4-2. Transformational Leadership

106
To summarize results from Model 4 and 5, either transactional or transformational
improves the log odds of goal achievement in Coping/Procedural and Production/Craft agencies,
although transformational leadership leads to more dramatic effects in the former while
transactional leadership does so in the latter. Nonetheless, neither type is found to be effective in
Mixed/Undetermined agencies. Goal achievement in Mixed/Undetermined agencies is not
associated with transformational leadership, and even worse, it is negatively associated with
transactional leadership. Wilson assumed that management of coping agencies would be the most
challenging, but the analysis nuances that management of Mixed/Undetermined agencies can be
more challenging.
Model 6 is a fully specified model, which includes both types of leadership and agency
interaction. First, Figure 4-3 illustrates that transactional leadership is associated with improved
performance only in Production/Craft agencies. It is associated with diminished performance in
Coping/Procedural. The negative impact of transactional leadership is even more severe in
Mixed/Undetermined agencies. Markedly, the positive association between transactional
leadership and performance in Coping/Procedural agencies in Model 3 changes to a negative
association when both leadership types are implemented.
107
Figure 4-3. Full-Model: Transactional Leadership

Second, transformational leadership is found to be associated with increased odds of goal
achievement in all three models. It is the only model that performance increases in the
Mixed/Undetermined group. Coefficients for the other two agency types are not statistically
different from the Mixed/Undetermined group at 95% significance level, although at 90%
significance level, the log odds coefficient for Production/Craft agencies decreases. Figure 4-4
shows that performance trends in Mixed/Undetermined and Coping/Procedural types are almost
identical. Performance in Production/Craft agencies increases with transformational leadership,
but at a lower rate.

108
Figure 4-4. Full Model: Transformational Leadership

It is worth mentioning that some variables are found to be significant across all six models.
Goal observability is positively associated with performance, meaning that goals that are
attributable directly to agency efforts are more likely to be achieved than the goals outside of the
agency's control. In conformity with previous studies (e.g., Chun & Rainey, 2005b), priority
ambiguity is associated negatively with the odds of goal achievement. Increasing levels of
presidential attention to the agency are associated with an increase in performance. As found in
Chapter 3, politicization is found to be a significant predictor of goal performance. Percent of
professional workers is significant and positively associated with performance in the models
without agency typology interactions, but they become insignificant in the models that include
agency typology.

109
4.5 Conclusion and Discussion
In sum, this article finds that leadership makes a difference in organizational performance.
It corroborates existing literature on the positive effects of transformational leadership. The effects
of transactional leadership are complicated. When used alone, its effects on performance are not
statistically significant. When implemented simultaneously with transformational leadership,
increased levels of transactional leadership undermine organizational performance. This result is
in contraction to the augmentation theory that predicts amplification of positive effects when both
leadership types are implemented together. However, the findings do not suggest that public
managers should deter the implementation of transactional leadership. Although the increasing
levels of transactional leadership decrease organizational performance, it intensifies the positive
effects of transformational leadership. Thus, public managers need to consider employing low
levels of transactional leadership and high levels of transformational leadership.
When Wilson’s agency typology is considered, each agency type responds differently to
FRL. In Production/Craft agencies, both types of leadership contribute to improved odds of goal
achievement. In Coping/Procedural agencies, transformational leadership is associated with
improved performance. Transactional leadership is positively associated with performance when
it is implemented alone, but when implemented simultaneously with transformational leadership,
it is negatively associated with performance. Leadership practices have the least impact on
performance in the Mixed/Undetermined agencies. Transactional leadership is associated with
decreasing performance, and transformational leadership is not associated with performance when
implemented alone. Only when FRL is taken into consideration, performance improves in the
Mixed/Undetermined agencies.
110
I recommend that future studies investigate effective leadership practices for the
Mixed/Undetermined agencies. They are least responsive to the leadership practices examined in
this article. Nevertheless, because there are many other theories in leadership, other types of
leadership may be effective in managing the Mixed/Undetermined agencies.
Findings in this article provide optimistic news to practitioners. Although public managers
cannot change institutional arrangements of the agency, how they motivate operators do make
significant differences in agency performance. Field experiment studies found that managers’ use
of transformational and transactional leadership can improve with training (An et al., 2019;
Jacobsen et al., 2021), which implies that leadership training programs for public managers can
enhance organizational performance.
The biggest limitation of the paper lies in the agency typology used in the analysis.
Because the GPRA goals only represent the observability of outcomes and not outputs, four of
Wilson’s original categories are grouped into two categories. If agency classification that
accommodates all four categories were used, effective leadership for agency each type could have
been examined. Future studies that include the output dimension of Wilson’s typology need to
reexamine the topic in order to provide a full picture.
111
Chapter 5
Concluding Remarks

5.1 Summary and Contributions
This dissertation is inspired by James Q. Wilson’s seminal book, Bureaucracy: What
Government Agencies Do and How They Do It. Many students of public administration and
political science have accepted Wilson’s propositions as a comprehensive theory of public
organizations (Rourke & Doig, 1991). However, Wilson put an explicit warning that his book does
not entail a theory. In the preface of the original edition where Wilson expressed his view on
establishing organizational theory, Wilson says, “I have come to have grave doubts that anything
worth calling “organization theory” will ever exist. Theories will exist, but they will usually be so
abstract or general as to explain rather little. Interesting explanations will exist, some even
supported with facts, but these will be partial, place- and time-bound insights (p. xix).”
My dissertation makes contributions to the literature by validating Wilson’s famous
agency typology and propositions about leadership that are often regarded as a theory yet left
largely untested. First, I provided definitions of key terminologies (observability, outputs, and
outcomes) and evaluation standards to assess the outcome observability in order to address
criticisms that Wilson’s typology lacks a clear definition and criteria to determine observability
(Trommel et al., 2012). I establish that outcome observability refers to the extent to which an
outsider can establish a causal inference between agency activities and societal outcomes that the
agencies try to achieve. Moreover, using the definition, I use agency performance measures to
112
determine the observability of agency outcome. As agencies are required by law (GPRA 1993) to
set strategic objectives and annual indicators that reflect agency policy priorities, organizational
goals, when aggregated to the agency level, serve as a good tool to determine agency outcome
observability.
Second, my work contributes to the field by providing extensive data on federal
government performance measures. To my knowledge, this is the most comprehensive data for
GPRA goals. This dataset not only includes text data for agency strategic objectives and annual
performance indicators, but it also includes performance information and goal observability for
each performance measure. In order to avoid sampling bias, I attempted to collect the entire
population of performance goals. However, due to the data availability, goals were excluded from
the analysis. Still, the dataset includes more than 20,000 performance measures in the 28 federal
agencies over a 10-year time period.
Third, using the above-mentioned dataset, I provide an empirical measurement of agency
outcome observability on a continuous scale. Although some efforts were made previously to
classify several government agencies into Wilson’s typology, this is the first systematic effort to
use empirical data to classify a large number of agencies using clearly defined criteria. Based on
the aggregated average observability score of performance measures for each agency, I classify 28
agencies into three groups: Coping/Procedural, Mixed/Undetermined, and Production/Craft types
in the order of increasing outcome observability. I conducted multiple tests to ensure the reliability
and validity of the measurement.
Fourth, I utilize the agency typology to test Wilson’s propositions on agency leadership
and performance. By incorporating agency typology as a moderating variable, the findings elicit
that the effect of leadership varies by agency type. In terms of who should be in the leadership
113
position (appointees vs. careerists), existing literature attests to the negative association between
increased appointee leadership and agency performance. However, I validate Wilson’s proposition
that appointee leadership can help increase agency performance depending on agency outcome
observability. Empirical data demonstrates that appointee leadership is associated with decreased
performance in Production/Craft agencies, but it helps performance in Coping/Procedural agencies.
Conversely, the increased influence of careerist leadership leads to enhanced agency performance
in Production/Craft agencies while it leads to decreased performance in Coping/Procedural
agencies.
Likewise, I validated Wilson’s proposition on how leaders in each agency category should
motivate employees to improve agency performance. Because the majority of existing studies
focus exclusively on transformational leadership, studying transformational and transactional
leadership together provides insights into the Full Range of Leadership theory. The findings
suggest that transactional leadership is effective only in Production/Craft agencies, while
transformational leadership is effective across agency types.

5.2 Limitations and Future Research Agenda
Several limitations exist. First, this study examines only one dimension of Wilson’s
typology. Outcome observability is quantified by taking the aggregated average of goal
observability scores from agency performance measures, but output observability is not measured.
Hence, I combined Wilson’s categories into two groups based on outcome observability:
Production/Craft and Coping/Procedural. In order to fully validate Wilson’s typology, I
recommend that future studies quantify output observability. Because output observability refers
114
to the degree operator activities is visible to managers, a survey to public managers about how they
monitor their subordinate employees’ work will assist in measuring output observability.
Another limitation is that the dataset does not include all performance goals. In order to
avoid sampling bias, I attempted to collect the entire population of performance goals. However,
due to the data availability, some goals were not located. I recommend that future studies locate
the missing goals in order to provide a complete list of goals and observability scores. Also, I
recommend that future studies extend the data collection efforts beyond 2011 to reflect the latest
performance goals.
Still, the dataset includes text information for more than 20,000 performance measures
and their strategic indicators, along with the information on each goal’s performance status (met
or unmet) and goal observability. This dataset provides exciting opportunities to explore
performance management in federal agencies. One possibility is to replicate the studies that used
PART data. Because the GPRA is a legislative initiative and while the PART is a presidential
initiative, a comparison of the agency performance evaluations by the GPRA ad PART will help
us understand the differences in the two branches ’ approach to agency performance. Also, because
GPRA takes a bottom-up approach to goal setting and PART takes a strict top-down approach, the
differences in the GPRA and PART agency performance evaluation can be examined through the
lens of organizational behavior.

115
List of References
Abdi, H. (2003). Factor Rotation. In Encyclopedia ofSocial Sciences Research Methods (pp.
2144–2144). https://doi.org/10.1007/978-94-007-0753-5_101312
Aberbach, J. D. (2002). What’s Happened to the Watchful Eye? Congress & the Presidency,
29(1), 3–23. https://doi.org/10.1080/07343460209507847
Ali, S. B. (2018). Is all turnover intent the same? Exploring future job preference and
environmental considerations. Public Management Review, 20(12), 1768–1789.
https://doi.org/10.1080/14719037.2017.1417464
Allison, G. T. (1980). Public and Private Management: Are They Fundamentally Alike in All
Unimportant Respects? John F. Kennedy School of Government, Harvard University.
An, S.-H., Meier, K. J., Bøllingtoft, A., & Andersen, L. B. (2019). Employee Perceived Effect of
Leadership Training: Comparing Public and Private Organizations. International Public
Management Journal, 22(1), 2–28. https://doi.org/10.1080/10967494.2018.1497739
Andersen, L. B., Bjørnholt, B., Bro, L. L., & Holm-Petersen, C. (2017). Achieving High Quality
Through Transformational Leadership: A Qualitative Multilevel Analysis of
Transformational Leadership and Perceived Professional Quality. Public Personnel
Management, 47(1), 51–72. https://doi.org/10.1177/0091026017747270
Andersen, L. B., & Pallesen, T. (2008). “Not Just for the Money?” How Financial Incentives
Affect the Number of Publications at Danish Research Institutions. International Public
Management Journal, 11(1), 28–47. https://doi.org/10.1080/10967490801887889
Andrews, R., Boyne, G. A., Moon, M. J., & Walker, R. M. (2010). Assessing Organizational
Performance: Exploring Differences Between Internal and External Measures. International
Public Management Journal, 13(2), 105–129. https://doi.org/10.1080/10967491003766533
Andrews, R., Boyne, G. A., & Walker, R. M. (2006). Subjective and objective measures of
organizational performance: An empirical exploration. Public Service Performance:
Perspectives on Measurement and Management, 14–34.
https://doi.org/10.1017/CBO9780511488511.002
Asencio, H. D., & Sun, R. (2020). The Effects of Leadership on Employee Trust: A Longitudinal
Study of United States Federal Agencies. International Journal of Public Administration,
43(14), 1237–1251. https://doi.org/10.1080/01900692.2019.1668809
Bass, B. M. (1985). Leadership and Performance Beyond Expectations. Free Press.
Bedeian, A. G. (1984). Organizations: Theory and Analysis : Text and Cases. Dryden Press.
Bertelli, A. M. (2007). Determinants of bureaucratic turnover intention: Evidence from the
department of the treasury. Journal of Public Administration Research and Theory, 17(2),
235–258. https://doi.org/10.1093/jopart/mul003

116
Bertelli, A. M., & Grose, C. R. (2011). The Lengthened Shadow of Another Institution? Ideal
Point Estimates for the Executive Branch and Congress. American Journal of Political
Science, 55(4), 767–781. http://www.jstor.org/stable/23025119
Bertelli, A. M., & Lewis, D. E. (2013). Policy Influence, Agency-Specific Expertise, and Exit in
the Federal Service. Journal of Public Administration Research and Theory, 23(2), 223–
245. https://doi.org/10.1093/jopart/mus044
Bohte, J., & Meier, K. J. (2000). Goal Displacement: Assessing the Motivation for
Organizational Cheating. Public Administration Review, 60(2), 173–182.
http://www.jstor.org/stable/977649
Buch, R., Martinsen, Ø . L., & Kuvaas, B. (2015). The Destructiveness of Laissez-Faire
Leadership Behavior: The Mediating Role of Economic Leader–Member Exchange
Relationships. Journal of Leadership and Organizational Studies, 22(1), 115–124.
https://doi.org/10.1177/1548051813515302
Burns, J. M. G. (1978). Leadership. Harper & Row.
Caillier, J. G. (2012). Agency Retaliation Against Whistle-Blowers. Public Integrity, 15(1), 29–
50. https://doi.org/10.2753/PIN1099-9922150102
Caillier, J. G. (2014). Do Transformational Leaders Affect Turnover Intentions and Extra-Role
Behaviors Through Mission Valence? The American Review of Public Administration,
46(2), 226–242. https://doi.org/10.1177/0275074014551751
Caillier, J. G. (2016). Linking Transformational Leadership to Self-Efficacy, Extra-Role
Behaviors, and Turnover Intentions in Public Agencies: The Mediating Role of Goal
Clarity. Administration and Society, 48(7), 883–906.
https://doi.org/10.1177/0095399713519093
Caillier, J. G. (2018). Can Changes in Transformational-Oriented and Transactional-Oriented
Leadership Impact Turnover over Time? International Journal of Public Administration,
41(12), 935–945. https://doi.org/10.1080/01900692.2017.1300918
Carpenter, D. (2020). On Categories and the Countability of Things Bureaucratic: Turning From
Wilson (Back) to Interpretation. Perspectives on Public Management and Governance,
3(2), 83–93. https://doi.org/10.1093/ppmgov/gvz025
Chun, Y. H., & Rainey, H. G. (2005a). Goal Ambiguity in U.S. Federal Agencies. Journal of
Public Administration Research and Theory, 15(1), 1–30.
https://doi.org/10.1093/jopart/mui001
Chun, Y. H., & Rainey, H. G. (2005b). Goal Ambiguity and Organizational Performance in U.S.
Federal Agencies. Journal of Public Administration Research and Theory, 15(4), 529–557.
https://doi.org/10.1093/jopart/mui030
Cohen, D. M. (1998). Amateur Government. Journal of Public Administration Research and
Theory, 8(4), 450–497. https://doi.org/10.1093/oxfordjournals.jpart.a024393

117
Corrêa d’Almeida, A., & Klingner, D. E. (2008). FEMA and the Witt Revolution: Testing the
Hypothesis of “Bureaucratic Autonomy.” Public Organization Review, 8(4), 291–305.
https://doi.org/10.1007/s11115-008-0060-9
Dahl, R. A., & Lindblom, C. E. (1953). Politics, Economics, and Welfare: Planning and
Politico-economic Systems Resolved Into Basic Social Processes. Harper.
Davidson, M. (2014). Known-Groups Validity BT - Encyclopedia of Quality of Life and Well-
Being Research (A. C. Michalos (ed.); pp. 3481–3482). Springer Netherlands.
https://doi.org/10.1007/978-94-007-0753-5_1581
DeGroff, A., Schooley, M., Chapel, T., & Poister, T. H. (2010). Challenges and strategies in
applying performance measurement to federal public health programs. Evaluation and
Program Planning, 33(4), 365–372. https://doi.org/10.1016/j.evalprogplan.2010.02.003
Dirsmith, M. W., Jablonsky, S. F., & Luzi, A. D. (1980). Planning and Control in the U.S.
Federal Government: A Critical Analysis of PPB, MBO and ZBB. Strategic Management
Journal, 1(4), 303–329. http://www.jstor.org/stable/2486258
Downs, A. (1964). Inside Bureaucracy. Rand Corporation.
Drucker, P. F. (1976). What Results Should You Expect? A Users’ Guide to MBO. Public
Administration Review, 36(1), 12–19. https://doi.org/10.2307/974736
Dull, M. (2006). Why PART? The institutional politics of presidential budget reform. Journal of
Public Administration Research and Theory, 16(2), 187–215.
https://doi.org/10.1093/jopart/muj004
Dull, M. (2009). Results-model reform leadership: Questions of credible commitment. Journal of
Public Administration Research and Theory, 19(2), 255–284.
https://doi.org/10.1093/jopart/mum043
Dull, M., Roberts, P. S., Keeney, M. S., & Choi, S. O. (2012). Appointee Confirmation and
Tenure: The Succession of U.S. Federal Agency Appointees, 1989–2009. Public
Administration Review, 72(6), 902–913. https://doi.org/https://doi.org/10.1111/j.1540-
6210.2012.02676.x
Fernandez, S., & Moldogaziev, T. (2013). Using Employee Empowerment to Encourage
Innovative Behavior in the Public Sector. Journal of Public Administration Research and
Theory, 23(1), 155–187. https://doi.org/10.1093/jopart/mus008
Fernandez, S., Resh, W. G., Moldogaziev, T., & Oberfield, Z. W. (2015). Assessing the Past and
Promise of the Federal Employee Viewpoint Survey for Public Management Research: A
Research Synthesis. Public Administration Review, 75(3), 382–394.
https://doi.org/10.1111/puar.12368
Folwer, L. (2013). Measuring organization: Performance in environmental agencies.
International Journal of Organization Theory & Behavior, 16(3), 326–358.
https://doi.org/10.1108/IJOTB-16-03-2013-B002
Frederickson, D. G. (2006). Measuring the Performance of the Hollow State. Georgetown
University Press.
118
Frederickson, H. G., Smith, K. B., Larimer, C., & Licari, M. J. (2018). The Public
Administration Theory Primer. Taylor & Francis.
Gallo, N., & Lewis, D. E. (2012a). The consequences of presidential patronage for federal
agency performance. Journal of Public Administration Research and Theory, 22(2), 219–
243. https://doi.org/10.1093/jopart/mur010
Gallo, N., & Lewis, D. E. (2012b). The Consequences of Presidential Patronage for Federal
Agency Performance. Journal of Public Administration Research and Theory, 22(2), 219–
243. https://doi.org/10.1093/jopart/mur010
General Accounting Office. (1979). Streamlining Zero-base Budgeting Will Benefit
Decisionmaking: Report to the Congress.
Gilmour, J. B., & Lewis, D. E. (2006). Political appointees and the competence of federal
program management. American Politics Research, 34(1), 22–50.
https://doi.org/10.1177/1532673X04271905
Goldenkoff, R. (2015). The Federal Employee Viewpoint Survey: A Practitioner’s View of
Using the Data. Public Administration Review, 75(3), 397–398.
https://doi.org/10.1111/puar.12375
Government Accountability Office. (2004). Results-oriented Government: GPRA Has
Established a Solid Foundation for Achieving Greater Results : Report to Congressional
Requesters (GAO-04-38). U.S. Government Accountability Office.
Greenhouse, S. M. (1966). The Planning-Programming-Budgeting System: Rationale, Language,
and Idea-Relationships. Public Administration Review, 26(4), 271–277.
https://doi.org/10.2307/973298
Gregory, R. (1995). THE PECULIAR TASKS OF PUBLIC MANAGEMENT: TOWARD
CONCEPTUAL DISCRIMINATION. Australian Journal of Public Administration, 54(2),
171–183. https://doi.org/https://doi.org/10.1111/j.1467-8500.1995.tb01124.x
Gueorguieva, V., Accius, J., Apaza, C., Bennett, L., Brownley, C., Cronin, S., & Preechyanud, P.
(2008). The Program Assessment Rating Tool and the Government Performance and
Results Act: Evaluating Conflicts and Disconnections. The American Review of Public
Administration, 39(3), 225–245. https://doi.org/10.1177/0275074008319218
Gwet, K. L. (2014). Handbook of Inter-Rater Reliability, 4th Edition: The Definitive Guide to
Measuring The Extent of Agreement Among Raters. Advanced Analytics, LLC.
Hammond, T. H., Knott, J. H., Heimann, L., Hill, J., Koford, K., Krehbiel, K., Maser, S., Miller,
G., Moe, T., Quirk, P., Roberts, B., Robinson, G., Smith, S., Tsebelis, G., & Weingast, B.
(1996). Who Controls the Bureaucracy?: Presidential Power, Congressional Dominance,
Legal Constraints, and Bureaucratic Autonomy in a Model of Multi-Institutional Policy-
Making. Source: Journal of Law Economics, & Organization, 12216163(1), 119–166.
https://doi.org/10.2307/765041
Hanbury, G. L., Sapat, A., & Washington, C. W. (2004). Know Yourself and Take Charge of
Your Own Destiny: The “Fit Model” of Leadership. Public Administration Review, 64(5),
566–576. https://doi.org/https://doi.org/10.1111/j.1540-6210.2004.00404.x
119
Heclo, H. (1977). A Government of Strangers: Executive Politics in Washington. Brookings
Institution Press.
Heinrich, C. J. (2012). How Credible Is the Evidence, and Does It Matter? An Analysis of the
Program Assessment Rating Tool. Public Administration Review, 72(1), 123–134.
https://doi.org/https://doi.org/10.1111/j.1540-6210.2011.02490.x
Hollibaugh, G. E. (2015). Naï ve cronyism and neutral competence: Patronage, performance, and
policy agreement in executive appointments. Journal of Public Administration Research
and Theory, 25(2), 341–372. https://doi.org/10.1093/jopart/muu016
Hollibaugh, G. E., Horton, G., & Lewis, D. E. (2014). Presidents and Patronage. American
Journal of Political Science, 58(4), 1024–1042. https://doi.org/10.1111/ajps.12083
Hsieh, A.-T., Chou, C.-H., & Chen, C.-M. (2002). Job standardization and service quality: A
closer look at the application of total quality management to the public sector. Total Quality
Management, 13(7), 899–912. https://doi.org/10.1080/0954412022000017012
Jacobsen, C. B., Andersen, L. B., Bøllingtoft, A., & Eriksen, T. L. M. (2021). Can Leadership
Training Improve Organizational Effectiveness? Evidence from a Randomized Field
Experiment on Transformational and Transactional Leadership. Public Administration
Review, n/a(n/a). https://doi.org/https://doi.org/10.1111/puar.13356
Jacobsen, C. B., & Bøgh Andersen, L. (2015). Is Leadership in the Eye of the Beholder? A Study
of Intended and Perceived Leadership Practices and Organizational Performance. Public
Administration Review, 75(6), 829–841. https://doi.org/https://doi.org/10.1111/puar.12380
Jensen, U. T., Andersen, L. B., Bro, L. L., Bøllingtoft, A., Eriksen, T. L. M., Holten, A.-L.,
Jacobsen, C. B., Ladenburg, J., Nielsen, P. A., Salomonsen, H. H., Westergård-Nielsen, N.,
& Würtz, A. (2016). Conceptualizing and Measuring Transformational and Transactional
Leadership. Administration & Society, 51(1), 3–33.
https://doi.org/10.1177/0095399716667157
Jensen, U. T., & Bro, L. L. (2017). How Transformational Leadership Supports Intrinsic
Motivation and Public Service Motivation: The Mediating Role of Basic Need Satisfaction.
The American Review of Public Administration, 48(6), 535–549.
https://doi.org/10.1177/0275074017699470
Jensen, U. T., Moynihan, D. P., & Salomonsen, H. H. (2018). Communicating the Vision: How
Face-to-Face Dialogue Facilitates Transformational Leadership. Public Administration
Review, 78(3), 350–361. https://doi.org/10.1111/puar.12922
Joyce, P. G. (2011). The Obama Administration and PBB: Building on the Legacy of Federal
Performance-Informed Budgeting? Public Administration Review, 71(3), 356–367.
https://doi.org/https://doi.org/10.1111/j.1540-6210.2011.02355.x
Judge, T. A., & Piccolo, R. F. (2004). Transformational and Transactional Leadership: A Meta-
Analytic Test of Their Relative Validity. Journal of Applied Psychology, 89(5), 755–768.
https://doi.org/http://dx.doi.org/10.1037/0021-9010.89.5.755

120
Jung, C. S. (2014a). Why Are Goals Important in the Public Sector? Exploring the Benefits of
Goal Clarity for Reducing Turnover Intention. Journal of Public Administration Research
and Theory, 24(1), 209–234. https://doi.org/10.1093/jopart/mus058
Jung, C. S. (2014b). Organizational Goal Ambiguity and Job Satisfaction in the Public Sector.
Journal of Public Administration Research and Theory, 24(4), 955–981.
https://doi.org/10.1093/jopart/mut020
Jung, C. S., & Rainey, H. G. (2011). Organizational goal characteristics and public duty
motivation in U.S. federal agencies. Review of Public Personnel Administration, 31(1), 28–
47. https://doi.org/10.1177/0734371X10394404
Kamensky, J. M. (2011). The Obama Performance Approach. Public Performance &
Management Review, 35(1), 133–148. https://doi.org/10.2753/PMR1530-9576350107
Kervasdoué, J. de. (2008). Rating and Evaluating Health Systems: The Value of the Life
Expectancy Approach. International Public Management Journal, 11(3), 329–343.
https://doi.org/10.1080/10967490802301328
Kindarto, A., Zhu, Y. Q., & Gardner, D. G. (2020). Full Range Leadership Styles and
Government IT Team Performance: The Critical Roles of Follower and Team Competence.
Public Performance and Management Review, 0(0), 1–29.
https://doi.org/10.1080/15309576.2020.1730198
Klotz, I. (2020). NASA Chief Plans To Step Aside Under Biden. Aviation Week.
https://aviationweek.com/defense-space/space/nasa-chief-plans-step-aside-under-biden
Kotter, J. P. (1990). Force For Change: How Leadership Differs from Management. Free Press.
Krause, G. A., Lewis, D. E., & Douglas, J. W. (2006). Political Appointments, Civil Service
Systems, and Bureaucratic Competence: Organizational Balancing and Executive Branch
Revenue Forecasts in the American States. American Journal of Political Science, 50(3),
770–787. https://doi.org/https://doi.org/10.1111/j.1540-5907.2006.00215.x
Kroll, A., & Moynihan, D. P. (2018). The Design and Practice of Integrating Evidence:
Connecting Performance Management with Program Evaluation. Public Administration
Review, 78(2), 183–194. https://doi.org/https://doi.org/10.1111/puar.12865
Kroll, A., & Moynihan, D. P. (2020). Tools of Control? Comparing Congressional and
Presidential Performance Management Reforms. Public Administration Review,
puar.13312. https://doi.org/10.1111/puar.13312
Landis, J. R., & Koch, G. G. (1977). The Measurement of Observer Agreement for Categorical
Data. Biometrics, 33(1), 159–174. https://doi.org/10.2307/2529310
Lapenta, A., Fattore, G., & Dubois, H. F. W. (2012). Measuring New Public Management and
Governance in Political Debate. Public Administration Review, 72(2), 218–227.
http://www.jstor.org/stable/41433296
Lapuente, V., & Suzuki, K. (2020). Politicization, Bureaucratic Legalism, and Innovative
Attitudes in the Public Sector. Public Administration Review, 80(3), 454–467.
https://doi.org/https://doi.org/10.1111/puar.13175
121
Lavertu, S., & Moynihan, D. P. (2013). Agency Political Ideology and Reform Implementation:
Performance Management in the Bush Administration. Journal of Public Administration
Research and Theory, 23(3), 521–549. https://doi.org/10.1093/jopart/mus026
Lavertu, Stéphane, Lewis, D. E., & Moynihan, D. P. (2013). Government Reform, Political
Ideology, and Administrative Burden: The Case of Performance Management in the Bush
Administration. Public Administration Review, 73(6), 845–857.
http://www.jstor.org/stable/42003132
Lavertu, Stéphane, & Moynihan, D. P. (2013). Agency Political Ideology and Reform
Implementation: Performance Management in the Bush Administration. Journal of Public
Administration Research and Theory, 23(3), 521–549.
https://doi.org/10.1093/jopart/mus026
Lee, A., & Carpenter, N. C. (2018). Seeing eye to eye: A meta-analysis of self-other agreement
of leadership. The Leadership Quarterly, 29(2), 253–275.
https://doi.org/https://doi.org/10.1016/j.leaqua.2017.06.002
Lee, J. W., Rainey, H. G., & Chun, Y. H. (2010). Routineness in Federal Agencies. The
American Review of Public Administration, 40(3), 284–308.
Lee, S.-Y., & Hong, J. H. (2011). Does Family-Friendly Policy Matter? Testing Its Impact on
Turnover and Performance. Public Administration Review, 71(6), 870–879.
https://doi.org/https://doi.org/10.1111/j.1540-6210.2011.02416.x
Lee, S.-Y., & Whitford, A. B. (2013). Assessing the Effects of Organizational Resources on
Public Agency Performance: Evidence from the US Federal Government. Journal of Public
Administration Research and Theory, 23(3), 687–712.
https://doi.org/10.1093/jopart/mus050
Lewis, D. E. (2007). Testing pendleton’s premise: Do political appointees make worse
bureaucrats? Journal of Politics, 69(4), 1073–1088. https://doi.org/10.1111/j.1468-
2508.2007.00608.x
Lewis, D. E. (2009). Revisiting the Administrative Presidency: Policy, Patronage, and Agency
Competence. Presidential Studies Quarterly, 39(1), 60–73.
http://www.jstor.org/stable/23044875
Lewis, D. E. (2010). The Politics of Presidential Appointments: Political Control and
Bureaucratic Performance. Princeton University Press.
Lewis, D. E., & Richardson, M. D. (2021). The Very Best People: President Trump and the
Management of Executive Personnel. Presidential Studies Quarterly, n/a(n/a).
https://doi.org/https://doi.org/10.1111/psq.12697
Lewis, D. E., & Waterman, R. W. (2013). The Invisible Presidential Appointments: An
Examination of Appointments to the Department of Labor, 2001-11. Presidential Studies
Quarterly, 43(1), 35–57. https://doi.org/10.1111/psq.12002
Locke, E. A., & Latham, G. P. (2002). Building a practically useful theory of goal setting and
task motivation: A 35-year odyssey. American Psychologist, 57(9), 705–717.
https://doi.org/10.1037/0003-066X.57.9.705
122
Locke, E. A., Smith, K. G., Erez, M., Chah, D.-O., & Schaffer, A. (1994). The Effects of Intra-
individual Goal Conflict on Performance. Journal of Management, 20(1), 67–91.
https://doi.org/10.1177/014920639402000104
Long, E., & Franklin, A. L. (2004). The Paradox of Implementing the Government Performance
and Results Act: Top-Down Direction for Bottom-Up Implementation. Public
Administration Review, 64(3), 309–319. http://www.jstor.org/stable/3542596
Lonti, Z., & Gregory, R. (2007). Accountability or Countability? Performance Measurement in
the New Zealand Public Service, 1992–2002. Australian Journal of Public Administration,
66(4), 468–484. https://doi.org/https://doi.org/10.1111/j.1467-8500.2007.00557.x
Lowande, K. (2018). Politicization and Responsiveness in Executive Agencies. The Journal of
Politics, 81(1), 33–48. https://doi.org/10.1086/700270
Lowi, T. J., & Lowi, T. J. (1969). The End of Liberalism: Ideology, Policy, and the Crisis of
Public Authority (Issue v. 10). Norton.
Lynn, L. E. (1999). Requiring bureaucracies to perform : what have we leamed from the U.S.
Government Performance and Results Act (GPRA) ? Politiques et Management Public,
17(2), 1–20. https://doi.org/10.3406/pomap.1999.2227
Maranto, R, & Pfiffner, J. P. (2005). Beyond a Government of Strangers: How Career Executives
and Political Appointees Can Turn Conflict to Cooperation. Lexington Books.
Maranto, Robert, & Wolf, P. J. (2013). Cops, Teachers, and the Art of the Impossible:
Explaining the Lack of Diffusion of Innovations That Make Impossible Jobs Possible.
Public Administration Review, 73(2), 230–240. https://doi.org/10.1111/j.1540-
6210.2012.02626.x
Martz, W. (2013). Evaluating Organizational Performance: Rational, Natural, and Open System
Models. American Journal of Evaluation, 34(3), 385–401.
https://doi.org/10.1177/1098214013479151
Meier, K. J., & O’Toole, L. J. (2013). I Think (I Am Doing Well), Therefore I Am: Assessing
the Validity of Administrators’ Self-Assessments of Performance. International Public
Management Journal, 16(1), 1–27. https://doi.org/10.1080/10967494.2013.796253
Melnick, R. S. (2012). Political Science as a Vocation: An Appreciation of the Life and the Work
of James Q. Wilson. The Forum, 10(1). https://doi.org/https://doi.org/10.1515/1540-
8884.1502
Milakovich, M. E. (2006). COMPARING BUSH–CHENEY AND CLINTON–GORE
PERFORMANCE MANAGEMENT STRATEGIES: ARE THEY MORE ALIKE THAN
DIFFERENT? Public Administration, 84(2), 461–478.
https://doi.org/https://doi.org/10.1111/j.1467-9299.2006.00012.x
Miller, B., & Curry, B. (2018). U.S. Attorneys, Political Control, and Career Ambition. Oxford
University Press. https://doi.org/10.1093/oso/9780190928247.001.0001

123
Miller, G., & Whitford, A. B. (2007). The Principal’s Moral Hazard: Constraints on the Use of
Incentives in Hierarchy. Journal of Public Administration Research and Theory, 17(2),
213–233. https://doi.org/10.1093/jopart/mul004
Min, K. R., Ugaddan, R. G., & Park, S. M. (2016). Is the Creative Tendency Affected by
Organizational Leadership and Employee Empowerment? An Empirical Analysis of U.S.
Federal Employees. Public Performance and Management Review, 40(2), 382–408.
https://doi.org/10.1080/15309576.2016.1230503
Moe, T. (1985). The Politicized Presidency. In J. Chubbs & P. Peterson (Eds.), The New
Direction in American Politics. Brookings Institution Press.
Moynihan, D. P. (2013). Advancing the Empirical Study of Performance Management: What We
Learned From the Program Assessment Rating Tool. American Review of Public
Administration, 43(5), 499–517. https://doi.org/10.1177/0275074013487023
Moynihan, D. P., & Pandey, S. K. (2005). Testing how management matters in an era of
government by performance management. Journal of Public Administration Research and
Theory, 15(3), 421–439. https://doi.org/10.1093/jopart/mui016
Moynihan, D. P., Pandey, S. K., & Wright, B. E. (2012). Setting the table: How transformational
leadership fosters performance information use. Journal of Public Administration Research
and Theory, 22(1), 143–164. https://doi.org/10.1093/jopart/mur024
Moynihan, D. P., Wright, B. E., & Pandey, S. K. (2012). Working Within Constraints: Can
Transformational Leaders Alter the Experience of Red Tape? International Public
Management Journal, 15(3), 315–336. https://doi.org/10.1080/10967494.2012.725318
Musso, J. A., Young, M. M., & Thom, M. (2019). Volunteerism as co-production in public
service management: application to public safety in California. Public Management Review,
21(4), 473–494. https://doi.org/10.1080/14719037.2018.1487574
Nathan, R. P. (1983). The Administrative Presidency. Wiley.
National Research Council. (1999). Health Performance Measurement in the Public Sector:
Principles and Policies for Implementing an Information Network (E. B. Perrin, J. S. Durch,
& S. M. Skillman (eds.)). The National Academies Press. https://doi.org/10.17226/6487
Newcomer, K. (2007). How does Program Performance Assessment Affect Program
Management in the Federal Government? Public Performance & Management Review,
30(3), 332–350. https://doi.org/10.2753/pmr1530-9576300302
Nielsen, P. A., Boye, S., Holten, A.-L., Jacobsen, C. B., & Andersen, L. B. (2019). Are
transformational and transactional types of leadership compatible? A two-wave study of
employee motivation. Public Administration, 97(2), 413–428.
https://doi.org/https://doi.org/10.1111/padm.12574
Northouse, P. G. (2018). Leadership: Theory and Practice. SAGE Publications.
Oberfield, Z. W. (2014). Public management in time: A longitudinal examination of the full
range of leadership theory. Journal of Public Administration Research and Theory, 24(2),
407–429. https://doi.org/10.1093/jopart/mus060
124
Olsen, J. P. (2015). Democratic Order, Autonomy, and Accountability. Governance, 28(4), 425–
440. https://doi.org/https://doi.org/10.1111/gove.12158
Park, S., & Lake, E. T. (2005). Multilevel Modeling of a Clustered Continuous Outcome.
Nursing Research, 54(6), 406–413. https://doi.org/10.1097/00006199-200511000-00007
Park, S. M., & Rainey, H. G. (2008). Leadership and Public Service Motivation in U.S. Federal
Agencies. International Public Management Journal, 11(1), 109–142.
https://doi.org/10.1080/10967490801887954
Pitts, D. (2009). Diversity Management, Job Satisfaction, and Performance: Evidence from U.S.
Federal Agencies. Public Administration Review, 69(2), 328–338.
https://doi.org/https://doi.org/10.1111/j.1540-6210.2008.01977.x
Radin, B. A. (1998). The Government Performance and Results Act ( GPRA ): Hydra-Headed
Monster or Flexible Management Tool ? Published by : Wiley on behalf of the American
Society for Public Administration Stable URL : https://www.jstor.org/stable/977560
ResdltsAct ( GPRA ): Public Administration Review, 58(4), 307–316.
Radin, B. A. (2000). The Government Performance and Results Act and the Tradition of Federal
Management Reform: Square Pegs in Round Holes? Journal of Public Administration
Research and Theory, 10(1), 111–135. https://doi.org/10.1093/oxfordjournals.jpart.a024260
Radin, B. A. (2003). Caught between agendas: Gpra, devolution, and politics. International
Journal of Public Administration, 26(10–11), 1245–1255. https://doi.org/10.1081/PAD-
120019930
Rainey, H. (2012). Goal Ambiguity and the Study of American Bureaucracy. In R. F. Durant
(Ed.), The Oxford Handbook of American Bureaucracy (pp. 231–251). OUP Oxford.
Randall, R. (1982). Presidential use of management tools from PPB to ZBB. Presidential Studies
Quarterly, 12(2), 186–194.
Raudenbush, S. W., Bryk, S. W. R. A. S., Bryk, A. S., & Coaut, B. A. S. (2002). Hierarchical
Linear Models: Applications and Data Analysis Methods. SAGE Publications.
Resh, W. G. (2015). Rethinking the Administrative Presidency: Trust, Intellectual Capital, and
Appointee-Careerist Relations in the George W. Bush Administration. Johns Hopkins
University Press.
Resh, W. G., & Pitts, D. W. (2013). No Solutions, Only Trade-Offs? Evidence about Goal
Conflict in Street-Level Bureaucracies. Public Administration Review, 73(1), 132–142.
https://doi.org/https://doi.org/10.1111/j.1540-6210.2012.02623.x
Resh, W., Moldogaziev, T., Fernandez, S., & Leslie, C. A. (2019). Reversing the Lens:
Assessing the Use of Federal Employee Viewpoint Survey in Public Administration
Research. Review of Public Personnel Administration, 1–31.
https://doi.org/10.1177/0734371X19865012
Richardson, M. D. (2019). Politicization and expertise: Exit, effort, and investment. Journal of
Politics, 81(3), 878–891. https://doi.org/10.1086/703072
125
Roberts, N. (2000). The Synoptic Model of Strategic Planning and the GPRA: Lacking a Good
Fit with the Political Context. Public Productivity & Management Review, 23(3), 297.
https://doi.org/10.2307/3380721
Rosenbloom, D. H. (1983). Public Administrative Theory and the Separation of Powers. Public
Administration Review, 43(3), pp.-219-227. https://doi.org/10.2307/976330
Rourke, F. E., & Doig, J. W. (1991). James Q. Wilson’s “Bureaucracy”: Two Reviews. Journal
of Public Administration Research and Theory: J-PART, 1(1), 90–99.
http://www.jstor.org/stable/1181768
Ryan, J. C., & Tipu, S. A. A. (2013). Leadership effects on innovation propensity: A two-factor
full range leadership model. Journal of Business Research, 66(10), 2116–2129.
https://doi.org/https://doi.org/10.1016/j.jbusres.2013.02.038
Schick, A. (1978). The Road from ZBB. Public Administration Review, 38(2), 177–180.
Selin, J. L. (2015). What Makes an Agency Independent? American Journal of Political Science,
59(4), 971–987. https://doi.org/https://doi.org/10.1111/ajps.12161
Shipan, C. R., & Shannon, M. L. (2003). Delaying Justice(s): A Duration Analysis of Supreme
Court Confirmations. American Journal of Political Science, 47(4), 654–668.
https://doi.org/10.1111/1540-5907.00046
Simon, H. A. (1997). Administrative Behavior, 4th Edition. Free Press.
Skogstad, A., Einarsen, S., Torsheim, T., Aasland, M. S., & Hetland, H. (2007). The
destructiveness of laissez-faire leadership behavior. Journal of Occupational Health
Psychology, 12(1), 80–92. https://doi.org/10.1037/1076-8998.12.1.80
Slocum, J. W., Cron, W. L., & Brown, S. P. (2002). The Effect of Goal Conflict on Performance.
Journal of Leadership & Organizational Studies, 9(1), 77–89.
https://doi.org/10.1177/107179190200900106
Somers, M. J. (2018). Strategies for Improving Measurement Models for Secondary Data in
Public Administration Research: Illustrations from the Federal Employee Viewpoint
Survey. Public Administration Review, 78(2), 228–239. https://doi.org/10.1111/puar.12805
Song, M., & Meier, K. J. (2018). Citizen Satisfaction and the Kaleidoscope of Government
Performance: How Multiple Stakeholders See Government Performance. Journal of Public
Administration Research and Theory, 28(4), 489–505.
https://doi.org/10.1093/jopart/muy006
Steenbergen, M. R., & Jones, B. S. (2002). Modeling Multilevel Data Structures. American
Journal of Political Science, 46(1), 218. https://doi.org/10.2307/3088424
Sun, R., & Henderson, A. C. (2017). Transformational Leadership and Organizational Processes:
Influencing Public Performance. Public Administration Review, 77(4), 554–565.
https://doi.org/10.1111/puar.12654

126
Tang, A. S., Robertson, P. J., Lane, C. E., Productivity, S. P., Review, M., Mar, N., Tang, S.,
Robertson, P. J., & Lane, C. E. (1996). ORGANIZATIONAL TYPES , COMMITMENT ,
AND. 19(3), 289–312.
Tavich, D. (2017). New Zealand Productivity Commission internship: social sector productivity.
Policy Quarterly, 13(4), 80–83. https://doi.org/10.26686/pq.v13i4.4584
Thompson, J. R., & Fulla, S. L. (2001). Effecting Change in a Reform Context. Public
Performance & Management Review, 25(2), 155–175.
https://doi.org/10.1080/15309576.2001.11643652
Ting, M. M. (2001). The “Power of the Purse” and Its Implications for Bureaucratic Policy-
Making. Public Choice, 106(3/4), 243–274. http://www.jstor.org/stable/30026219
Trommel, W., Honingh, M., & Brandsen, T. (2012). Professional Pride: A Powerful Force. 23,
June 2015.
Trottier, T., Van Wart, M., & Wang, X. (2008). Examining the nature and significance of
leadership in government organizations. Public Administration Review, 68(2), 319–333.
https://doi.org/10.1111/j.1540-6210.2007.00865.x
Vandenabeele, W. (2014). Explaining Public Service Motivation: The Role of Leadership and
Basic Needs Satisfaction. Review of Public Personnel Administration, 34(2), 153–173.
https://doi.org/10.1177/0734371X14521458
Walker, R. M., Chandra, Y., Zhang, J., & van Witteloostuijn, A. (2019). Topic Modeling the
Research-Practice Gap in Public Administration. Public Administration Review, 79(6), 931–
937. https://doi.org/10.1111/puar.13095
Waterman, R. W., & Ouyang, Y. (2020). Rethinking Loyalty and Competence in Presidential
Appointments. Public Administration Review, 9999, 1–16.
https://doi.org/10.1111/puar.13212
Weibel, A., Rost, K., & Osterloh, M. (2010). Pay for Performance in the Public Sector—Benefits
and (Hidden) Costs. Journal of Public Administration Research and Theory, 20(2), 387–
412. https://doi.org/10.1093/jopart/mup009
Wenger, J. B., Toole, L. J., & Meier, K. J. (2008). Trading speed for accuracy? Managing goal
conflict and accommodation in the U.S. unemployment insurance program. Policy Studies
Journal, 36(2), 175–198. https://doi.org/10.1111/j.1541-0072.2008.00261.x
West, W. F. (2005). Neutral competence and political responsiveness: an uneasy relationship.
Policy Studies Journal, 33, 147+.
Whitford, A. B. (2005). The Pursuit of Political Control by Multiple Principals. The Journal of
Politics, 67(01), 29–49. https://doi.org/10.1111/j.1468-2508.2005.00306.x
Whitford, A. B., Lee, S.-Y., Yun, T., & Jung, C. S. (2010). Collaborative Behavior and the
Performance of Government Agencies. International Public Management Journal, 13(4),
321–349. https://doi.org/10.1080/10967494.2010.529378

127
Wildavsky, A. B. (1979). Speaking Truth to Power: The Art and Craft of Policy Analysis. Little,
Brown.
Wilson, J. Q. (1989). Bureaucracy. Basic Books.
Wong, S. I., & Giessner, S. R. (2016). The Thin Line Between Empowering and Laissez-Faire
Leadership: An Expectancy-Match Perspective. Journal of Management, 44(2), 757–783.
https://doi.org/10.1177/0149206315574597
Wood, A. K., & Lewis, D. E. (2017). Agency performance challenges and agency politicization.
Journal of Public Administration Research and Theory, 27(4), 581–595.
https://doi.org/10.1093/jopart/mux014
Woodrow Wilson. (1887). The Study of Administration Author ( s ): Woodrow Wilson Source :
Political Science Quarterly , Vol . 2 , No . 2 ( Jun ., 1887 ), pp . 197-222 Published by : The
Academy of Political Science Stable URL : http://www.jstor.org/stable/2139277. Political
Science Quarterly, 2(2), 197–222.
Wright, B. E., Moynihan, D. P., & Pandey, S. K. (2012). Pulling the Levers: Transformational
Leadership, Public Service Motivation, and Mission Valence. Public Administration
Review, 72(2), 206–215. https://doi.org/https://doi.org/10.1111/j.1540-6210.2011.02496.x
Wright, B. E., & Pandey, S. K. (2010). Transformational leadership in the public sector: Does
structure matter? Journal of Public Administration Research and Theory, 20(1), 75–89.
https://doi.org/10.1093/jopart/mup003
Yammarino, F. J., Spangler, W. D., & Bass, B. M. (1993). Transformational leadership and
performance: A longitudinal investigation. The Leadership Quarterly, 4(1), 81–102.
https://doi.org/https://doi.org/10.1016/1048-9843(93)90005-E
Yang, I. (2015). Positive effects of laissez-faire leadership: conceptual exploration. Journal of
Management Development, 34(10), 1246–1261. https://doi.org/10.1108/JMD-02-2015-0016

Abstract (if available)

Abstract This dissertation is motivated by James Q. Wilson’s seminal book, Bureaucracy: What Government Agencies Do And Why They Do It. Although Wilson explicitly mentions that the propositions made in the book are not a theory but rather observations that need further empirical validation, studies in public administration and political science cite his propositions without any further validation. In order to examine whether Wilson’s propositions are applicable beyond his time and observations, this study establish Wilson’s famous agency typology based one empirical data. It uses a novel data set based on approximately 21,000 performance measures from 28 agencies across a 10-year time span with each goal coded by its outcome observability, or the extent to which the achievement of a given goal can be attributable largely to the agency efforts. As a result of the analysis of average goal observability, 28 federal agencies are placed on the continuum of agency outcome observability scores and grouped into three categories (low, medium, and high). Next, the study hypothesizes that performance in federal agencies varies as a function of agency leadership—(1) who runs the agency (political appointees and/or careerists)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

The functions of the middleman: how intermediary nonprofit organizations support the sector and society

PDF

The effects of interlocal collaboration on local economic performance: investigation of Korean cases

PDF

How can metrics matter: performance management reforms in the City of Los Angeles

PDF

A study of Chinese environmental NGOs: policy advocacy, managerial networking, and leadership succession

PDF

Three essays on the causes and consequences of China’s governance reforms

PDF

Performance management in government: the importance of goal clarity

PDF

Mission match, reputation, and bureaucratic motivation

PDF

Governing public goods: how representation and political power in local and regional institutions shape inequalities

PDF

The risks and rewards of city management: how city managers evaluate the nature of the job and compensation

PDF

Comparative study of organizational commitment in the public and private sectors: The case of transportation agencies in Thailand

PDF

The influence of organizational ethics on job attitudes and government performance

PDF

Exploring heterogeneous effects of performance-based funding: Implications for equity and policy

PDF

Essays on fiscal outcomes of cities in California

PDF

Structure, agency, and the Kuznets Curve: observations and implications for sustainability planning in U.S. cities

PDF

Does collaborative R&D policy work? The effect of collaborative R&D policy on innovative activities of firms in Korea

PDF

Why go green? Cities' adoption of local renewable energy policies and urban sustainability certifications

PDF

A framework for evaluating urban policy and its impact on social determinants of health (SDoH)

PDF

Does organizational culture play a role in aviation safety? A qualitative case study analysis

PDF

The economy-environment trade-off in China's air pollution policies

PDF

Three essays on the political economy of federal spending

Asset Metadata

Creator Cho, Heejin (author)

Core Title Goal setting and performance in federal agencies

School School of Policy, Planning and Development

Degree Doctor of Philosophy

Degree Program Public Policy and Management

Publication Date 04/26/2021

Defense Date 03/04/2021

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag agency performance,appointee politics,OAI-PMH Harvest,performance measurement,public leadership

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Resh, William (committee chair), Robertson, Peter (committee member), Tang, Shui Yan (committee member)

Creator Email heejinc@usc.edu,heejincho121@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c89-453279

Unique identifier UC11668417

Identifier etd-ChoHeejin-9545.pdf (filename),usctheses-c89-453279 (legacy record id)

Legacy Identifier etd-ChoHeejin-9545.pdf

Dmrecord 453279

Document Type Dissertation

Rights Cho, Heejin

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA