University of Southern California Dissertations and Theses

How urban school superintendents effectively use data-driven decision making to improve student achievement

(USC Thesis Other)

How urban school superintendents effectively use data-driven decision making to improve student achievement

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

HOW URBAN SCHOOL SUPERINTENDENTS EFFECTIVELY USE DATA

DRIVEN DECISION MAKING TO IMPROVE STUDENT ACHIEVEMENT

by

Lonny Gene Root

___________________________________________________________

A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION

May 2010

Copyright 2010 Lonny Gene Root

ii

TABLE OF CONTENTS
LIST OF TABLES vi
ABSTRACT vii
Chapter 1: INTRODUCTION 1
The Changing Role of the Superintendent 1
Background of the Problem 2
The Changing Role of the Superintendent 2
Superintendents’ Response to Increased Accountability 3
Importance of Data-Driven Decision Making 3
Statement of the Problem 9
Purpose of the Study 11
Research Questions 11
The Importance of the Study 12
Assumptions of the Study 13
Limitations of the Study 13
Delimitations of the Study 13
Definitions of Terms 14
Organization of the Study 15
Chapter 2: REVIEW OF THE LITERATURE 16
Organization of Literature Review 16
The History of Data and Accountability 17
Using Data Effectively and Determining their Value 24
The Technology and Skills Necessary for Data Driven Decision Making 28
Increasing the Use of Data throughout the District 33
Translating Data into Changes in the Classroom 39
Conclusion 45
Chapter 3: METHODOLOGY 47
Research Questions 48
Design of the Study 49
Participants 50
Selection of Participants 51
Characteristics of Participating Superintendents 52
Measures and Survey Instruments 53
Validity and Reliability 55
Procedure for Data Collection 55
Structured Interview Data Collection 56
Data Analysis 57
Superintendent Survey Data Analysis 57
Structured Interview Data Analysis 57
Validity Concerns 58

iii

Summary 59

Chapter 4: FINDINGS 60
Introduction 60
Research Questions 60
Results of the Superintendent Data-Driven Decision Making
Survey Questionnaire 61
Demographics of Survey Questionnaire Sample 63
Urban Superintendent Population and Sample 63
The Superintendents and Their Districts 64
Superintendents Participating in Individual Personal Interviews 65
Findings by Research Questions 67
Research Question One: Competencies Superintendents Need to
Effectively Use Student Data in Making the Decisions Affecting
Student Achievement 67
Results from Surveys 67
Competencies Superintendents Rated Highest 70
Competencies Superintendents Rated Lowest 71
Acquisition of Data Use Competencies 72
Discussion of Survey Results for Research Question One 73
Research Question One Results from Interviews 74
Discussion of Interview Responses 77
Research Question Two: What Sources of Data Superintendents
Use and How They Determine the Value of That Data 79
Types of Data Superintendents Value and Use in
Decision Making 81
Types of Data Superintendents Value and Use in
Communication 82
Types of Data Superintendents Value in School
Site Decisions 84
Discussion of Survey Results for Research Question Two 86
Results from Interviews for Research Question Two 88
Discussion of Interview Results –Research
Question Two 90
Research Question Three: Specific Strategies and Policies
Superintendents Use to Increase the Use of Student
Data by Educators in Their District 92
Actions Superintendents Engage in to Increase
Data-Driven Decision Making in Others 92
Discussion of Survey Results for Research
Question Three 93
Interviews Results for Research Question Three 95

iv

Discussion of Interview Results for Research
Question Three 98
Research Question Four: How Superintendents Use Data to
Determine the Effectiveness of Their Educators,
Administrators, and Programs 100
Superintendents’ Level of Confidence in Using
Data in Evaluations 101
Types of Data Superintendents Use and Value
in Evaluations 102
Discussion of Survey Results of Research
Question Four 103
Interview Results for Research Question Four 105
Discussion of Interview Results of Research
Question Four 108
Summary 110

Chapter 5: DISCUSSION 112
Introduction 112
Significant Findings 115
The Competencies Superintendents Need and Apply Effectively
in the Use of Student and Staff Data in Making the Decisions
That Will Effect Student Achievement 115
Competencies Required for Effective Data-Driven
Decision Making 116
Effective Data-Driven Decision Making Competencies
Possessed by Superintendents in the Study 117
How Effective Data-Driven Decision Making
Competencies Were Acquired 119
What Sources of Data Superintendents Use and How They
Determine the Value of That Data 121
Types of Data that Superintendents Use 121
California Standards Test (CST) 121
Benchmark and Program Data 122
Demographic Data 122
The Specific Strategies and Policies Superintendents Use to
Increase the Acceptance of Using Student Data by Educators in
Their District 123
Acquisition of a Computerized Data Program
and Technology 124
Providing Professional Development in the Use of Data 125
Communicating Clear Expectations of Data Use
by Educators 125
Promoting the Use of Data and Providing
Opportunities to Use Data 126

v

How Superintendents Use Data to Determine the Effectiveness
of Their Educators, Administrators, and Programs 126
Use of Data to Evaluate Principals 127
Use of Data to Evaluate Teachers 128
Use of Data to Evaluate Programs 129
Effective Data-Driven Decision Making Competencies 130
Practices Superintendents Use in Effective Data-Driven
Decision Making 132
Effective Use of Data in the Evaluation of Educators 134
Implications for Practice 134
Formal Program for the Acquisition of Data-Driven Decision
Making Skills 135
Changing Skill Requirements in the Use of Data 135
Using Data to Evaluate Teachers and Principals 136
Implications for Research 136

REFERENCES 138

APPENDICES:
Appendix A: Interview Questions 141
Appendix B: Data- Driven Decision Making Survey Questions 142
Appendix C: Research Introduction Letter 144
Appendix D: Information Sheet for Non-Medical Research 145
Appendix E: Results from Superintendent Surveys – Data Use
Competencies 147
Appendix F: Results from Superintendent Surveys – Data That
Superintendents Use and Value 149
Appendix G: Results from Superintendent Surveys –Frequency
of Engaging in Data-Driven Decision Making
Activities 151
Appendix H: Data-Driven Decision Making Framework 152

vi

LIST OF TABLES
Table 1: District Information of Superintendents Participating in the Study 51
Table 2: Descriptive Statistics of the Superintendents Participating
in Surveys 64
Table 3: Descriptive Statistics of the Superintendents’ Districts 64
Table 4: Descriptive Statistics of the Superintendents Participating in
Personal Interviews and Their Districts 66
Table 5: Rating of Superintendents Data Use Competencies 70
Table 6: Superintendents Level of Competencies Levels in Data Use
Skills from Surveys 71
Table 7: Where and How Superintendents Acquired Their Data-Driven
Decision Making Competencies 73
Table 8: Trends from Interviews with Superintendents 77
Table 9: Types of Data Identified in Surveys That Are Used by
Superintendents to Make Decisions 82
Table 10: Types of Data Identified in Surveys That Are Used by
Superintendents to Communicate with Staff and the
Community 83
Table 11: Types of Data Identified in Surveys That Superintendents
Expect Their Educators to Use to Make Decisions 85
Table 12: Summary Averages of Types of Data Identified in Surveys
That Are Used by Superintendents 86
Table 13: Data Use Trends from Interviews with Superintendents 90
Table 14: Frequency of Data Use Actions by Superintendents 93
Table 15: Trends in Use of Data from Interviews with Superintendents
from the Conceptual Framework for Effective Data-Driven
Decision Making 97
Table 16: Superintendent Competency Levels in Using Data to Evaluate
Educators 102
Table 17: Data Valued by Superintendents When Using Data to Evaluate
Educators 103
Table 18: Trends in Use of Data for Evaluations from Interviews with
Superintendents from the conceptual framework for effective
data-driven decision making 108

vii

ABSTRACT
With the passage of the No Chile Left Behind (NCLB) Act of 2002, schools,
districts, and therefore, superintendents have been held increasingly accountable for
the achievement of the students. The states and federal governments have used
student achievement data to measure the progress and success of schools and
districts and have held districts accountable to this data. This study examined the use
of data by superintendents in their decision making related to student achievement.
The study looked at what competencies they possessed in the use of data and how
they learned those competencies, and what types of data they used and valued. The
study also identified what actions, policies, and communications superintendents
used to promote data use among all of the educators in their districts.
Superintendents’ use of data in the evaluation of principal, teachers, and educational
programs was also included in this study.
This study used a mixed-methods (quantitative and qualitative) approach to
examine a population of 23 superintendents from urban school districts in California.
All of the participating superintendents had been in their position for at least two
years and demonstrated improved student achievement as measured by their API
scores. All 23 superintendents responded to a 45 questions Data-Driven Decision
Making Survey Questionnaire designed to measure and identify specific skills they
possessed, the level of those skills, and for what types of decisions did they use those
skills. It identified which types of data they believed valuable and which they used in
a set of decisions that superintendent often make. It also measured the frequency that
they engaged in certain data-driven decision making actions. Five superintendents

viii

whose responses and reputations indicated that they effectively used data in their
decision making were selected from the group of 23 and participated in personal
interviews. They responded to 13 pre-selected questions designed to elicit detailed
information in the same areas as the surveys.
The research study made several significant findings. Participating
superintendents reported that they possess nearly all of the competencies needed to
make effective-data driven decisions and that they learned most of these
competencies through on-the-job experiences rather than any type of formal training.
These superintendents identified CST data, benchmark data, and program data as the
types that they used and valued in making most of their decisions. The results
indicated that the actions superintendents were most effective in for the purpose of
promoting data use among all educators in their districts were: communicating
district goals, expectations, and results using data, and providing the time and
resources for using data to teachers. The study revealed that the superintendents
used data effectively in evaluating their principals and programs, but did not use data
in the evaluation of their teachers.

1

CHAPTER ONE
INTRODUCTION
Educators today are under ever increasing pressure to increase student
achievement. Teachers, school administrators, all the way up to the district
superintendents are looking for new and better ways to help more of their students
achieve. Superintendents and their directors and administrators must be able to use
student data to choose or design curricular programs, textbooks, interventions, and
even to hire teachers that will specifically address achievement gaps.
The expectations of the superintendency have changed since public school
districts first created the position in 1865. Initially the duties of the position were as a
supervisor of teachers and principals to ensure that they were carrying out the
policies and vision of the school board members. They communicated the board’s
decisions to the school sites and ensured they were carried out, and they discussed
with the board members the needs of the teachers and schools. In 1910 the
expectations of the position changed to a more managerial one in which the
superintendent worked with the school board to create policy and directed principals
and teachers to carry out their policies. The communication, for the most part, was in
one direction, from the top, down. By the 1950’s the superintendency had evolved
into a more political position that required true leadership rather than just managerial
skills. By the 1960’s successful superintendents had become true social scientists,
able to solve real economic, political, and academic problems (Kowalski, 2005).

2

Superintendents have started looking at data in new and different ways in an
attempt to develop methods that use such data to help them make effective decisions
related to student achievement. The Professional Standards for the Superintendency
that have been adopted by the American Association of School Administrators
[AASA], (1993) identifies eight standards and indicators of competencies that
superintendents must meet to be effective in their positions. These Standards address
the following: Standard 1- leadership and district culture; Standard 2 – policy and
governance; Standard 3 – communications and community relations; Standard 4 –
organizational management; Standard 4 – curriculum planning and development;
Standard 6 – instructional management; Standard 7 – human resources management;
Standard 8 – values and ethics of leadership. Standard 4 specifically identifies the
use of data in making decisions. The complete Standard reads as the following:
Standard 4: Organizational Management, Exhibit an understanding of the
school district as a system by defining processes for gathering, analyzing,
and using data for decision making; manage the data flow; frame and solve
problems; frame, develop priorities, and formulate solutions; assist others to
form reasoned opinions; reach logical conclusions and make quality
decisions to meet internal and external customer expectations; plan and
schedule personal and organization work; establish procedures to regulate
activities and projects; delegate and empower at appropriate organizational
levels; secure and allocate human and material resources; develop and
manage the district budget; maintain accurate fiscal records. (American
Association of School Administrators, 1993, p. 8)
According to this Standard, superintendents are expected to know how the use of
data fits into the operation of their districts and to be able to use that data to make
decisions on student achievement, organizational structure, and fiscal and human
resources. In California the data required under the No Child Left Behind Act

3

(NCLB, 2001) is frequently the data used by superintendents and other educators.
These data are identified in the Adequate Yearly Progress (AYP), which measures
the level of math and English proficiency of all students by subgroups, and the
Academic Performance Index (API), which measures the overall proficiency levels
of the students in a school or district. Data are collected by each state to determine
the level and percentage of proficiency of all students in a district. According to
NCLB, all students must be proficient in English and Math by 2014. All students
really means all students, regardless of gender, race or ethnicity, socio-economic
status, home language, or disability. Under NCLB schools and districts are held
accountable for the achievement of students who have historically achieved at levels
below the general population.
The level of accountability being experienced by schools and districts is
unprecedented, with states taking control of schools and districts that fail to reach the
required levels of achievement year after year. Superintendents must know exactly
how all of their students are achieving, not only at each of their schools but also by
each subgroup of students. They have to know exactly at what level each group of
students is achieving and compare that with the level of achievement required by
NCLB. The achievement gaps that arise between the groups of students who are
underachieving and their peers in the general population are what many schools and
districts look at and try to address.
The use of data is an essential part of effectively addressing the problems of
student achievement. Data gathering alone is meaningless without the educators’
ability to understand how it links to what they are trying to achieve. It is important

4

that educators place the appropriate relative value on the large variety of possible
student data. Data’s value comes from the insight the information offers educators to
find solutions to the problems they face or determine the effectiveness of strategies
already in place. Data can have multiple meanings depending on who is interpreting
the information and for what purpose. Deciding what questions they want answered
will determine what data an educator should use (Lashway, 2002). Collecting and
looking at the data are only the first steps and add little value to what educators do
without the ability to analyze the data correctly and translate the meaning provided
into appropriate actions. Data can be used from many sources to improve schools
from various angles. Many superintendents use fiscal resource data to determine the
best way to allocate funds so that student learning is supported in the most effective
manner. Districts with fewer students and fewer financial resources have been able to
use data-driven decision making to employ those resources in a way that allows them
to sustain or improve student achievement on par with districts with greater
resources (Pan, Rudo, and Smith-Hanson, 2002).
Data have become an important part of this examination into student
achievement. Superintendents must not only use their students’ achievement results
themselves but they must also make sure that all educators in their district can
effectively use this data. Superintendents and their directors and administrators must
be able to use student achievement outcomes to choose or design curricular
programs, textbooks, interventions, and even hire teachers that will specifically
address achievement gaps. Organization-related factors, such as school district
requirements and accessibility of data, tend to have more direct influence on data use

5

by teachers and principals where data were used infrequently. High school
principals without background in research and measurement have difficulty in
understanding and interpreting the data presented to them for their decision making
and are less likely to use it (Luo, 2008). In other words, increased accessibility of
meaningful data and the expectation from district leadership that the data were to be
used increased the use of data by principals and teachers. Superintendents and their
districts had to invest in professional development for their principals and teachers
for them to learn how to analyze data and turn it into actions. Each superintendent
uses data differently and places varying degrees of emphasis on the use of data. How
data are disaggregated, either by demographics, or by time, is important in providing
meaning and use to the data. Many districts and superintendents are able to
successfully focus their resources and efforts where it will make the greatest impact
using student assessment outcomes. Data becomes critical in the decision making
process for superintendents who often have to change past practices, reallocate
resources, and increase emphasis on failing students. Superintendents must have a
basic understanding of applied statistics, data analysis, and the necessary computer
skills to effectively use the data that are available. Successful superintendents are
skillful at interpreting and conducting research, evaluating programs, and planning
for the future (Luo, 2008). Information is more likely to be used by decision makers
if it is from a source deemed as credible or trustworthy and central to the user’s
functioning; better quality information is generally associated with improved
decision-making performance.

6

How data can be collected in a valid and reliable form is one of the key elements for
school administrators in using data for their decision making.
There are three types of data that districts use: assessment, demographics, and
program. All these can be linked to student achievement, but assessments are the
way that educators measure student achievement. Data can be used to reveal the
needs of a district or school or to determine the effectiveness of the district’s
programs or curriculum. Many superintendents and their districts have implemented
additional assessments of their students in order to obtain and analyze student
achievement data before those mandated by NCLB and/or part of federal and state
accountability. The additional assessments include formative assessments designed
by teachers, benchmark exams at the district level, and final course exams. Most
districts also conduct a variety of school, district, and state level surveys to provide
data on school climate and safety, student/parent satisfaction, and adequacy of
resources. Data from these sources can often be linked directly, or indirectly, to
student achievement. Formative assessments require educators to determine a critical
measuring point in the curriculum and then develop a way a measuring students’
mastery of the curriculum to that point. This provides immediate data on the level of
understanding by students at that point and is timely enough to allow the teacher to
re-teach or provide intervention to students who did not demonstrate mastery.
Precautions must be made to ensure that only what is important is being measured
since assessments and data analysis can be time consuming (Kroeger, Blaser, Raack,
Cooper, & Kinder, 2000).

7

Superintendents can require these interim assessments and the collection of the data
or they can work toward creating a culture in which the educators in their district
routinely use data and know how to make decisions based on that data.
While technology is not necessary to collect and analyze data, it can play an
important part in using data effectively. The state provides achievement data to
schools and districts from the NCLB assessments each year, but it takes someone or
a group of people within the district or school site to make meaning of that data. Data
that comes from the district and school assessments have to be collected, compiled
and analyzed by personnel within the district. Many districts have acquired
technology and computer programs to enable them to collect and use this data
efficiently and effectively. Whichever software is chosen, four factors must be
considered in choosing it: functionality, data storage capacity, format and training.
Superintendents must be able to determine which computer programs and technology
will provide their educators with the data they need and in a functional mode before
expending district resources on this, often expensive, equipment and software.
Superintendents must evaluate the cost effectiveness of using technology to provide
and analyze data to decide if the amount of money and resources spent on the
technology and the professional development related to it will result in an adequate
increase in student achievement.
When a superintendent implements an effective and easy-to-use data collection
system and either sets a policy for it use or creates an expectation of data use,
principals, teachers and other educators in their districts are more likely to use it and
recognize how the use of data can help them improve what they do in the

8

classrooms. As principals bear ultimate responsibility for effective data-driven
decision making, the district should have the appropriate policies in place to
guarantee the implementation of data-driven improvement (American Association of
School Administrators [AASA], 2002). A difficult to use or less effective system
would increase teacher resistance to the use of data. Teachers and principals need to
have a basic understanding of applied statistics, data analysis skills, and other
necessary computer skills to be comfortable using data before they will be able to
use it effectively (Luo, 2008). Many teachers, and their unions, believe that data will
be used by administrators to evaluate them. It is important that superintendents
establish a practice of teachers using data to focus on student achievement and
classroom practices and that it is well communicated to the teachers that the process
isn’t an attempt to evaluate them. Educators can become overwhelmed when they
first start to use data and superintendents and district administrators must guide them
in what data to use and engage in dialogue about what goals the data can address
(Lashway, 2002). There must be some up-front work done to create a data friendly
district in which data is seen as enlightening and useful for guiding instruction rather
than as irrelevant and necessary only for completing mandatory reports.
When superintendents infuse the use of data into their districts and it becomes an
accepted practice among their educators, then professionalism increases among those
educators. The use of data is important to making teaching and learning a result of
proven researched-based practices and curriculum. A supportive administrative
organization structure plays a key role in the practice of data-driven decision making
(Luo, 2008). Teaching is becoming more and more a profession on par with the

9

medical and legal fields and the use of data and research is an important part of that
transformation. Districts and schools whose educators have developed the ability to
analyze data and understand how it is essential to improving instruction have been
able to develop programs that demonstrated increased student achievement. Schools
and districts can use data to coordinate various programs and determine which ones
support or rely on other programs. District leaders have found that data guides
interventions, informs program refinements and drives staff development. Data
collection must be on going and its use timely. Once a need is identified and a
remedy decided on, it should be implemented (Walpole, Justice, & Invernizzi, 2004).
Statement of the Problem

Superintendents are under increasing pressure to improve student achievement for
all students in their districts. Improving student achievement often means changing
or improving what is being done in the classroom. Knowing what to change or do
differently requires meaningful student data, achievement gap data and research
proven teaching practices. Superintendents must be able to identify which data are
meaningful and how to provide that data to those who will use them.
Data provide an important tool in determining the achievement gaps and how to
address them, but many educators do not know how to effectively utilize data to
impact instruction. Superintendents must be able to successfully create or sustain a
culture of data-driven decision making, but how to do this is not standard practice.
Superintendents must be able to decide which way is the most appropriate for
providing the professional development that enables teachers and administrators to
understand student data and make connections to classroom practices.

10

Superintendents have limited resources and must know how to choose systems
necessary to collect and analyze data in an effective and efficient manner.
For a superintendent to ensure improvement in student achievement, educators
need to know how to use data from multiple sources. They must identify sources of
data that they can rely on and find most useful. How superintendents choose the
sources of the data their districts use varies from superintendent to superintendent
and varies in success.
Teacher unions are suspicious that student achievement data will be used to
evaluate teachers and have historically put limits on the use of such data.
Superintendents will have difficulty getting teacher acceptance and trust of student
data if they cannot get their teachers to trust that it will not be used to evaluate them.
Such reluctance will reduce the effectiveness of that data and jeopardize student
achievement. Superintendents often face challenges with teacher unions if they feel
teachers are being evaluated based on student achievement. There is resistance to the
use of this data because many educators believe that there is currently no method to
establish a teacher’s teaching practices as the direct cause of a student’s achievement
or lack of achievement on a state standardized test. Even if a causal link could be
established, most believe that there are many other factors that affect an individual
student’s achievement that may be beyond a teacher’s or school’s control. Data that
establish a link to student achievement are the type that superintendents must get
their teachers and educators to use and to use for the purpose of examining student
learning.

11

Superintendents must use the data to make decisions about instruction,
curriculum, programs, resources, and other instructional and administrative issues.
They must effectively translate the data into decisions about these issues. Translating
data into actions that help students takes skill and knowledge. Superintendents must
identify what their district’s specific achievement gaps are and then take that data
and match it with the most effective research based teaching strategies, and then
promote those strategies throughout the district.
Purpose of the Study

The purpose of this study is to examine how superintendents of urban school
districts use data to make decisions, especially those related to student achievement
and considered part of the NCLB accountability criteria. This study examines what
criteria the superintendents use to select sources of data from the wide variety of
available information. It also inquires about the impact that technology may or may
not have on the accessibility and ease of analysis of the data. Additionally, it
explores the policies the superintendents put in place and the resources they supply
their educators to increase the effective use of data among all administrators and
teachers. Finally, it asks how they use data to determine the effectiveness of their
principals, teachers, and programs.
Research Questions

The superintendents of urban school districts that participated in this study
completed a comprehensive survey and/or took part in an interview both of which
were designed to answer the following questions:

12

1. What competencies do superintendents need to effectively use student
and staff data in making the decisions that will affect student achievement?
2. With many different sources and types of student data, what sources of data do
urban superintendents use and how do they determine the value of that data?
3. What specific strategies or policies do urban superintendents use to increase
the acceptance of using student data by all educators in their district?
4. How do superintendents use data to determine the effectiveness of their
educators, administrators, and programs?
Importance of the Study

This study clarified the decision-making process that superintendents utilize to
determine the most effective types and uses of student data. As accountability for
student achievement increases and the ability of school districts to meet the
requirements of NCLB becomes more difficult, superintendents are constantly
looking for ways to improve student outcomes. At the same time, fiscal resources
available to districts are being restricted by state and local budget deficits. It
becomes critical that superintendents know how to use data effectively and make
successful decisions based on that data. Numerous studies illustrate the importance
of using data to improve student achievement and that schools or districts that use
student data effectively show improvement (WestEd, 2002). This study will provide
information to district and school leaders about the skills necessary to effectively use
data to make decisions regarding student achievement. Superintendents and other
district leaders will be able to use the information from this study to make data useful
to decision making and to determine the best methods for making that data

13

accessible. The most effective way to enlist school leaders and teachers into the use
of data will be identified. This will provide the actions that superintendents can take
to increase the use of data among their district’s educators.
Assumptions

This study assumes that all school districts and their superintendents are operating
under goals or objectives of improving student achievement as measured by various
state and district assessments. It is also assumed that actions taken and policies
created by those superintendents have an effect on student achievement.
Limitations

Due to a limited amount of time and resources available to conduct the analysis of
this study, significant portions of the data were restricted to a self-reporting survey
submitted to twenty-nine urban superintendents in the state of California. A more in-
depth personal interview, seeking specific responses related to urban superintendents
and their use of data for decision making, was conducted with five superintendents.
This small sample size and the qualitative nature of this data limited the
generalization of the findings.
Delimitations

1. The superintendents must have been in their position for at least two years.
2. The superintendents must have been leading a district of at least fifteen
thousand students.
3. The superintendents must have been leading a district with a diverse student
population.

14

4. The superintendents must use some system or software for collecting and
analyzing data.
Definition of Terms

California Standards Test (CST): Standardized assessment administered annually to
public school students intended to measure their understanding of the California
State standards in English, math, science, and social studies. It is used to measure
student progress under NCLB.
Annual Performance Index (API): A calculation of student CST scores to determine
a school or district’s overall progress and level of student achievement connected to
NCLB.
Annual Yearly Progress (AYP): A measurement of the percentage of students who
have met the level of proficiency determined by the state, disaggregated by all sub-
groups of students.
Data Warehousing System: A computer system that stores student data;
demographic, personal, program, and achievement. Used by personnel to record,
track, and analyze student achievement and trends.
Student Sub-group: A group of students who share a common characteristic such as
race, sex, socio-economic level, English fluency, etc. Used by NCLB to ensure that
all students are achieving at the same rate and that no group is being “left behind”.
Collective Bargaining Agreement: A contractual agreement between the teachers
union/classified personnel’s union and the Board of Education that identifies the
rights and responsibilities of the teachers and administrators and included salary and
benefits.

15

Professional Learning Communities: A professional development model that trains
teachers to be the teachers and learners of each other. Teachers and educators work
together to determine the needs of their school and research methods to address those
needs through a collegial collaborative process.
Organization of the Study

Chapter One of this dissertation began with an introduction of the study, a
statement of the problem, the purpose of the study, the research questions that the
study answered, the importance of the study, the assumptions, the limitations, the
delimitations, and a list of the definitions of the terms key to understanding the
study. Chapter Two consists of a review of the most current literature relevant to the
study’s topics: The use of data for decision making by superintendents, the
connections between use of data and student achievement, and the manner in which
superintendents choose data systems and enlist the use of those systems by their
educators. Chapter Three identifies the research methodology used in the study and
clarifies how the study was designed and the participants selected. Chapter Four
presents the findings of the study with an analysis and discussion of the data. Chapter
Five provides a summary of the study and the implications for superintendents and
other educators that were revealed by the study.

16

CHAPTER TWO

REVIEW OF THE LITERATURE

This chapter will be presented in five sections, each examining a different
component of data-driven decision making by superintendents and its link to student
achievement. The first section will look at how superintendents have used data to
make decisions about student achievement historically and the link between data and
accountability from before the federal NCLB legislature of 2001 until today. The
section identifies various reform movements and the increasing level of
accountability, along with the concomitant rise in the use of data to address
accountability concerns. The second section reviews what data superintendents and
educational leaders use when making decisions, and what value they place on
different types of data. The section will make the point that the value that a type of
data has depends on the position or goals of the educator. This section will also
report the identified obstacles to increasing the use of data and what steps district
leaderships have taken to overcome these barriers. The third section will examine
what the literature states are the technology skills and competencies needed by
superintendents and district leaders to access and analyze data effectively, as well as
how those skills and competencies are acquired and disseminated. The section shows
that some of the needed skills are identified but reveals the lack of sufficient research
in the area of how superintendents have gained these necessary skills and requisite
technological knowledge. The fourth section will be a look at what research has
shown regarding how superintendents increase and or improve the use of data-driven

17

decision making throughout their districts, not only at the district level but down to
the classroom teachers. Between 1983 and 2008, at least fifteen studies targeted
school leaders and their self-efficacy in improving student learning. Of these fifteen
studies all were done on school site leaders, usually principals. No studies were
found that examined the efficacy of superintendents or district leaders in regard to
improving student learning (Leithwood & Jantzi, 2008). The last section looks at the
processes identified in the literature that superintendents and educational leaders
employ to translate the data into actions or policies on their part and the part of
personnel throughout the district. The section explores research that identifies the
importance of using data to drive instruction in the classroom and how teachers and
school administration are carrying that out. It will be noted that research studying
how superintendents are using data to make instructional changes is a relatively new
area of inquiry.
The History of Data and Accountability
Throughout the history of public education in the United States the superintendent
has been held accountable, either to the community, the school board, the state, or a
combination of all of the above. High stakes accountability has caused a significant
increase in accountability data. These data are used by educators to determine if their
students have met or are exceeding the standards. There is still little evidence that
these accountability data are being used to change practices or to monitor those
changes even when policies are in place regarding the use of data for decision
making on a large scale throughout most school districts. (Ingram, Louis, &
Schroeder, 2004).

18

Different than their counterparts just a few decades ago, superintendents today are
under increasing pressure to use data to impact classroom instruction, as they work
in an accountability driven world, where they are held personally responsible for
student test scores and are pressured to educate every child. The politicalization of
test scores by the media and by people running for government office requires the
superintendent to adopt a more hands-on approach to student achievement and
accountability (Orr, 2006). Indeed, a survey of over 2,200 superintendents (reported
in The 2000 American Association of School Administrators Ten–Year Study of the
American School Superintendent), revealed that their self-identified most profound
challenges were assessing learner outcomes and the accompanying accountability
(Moore, Dexter, Berube, & Beck, 2005). This increased accountability means that it
is imperative that superintendents take the lead in improving student achievement
through improved classroom instruction and interventions. Since the 1990’s
superintendents have been increasingly expected to be instructional leaders in
addition to being organizational managers. Student assessment and accountability in
curriculum and instruction are now at the heart of district level leadership.
Superintendents today must be knowledgeable about classroom assessments, school
and district level assessments, and the management and analysis of student
assessment data to make teaching and school decisions. This expertise in assessment
is necessary because superintendents must be careful not to place too much emphasis
on outside mechanisms of accountability, which are often highly prescriptive in
nature and can undermine personal and professional interest in internal
accountability. District-developed assessments and accountability measures often

19

have more value for teachers than state or federal assessments. NCLB has intensified
accountability and the need for assessments at the state level but they should not
replace local assessments or methods of accountability (Bredeson & Kose, 2007).
This is the current state of the superintendency, but when school districts first
created the position in 1865 they initially wanted a person who would be a
supervisor of teachers and principals to ensure that they were carrying out the
policies and vision or the school board members. The need for a more managerial
position in 1910 changed the superintendent position to one that not only carried out
the school board’s policies but one that was also responsible for creating policy. By
the late 1920s, most states clearly specified in statute the legal responsibilities of
superintendents and school boards, and larger school boards viewed their
superintendents as corporate managers (Alsbury & Whitaker, 2007).
By the 1950’s the superintendency had evolved into a more political position that
required true leadership rather than just managerial skills. The era of the current
superintendent model began in the 1960’s when they had become true social
scientists, able to solve real economic, political, and academic problems (Kowalski,
2005). We are now moving into the next model of the superintendency, one in
which the superintendent is an instructional leader and is making decisions regarding
instruction and student achievement.
Since the passage of NCLB, states have grown more influential in the curriculum,
assessments, and student performance at the local and school site level. Districts
have had to yield much of their autonomy due to the accountability the states hold
them to regarding student achievement and the consequences if schools or districts

20

fail to meet those standards. Additionally, local stakeholders such as parents and
community leaders, are also increasing their expectations of student performance
along with the state and federal government. Superintendents and the school boards
must establish goals to meet their needs as well (Marks & Nance, 2007).
There have been educational reform movements throughout the history of
education with the scientific management of the 1920’s, the progressive education of
the 1930’s, to the Sputnik scare of the 1950’s, but these were not as far reaching or
long lasting as the standards reform of the 1980’s and 90’s. When the current school
reform movement first started after the publication of A Nation at Risk in 1983, many
states began a lot of top-down school reform measures. One study found that by
1988, most school districts had already implemented most of these measures before
the mandates from the state level were handed down and had already met the
standards set by the state. In many cases, districts used the state policy to advance
even further reforms, with the help of the state funding provided (Marks & Nance,
2007). However, these reforms appeared to have little impact on increasing student
achievement, and site-based management became the next phase of school reform
starting in the late 1980’s. Under this reform, districts and the state had less
influence on decisions regarding curriculum and instruction. This bottom-up decision
making also resulted in insignificant improvement in student achievement (Marks &
Nance, 2007).
The next version of reform, started in the early 1990’s, was restructuring, which
sought to coordinate and standardize the curriculum, performance standards,
policies, and professional development among schools within the state.

21

In conjunction with this standardization was accountability. Districts and schools
that failed to meet the established standards were ranked, classified, and sometimes
sanctioned based on their performance (Marks & Nance, 2007). Alsbury and
Whitaker (2007) found that the increased accountability through state mandates in
the 1990’s was worthwhile and that the results-based accountability benefited school
districts. However, these versions of educational reform were called “single-loop
learning” by Cibulka in 2000, because they created a superficial, technical type of
solution to education problems rather than allowing for a deeper examination of the
organization and creating a source of culture change that could result in long term
improvement. He stated that single-loop learning can produce short-term results but
does not change the underlying causes of the educational problems.
The latest version of school reform has the states setting up an accountability
system, usually through assessment data and other performance standards, and
imposing a system of sanctions for schools and districts that fail to meet the
established standards, but they leave how the schools or districts will meet those
standards up to the local leadership. This forces the schools and districts themselves
to look at their own student achievement data and examine what methods they will
use to improve their student performance (Marks & Nance, 2007).
The passage of NCLB in 2001 was the first time accountability reforms focused
on the achievement gaps between groups of students. Many states had made
progress toward improving student achievement overall but not until NCLB did they
start looking at the gaps between groups of students. The American educational
system has often been criticized for this continuation of achievement gaps and has

22

often been described by many as nothing more than a societal sorting mechanism.
(Sherman, 2008). Some educational leaders believe that the main problem with
NCLB is that legislators make the mistake of assuming that measuring school
performance is the same as fixing gaps in the performance. They assume that schools
have the capacity to reform if their mistakes are just pointed out to them. They don’t
seem to understand that many factors can lead to poor performance by students and
that there are many factors needed to bring about positive school reform (Sherman,
2008). Since the passage of NCLB, superintendents have been required to make the
shift to instructional leaders with a focus on identifying and eliminating the
achievement gaps between groups of students. They must demonstrate competency
as instructional leaders and as catalysts for change (Sherman, 2008). According to
the AASA (Virginia Department of Education, 2000), superintendents must serve
their districts in curriculum planning and development and in instructional
management. Prior to NCLB very few superintendents and their school boards knew
the exact extent of the achievement gap(s) between their groups of students and
fewer still had any strategies or policies in place to address those gaps (Sherman,
2008). As long as their district’s graduation rate was acceptable or continued to
improve and student data as a whole met the established standards, then most
superintendents and school boards saw no reason to implement special programs or
provide specialized professional development for their teachers. The disaggregated
data provided under NCLB and the fact that they were now being held accountable
for student achievement in all subgroups led superintendents to use data to identify
the specifics of the problem and examine possible solutions. Superintendents who

23

are able to use data to recognize achievement gaps become competent at analyzing
data and using it to focus instruction. This competence may be able to eliminate
achievement gaps over time, if it is coupled with effective school reform (Sherman,
2008).
Past studies revealed that the amount of influence superintendents have on the
academic performance of their district’s students was small. They found the major
role of the superintendent to be managerial, and that their instructional influence was
an indirect one, mainly through communication, decision-making procedures, and
evaluation processes (Alsbury & Whitaker, 2007).
By 2004 the picture was different. Studies by Archer (2005) found that successful
school districts had superintendents that were emerging as instructional leaders. As
instructional leaders these superintendents were making decisions regarding training
for teachers and principals on the use of student performance data: common district
curriculum, instructional walkthroughs, the standardization of school improvement
plans, induction plans for new teachers and assistant principals, common reading and
math programs, common planning time for teachers, and the development of district-
wide assessments. While many superintendents voiced their aim to continue to
develop as instructional leaders, they felt that there are numerous obstacles hindering
their ability to be as effective as they should be. Most superintendents reported they
felt that the barriers to being effective instructional leaders are, in order of
importance, the lack of funds, competing priorities, lack of district level staff, teacher
concerns about loss of creativity, lack of proven instructional strategies, union
contracts, and principal concerns about lost autonomy (Archer, 2005).

24

The 2000 survey by the Association of School Superintendents rated “assessing
learner outcomes” as the second most common challenge for superintendents, right
after, “financing schools,” and just ahead of “accountability” (Orr, 2006, p. 1366).
States and districts have been collecting and examining student data for decades
but until recently they used this data to determine if their students at the district or
school level were meeting a pre-determined goal or level of competency. As long as
their districts showed growth or met their goal, they saw no need to change the
process. After NCLB, and being held accountable for all subgroups of students,
districts could no longer be satisfied with these general data. Using data meant
examining the achievement levels of each group of students and looking for trends
and discrepancies to determine solutions to achievement gaps.
Using Data Effectively and Determining Its Value
As data have become a part of the accountability process of NCLB,
superintendents cannot move their districts forward without being able to effectively
analyze data, especially student achievement data. In a 2005 national survey of
superintendents respondents frequently identified the use of data to guide decisions,
as the most important strategy for improving student achievement. What these
superintendents meant by data being an important strategy is not known and
educational leaders at different levels see a variety of data as valid evidence of
student achievement (Coburn & Talbert, 2006).
One of the necessary skills superintendents need is the ability to discriminate
between useful or important data and superfluous data or data that have little impact
on student achievement. It is not enough to know how to use data but what data to

25

use. For a superintendent to be effective at data-driven decision making, they must
be able to select the most appropriate data for making that decision. The relevance
of data depends on the purpose for which they are being analyzed. The value of data
can change from one location to another or from one level of leadership in a district
to another level. Data that may be useful to one superintendent may have little
meaning to another. Data that is meaningful to a superintendent may be of little help
to a school principal or the other way around. To increase data use and data-driven
decision making in their district, superintendents must ensure that everyone has
access to the data that are useful to them. If the superintendents’ objective is to
increase and to support the use of data-driven decision making they must make
different types of data accessible to the different levels of the school system.
Individuals with different roles in the district have different needs for data (Coburn
& Talbert, 2006). Researchers have found four steps districts must go through in
order to ensure the effective use of data by their educators. First, these districts must
agree upon which data indicators to collect. Second, the people responsible for
collecting the data must be able to validate the information’s reliability. Once the
data is considered valid and reliable, educators must have the opportunity to use the
data and they must know how to use it (Wolf, 2006).
A recent survey by the Data Quality Campaign and the National Center for
Educational Accountability of all 50 state departments of education on the ten
essential elements defined by the NCEA for using data to improve student
performance found that only seven states had systems that addressed at least eight of
the elements and no state addressed all ten. These elements addressed such processes

26

as the appropriate data to be collected and the most appropriate use of student data.
This indicates that there has been progress in the area of data use by educational
leaders but that there is still much more to be done (Wolf, 2006).
Top-level district administrators report a high degree of faith in research, while
frontline administrators working at the school sites had a more mixed response.
Principals were more skeptical of the value of research to inform policy and practice.
This may be due to the fact that district staff are under pressure to use research in
making their decisions and therefore express a greater belief in the efficacy of this
type of decision-making process (Coburn & Talbert, 2006). Many teachers also
express skepticism about data, but they must buy in for data-driven decision making
to be effective. There are several widely held teacher attitudes and beliefs that are
identified as incompatible with data-driven inquiry. The more common negative
attitudes were\ teachers often discounted assessment data because they used personal
metrics that they had developed to determine student success, and teachers who had
a low sense of efficacy and did not believe that they had an influence on their
students’ achievement data were unlikely to buy into data-based decision making
(Kerr, Marsh, Ikemoto, Darilek, & Barney, 2006). Teachers in high performing
schools found data use empowering, while teachers from low-performing schools
with more diversity felt devalued and disenfranchised by data use. It is important for
school and district leaders to emphasis data use as nonthreatening and a positive tool
for improvement to help alleviate these problems. Many teachers and staff lack the
technical capacity to effectively use data, even in schools identified as active data-
driven decision makers. This can create a barrier to successful use of data as well.

27

The technical skills needed are not just the skills needed to run computers or
software programs, but also the skills necessary to formulate research questions,
interpret results, and effectively develop and use classroom assessments (Kerr et al.,
2006).
Educational leaders need to be able to use the data to make decisions about
teaching and student achievement without losing sight that those numbers represent
real students. Various studies found that school personnel valued data other than
those provided by standardized tests created for the state accountability policies.
Many teachers are concerned with data quality. They do not feel that the data
provided through the state accountability system was adequate for their needs
(Ingram et al, 2004). They also found that because of different roles and
responsibilities, teachers and administrators view data differently and the appropriate
use of that data. Also, views of data differed at the district level based on what
department they worked in. The results were only suggestive as no research has
looked specifically at how people think about research and data across different parts
of the educational system (Coburn & Talbert, 2006).
Having meaningful data available is a necessary component of effective data
driven decision making, but even if all of the data needed is made available often
other factors can interfere with using data effectively. Numerous articles were found
by Coburn and Talbert (2006), that identified these obstacles to using data. They
were lack of time, lack of technological infrastructure, lack of access to appropriate
data, and a culture of teaching that is counter to ongoing evidence use. These articles
focused on the school and classrooms level and did not examine the data used at the

28

district level or at all levels. All of these obstacles within a district can be addressed
by superintendents. How addressed these obstacles was an important part of this
study and believed to have a direct effect on the quality of data driven decision
making by a superintendent and educational leaders within their districts.

The Technology and Skills Necessary for
Data Driven Decision Making

The use of data by superintendents to make decisions about student achievement
and state mandated accountability measures is essential and an indispensable
component of being an instructional leader. Superintendents need to be able to do
more than just read and understand data; they must be able to analyze it and place the
appropriate value on it before making decisions based on those data. Student-
centered accountability requires superintendents to look at qualitative and
quantitative data that looks at all of the steps teachers and students take to improve
learning. Three factors of assessment knowledge that are needed by superintendents
are planning, implementation of data-driven decision making, and technical
assessment knowledge. These factors align with various leadership standards related
to student assessment knowledge identified by educational leadership professional
organizations (Moore et al., 2005). Several organizations and researchers have
identified the skills necessary to effectively use student achievement data to make
data-driven decisions that will improve classroom instruction and thereby student
achievement.

29

Twelve assessment competencies need to effectively use data for decision making
were identified. They are:
1. Know the attributes of sound student assessment; 2. Know the attributes of
a sound student assessment system and how to apply them; 3. Know the
issues related to the ethical and inappropriate use of assessment information
and how to protect students and staff from misuses; 4. Know the importance
and features of assessment policies; 5. Know how to work with staff
members for integration of assessment into instruction; 6. Know and be able
to evaluate teachers’ classroom assessment competencies; 7. Know how to
plan staff development to use assessments for decision making; 8. Know how
to use assessment results for instructional improvement; 9. Know how to
accurately analyze and interpret assessment information; 10. Be able to act
effectively upon assessment information; 11. Create the conditions necessary
for the appropriate use of achievement information; and 12. Be able to
communicate effectively with all interested members of the school
community about assessment results and their relationship to instruction.
(Moore, et al, 2005, 74)

Several professional organizations such as AASA, the Association for Supervision
and Curriculum Development (ASCD), and NAESP have published explicit criteria
for educational leaders in student assessment. The responsibilities frequently
identified by superintendents were: determining procedures regarding the collection
of student achievement data and communicating those procedures to staff, making
decisions about which assessments were to be used, skilled in communicating
student achievement results to the parents and community, relaying an expectation
of student learning that drives decisions about resource allocation and the regular use
of student data, and the acquisition and maintenance of a computerized data system
(Moore, et al, 2005). Superintendents don’t just need to know how to analyze student
data; they also need to be able to understand where that data comes from. They need
to be knowledgeable in the development and implementation of assessments. In a

30

survey of superintendents asking them about the skills and knowledge necessary for
creating and using student assessments they ranked the skills by importance.
The five areas of assessment knowledge and skills receiving the highest
importance ratings by respondents were ability to (a) evaluate school or
system assessment or testing programs, (b) be aware of changes in testing
and assessment practices, (c) communicate test/assessment results to the
media and general public, (d) develop a plan for assessment
implementation for your building or school system, and (e) read the current
literature on assessment. (Moore et al., 2005, 75)

Superintendents should keep in mind the cardinal rule of assessment which states
that it is more effective to measure a few things frequently rather than many things
once a year (Moore, et al, 2005). The availability of the data and how easily it can
be accessed and analyzed is determined by the type of data gathering and storing
system a district uses. Many districts employee a combination of computer hardware
and software systems that enables them to gather, analyze, and report a wide variety
of student data from both the state level and at the school level. The literature was
searched for what data analysis and technology skills a superintendent needs in order
to be effective at data driven decision making and how they acquire those skills.
Most superintendents acquire the skills they use through the successive steps of
their career path. However, these skills are incomplete and tend to reinforce current
practices. The opportunities needed by aspiring superintendents to fill needed gaps
in skills are not currently available through any single university or organization
training program. Those that do offer some type of training do not cover all of the
areas required of a superintendent (Glass, Bjork, & Brunner, 2000). Preparation
programs for superintendents are often criticized for not adequately preparing them
for the job. In 2000, 73% of superintendents rated their preparation program as good

31

or excellent. The weaknesses of these programs were identified as hands-on
application, access to technology, and links to practice (Glass, et al., 2000). This
means that at least 27% of these superintendents felt that they did not get the proper
training. These superintendents rated their university or state provided programs
more positively than they did the mentoring programs that some were in.
Bjork (2000) concluded that the superintendents learn the skills and knowledge
they need primarily from being on the job. He described this process as incomplete
and that it prepares superintendents to reproduce existing actions. Formal leadership
preparation programs seem to be only partially adequate in preparing educators for
the superintendency. Their strength seems to have been in providing theoretical
frameworks that guide vision building and problem solving. Superintendent seminars
and academies sponsored by states and professional associations also seem partially
helpful because they provide a safe, supportive environment for orientation to the
superintendency, problem solving, and networking. Existing leadership preparatory
experiences were found to be increasingly insufficient because of the lack of
attention to role development, socialization, and support in learning new
and creative forms of leadership, specifically for the superintendency (Orr, 2006).
Only recently have university programs for educational leadership incorporated
the use of data-driven decision making and even now it still only addresses
foundational skills. Several studies conducted in the 1980’s and 90’s concluded that
the weakest link in school and district leadership preparation were the university
programs. These programs usually consisted of a series of loosely related courses
that had little to no field experience components (Glass et al., 2000). In the late

32

1980’s the shift of district leadership moved toward student achievement requiring
them to develop knowledge of curriculum, instruction, teaching and learning. (Glass
et al., 2000) How superintendents acquire the skills necessary for data-driven
decision making appears to be as individual as the superintendents themselves. Even
though many university programs are providing some of the foundational skills,
there have not been any formal training provided that specifically prepares
superintendents to effectively use student data to make instructional decisions. In a
2003 study superintendents identified data analysis as the most frequent task they
performed in the area of evaluation of student learning; management of curriculum
and instruction was a distant second (Halverson, 2003). When you compare this
study to a similar one conducted in 1994, there is strong evidence that the
expectations of superintendents related to instructional leadership, curriculum
development, and student learning have increased dramatically. One example is data
analysis. The 2003 study indicated it was the number one task performed by
superintendents related to evaluation of student learning. Data analysis was not
mentioned as one of the critical tasks performed by superintendents in the 1994 study
(Bredeson & Kose, 2007).
In addition to skills needed for analyzing data are skills used for technology. Less
than ten years ago when states and districts decided to start keeping assessment data
on individual students for each standard so that instruction could be individualized to
meet their needs, the idea was an excellent one. However, the technology and the
technology skills needed to support this were not available in most districts. The
understanding of the data or even what data to use was not developed either.

33

Gathering, looking at, and trying to make use of the data that was available took a lot
of time and had little meaning in changing student achievement (Wolf, 2006).
Superintendents would not be the person to personally set up or run the computers
and other technology equipment needed for data analysis, but superintendents need
knowledge about computer hardware and software and the system development that
are an integral part of assessment and accountability. Superintendents need to have
technology skills to be able to effectively use data and they also need to have more
complex knowledge related to what data to gather, and how to manage these data in
such a way that they serve the accountability and decision-making needs of the
district (Moore et al., 2005). Leadership practices of superintendents that have
demonstrated significant effects on student achievement included planning and
supervising instruction, providing instructional support, monitoring their schools’
and students’ progress, and buffering staff from external demands unrelated to the
schools’ and district’s goals (Leithwood & Jantzi, 2008).
Increasing the Use of Data Throughout the District
The federal government has been a key player in the process of increasing the use
of student data to measure student performance, as well as, for accountability. For
many years now, the US Department of Education has worked to define which data
elements and indicators should be collected at the state and district levels. NCLB has
played a major role in defining data, collecting data, and generally raising the
attention given to data (Wolf, 2006). The federal government, states, and school
districts are demanding more testing and measuring of students than ever before.

34

All of this testing is creating a multitude of data but due to lack of skills and time to
use it effectively, there is little information reaching the teachers and classrooms
where it is needed. Many teachers and school leaders feel overwhelmed by all of the
data and the expectations connected to it. Educators are moving forward with or
without guidance with, as expected, a wide variety of success levels (Wayman &
Springfield, 2006). Successful school districts are ones with strong leadership in the
areas of assessment and the use of student data to drive instruction.
The most effective way for superintendents to use data to make decisions that
improve student achievement is for them to make decisions that propagate the use of
data for decision making throughout their district. For superintendents to have their
educators at all levels in the district using data, there are several factors that must be
addressed. The literature has identified these factors in four categories. The
educators must trust the data and trust that the use of the data will have no negative
impact on them. The data must be relevant and useful for the purpose the educators
need it. The data must be easy to access and available in a form that is easy to
understand, usually by the use of some type of computer software. A culture of data
use must be established at all levels of the district to support teachers and school site
administrators and this includes professional development in the use of data (Luo,
2008). Superintendents should include board members, principals, and teachers in
the data-driven decision making process so that they can model the process for them.
This will help them see how data can help educators make decisions about
instructional strategies, resources, and classroom teacher support. Superintendents
are responsible for ensuring that data is used, not just collected.

35

They are responsible for building a data-friendly culture, ensuring that the school
community members understand their roles and responsibilities, providing ongoing
professional development to foster new skills, and establishing a system focused on
improvement (Moore et al., 2005).
Attitudes and competencies related to data use vary throughout a district but the
superintendent must make decisions that increase the use of data-driven decision
making at all levels. To be able to do this the superintendents must have the ability to
use data themselves and translate that into policies and programs that increase the
use of data among all educators. State policy has pressured school and district
leaders to improve their schools and the use of data in school improvement has also
increased. Districts should have policies in place to ensure the implementation of
data-driven improvements (Luo, 2008).
As stated earlier, school site leadership has a different attitude toward data and
accountability than does central district leadership. School leaders are more likely to
use data for decision making in the following four areas: school instruction, school
organizational operation and moral perspective, school vision, and collaborative
partnerships. They were less likely to use data for decisions regarding administrative
problems and politics. Districts that create policies that require or reinforce data use
among their principals are key in ensuring data-driven decision making practices by
increasing awareness of issues of using data to solve problems (Luo, 2008).
Principals usually think of the state’s influences in negative terms because of the
many mandates, regulations, and sanctions the state uses to promote its policies.
Principals tend to view district influences in a generally positive manner because the

36

district’s policies are more supportive and use capacity building to reach their
objectives. The more teachers are part of the decision making process, the less
districts are perceived by principals as having influence on decisions. Districts can
increase principals’ influence on decision making through professional development
for teachers and other supports for teachers in the area of curriculum and instruction
(Marks & Nance, 2007). Teachers themselves are often critical of accountability
data but will usually embrace a data initiative when it is implemented in a sound
manner and it responds to the educational needs of their students.
Seven barriers to establishing a school culture that supports data driven decision
making were identified by Ingram et al., (2004) as: 1. teachers and administrators
who have already established a personal metric for determining effectiveness, 2.
educators relying on anecdotal information rather than systematic data, 3.
disagreement over which student outcomes are the most important, 4. many teachers
do not associate what they do with students performance, 5. data that teachers want
is not always available, 6. the time needed to collect and analyze data is rarely
provided by the school, and 5. data is often used for political purposes foster distrust
of the data. As superintendents and district leadership work to increase the things
that expand the use of data and data-driven decision making they must also work to
reduce or eliminate barriers to its use. It is important for superintendents to help
principals and teachers use student assessment data effectively to improve student
achievement. This is the main reason that districts collect data and it is imperative
that they develop a plan for a data-driven culture. The superintendent should involve

37

teachers in self-assessment, reflection, and staff development to meet the needs of
their students using student data (Moore et al., 2005).
Involving teachers in using data can be difficult because the amount of work it
takes to extract usable data from available systems often causes frustration (Wayman
and Stringfield, 2006). Most teachers are not prepared to use the large amounts of
available data efficiently and must be supported by professionals who can help them
turn student data into information they can use to inform classroom practice. Making
the problem even more complicated is the fact that many data systems are being
implemented quickly to address accountability pressures but without the necessary
professional development for the principals and teachers (Wayman and Stringfield,
2006). The proper approaches to professional development related to data use are
ones that provide timely, relevant, and accurate data that can be used in the
classroom and will change the negative perception that many educators have about
data. They will stop seeing data as something that is done to them and see it as
something done for them (Wolf, 2006).
Data use by teachers and school principals has been historically hampered by lack
of accessible data systems. The systems that are often used usually store large
amounts of data in ways that most teachers find difficult to extract or requiring an
amount of effort that stifles its use. These systems usually “warehouse” student data
from multiple sources and integrate it across different domains making it possible to
examine the relationships between different types of data. A study by Wayman and
Stringfield ( 2006) indicated that when user-friendly technology systems are used to
store and access data ,the use of student data by teachers increases. The use of

38

efficient student data technology systems often increased school capacity to use data
and improved involvement of all faculty. The supports provided by the districts and
principals were vital in promoting faculty involvement. In schools in which the
faculty used data, they reported an increased feeling of efficacy and improved
instruction related to effective differentiation and learning through collaboration
(Wayman and Stringfield, 2006). Classroom data that are difficult to access can
prevent school leaders from conducting efficient data-driven decision making. Data
that are stored in an incompatible manner or are not organized in a usable manner
can cause the evolution to data driven decision making to be difficult. (Luo 2008)
The time it takes to analyze data and collaborate with other teachers and
administrators to develop programs and make decisions is something that is in short
supply. Many teachers feel that collecting and analyzing data competes with their
“real” jobs, which is working with students. They don’t always see the value in data
and how to use those facts to change their practice and make them a more effective
teacher. Many schools that have increased the expectation that teachers use data to
make decisions about curriculum and student achievement have not provided extra
time to teachers for this purpose. Teachers, therefore, do not see data analysis as a
priority (Ingram et al., 2004). Schools and districts have scores of data but it is often
in the wrong form or teachers have to spend an inordinate amount of time to get it or
translate it into a usable form. In addition to the lack of time to translate data, there is
little time for team learning in connection with the data. Teachers are more likely to
use systematic data when they are involved, as part of a group, in studying
something that needs improvement (Ingram et al., 2004).

39

Although there has been much research in the area of data use and how to
increase its use among educators at the school site level and even some research into
how to increase the effective use of it at the district level, there is little research on
how superintendents and the decisions and policies that make can or do increase the
use of data within their districts. One of the four areas that has shown an active
increase is the area of making data more available. Many districts are investing in
data systems that help make data accessible and useful to their teachers and
administrators. Districts that try to use data to improve teaching and learning usually
have the following processes in place: human and fiscal resources dedicated to
developing data systems, the right team involved in the development and
implementation of the systems that included representatives from various
departments and levels, and ownership by and professional development for teachers
and administrators (Wolf, 2006).
Translating Data into Changes in the Classroom
Superintendents frequently talk about the problems and difficulties in determining
what are both ethically and morally in the best interest of their students. They
reported that they want to make decisions that will benefit their students but that
different research supports different decisions and making the right decisions for
their students and their situations was difficult. The most important question for them
was determining what works in what situations. District leaders stressed the
importance of making decisions based on data as much as possible. Superintendents
stress the power of using statistics gathered by their districts and converting those
data into real students (Alsbury & Whitaker, 2007). Data-driven decision making

40

has become a growing practice among school and district leaders and has become a
focus of educational policy and practice (Luo, 2008). There is a strong positive
relationship between the sense of self-efficacy of superintendents in managing the
instructional program and student achievement. The superintendent’s feeling that the
personnel within their district were efficacious at managing the instructional program
also showed a positive correlation with student achievement (Leithwood, & Jantzi,
2008). This same study suggests that the effects of district leadership on student
achievement are largely indirect. The superintendent creates the conditions within
the district that enhances and supports the work of the leaders at the school sites
(Leithwood & Jantzi, 2008). Schools today are under more pressure to use data-
driven decision making and most are using data more frequently, but several case
studies of schools attempting to enact data-driven inquiry and decision making
indicates that they are not always successful in implementing it. The same studies
conclude that there are enabling factors that need to be in place before they can be
successful in data-driven decision making. These factors are strong leadership, up-
front planning for data collection and use, and strong human capacity for data-driven
inquiry (Kerr et al., 2006).
The literature indicates that many superintendents understand the use of data and
grasp the importance of data driven decision making, but very little research
indicates how they actually translate that data into the decisions they are making. Are
they using the data to make individual private decisions or are they using the data to
influence groups of educators or the school board? One set of indicators for the use
of data for decision making was developed by Taylor (1991) and described in the

41

form of a theoretical framework. This framework indicates that information use
depends not only on the subject matter or how well the information content fits a
topic but also on the situational characteristics that are contingent on the user’s work
and organizational contexts. Information behaviors such as principals’ data driven
decision making are influenced by (1) the sets of people who share assumptions
about the nature of their work and the role of information unit, (2) the problems
characterized by dimensions that are applied to judge the usefulness of information,
(3) the work settings that influence an individual’s attitude toward information as
well as the availability and value of information, (4) and the perceptions about
problem resolution that regulate the intensity of an individual’s information search
and his or her expectations about the kinds of information he or she needs. Taylor’s
(1991) model is based on the notion that a person’s use of data is the result of an
interaction between who the person is and the work environment (Luo, 2008).
The educational background of principals and educational leaders affects the use
of data in decision making. Educators without a background in research and
measurement had more difficulty in understanding and interpreting the data available
for decision making. The following factors determined educational leader’s ability
to effectively use data: experience, data analysis skills, problem dimensions, school
district requirements and supports, accessibility of data, and perception of data
quality (Luo, 2008). These factors determined the ability of a superintendent to use
data to make decisions but it did not indicate how they used these skills to use data to
make decisions. Teachers and administrators described systematic data as the type
of data they were more likely to use when trying to bring about improvement.

42

Anecdotal data was used at times but usually in combination with systematic data
(Ingram et al., 2004). But the manner in which they use this data and what actions
they take in connection to this data was not identified, nor was it identified what role
the superintendent took in ensuring teachers used the data.
Using data to make changes in the classroom can mean negative things to many
teachers. They believe that data will be used to evaluate their teaching practices and
bring about negative consequences if their students perform poorly on state
standardized tests. They also fear that the use of data will mean the standardization
of classroom practices and eliminate academic freedom. Considering these fears it is
important for superintendents to keep in mind that it is common for teachers to hide
or distort data if there are judgmental consequences or punitive retributions. They
will not trust the data or see data as useful. Many school administrators are
increasing the use of data in their decision making and are often attempting to
include teachers in this practice. They struggle with teachers not willing to make a
commitment to using data for accountability purposes and continuous improvement.
(Ingram et al, 2004) There are three main factors that determine effective data-
driven decision making. First, teachers, schools, and districts need to be able to
access data that are timely, valuable, and presented in a user-friendly format.
Teachers and educators face challenges with enough time to analyze and interpret
data and it is important to make the use of data as efficient as possible, while at the
same time providing time to teachers to use data. Second, districts must provide
flexibility to address student needs. Many teachers felt that district and state
curriculum guides and instructional expectations did not allow them to address the

43

students’ needs that were identified by the data. If they were to spend time reteaching
concepts that data showed students did not master then they would fall behind in the
district-directed pacing plan. Districts must consider reforming their curriculum
coverage and pacing to provide teachers with more flexibility. The third practice
that districts should engage in are assisting school-level staff with data analysis and
identifying appropriate interventions or changes to instructional practices based on
those data. Many school level staff lack the capacity to successfully engage in data
use, both technical and in the analysis and questioning associate with data use (Kerr
et al., 2006). Superintendent Pat Forgione of Austin, Texas, had to start from scratch
to initiate the use of data when he took the lead of their school district. He started by
building support for a data system and data use among the board members,
administrators and teachers. He then secured financial backing for his plan. The
system provided teachers with summative and formative data on student
achievement by specific standards. The system linked the problem areas to
particular instructional materials, digital content, and other interventions. The ability
of superintendent to obtain the support of the educators and instill ownership in the
process was key in the success of the program. Student achievement increased
dramatically and the teachers feel that the data is making the difference. (Wolf,
2006)
Numerous studies link data-driven decision making to changes in school culture
and teacher practices that research has also linked to increased student achievement.

44

The practices that have resulted from data-driven decision making are; greater
differentiation of instruction, greater collaboration among educators, improved
identification of students’ learning needs (Wayman and Stringfield 2005; Kerr et al.,
2006).
Using student data to change classroom practice can take place on the individual
level or at the district level. The most effective way of making changes to classroom
practices that will positively affect student achievement is for groups of educators to
analyze student data and collaboratively develop practices and programs to reach
their goals. When only top leadership is determining changes to classroom practices
based on data then organizational learning is dependent on a favorable environment
for accepting of new knowledge and ideas. A politicized process that does not
involve serious deliberation and discussion can serve to distort facts and to make
power a more important ingredient of decision-making than data. The relationship
between the principals and their teachers is the primary connection between the
superintendent’s vision regarding data-driven decision-making and the
implementation of these innovative practices by classroom teachers. However, the
largest impact on classroom pedagogy comes from the trust, cultivated by the
superintendent, that his or her vision will benefit the district's teachers and the
students (Acker 2007). There are many documented purposes toward which schools
have successfully applied data-based inquiry. Data are used to determine annual and
intermediate goals for school improvement. Data has also been used to depict
progress toward goals, motivate students and staff, and instructional decisions.
Schools and districts are using data to make instructional decisions such as aligning

45

instruction with standards, indentifying low-performing students, monitoring student
progress, and individualizing instruction. How schools are to be structured, policies
developed, and resources used are also becoming dependent on data. (Kerr, et al,
2006)
Data, of course, still have a purpose. But the purpose is changed in the face of an
adversarial political process. In this case data are used to support a decision or course
of action rather than to uncover problems and to objectively determine the best
course of action (Ingram et al, 2004).
Conclusion

The role of the superintendent has changed over the past decade to one in which
they are required to be more than just organizational managers or educational
leaders. Today superintendents must be instructional leaders as well, not just leading
and managing the people in their district but leading and managing the curricular
programs and practices. The ever increasing accountability that comes with NCLB
and the expectations of the community and state means that superintendents
themselves must take the lead on data analysis and making decisions based on that
data. The literature confirms that the expectation is for superintendents to become
instructional leaders and that data-driven decision making is an integral part of that
process. The literature also identifies several challenges that superintendents face in
taking on this role, such as, lack of data using skills and knowledge regarding the
correlation between student data and their teaching practices, among the educators in
their district, attitudes among educators that conflict with using data, and lack of
systems for using data. The literature also identifies the skills necessary for a

46

superintendent to effectively use assessment data to make decisions about instruction
but that they often gain these skills as part of their role. There was no single method
or program used by superintendents to gain the skills and knowledge needed to use
data. Which leaves the question, what is the most effective way for superintendents
to acquire the skills and knowledge related to data?
The literature did not identify how superintendents select the data they will use or
what process they use to analyze it before basing instructional decisions on that data.
Another question that the literature did not answer was how they gain teacher
commitment to the process. The research was very clear on the importance of teacher
buy-in relative to effective data-driven decision making, but not what role the
superintendent plays in getting this commitment.

47

CHAPTER THREE
METHODOLOGY
The role of the superintendent has changed significantly in the past ten years.
They can no longer be considered successful if they balance their budgets, ensure the
students’ safety, and keep their district running smoothly. They must also make sure
that all of the students in their district are learning and must take a direct role in the
academic achievement of their students. They must be able to accomplish these
goals, as well as attend to their historical responsibilities, to be considered
successful. One of the most effective tools superintendents use to increase student
achievement is data. Use of student data to make decisions about curriculum, use of
resources, programs, interventions, teaching practices, and evaluation of staff are
effective practices in increasing student achievement. There have been several
practices and competencies identified that increase the effectiveness of data-driven
decision making (Cromey, van der Ploeg, & Masini, 2000; Luo, 2008; Reichardt,
2000). This study will reveal which of those, or possibly others, that urban
superintendents use in their data-driven decision making. The purpose of this study
is to examine how superintendents of urban school districts use data to make
decisions, especially those related to student achievement and considered part of the
NCLB accountability criteria to address the ever increasing level of accountability
that they encounter. The study asks the following questions of superintendents: How
did they select sources of data and determine them as reliable and useful from all of
the data available to them? Did the technology increase the usefulness of that data?

48

What policies did they put in place and what resources do they supply their educators
to increase the effective use of data among all administrators and teachers? The use
of data by all educators in a district is what is necessary to improve student
achievement; use of data by the superintendent alone will not be sufficient (AASA,
2002). The type of data that would need to be collected for this study to determine
the practices and competencies of the superintendents that led to effective data-
driven decision making was determined from a review of the literature on data-
driven decision making and practices and competencies of educational leaders,
including principals and district level administrators. This chapter will outline the
design, sample, instrumentation, data collection methods, and data analysis for the
study.
In order to determine how superintendents use data-driven decision making
effectively, and are able to use it to improve student achievement, the study was
designed to provide answers to the following four research questions which provided
the framework for the study:
1. What competencies do urban superintendents need to effectively use student
and staff data in making the decisions that will affect student achievement?
2. With many different sources and types of student data, what sources of data
do urban superintendents use and how do they determine the value of that
data?
3. What specific strategies or policies do urban superintendents use to increase
the acceptance of using student data by all educators in their district?

49

4. How do superintendents use data to determine the effectiveness of their
educators, administrators, and programs?
Determining what competencies urban superintendents needed in order to
effectively use data to improve student achievement was necessary for distinguishing
effective practices from ineffective practices and to identify the characteristics of
particular situations that might be especially suited to particular practices. The ability
of these superintendents to identify useful types of data and how they achieve this
relates to their ability to effectively and efficiently use data. In order for a
superintendent to improve student achievement through the use of data driven
decision making, they must be able to create and/or maintain a culture of student data
use by all educators in their district. Identifying how superintendents are able to do
this effectively is directly connected to their ability to use data driven decision
making to improve student achievement.
In order to answer all four research questions the researcher utilized a multi-
method qualitative approach to gather information from superintendents of urban
school districts in Southern California. The information gathered from these
superintendents was analyzed to identify common practices and to determine the
effectiveness of several of their key practices.
The results were gathered from the superintendents involved in this study through
a forty five question survey that twenty-three of the superintendents completed and
returned. Structured personal interviews with five superintendents provided more
detailed information in addition to clarifying questions that had arisen out of the
analysis of the survey data.

50

As outlined in the Information Sheet for Non-Medical Research approved by the
University of Southern California’s Review Board, any information connected with
this study or identifiable with the district, superintendent, school site, or any specific
participant remained completely confidential. Each participant’s interview or survey
responses were not released to any other party or discussed in conferences, and no
information was included revealing the participant’s identities. All participants were
informed of the purpose of the study, that their participation was completely
voluntary, and that they could choose to withdraw from the study at any time.
Participants/ Sample size/ Selection

This study utilized purposeful sampling in selecting the districts and their
superintendents for participation. Today’s urban superintendents are pressured to
improve student achievement and to maintain those achievement levels. Each
superintendent uses a variety of skills and practices as they endeavor to get their
district’s students to achieve. The use of student data to guide decisions regarding
student achievement can be one of the most valuable practices used by educational
leaders, if used effectively (Wolf, 2006). In order to obtain a clear understanding of
superintendents and their use of data, the researcher selected urban superintendents
who were currently engaged in data-driven decision making as part of their
leadership style and who have demonstrated a significant level of success in
increasing their district’s API scores.

51

Selection of Participants

The criteria for selecting the superintendents for participation in the study were
based on improved student achievement within their district and indicators of data-
driven decision making on the part of the superintendent. The researcher examined
school districts within Southern California that were located in urban areas, had a
student enrollment of fifteen thousand or more, and had a diverse population of
students. Selected were superintendents within those districts that had demonstrated
student achievement growth, as measured by API, since the superintendent began
leading the district. In addition, evidence existed that showed these superintendents
used data to make decisions in the district.
The demographic and achievement data of the district of each superintendent
who participated in the study were collected and the mean and median were derived
from that data. The mean and median of the 23 superintendents and their districts
were determined and placed in a chart. Figure 3.1 shows the data for each
characteristic of their districts.
Table 1
District Information of Superintendents Participating in the Study
Characteristics Mean Median Range
District Enrollment 34,749 26,613 17,400 to 132,000
District API 761 765 629 to 896
Percentage of English
Language Learners
25.57 24.4 7.6 to 60.5
Percent Socio-
economically
Disadvantaged
48.5 51.3
6.4 to 86.2
Percentage of minority
students
59.3 58.4 10.1 to 98.0
Enrollment, Title I, ELL, and Minority student data based on 2008 CBEDS
API score, from 2009 CDE report

52

Characteristics of Participating Superintendents

As stated above, purposeful sampling was used to select superintendents for
which there was evidence of improved student achievement and data-driven decision
making because they would be information-rich participants for the study. Based on
the above criterion, twenty-nine superintendents were identified in Southern
California. The twenty-nine superintendents meeting the criteria of the study worked
in districts that ranged in student populations from 17,400 to 132,000. Their API
scores for their districts ranged from 629 to 896. It was evident that all twenty-nine
superintendents engaged in data-driven decision making to some degree through
their district’s use of common data system and information obtained from district
newsletters and websites.
All twenty-nine superintendents selected were sent a survey instrument that was
designed to measure their understanding of data, and their skills and practices related
to data-driven decision making. Of these twenty-nine superintendents, five were
selected and solicited for a personal interview. Purposeful sampling was used to
select these five urban superintendents to ensure that they would provide data-rich
information for this study. These five superintendents were chosen from their
responses on the survey instrument that they completed and because of their
reputation among other urban superintendents and educational leaders as
superintendents that engaged in effective data-driven decision making.
The five superintendents chosen for personal interviews came from districts with
an average enrollment of 25,000 students, an average API of 799.2, and with an
average of seven years of experience as a superintendent.

53

All five superintendents indicated in their survey responses that they actively
engaged in behaviors and practices that indicate data-driven decision making and
continuing actions to promote and maintain a data-using culture in their districts.
Each of these five superintendents was contacted via a letter that included a
personal request from the researcher and one of the superintendent’s colleagues and
a telephone call for the purpose of arranging the personal interview.
Measures and Survey Instruments

The purpose of this study was to determine the knowledge and practices that
urban superintendents use to improve student achievement through data-driven
decision making. The data needed to determine this had to come directly from the
superintendents engaged in these practices in their districts. To gather this data from
them in the most effective manner while at the same time being mindful of their
demanding schedules, a survey instrument was designed. The survey was designed
so that it could be completed in less than 20 minutes and allowed for the responder to
answer with a selection from a list of possible answers. Additional spaces were
included that allowed the responder to answer with short responses if their response
was not included in the selection.
The survey instrument was designed to gather data that would examine the depth
of data-driven decision making and its connection to student achievement. The
survey consisted of a mixture of description, relationship, and judgment type
questions. The survey was composed of 45 questions that allowed for the responders
to choose their answers from a scale that was designed to measure the frequency and
depth of their practices and knowledge. Six of the questions asked superintendents

54

to rate their level of knowledge and skills in using data using a five part Likert-like
scale. There were twelve questions that required them to identify if they had acquired
relevant knowledge and skills from a university program or on-the-job experiences,
using the same scale as the first six questions. There was a series of nineteen
questions that asked the superintendents to identify the types of data they used in
their decision making or to communicate their goals and expectations. They were
also asked to identify which data they expected their employees to use in their
decision making process. They could choose one or more for each response and
write in a type of data if it was not in the selection offered. The final eight questions
asked the superintendents to identify the frequency that they engaged in various
actions related to data use and the promotion of data use in others.
The personal interviews with the five pre-selected superintendents used a
predetermined set of nine questions that sought information and data on their
experiences, practices, and knowledge pertaining to data and data-driven decision
making. There were also four questions inquiring about their policies and
communications with the educators in their districts related to creating and
maintaining a data using culture. The questions were designed to elicit specific
information regarding each element of the research questions, but the
superintendents were allowed to respond as they wished and add more information to
each response.
The surveys were mailed to each of the superintendents along with specific
directions along with a pre-addressed stamped envelope to return the completed
surveys. Each survey was given a code number that identified which superintendent

55

it was sent to. The superintendents were not required to identify themselves on the
surveys to protect their confidentiality.
Validity and Reliability

In order to ensure content validity, the superintendent survey instrument was first
piloted with experienced superintendents in California who did not participate in the
formal study. Each superintendent reviewed the instrument for wording, readability,
clarity, and validity. Feedback and recommendations were used to revise the survey
instrument and to ensure content validity. Based on the review, the instruments were
presented to members of the dissertation committee for approval. Feedback and
recommendations were taken from the three members of the dissertation committee,
and the data collection instrument was revised and approved for use.
Procedure for Data Collection

The first phase of the data collection and analysis began with the collection of
data on each district and their superintendents. District documents, data from the
California Department of Education, and websites created and maintained by those
districts were all examined. Student achievement data was collected from all three
sources and analyzed to determine which districts met the criteria for participation in
the study. District documents and websites were examined to determine the level of
data use by the educators in that district and for evidence of data-driven decision
making on the part of the superintendent. These information sources were analyzed
to determine which superintendents met the criterion for participation in the study, as
well as guide the type of questions to include in the surveys.

56

Each superintendent selected to participate in the study was asked to complete a
survey that consisted of 45 questions that were intended to collect data regarding the
research questions. Each of the survey questions was designed in a manner that
allowed for them to be coded with a value that was then used to determine a mean,
median, and mode for each of the responses as well as a percentage. The responses
were then analyzed to determine the knowledge, skills, and practices that correlated
with effective data-driven decision making on the part of the superintendents. The
response to each of the survey questions provided data for one or more of the
research questions and were identified and grouped according to those research
questions.
The prompts or questions that were used for the personal interviews of the five
selected superintendents were also designed to elicit responses that would provide
information to answer the study’s research questions. The responses to these
questions were recorded during the interview and later transcribed. Each of the
interview questions was designed in a manner that allowed for them to be coded with
a value that was then used to determine a mean, median, and mode for each of the
responses, as well as how they compared to a conceptual framework matrix
regarding effective data-driven decision making. The data from each response were
then analyzed to determine the knowledge, skills, and practices that correlated with
effective data-driven decision making on the part of the superintendents.
The response to each of the interview questions provided data for one or more of the
research questions and were identified and grouped according to those research
questions.

57

Notes were also taken during the interviews and the data and information from those
written notes were also incorporated into data related to the research questions.
Data Analysis

Data analysis methods are described in terms of the quantitative data gathered via
the survey questionnaire and the qualitative data collected via the structured
interviews.
Superintendent Survey Data Analysis

The methodology for this part of the study employed quantitative techniques.
Data analysis of the 45 close-ended items on the superintendent survey questionnaire
was conducted using a comparison of the superintendents responses to the Data-
Driven Decision Making framework and analyzing the frequency and depth of the
superintendents knowledge and practices . Each variable was defined and assigned a
variable label. The Likert-scale response (0 to 4 or 0 to 5) for each of the 45 closed-
ended items on each coded survey was entered into that data set.
Structured Interview Data Analysis

Data collected from the structured interviews were analyzed to identify the skills
and practices superintendents used that have been identified with data-driven
decision making. The data collected from the interviews were critically examined to
determine how urban superintendents should effectively utilize data to make decision
regarding student achievement, to communicate goals and results to their district’s
educators, and evaluate their staff and programs. To analyze the transcribed

58

interview data, the researcher used the following data analysis procedure. First, all
handwritten notes, reflections, and audiotapes of the interviews were transcribed.
Second, each transcription was read carefully, without taking notes. Following the
first reading, the transcripts were read for a second and third time and notes and
observations regarding recurring themes and patterns were recorded in the margins
(inductive analysis). Next, the transcripts were reread to identify aspects of effective
data-driven decision making practices that were identified in the literature review
(deductive analysis) and coded as follows:

BD3MC = Beginning data-driven decision making culture
ED3MC = Established data-driven decision making culture
SD3MC = Successful data-driven decision making culture
DAS = Data analysis skill
DUS = Data utilization skill
DCS = Data communication skill
DES = Data evaluation skill
TS = Technology skill
EV = Evaluation skill
DUC= Data Using Culture Actions

Validity Concerns

The current study employed triangulation to increase the validity of its findings.
Patton (2002) identified four basic types of triangulation: use of multiple data
sources, researchers, frameworks, and methodologies. The current study used three

59

forms of triangulation to ensure the validity of findings. First, a mixed
methodology—combining both quantitative and qualitative—was used to collect,
analyze, and report data. Survey questionnaires and in-depth interviews were
conducted to provide insight into the knowledge and practices successful urban
superintendents in California have used to improve student achievement through use
of data. The study employed theory triangulation by using multiple data-driven
decision making frameworks to develop the survey questionnaire and the interview
protocol, as well as to interpret the collected data. The study used data triangulation
by surveying and interviewing multiple urban superintendents and by using both
written responses on the survey questionnaire and open-ended verbal responses to
the interview questions. Overall, these three triangulation techniques were used to
ensure the validity of the findings.
Summary
This chapter described the methodology this study will use to determine the
skills, knowledge, and practices that successful urban superintendents employ for
effective data-driven decision making. This chapter described where this data would
be collected and how superintendents were selected to participate in the study. The
mixed methodology approach that was to be used, with both quantitative and
qualitative data being collected and analyzed, was explained. The validity of the data
was ensured by the use of sample surveys and feedback from experts in the field, as
well as by using a triangulation of that data that was collected. The methodology
used for this study was designed to gather the most accurate data possible that would
provide answers to the study’s research questions.

60

CHAPTER FOUR
FINDINGS
This chapter will present the finding of this study, which examined how
superintendents of urban school districts use data to make decisions, especially those
related to student achievement and considered part of the NCLB accountability
criteria. It examined how they selected sources of data and determined them to be
reliable and useful from all of the data available to them, and whether the use of
technology increased the usefulness of that data. It also asked what policies they put
in place and what resources they supply their educators to increase the effective use
of data among all administrators and teachers. Finally, it asked how they use data to
determine the effectiveness of their principals, teachers, and programs.
The findings from this research study, along with a detailed analysis and
discussion of those findings, are presented. This chapter will describe the
information that was collected and the analysis of that information as it relates to the
four research questions and other significant findings.
Research Questions

The superintendents of urban school districts that participated in this study
completed a comprehensive survey and/or took part in an interview that was/were
designed to answer the following questions:
1. What competencies do superintendents need to effectively use student and
staff data in making the decisions that will effect student achievement?
2. With many different sources and types of student data, what sources of data do
urban superintendents use and how do they determine the value of that data?

61

3. What specific strategies or policies do urban superintendents use to increase
the acceptance of using student data by all educators in their district?
4. How do superintendents use data to determine the effectiveness of their
educators, administrators, and programs?
Results of the Superintendent
Data-Driven Decision Making Survey Questionnaire

The questionnaire was designed to gather information from superintendents
regarding their level of competency in using data and where they acquired those
skills, to what level they relied on data when making decisions, which data they
relied on most in making those decisions, and how they used data in their decision-
making process. The research-based survey consisted of forty-five questions divided
into seven sections. The first section consisted of six questions asking the
superintendent to rate their current level of skills in using data. They rated
themselves as one of the following: Novice - still learning most components of this
aspect of data use;, Developing - knowledgeable of many components of this aspect
of data use, but not all;, Proficient - knowledgeable of most components of this
aspect of data use; Skilled - knowledgeable of all components of this aspect of data
use; or Expert - could train others on this aspect of data use. They rated themselves
in six different areas of data use frequently used by superintendents.
The second section asked the superintendent to respond to six questions regarding
where they acquired their level of data use skills. The third section consisted of six
questions asking the superintendents to rate the how effective various data training
methods were for them. Both the second and third set of questions used the same

62

rating scale as the first set but asked the superintendents to rate the level of
proficiency they attained specific to a formal training program or through on-the-job
experience. The fourth section was made up of seven questions asking
superintendents to identify which types of data they use for various decisions they
might make. They were given four types of data to choose from: California
Standards Test, district created assessments, program data, and demographic data.
There was also a space to write in a fifth type of data if they used a type of data other
than these. The fifth set consisted of six questions that asked the superintendents to
identify which data they use to communicate their goals, priorities, policies, and
results to schools staff, the school board, and the community. The sixth section asked
the superintendents six questions about their expectations regarding the use of data
by teachers and principals in making decisions about curriculum and teaching. The
fifth and sixth set of questions used the same five choices of responses as section
four but asked them to identify which type of data they used regarding
communication and expectations of school and district staff. The last section of
questions consisted of eight questions that asked the superintendent to rate how often
they engaged in a variety of behaviors that encourages and promotes the use of data
by school and district staff. The superintendents were able to choose between four
levels of frequency when asked questions regarding how often they engaged in a
particular behavior. The four choices were: Never - have not done this; Sometimes -
once or twice a year; Occasionally - more than twice a year but not at every
opportunity; Frequently - every opportunity.

63

Demographics of Survey Questionnaire Sample
Twenty-nine superintendents who were currently leading urban school districts in
Southern California during the time frame of the study (2009 -2010 school year) met
the criteria of the study and were asked to participate in the survey part of this
research study. These superintendents had been in their current positions for a
minimum of two years. Their districts had an enrolled student population of at least
15,000, and had a significant population of minority students, English language
learners, and socio-economically disadvantaged students.
Urban Superintendent Population and Sample (n=23)
The return rate for the surveys was 79.3%, with 23 of the 29 surveys sent out
being returned completed by the superintendents. Three superintendents returned
surveys that were not completed and were not counted or considered in this study.
The remaining three superintendents did not respond after four attempts and no
further attempts were made. Of the superintendents that returned their completed
surveys, 15 were male and 8 were female, and all were presently leading urban
school districts in the southern region of California. They had an average of 4.3
years in their position as superintendent, with the shortest time as a superintendent
being two years and the longest being 15 years. Eighteen of the twenty-three had
earned a doctoral degree, but all had earned at least a master’s degree. One
superintendent held both a Ph.D. and an Ed.D.

64

Table 2:
Descriptive Statistics of the Superintendents
Participating in Surveys
Superintendents n =23
Gender Ed. Level
Years as
Superintendent
Years in
Current
District
65%
Male
78%
Doctorate
Least
2 years
Least
2 years
35%
Female
22%
Masters only
Most
15 years
Most
36

The Superintendents and Their Districts
The average enrollment for the district of the participating superintendents was
31,591 students, with the largest district enrolling 56,700 students and the smallest
16,480. Their average student populations consisted of 55.5% minority students,
27% English language learners, and 43.4% socio-economically disadvantaged. The
average API score for their districts was 778.8, with the highest API being 896 and
the lowest 659. These districts had an average increase in their API’s over the past
two years of 15.2 points.
Table 3:
Descriptive Statistics of the Superintendents’ Districts
Districts n=23

District Student
Enrollment
Annual
Performance
Index
Percent Socio-
economically
Disadvantaged
Percent English
Language
Learners
Percent
Minority
Students
Average
31,591 778.8 43.4 27.0 55.5
Median
27,500 795 47.2 26 58.4
Low
16,480 659 6.4 7.7 10.1
High
56,700 896 86.2 60.5 94.5

65

Superintendents Participating in Individual Personal Interviews
Five superintendents who responded to the written surveys were asked and agreed
to participate in a one-on-one personal interview. The interview was a series of 14
open-ended questions designed to elicit responses that would answer this study’s
research questions. There were three questions asking superintendents to provide
information on what competencies were important in using data in decision making,
what competencies they possessed in using data, and how they believed they
acquired those competencies. The next question asked the superintendents to identify
which data they used for the decisions they make and how they determined the value
of that data. This was followed by four questions asking for information to determine
what actions, policies, and communications the superintendents engaged in to
increase data use among the educators in their districts. There were three questions
asking superintendents about their use of data and their expectations of data use by
principals in evaluating faculty and programs. The interview concluded with two
questions, one asking how each superintendent used data to communicate with their
boards of education and the other asking for any important data-related information
that they had not yet discussed. If the superintendents discussed in-depth
information that answered another question in the survey while answering a different
question, all or part of that question may have been omitted as the superintendent had
already answered it.
The five superintendents that participated in the interviews were all leading urban
school districts in Southern California. Four of them were men and one was a
woman. They all had earned a doctoral degree and had been a superintendent for at

66

least six years. They lead districts that averaged 25,000 students with demographic
make-ups that averaged 54.1% minority students, 30.3% ELL students, 32.9% low
income students, and an average API of 799.2 point.
Table 4:
Descriptive Statistics for the Districts of the Superintendents
Participating in Personal Interviews
Districts n=5

District Student
Enrollment
Annual
Performance
Index
Percent Socio-
economically
Disadvantaged
Percent ELL
Percent Minority
Students
Average 25,000 799.2 32.9 30.3 54.1
Mode 22,300 796 47.2 26 43.3
Low 20,800 744 15.8 12.4 23
High 32,900 844 47.2 60.5 91

Their responses to the questions were digitally recorded, transcribed, and coded.
The coding connected the response to the research question(s) that it addressed and
where the response placed the superintendent’s knowledge, skills, and practices in
the effective data-driven decision making framework. After their responses were
coded, commonalities and trends were looked at, as well as knowledge, skills, or
actions that were unique to an individual superintendent. These commonalities,
trends, and singularities are identified and discussed in connection to the research
questions below.

67

Findings by Research Questions
Research Question One: Competencies Superintendents Need to Effectively Use
Student Data in Making the Decisions Affecting Student Achievement

Results from Surveys
This question was developed with the assumption that for a superintendent to
effectively use data in their decision making process there would be a set of skills
and competencies that they needed to possess. This question was designed to identify
those skills, their prevalence among superintendents, and how those competencies
were acquired.
Superintendents responded by rating their current competency levels in six areas
of data use skills and what level of competency they achieved in those same areas
from a formal program such as a university degree program or a Superintendents
Academy and what level they achieved from experience or collaboration with others
while on the job. Superintendents could choose - Novice - still learning most
components of this aspect of data use, Developing - knowledgeable of many
components of this aspect of data use, but not all, Proficient - knowledgeable of most
components of this aspect of data use, Skilled - knowledgeable of all components of
this aspect of data use, or Expert - could train others on this aspect of data use. The
superintendents rated each item using a 5-point Likert-type scale. Novice was given
a value of 1, Developing a value of 2, Proficient a value of 3, Skilled a value of 4,
and Expert a value of 5.
The results of the survey indicated that the superintendents believed themselves
to be Skilled at analyzing raw student achievement data and that most learned their

68

skills in this area from on-the-job training. Fourteen of the 23 rated themselves as
Skilled and four as Expert in this skill, with the remaining five superintendents rating
themselves as Proficient. Seven superintendents indicated that they learned their
level of skills mostly through a formal program and 16 reported that they learned the
majority of their skills from on-the-job training.
The superintendents’ responses indicate that most perceive themselves as Skilled
or Expert in disaggregating student data by sub-groups. Fifteen of the
superintendents indicated that they were Skilled and three Expert, while five said
they were Proficient. The majority of the superintendents, 17 of them, stated that
they gained their level of proficiency in this skill from on-the-job experiences and
seven stating they gained these abilities from a formal program.
The use of a data warehousing computer software and/or hardware was the area
that superintendents reported to have the lowest level of competency. Nine
superintendents rated themselves Skilled and one as an Expert, while nine rated
themselves Proficient and the remaining four as Developing or Novice in this skill.
Slightly over half of the superintendents felt that they were still developing their
skills in this area with 11 reporting that they had achieved a level of Skilled or
Expert through on-the-job experiences and 12 feeling that they were Proficient or
less in this skill area.
The next data skill the superintendents rated themselves on was the use of data to
evaluate teacher and principal performance. Fourteen of the superintendents rated
themselves as Skilled and three as Expert, with the remaining six superintendents

69

rating themselves Proficient. The majority of the superintendents (20) reported that
they gained a proficiency level of Skilled or Expert from on-the-job experiences.
The survey results revealed that nearly all of the superintendents felt their ability
to use data to communicate to parents/community was Skilled or Expert with only
two responding that their skill level was in the Proficient range. This area had the
most superintendents rating themselves as Experts also, with 11 identified as such.
Twenty of the superintendents identified on-the-job experiences as the route by
which they achieved a proficiency level of Skilled or Expert.
Superintendent responses to survey questions regarding the use of data to
determine the effectiveness of a district program and associated changes in student
achievement indicated that 13 felt that they were Skilled and four were Expert, with
the remaining six at a skill level of Proficient. Eighteen superintendents identified
on-the-job experience as the method by which they achieved a proficiency level of
Skilled or Expert, while nine identified a formal program as their avenue.

70

Table 5:
Rating of Superintendents Data Use Competencies
Survey Questions on
Superintendent Competencies
Analyze Raw Student
Data
Disaggregating Student
Data by Subgroups
Ability to Use Computer
Data System
Use of Data to Evaluate
Principals and Teachers
Use of Data to
Communicate Goals and
Objectives
Use of Data to Evaluate
the Effectiveness
Programs
Novice
0 0 1 0 0 0
Developing
0 0 3 0 0 0
Proficient
5 5 9 6 2 6
Skilled
14 15 9 14 10 13
Expert
4 3 1 3 11 4
Average
4.0 4.0 3.3 3.9 4.4 4.0

Competencies Superintendents Rated Highest
Superintendents rated themselves on their ability to perform the six different sets
of skills employed in the use, analysis, and understanding of data (see Table 4). They
rated themselves from Novice, meaning they have little to no ability in that skill set,
through five levels of competency to Expert, meaning they knew the skill set in
depth and could teach those skills to others. A rating by the superintendent of Skilled
or Expert means that they believe that they knew all of the skills and knowledge
required to perform that set of skills. The set of data use skills that the
superintendents participating in the survey felt most confident about was their ability
to use data to communicate with staff and the community. Nearly all of the
superintendents, 21 of the 23, rated themselves Skilled or Expert.

71

Eleven superintendents felt they were Experts in this area, and 10 felt that they were
Skilled in this area.
Table 6:
Superintendents Level of Competencies Levels in Data Use Skills from Surveys
0
2
4
6
8
10
12
14
16
Novice DevelopingProficient Skilled Expert
Analyzing Raw Data
Disaggregating Data
Use of Data Systems
Evaluating People Using
Data
Communicating Using
Data
Evaluating Programs
Using Data

Competencies Superintendents Rated Lowest
The survey results showed that the superintendents had a similar distribution of
skill levels in four of the remaining five sets of skills. In these four skill sets,
analyzing raw data, disaggregating student data, using data to evaluate, and using
data to determine correlations between programs, sixteen to eighteen superintendents
felt that that were Expert or Skilled. There were no superintendents who reported
that they had only some, little, or no knowledge in those areas. The area of skills
that superintendents rated themselves lowest in was their ability to use computer
software and/or hardware in the use of data.

72

Only one superintendent felt s/he was an Expert in this area and four superintendents
reported that they knew little to nothing about this set of skills.
Acquisition of Data Use Competencies
The survey asked superintendents to identify where they felt they learned the
most about each data skill set. They rated the level of knowledge they gained in each
area from formal university training or from on the job experiences and collaboration
with others. An average of 7.7 superintendents reported that they learned all or most
of their knowledge in each skill set from a formal educational process, while more
than double that average, 17.2, felt they learned all or most of their knowledge from
on-the-job experiences or collaboration with others. The knowledge needed to use
computerized data systems was rated low in both areas, formal education and on the
job experiences, and this links with how superintendents rated their current
competency levels in this area. The areas that superintendents felt that on-the-job
experiences provided them more knowledge than formal training was in the area of
evaluating educators and communicating with educators and the community using
data. Fifty percent more of the superintendents felt they learned their skills in these
areas from on-the-job training, compared to 23 - 41% of superintendents in the other
skill set areas.

73

Table 7:
Where and How Superintendents Acquired Their Data-Driven Decision Making
Competencies. Level of Competency: 1= Novice, 2= Developing, 3=Proficient, 4= Skilled,
5=Expert
Competencies Needed Current Level Level from Program Level from
On the Job

Medium Range Mode Medium Range Mode Medium Range Mode
Analyzing raw student
achievement data
4.0 3-5 4 3.9 1-5 3 4.0 3-5 4
Disaggregating student
data by sub-groups
3.95 3-5 4 3.18 1-5 3 3.95 3-5 4
Use of data
warehousing computer
software/hardware
3.27 1-5 3 3.82 1-5 3 3.41 1-5 3/4
Use of data to evaluate
teacher/principal
performance

3.9

3-5

4

3.18

1-5

3

4.0

3-5

4
Ability to use data to
communicate to
parents/community

4.36

3-5

4

3.18

1-5

3

4.45

3-5

5
Use of data to
determine a correlation
between a district
program and changes in
student achievement

3.95

3-5

4

3.14

1-5

3

4.09

3-5

4

Discussion of Survey Results for Research Question One
All of the superintendents that participated in this study believed that they have
the ability to use data effectively, with well over half of them believing themselves to
be Skilled or Expert in every area of data use except for their ability to use the
technology systems their districts have acquired. Superintendents do not typically
use computerized data systems themselves to get the data they need to make
decisions; instead, to their staff and principals they will pose questions that require
data to answer and their staff will acquire and analyze the data either for or with the
superintendent. Therefore, it is not always necessary for the superintendent to be
highly skilled in the use of a computerized data system to be able to make effective

74

data-driven decisions. The superintendent needs the other data use skills in order to
ensure the data and information that are provided to them is accurate and meets their
needs for the purpose of making the decisions.
Only a few superintendents identified a formal program that they attended as the
source of their data use skills; instead, the majority stated that they learned their
skills at using data from on-the-job experience. Their university programs taught
them how to disaggregate data and to perform basic analysis of data, but their ability
to analyze data in a way needed to make the decision in running their districts and
affecting student achievement come from experiences in their profession. They
learned these skills from training provided by outside organizations, from other
district employees with greater skills, from collaboration with others in the same
position, and sometimes by trial-and-error.
Research Question One Results from Interviews
Each of the five superintendents who participated in the individual interviews
answered three open ended questions that asked them to identify skills and
knowledge they felt were important in using data effectively, evaluate their current
level of skills in using data to make decisions, and to identify where they believe
they acquired those skills.
When asked about what competencies were important for superintendents to have
for them to make effective data-driven decisions, all five identified the abilities to
analyze data, and technical skills as necessary. Three of them described the ability to
formulate questions about student achievement, programs, or practices that required

75

data to answer and being able to formulate an answer as a very important skill.
Other knowledge and skills that were identified by them were ability to disaggregate
data, enable others to use data, and communicate using data.
All superintendents made repeated connections between what competencies they
felt were important and what competencies they presently possessed. When
discussing the competency, they would identify their use of that skill or their ability
in that area. There was only one area where their ability level did not match the level
they felt was important. Four of them rated their ability to use technical data
programs as below what they felt was necessary. Their explanation for this was due
to the rapid changes that occur in technology and the fact that as superintendents
usually others use the computer systems and provide them with the data they request,
rather than they doing it themselves.
All five superintendents identified their university doctoral program as giving
them foundational skills in analyzing data and technical skills in analyzing data, but
identified no other skills or knowledge pertaining to using data that were obtained
through their university programs. All five superintendents identified experiences on
the job as providing the majority of their data use knowledge and skills. Two
superintendents identified specific people with whom they worked in the past as their
source. Two others identified that they learned their competencies from practice, by
being faced with problems or questions and using data to attempt to come up with a
solution or answer. One superintendent identified a skill that they acquired from
being in the same district working through the ranks from teacher to superintendent.

76

The identified skill was knowing who created particular types of data and who had
that data, the knowledge of which enabled the superintendent to get multiple types of
data quickly.
When comparing the responses of the superintendents to the conceptual
framework for effective data-driven decision making, there were numerous
commonalities and trends in their level of competencies in data use and technical
skills. All five superintendents identified data use skills and knowledge that placed
them in the very effective range of data-driven decision making. Only two of them
identified skills that place them in this range for technical data skills; the other three
had less skill and were placed in the somewhat effective range. All five were rated
in the very effective range for their ability to use standardized student assessment
data, but only two were rated in the same range in their use of local data or bench
mark assessment data. The other three were rated in the somewhat effective range
because they used local data but did not identify using benchmarks, common
formative assessments, or other local type of student achievement data.

77

Table 8:
Trends from Interviews with Superintendents- from the conceptual framework for
effective data-driven decision making.
Actions or Skills
Superintendents
Average
A B C D E
Have Data Use Skills- ability to analyze and
disaggregate data, determine which data to use to
answer specific questions.
Yes
3
Yes
3
Yes
3
Yes
3
Yes
3
3
Have Technical Skills – ability to use a computer
system to extract needed data.
Yes
3
Yes
2
Yes
3
Yes
2
Yes
2
2.4
Use of Standardized Test Data – California
Standards Tests, CELDT.
Yes
3
Yes
3
Yes
3
Yes
3
Yes
3
3
Use of Local Data/Benchmarks – benchmark,
teacher grades, surveys, etc.
Yes
2
Yes
2
Yes
3
Yes
1
Yes
3
2.2

Discussion of Interview Responses
All of the superintendents stated that the ability to analyze and disaggregate
student data was essential to their jobs and their ability to use data in their decision
making. They needed to be able to make sense of the numbers that were before them
and to understand the relationship these numbers have with student achievement and
what was or was not happening in the classrooms. The people in the position of
superintendent rarely sit in front of a computer or spread sheet and analyze the
numbers that represent student scores and levels of achievement; instead, they ask
questions regarding student achievement and academic programs, while their
assistant superintendents, someone in educational services, or technology and data
departments then provides the data they need.

78

Superintendent C explained it this way:
“The major role I play is enabling the decision makers to use data. I want to
enable talented people to analyze, reconfigure, trace, anticipate, dig deep, and
ask persistent questions about achievement. So my role is, what I can do to
enable them to make those decisions. So I look for the systems of data, the
procedures of data, mechanism of data, and software programs. A significant
part, however, is I have to hold all of us accountable.”

However, through the analysis of their responses during the interview, it was
determined that they all still have a high ability level in the area of analyzing and
disaggregating student data.
These superintendents all stated that they learned the majority of their data
analysis skills from on-the-job training, that training being provided by their districts,
by a person in their district who was skilled in data use and trained them, or by
figuring it out themselves. Superintendent A described their learning experience:
“The detailed use of it [data] and the analysis of it really came from practice,
from being a superintendent, looking at scores and trying to figure out
schools that are making progress, schools that are not making progress. Why
they are or why they are not. So, really the detailed look at it came from
being a working superintendent.”

Since they all had to analyze and disaggregate data in their positions before
becoming superintendents, all stated that they started to seriously analyze data while
they were principals of schools. They learned the data use needs of each position as
they moved up into the position of superintendent. The one area where three of them
stated that their skill level was only somewhat effective was the technical skills
required to use data analysis and warehousing computer programs. This lower skill
level was due to two factors: the speed at which technology changes, and, as stated

79

above, superintendents rarely analyze their own raw data but instead have others in
the district provide the data they need to answer the questions that they posed. They
all stated that the one skill they that still needed to acquire when they got into the
position of superintendent was how to ask the right questions related to data. When
solving a problem or determining the course of action related to student achievement,
the superintendents needed to understand what data they needed to make these
decisions and how to formulate a question that would provide them with the most
appropriate data.
Research Question Two: What Sources of Data Superintendents Use and
How They Determine the Value of That Data

Research question two was written for the purpose of identifying what types of
data superintendents relied on in their decision making process and if there was a
common criteria among superintendents for determining which data to use to answer
specific questions. The survey listed nineteen areas of responsibility in which
superintendents might use data to make decisions and then asked the superintendents
to identify which types of data they use in making decisions in each area. They could
choose more than one type of data if more than one type factored into their decision
making process. The nineteen areas of responsibility were divided into three
sections that shared like areas of their job responsibilities. One area, consisting of
seven responsibilities, asked superintendents to identify which data they used in
decisions regarding management of the district and evaluation of staff and programs.
The second area, consisting of six responsibilities, asked superintendents to identify
which data they used in communicating goals, results, and challenges to staff, the

80

board of education, and the community. The third area, consisting of six questions,
asked superintendents to identify what types of data they expected district and school
site educational leaders to use in making different types of decisions. Four types of
data that are commonly used and available in school districts were listed for the
superintendents to choose from with space provided for writing in any other type of
data that they may use. The four listed types of data were: California Standards Test
(CST) data, data from district created tests or benchmark assessments, instructional
program data, and demographic data. Data from the CST is used by the State of
California and districts to determine student progress toward NCLB student
proficiency standards and other measures. District-created tests or benchmark
assessments are used by schools and districts to measure student progress several
times during the school year. Instructional program data is used to look at the
effectiveness of a various types of programs. Demographic data is used to measure
student achievement, program effectiveness, and other factors by grade level, race,
sex, language, socio-economic status or other types of data. Only four
superintendents wrote in a type of data that they used other than the four types
identified in the survey. Two checked the box indicating they used another type of
data but did not identify that type of data and two identified customer satisfaction
survey data as information they used in their decision making.

81

Types of Data Superintendents Value and
Use in Decision Making

The survey asked the superintendents what types of data they use in making
decisions regarding district level staff assignments, school site staff assignments,
budget allocations, their annual focus or priorities, evaluating teacher/principal
performance, which programs, textbooks, or consultants to purchase, and type of
intervention or remedial programs and materials to be used district wide.
Superintendents could choose one or more types of data to indicate which data they
used.
Out of the 23 superintendents included in the survey, an average of 17.3 used
CST data in all seven decision-making areas. All 23 superintendents used CST data
to determine their annual focus or priorities, whereas only 12 used CST data to make
decisions about school site staff assignments, and 13 used CST data in making
decisions about budget allocations and district staff assignments.
Demographic data was used the least by superintendents, with an average of only
14.3 superintendents using this type of data in the seven areas of decision making.
The area that they used demographic data the most was in deciding what type of
intervention or remedial program to use district wide, with 17 superintendents using
this type of data.
Superintendents could choose any of the types of data in their responses and 16
of the superintendents indicated that they use all four types of data in their decision
making regarding the types of interventions or remedial programs to use in their
districts.

82

Only one superintendent used just one type of data to make decisions in this area.
Fourteen superintendents used all four types of data when determining their districts
focus or priorities for the year.
Tables 9:
Types of Data Identified in Surveys That Are Used by Superintendents to Make
Decisions
Types of Data
District Staff
Assignments
School Site Staff
Assignments
Budget
Allocations
Annual Focus
and Priorities
Evaluating
Teachers/
Principals
Acquisition
of Programs,
textbooks
Types of District
Wide
Interventions
Averages Area 1:
Decision Making
CST Data
14 13 14 23 19 18 20
17.3
District Assessment/
Benchmarks
10 12 12 19 17 14 21
15
Program Data
13 13 15 18 17 16 22
16.3
Demographic Data
12 14 14 16 13 14 17
14.3
Other Type of Data
4 4 2 3 2 1 3
2.7

Types of Data Superintendents Value and
Use in Communication

The survey asked the superintendents to identify which type(s) of data they used
when communicating a selection of goals and decisions to school and district staff
and the board of education. They were asked what data they used in
communications in six areas regarding district goals and objectives, district
initiatives, budget decisions, student achievement, challenges the district faces, and
initiation or closing of a program. As in the previous set of questions, they could
choose any or all of four types of data: CST data, data from district created tests or

83

benchmark assessments, instructional program data, and demographic data. They
also could write in any other type of data that they used.
The superintendents’ responses to the survey indicated that they used all four
types of data in their communications. All 23 superintendents used CST data in
communications about district goals and objectives and student achievement. CST
data was also used by 22 superintendents in communications regarding district
initiatives. All 23 also used program data in communication about the initiation or
closure of a program. CST and demographic data were the most used types of data in
all six communication categories. CST was used by an average of 21.2
superintendents across all six categories and demographic data averaged 19.5.
Program data averaged 18.8 superintendents in each category.
Table 10:
Types of Data Identified in Surveys That Are Used by Superintendents to
Communicate with Staff and the Community
Types of Data
District Goals
and Objectives
District
Initiatives
Budget
Decisions
Student
Achievement
Challenges the
District Faces
Beginning
/Ending a
Program
Averages Area 2:
Communicating
Vision and
Expectations
CST Data
23 22 19 23 21 19
21.2
District Assessment/
Benchmarks
16 18 17 20 15 18
17.3
Program Data
19 21 18 20 17 23
18.8
Demographic Data
18 21 14 19 19 17
19.5
Other Type of Data
3 3 4 2 3 3
3.0

84

The least used data for communicating was demographic data in regard to budget
allocations. Fourteen superintendents indicated that they used this type of data when
communicating about budget decisions. Fifteen superintendents indicated that they
use district benchmark data when communicating about purchasing programs,
textbooks, or consultants. Benchmark data was the least used on average by the
superintendents, with an average of 17.3 superintendents using this data in their
communications in all six categories.
Types of Data Superintendents Value in
School Site Decisions
Superintendents responded to six questions asking them to identify which of
the four types of data they expected school site and district staff to use when making
decisions about six school site or program decisions that are made. The six
categories were the general areas that educational administrators make on a regular
basis: the types of interventions to implement, the placement of students in
programs, budget decisions, the hiring and placement of teachers and staff,
classroom instructional practices, and the evaluation of teachers and
paraprofessionals.
Twenty-two of the superintendents identified district benchmark data as an
expectation in school site decisions about the types of interventions to implement
and classroom instructional practices. Twenty-two of them also said that program
data was expected in making decisions about intervention programs. Twenty-one
superintendents expected site staff to use CST data in intervention program
decisions. In this aspect of data-driven decision making the majority of

85

superintendents indicated that district benchmark data and program data were the
most valuable, with an average of 18.2 of them choosing these types of data in each
area. Superintendents’ expectations about what data to use in making decisions
about intervention programs was the most comprehensive with 17 of the 23
expecting school sites to use all four types of data in these decisions, and the
remaining six superintendents expecting them to use at least three types of data.
Table 11:
Types of Data Identified in Surveys That Superintendents Expect Their Educator to
Use to Make Decisions
Types of Data
Type(s) of
Interventions to
Implement
Placement/Progra
mming of
Students
Budget Decisions
Hiring /
Placement of
Teachers/Staff
Classroom
Instructional
Practices
Evaluation of
Teachers/Staff
Averages Area 3:
Expectation of
Data Use by
Others
CST Data
21 19 18 16 18 17
18.2
District Assessment/
Benchmarks
22 19 16 9 22 18
17.6
Program Data
22 17 18 15 16 18
17.6
Demographic Data
20 13 15 14 14 10
14.3
Other Type of Data
3 3 3 2 2 2
2.5

The use of benchmark data by school site staff in the hiring and placement of
teachers and staff was expected by just nine superintendents and only10 expected
them to use demographic data when evaluating teachers and paraprofessionals.
Demographic data was expected to be used the least with only an average of 14
superintendents expecting it to be used in each category.

86

Eight superintendents expected all four types of data to be used in making decisions
about placement of students, the hiring of educators, and the evaluation of teachers
and staff.
Table 12:
Summary Averages of Types of Data Identified in Surveys That Are Used by
Superintendents
Types of Data
Area 1:
Decision
Making
Area 2:
Communicating
Vision and
Expectations
Area 3:
Expectation of
Data Use by
Others
Average Over
All Three Areas
CST Data 17.3 21.2 18.2 18.9
District Assessment/ Benchmarks 15 17.3 17.6 16.6
Program Data 16.3 18.8 17.6 17.6
Demographic Data 14.3 19.5 14.3 16.0
Other Type of Data 2.7 3.0 2.5 2.7

Discussion of Survey Results for
Research Question Two

The majority of superintendents valued CST data in all of the areas that they
used data-- decision making, communicating, and evaluating. Data from the CST
provide generalized data on how a school, subgroup, or program has done and
provides a person with a direction to analyze the data deeper or what comparative
data to also look at. Although all of the superintendents used CST data for most or all
of their decisions, rarely did they use only CST data in making decisions,
communicating, or evaluating employees. Program and benchmark data was
frequently the other type of data used in making the decisions or evaluating people or
programs.

87

Benchmark data was valued by the superintendents because what they provided
was more closely connected to what teachers were doing in the classroom and the
superintendents could access this data quicker and periodically throughout the school
year. Superintendents and schools are still in the process of calibrating CST data
with benchmark data, meaning that students who score a certain level of proficiency
on benchmarks will score at that same level of proficiency on the CST. Once districts
and schools can do this, they can use the benchmark data to intervene more quickly
and precisely with students who may not do well on the CST assessments.
Superintendents valued program data in making decisions on which programs to
continue, expand, or discontinue. This type of data varied from number of students
enrolled in the program, to percentages of students successfully completing the
program, and correlative data connecting participation in the program and a desired
outcome. Often programs are connected in some manner with attempting to improve
CST scores, and the data provided by these programs are another indicator on how a
school or district may improve their CST scores as indicated by their API and AYP.
As accountability has intensified, superintendents cannot rely on an annual indicator,
like CST data, to determine how their students are doing, but instead must use
benchmark and program data throughout the year to monitor their students’ progress
or lack of it.

88

Results from Interviews for
Research Question Two
The five participating superintendents answered an open ended question asking
them to identify the types of data they use and how they determine the value of that
data. Many also added the types of decisions for which they use the data. Each
superintendent identified multiple types of data that they used in a variety of
situations.
All five superintendents participating in the interviews identified CST, as data
they use to get an overall determination of progress at the school and district level.
They used both the AYP and API to provide them the student achievement data they
needed. Three of the superintendents listed bench mark results as a type of data they
used frequently. They reported using this type of data to get a closer look at what
schools, teachers, and students were accomplishing in the classroom. They also used
it to get more frequent measurements of student achievement. Other types of data
that they identified using were graduation rates, demographic data, fiscal data,
survey data, attendance data, and program data. All five superintendents stated that
they use multiple sources or types of data when making the majority of their
decisions. They all stated that CST data gives them the “big picture” information,
while benchmark data provides them with detailed information. Other types of data
are then factored in to making the best decision on what action to take or information
to present.
When comparing the superintendents’ responses to the effective data-driven
decision making framework, all five of them were in the very effective range in how

89

they used standardized achievement data because they all used this type of data to
determine achievement gaps, school progress, and the effectiveness of principals. In
using benchmark data and other local assessment data only two of the five rated in
the very effective range because they use this type of data to analyze the progress of
specific groups of students or the results of specific principals or teachers. One
superintendent rated as ineffective in this area because they identified the use of
benchmark data as a good idea but admitted that it was not currently being used, nor
any other local types of data. Three of the superintendents were rated in the very
effective range in their ability to develop an inquiry type approach to using data to
make decisions. These three stated that they develop questions that they need to
answer regarding student achievement before making decisions, and then collect and
analyze that data to make an effective decision. All five superintendents rated very
effective for their use of multiple types of data because they all used a variety of data
in their decision making process, from CST data to demographic, or fiscal data. All
five superintendents also used data to communicate with their boards of education,
but four were rated as somewhat effective because they used the data to present
reports to their boards, while one superintendent was rated as very effective because
he used data to have discussions and solve problems with their boards, in addition to
using data to present information to them.

90

Table 13:
Data Use Trends from Interviews with Superintendents
Actions or Skills
Superintendents
Average
A B C D E
Have Data Use Skills- ability to analyze and
disaggregate data, determine which data to use to
answer specific questions.
Yes
3
Yes
3
Yes
3
Yes
3
Yes
3

3
Use of Standardized Test Data – California
Standards Tests, CELDT.
Yes
3
Yes
3
Yes
3
Yes
3
Yes
3
3
Use of Local Data/Benchmarks – benchmark,
teacher grades, surveys, etc.
Yes
2
Yes
2
Yes
3
Yes
1
Yes
3
2.2
Uses Multiple Measures to Make Decisions – uses
more than one type or source of data when making
decisions.
Yes
3
Yes
3
Yes
3
Yes
3
Yes
3
3
Uses Inquiry Method – develops a question that
around the district’s goal or what information is
needed and then uses data to provide the answer.
Yes
3
No
2
Yes
3
Yes
3
No
1

2.4
Uses Data to Communicate with the Board of Ed. –
uses data when presenting proposals or reporting
on the district’s progress to the board members.
Yes
3
Yes
2
Yes
2
Yes
2
Yes
2
2.2

Discussion of Interview Results –Research Question Two
The superintendents that participated in the interviews indentified the same types
of data as valuable to them as the superintendents from the surveys. They all said
that they used CST data to get an indication of how their students or programs were
progressing but that they needed other data to provide them with the details on how
and why students were or were not progressing. Superintendent A described the role
CST data and multiple measures:
“So you’re looking at CSTs, I think you start there and then when you start
looking at a school you start looking at multiple measures. If you’re looking
at a high school, for example, you’re going to look at graduation rates, the
percentage of kids that are taking AP [Advanced Placement], the pass rate for
APs, you’re going to look at CAHSEE data, you’re going to look at a lot of
different pieces at the high school.”

91

Four of the five superintendents used benchmark data to provide this detailed look at
student progress and expected all of their educators to be doing the same. The fifth
superintendent spoke about the benefits of this practice but had not instituted district
wide benchmark assessments at the time of the interviews. Superintendent E stated
his expectations for using benchmark data:
“From a district perspective I think the data that teachers generate through
benchmarking and common assessments is probably the most powerful data
that we use in regards to instruction, but as a superintendent I don’t look at
the data of my individual students in my classrooms.”

Three of the superintendents used and valued customer satisfaction survey data in
evaluating employees, policies, and programs. These three superintendents stated
that survey data was an important part of school and district management personnel’s
evaluations. One superintendent said that survey data was used by his board of
education to evaluate his performance during the year. All three of these
superintendents’ districts used customer satisfaction surveys of their communities,
while two also surveyed their teachers and students.
All five superintendents used demographic data, saying that it helped them
identify whose achievement was progressing and whose was not, and which students
are receiving intervention services. Accountability requires that districts and schools
ensure that all students are improving academically and demographic data is used by
superintendents and other administrators in conjunction with benchmark and
program data to make sure that all students are improving.

92

Research Question Three: Specific Strategies and Policies
Superintendents Use to Increase the Use of Student Data by Educators
in Their District

The third research question was developed with the understanding that to improve
student achievement district wide, all educators in that district must be able to use
student data to inform decisions about curriculum, teaching, and learning. This
research question sought to identify the practices, policies, and communications from
the superintendent that increased the use of student data by all educators in their
district.
Superintendents responded to a survey that asked them how frequently they
engaged in eight behaviors or actions that encourage the use and acceptance of data
use in their districts. Twenty–three superintendents responded to this survey using a
4-point rating scale that indicated the frequency of the behavior: Never, Sometimes,
Occasionally, and Frequently.
The difference between Sometimes and Occasionally was specified for them as
Sometimes meaning once or twice a year and Occasionally meaning more than twice
a year but not at every opportunity.
Actions Superintendents Engage in to Increase
Data-Driven Decision Making in Others

Superintendents identified communicating clear links between data, student
achievement, and the decisions that need to take place as the action they engaged in
most frequently with 19 of the 23 superintendents indicating they did this at every
opportunity, and the other four did it more than twice a year. Eighteen
superintendents also provided teachers with time to analyze data and collaborate with

93

one another about it at every opportunity. Four of the other five provided such time
more than twice a year. Seventeen of the 23 superintendents provided professional
development opportunities in data use, communicated clear expectations of data use,
and provided principals and teachers opportunities to create useful data at every
opportunity. An average of 16.5 superintendents frequently engaged in the identified
behaviors across all eight categories.
Table 14:
Frequency of Data Use Actions by Superintendents
Frequency
Provides up -to-date
data program
Provides Time for
Collaboration
Professional
Development on
Data Use
Clear Expectation of
Data Use
Provides
Opportunity to
Create Data
Data Provided to
Address Problems
Clear Connection
Made Between Data
and Action
Use Inquiry Method
When Using Data
Never
1 0 0 0 0 0 0 0
Sometimes
2 1 1 0 0 1 0 3
Occasionally
4 4 5 6 6 6 4 8
Frequently
16 18 17 17 17 16 19 12

Not all superintendents in this study engage in all of the identified actions; three
superintendents indicated that they provided an updated data analysis computer
system infrequently or never. Three also only sometimes used an inquiry method of
communicating and problem solving with staff.
Discussion of Survey Results for Research Question Three
All of the superintendents indicated in their surveys that they engaged in activities
that promoted a data-driven decision making culture in their districts. In their
position as superintendent of urban school districts, it is vital in this age of
accountability that they ensure all educators in the district are aware of where their

94

students are, where they want them to be, and how they plan on getting them there.
The superintendents cannot do this themselves but they can create a climate that
encourages, promotes, and clearly values a data-using culture. The survey results
indicated that all of the superintendents were engaging in the activities necessary for
creating this data-using culture in their districts.
The skills and actions that the most superintendents reported that they engaged in
focused on communicating the expectation and necessity of using data to improve
student achievement and providing the skills and opportunities for their educators to
effectively use data for this purpose. Providing time, usually through modified
schedules or professional development days, for teachers and principals to
collaborate on student data, and communicating a clear connection between student
data and the actions that are being engaged in or need to be engaged in, are examples
of such behavior.
The inquiry method of using data, asking an important question regarding student
achievement or practice and then analyzing data to determine the answer to that
question was the action least engaged in by superintendents. This action is usually
incorporated into the professional learning communities model of professional
development for educators and this study did not examine to what extent these
districts or superintendents engaged in professional learning communities in their
schools. Therefore, it is not possible to know if the reason they do not engage in this
action is related to their districts having or not having professional learning
communities.

95

Interviews Results for Research Question Three

The superintendents participating in the interviews answered five open-ended
questions regarding the actions and policies that they engage in to increase or
improve the use of data among all educators in their districts. These questions asked
about their acquisition of, access to, and training in a data management software
system. What amounts and types of professional development in the use of data are
provided at the district level? What types of data are made available to teachers and
principals? What policies and actions do they engage in to promote the use of data?
The last question, regarding the promotion of educators’ use of data, asked them if
they have identified any obstacles in their district to the use of data and what steps
they have taken to alleviate those obstacles.
All five superintendents either had a district wide data program or were in the
process of implementing one. Interestingly, all of them were using the same data
program, Data Director. This system is a web-based data program that allows anyone
with permission to access their student data from any computer connected to the
internet. It has numerous features that allow the user to compare and analyze the
data and print spreadsheet and reports. Four of the five superintendents stated that
they have clear expectations of data use among their principals and teachers that they
communicated to them numerous times a year. The fifth superintendent stated that
they expect data to be used by their principals in their annual goals, but it is up to the
principals if they expect their teachers to use data. Three of the superintendents said
that using data was written into one or more of their district annual goals or
objectives. Three of them also identified several ways that they provide professional

96

development training in the use of data for teachers and principals. Two
superintendents ensure that there is at least one person at each school site or a person
who can go to that school site and assist with getting data for teachers and provide
training for teachers in using data. Nearly all of the superintendents identified the
teachers union or the collective bargaining agreement as the only obstacle to using
data in their districts, particularly with regard to evaluations of teachers. One
superintendent said that the union attempts to keep them from using data but he
ignores them. This question was meant to identify attitudes, skills, or knowledge
gaps that might reduce the use or effectiveness of data use, but all of the
superintendents spoke about this obstacle in using data as an evaluation tool. This
will be examined further under question four.
When comparing the superintendents’ responses to the effective data-driven
decision making framework in regard to promoting a data-driven decision making
culture within their districts, all five of the superintendents scored in the very
effective range, rated a 3, for clear communication of their expectation of data use by
all educators in their districts. Four of them rated very effective for providing an
easily accessible district wide data system, with the fifth superintendent scoring
somewhat effective, rated a 2, because same system was not in place yet but had
been acquired. Two of the superintendents rated very effective and three somewhat
effective in level of training they provided district wide in the use of data. The two
who rated very effective had an ongoing training program that addressed the needs of
educators at various skill levels. Three that rated somewhat effective stated that the
district only provided training when the need arose or a new data system was being

97

implemented; otherwise, school sites provided data training. Four of five
superintendents scored in the ineffective range, rated a 1, for promoting the use of
data because they identified the teachers union or the teachers’ contract as an
obstacle that kept them from using data the way they thought it should and the only
suggestions they had for alleviating this obstacle was to get rid of the teachers’
unions. The fifth superintendent stated that his teachers union’s attempts to be an
obstacle in the use of data but he doesn’t allow them, while at the same time he feels
he is adhering to California’s Ed. Code.
Table 15:
Trends in Use of Data from Interviews with Superintendents from the conceptual
framework for effective data-driven decision making.
Actions or Skills
Superintendents
Averag
e
A B C D E
Communicates Expectations of Data Use – sets clear
expectations of their principals and other leaders to
use data in making decisions.
Yes
3
Yes
3
Yes
3
Yes
3
Yes
3
3
Acquisition of Educator Accessed Data Program –
has the superintendent made steps to acquire or
update a system that allows all educators in the
district to access student data.
Yes
3
Yes
3
Yes
3
Yes
2
Yes
3
2.8
Provides Training in Use of Data – professional
development in the use of data provided by the
district.
No
2
Yes
3
Yes
3
Yes
2
No
2
2.4
Use of Local Data/Benchmarks – benchmark, teacher
grades, surveys, etc.
Yes
2
Yes
2
Yes
3
Yes
1
Yes
3
2.2
Uses Inquiry Method – develops a question that
around the district’s goal or what information is
needed and then uses data to provide the answer.
Yes
3
No
2
Yes
3
Yes
3
No
1
2.4
Believe Teachers Union is Obstacle to Use of Data –
the collective bargaining agreement or teachers union
forbids or restricts the use of data in one or more
ways.
No
3
Yes
1
Yes
1
Yes
1
Yes
1
1.4
Uses Data to Communicate with the Board of Ed. –
uses data when presenting proposals or reporting on
the district’s progress to the board members.
Yes
3
Yes
2
Yes
2
Yes
2
Yes
2
2.2

98

Discussion of Interview Results for Research Question Three

During the interview discussions of how the five superintendents promoted a data
using culture in their districts several of them spoke of the importance of using data
not only to make decisions but also to support a decision they had made. They often
provided data to their staff, teachers, and their boards of education to provide
evidence as what needed to be done or what still needs to be done. They felt that it
reduced opposition to their decisions when they could back it up with good solid
data.
All of the superintendents also felt that they communicated clear expectations
regarding all educators using data to make decisions. They felt that it was important
that their principals use data in their decision making and held them accountable for
doing so. Superintendent A described how his principals were held accountable for
using data:
“When I saw a school that was struggling, fluctuating in its API and [I]
started talking to the principal about how the principal was going to make
improvements. [I asked] What the plan was to get themselves back on track
and when they couldn’t explain to me what the process was and how they
were going to use data in that process, I knew it was time for that principal to
go.”

They also held their principals accountable for ensuring that their teachers and other
staff used data to make decisions about teaching and learning. Superintendent D also
stated a clear expectation of data use by his principals:
“We have made it [use of data] clear to our principals, who we then expect to
carry back to their school sites that the decisions that they make with regards
to the instructional program will be based on the data they collect. Otherwise
what is the point of collecting it?”

99

This expectation was most evident in the goal setting and evaluation process for
administrators and in some cases even teachers. This will be discussed further under
research question four.
All of the superintendents interviewed had recently acquired an updated data
analysis and warehouse program and were in various stages of implementing and
using this program. All five were using the same program, Data Director. Data
Director provides readily available student data to any person with access to the
internet and access privileges to the system. It provides easy to understand and
analyze data from any source that is put into it, from CST to teacher-created
assessments. The reason for using Data Director was described by Superintendent B:
“We started with EduSoft when I first got here and then kind of morphed to
Data Director and because it is web based our teachers have access 24/7, so
they can go on line anytime anywhere through the staff portal on our website
and access their student data and information.”

Many districts like this program due to its ease of use and access but also its ability
to compare various sources of data. All five superintendents acquired this program
and have been providing training and access in its use.
Several of the superintendents reported having specific people or services
available to their teachers and school sites to provide generalized training in data use
and technical skills or specific support in finding the right data to answer a certain
data-related question. To ensure that their principals and teachers are using data in
their decision making, they are providing technical and professional assistance so
that lack of knowledge will not be an obstacle to using data.

100

One obstacle to data use that all but one superintendent identified was the
teachers union and/or the collective bargaining agreement. Superintendent E
expressed his feelings about the teacher collective bargaining agreement’s
restrictions on using student data:
“No, and as you know our evaluation tool is bargained and it does not have
any provision for utilizing data in the evaluation that we have negotiated,
those evaluations from my perspective have been watered down through the
process of collective bargaining over 40 years.”

They stated that their teachers unions were skeptical of the value of student data and
its connection to classroom practices. They reported that the teachers unions often
opposed increasing the use of data by teachers and administrators, and in one district
they opposed creating time for teachers to analyze data and collaborate around it.
Superintendent A stated that his teachers union often voiced opposition to using
student data but stated that he followed California’s Education Code and used data
and required others to used student data despite their opposition.
“For classroom teachers, I don’t believe the collective bargaining agreement
precludes us from using data and I don’t believe the California Ed Code
precludes us from using data, so do I want my principals using data as part of
a process for improving teacher’s instruction in the classroom? Absolutely!”

Research Question Four: How Superintendents Use Data to Determine the
Effectiveness of Their Educators, Administrators, and Programs
This research question was developed with the assumption that data can be used
to determine the effectiveness of people, programs, and policies.

101

This question was designed to determine if superintendents used data to evaluate
people, programs, and policies and if so, how that data weighed in their decisions
regarding these evaluations.
The survey did not specifically ask the superintendents how they used data in the
evaluation process of educators and programs, but information from the survey did
reveal whether they used data in the evaluation process and, if so, the types of data
they use and prefer. The survey results also indicated how confident they felt using
data as part of evaluations and where they learned to use it, along with their
expectations of school site management using data in evaluations.
The specifics on how they used data in the evaluation of principals were addressed in
one-on-one interviews.
Superintendents’ Level of Confidence in
Using Data in Evaluations
In responding to the survey question about their current level of competency in
using data to evaluate principals and teachers, 17 of the 23 superintendents
responded that they were skilled or Expert in this area, meaning that they felt they
knew everything necessary for this skill set. Only one other area had fewer
superintendents responding with this level of confidence, the use of data processing
software and/or hardware. All superintendents reported that they currently were
knowledgeable about most or all aspects of this skill set.
Eight superintendents indicated on the survey that they achieved a competency
level of Expert or Skilled from formal training in their university program, while 20
indicated that they achieved this competency level from on-the-job experiences and

102

collaboration. Four superintendents reported that they learned little in this
competency area from formal training, but no superintendents reported that they
acquired little to no skills in this area from on-the-job experiences.
Table 16:
Superintendent Competency Levels in Using Data to Evaluate Educators
Novice Developing Proficient Skilled Expert
Current Level of Proficiency in
Using Data to Evaluate Staff.
0 0 6 14 3
Level of Proficiency in Using Data to
Evaluate Staff Acquired from
University Program.
3 4 8 5 3
Level of Proficiency in Using Data to
Evaluate Staff Acquired from On the
Job Experience.
0 0 3 18 2

Types of Data Superintendents Use and
Value in Evaluations

Responses to the survey indicated that superintendents use multiple sources of
data when making evaluations of principals and other staff. All 23 superintendents
indicated they used at least two types of data in this area and nine indicated they use
all four types of data that were listed in the survey when making evaluations. CST
data was the most frequently used, with 19 of the superintendents using this type of
data, followed closely by benchmark exam data and program data, with 17
superintendents using this data in their evaluations. Demographic data was the least
used, with 13 of them indicating they use this data. Twenty-one of the
superintendents responded that they expected principals and district management to
use data in evaluation of teachers and staff. They expected them to use multiple
sources of data as well, with only one superintendent indicating the expectation of

103

one source of data. Eight of the superintendents expected their principals to use all
four sources of data and five expected them to use at least three. The use of district
benchmark exam data and program data were both expected to be used by 18 of the
superintendents and 17 of them expected the use of CST data in the evaluation
process. The data that was expected to be used by the least number of
superintendents was demographic data, with only 10 indicating they expected that
type of data to be used as part of the evaluation process. Two superintendents wrote
in ‘customer satisfaction survey data’ as a type of data they expected their principals
to use as part of the teacher and staff evaluations.
Table 17:
Data Valued by Superintendents When Using Data to Evaluate Educators

CST
Benchmark
Data
Program
Data
Demographic
Data
Other
Types of Data Used to Evaluate
Principal’s and Teacher’s
Performance.
19 17 17 13 2
Types of Data School Site
Administrators Are Expected to
Use in Evaluating Teachers.
17 18 18 10 2
Types of Data Used to Evaluate the
Effectiveness of Intervention and
Curricular Programs.
19 18 23 17 3

Discussion of Survey Results of Research Question Four
All of the superintendents participating in the surveys believed that they were at
least proficient in using data in their evaluation of staff, with the majority believing
they were Skilled or Expert in this area. The ability to use data to evaluate an
employee means that they can look at several sources of data that have a direct
connection to the employee’s performance and determine the effect the employee is

104

having on student achievement and/or meeting their goals. These data usually
include CST data, benchmark data, and at least one other source, such as survey data.
The survey results indicated that they relied heavily on CST and benchmark data
but also on program data, thereby providing a triangulation of the results before
making a decision. Program data was most important, according to the
superintendents, in evaluating the effectiveness of a program because these programs
often target a particular demographic of student, such as, ELL, Hispanic, or Socio-
economically disadvantaged. The superintendents would look at demographic data to
determine if the program was targeting the right students and if the desired effect
was being achieved.
The surveys indicated that this skill area was one that they particularly learned
from on-the-job experience instead of from a university program. This may be due to
the fact that using data as part of evaluations is a result of the accountability
movement that is part of NCLB and that most of these superintendents completed
their university program before or at the beginning of this movement. As the
superintendents were being held more accountable for student achievement as
indicated by data, they began holding others more accountable in the same way. For
them to hold their educators accountable, they need to be able to evaluate their
employees in a reasonable way using student data.

105

Interview Results for Research Question Four
There were three question asked in the interviews that solicited the
superintendents’ views, practices, and results in using data to evaluate their
administrators, teachers, and programs. The questions asked them if they used data
as part of their administrators’ evaluations, and if so, what type of data were used.
They were also asked the same question about their teachers and data use in their
evaluations. The third question asked them to describe the process they use when
evaluating a principal or teacher using data.
All five superintendents disclosed that they use data in the process of evaluating
their principals. The process they undertake in this evaluation method was very
similar among all five superintendents. They all discussed the pressure the districts
and schools are under to improve student achievement, and their evaluation process
has changed to meet this demand. They said that they use CST data to look at which
schools/principals are successful and which are not, as measured by increasing API
and AYP. All principals must use data as part of their goals to move their schools
forward and these goals are part of their evaluation. Three out of the five
superintendents said they would not fire principals for having low test scores, but
would fire them if they could not or would not use data to develop an effective plan
to improve student achievement in their schools. They all stated that this process
takes place over several years and one year of no progress would not cost principals
their jobs as long they could develop a plan to move forward, and this plan
incorporated measurable data. All of the superintendents in the interview identified
several types of data that would be used in determining the principals’ goals and

106

measuring their level of success. They identified benchmark data, customer
satisfaction survey data, graduation rates, enrollment in advanced placement classes,
and attendance data. Four of the superintendents placed strict importance on the need
for their principals to be able to use data to make decisions about their schools and
student achievement, and if their principals cannot demonstrate this ability, they
would not work for them for long.
All five superintendents expected the teachers in their district to use data to
determine the needs of their students and to drive instruction practices in their
classroom and in the school. Four of them said that they did not, or could not expect
their principals to evaluate teachers based on their student achievement data because
of provisions in their collective bargaining agreements, but they all said that they
expected principals to evaluate their teachers based on their ability to use data
effectively in addressing the needs of their students. One superintendent held that he
did expect his principals to use student achievement data as part of the teachers’
overall evaluation. The types of data they these principals expected to be used by
their teachers were CST data, benchmark data, and demographic data to identify
their sub-groups. They all stated that student achievement data and their ability to
use it is only part of the overall evaluation, and that like the evaluation of principals,
is a process that takes place over several years. A teacher who had poor student
achievement results one year would not be fired unless it occurred over a long period
of time. Two superintendents stated that they currently do use student achievement
data to evaluate which teachers or principals would work best with a specific set of
students or work best in a particular school.

107

When comparing the superintendents’ responses to the effective data-driven
decision making framework, all five superintendents stated they use student
achievement data as part of their evaluations of principals and four of the five scored
in the very effective range, rated a 3, in this area. One of them scored in the
somewhat effective range, rated with a 2, because although they stated that student
achievement data was part of the principal’s evaluation they stated that poor student
achievement data would not be a reason to change a principal’s assignment. Three
of the superintendents scored in the ineffective range, rated with a 1, in the area of
using data to evaluate teachers because they said they do not use student
achievement data or other types of data to evaluate their teachers. They gave the
reason as opposition from their teachers unions. One superintendent would evaluate
his teachers based on their use of data but not on the results of that data and that were
rated as somewhat effective. Another superintendent was rated as very effective
because they used student achievement data to evaluate their teachers’ effectiveness
and expected his teachers to use data to inform instruction.

108

Table 18:
Trends in Use of Data for Evaluations from Interviews with Superintendents from
the conceptual framework for effective data-driven decision making.
Actions or Skills
Superintendents
Average
A B C D E
Use of Standardized Test Data – California Standards
Tests, CELDT.
Yes
3
Yes
3
Yes
3
Yes
3
Yes
3
3
Use of Local Data/Benchmarks – benchmark, teacher
grades, surveys, etc.
Yes
2
Yes
2
Yes
3
Yes
1
Yes
3
2.2
Uses Multiple Measures to Make Decisions – uses
more than one type or source of data when making
decisions.
Yes
3
Yes
3
Yes
3
Yes
3
Yes
3
3
Uses Data as Part of Principal Evaluations – data
plays some part in their principals’ evaluations.
Yes
3
Yes
2
Yes
3
Yes
3
Yes
3
2.8
Uses Data as Part of Teacher Evaluations - data plays
some part in their teachers’ evaluations.
Yes
2
No
1
No
1
No
1
Yes
2
1.4
Believe Teachers Union is Obstacle to Use of Data –
the collective bargaining agreement or teachers
union forbids or restricts the use of data in one or
more ways.
No
3
Yes
1
Yes
1
Yes
1
Yes
1
1.4

Discussion of Interview Results of Research Question Four
For superintendents to be able to use data to effectively evaluate a person or
program, they must first know how to analyze data, know which data to use, and
know how to use data in the decision-making process. All of the superintendents
participating in the interviews provided information that demonstrated their ability to
do these things. They all stated that they used data to evaluate their principals and
other management personnel and stated that they were confident in their
effectiveness in doing this. However, only two stated that they use data in any way
when evaluating teachers and those two used it infrequently and only recently. The
other three did not use data for evaluation of teachers. So even though these

109

superintendents believed that data was important in evaluating teachers’ performance
and all felt that they had the ability to use data effectively for this purpose, none of
them did so more than occasionally. Superintendent D describe his transformation to
data-driven decision making:
“To be honest with you it [use of data in evaluations] didn’t used to be
important. I would say that we did not evaluate teachers or principals based
on student achievement. But I am changing because our world around us is
changing and I have this kind of comparison data and I know which are my
most efficient principals.”

Four of the five stated that the reason that they did not use data in the evaluation
of their teachers, regularly or at all, was due to their collective bargaining agreement
with their teachers unions. They reported that their collective bargaining agreements
prohibited the use of student achievement data in the evaluation of their teachers, but
all stated that they would like to change that. Apparently out of frustration, some
went as far as saying they would get rid of the collective bargaining agreement all
together if they could. One superintendent who was using data on a limited basis
believed that the collective bargaining agreement and California Education Code
permitted the use of data in evaluations if done correctly. The reason his district was
not using data on a regular basis was due to the training still needed for
administrators to use data effectively and correctly to evaluate their teachers. The
method of data use to evaluate teachers that this superintendent was implementing
was the same as the other superintendents, and the same method they were using to
evaluate principals. The teachers would set goals for themselves that were
measureable by student data and create a plan to improve student learning as

110

measured by that data. The evaluation would look at the results over a period of
years and not just one year. A bad evaluation of the teachers would only result if they
failed to use the data, respond to the data, or refused to use data. Superintendent A
described this process most succinctly:
“A teacher who consistently, year in and year out, when you looked at that
teachers CST results and compared it to the other teachers at that school
every single year his scores were at the bottom. You go in and found out why
is that. Are the kids not engaged? Does the teacher not use good practices?
You go through the whole litany of the reasons why. Then we work with that
person and work with that person. Finally we get that person out of there
because they are a detriment to kids.”

All five superintendents participating in the interviews believed that student
achievement data should be used in the evaluation of principals and teachers and all
identified the same or similar types of data that they would use. They differed on
how that data would be used and the process that they would implement, or have
implemented, in their data-based evaluations. Superintendent D stated that he would
first form a committee of teachers and administrators and agree on what types of data
would be used and how it would be used in the evaluation process. He felt that data
was important in evaluating teachers but it needed to be done in a fair way.
Summary

This chapter reviewed the findings from the superintendents’ survey responses,
the responses from personal interviews with five leading superintendents, and data
from various sources on their districts, personal backgrounds, and student
achievement levels. This chapter looked for commonalities, trends, and differences
in the ways superintendents of urban school districts use data in their decision

111

making, and how they learned those competencies needed to be effective in this
domain. When possible, the responses to the research questions were analyzed
through triangulation of the research literature, survey questionnaire findings, and
interview response results.
The results from these interview, surveys, and internet sources indicated the level
of competencies these participating superintendents currently possess and how they
acquired those competencies. The results also revealed what types of data they use,
how that data is used in their decision-making process, and what their expectations
are of others in their district in the use of data. Another finding was the identification
of the actions and policies that these superintendents employ to create a data-driven
decision making culture within their districts that are focused on student
achievement. The summary, conclusions, and implications of this study will be
presented in Chapter 5.

112

CHAPTER FIVE

DISCUSSION

The accountability movement in public education had its start in the United States
in the 1960’s. What schools and districts are being held accountable for and how
their progress is measured have changed several times from that time to now. Today,
public schools and districts must meet the requirements of the No Child Left Behind
Act of 2001. As the leaders of their districts, superintendents have had to meet the
increasing accountability required by NCLB and have found that they must now be
true instructional leaders if they are to be successful in their positions. Successful
superintendents have learned how to evaluate the level of achievement of the
students in their district. This level of achievement is not simply measured for the
entire district but by each sub-group and at each school. They are able to determine
which programs are effective, which teachers and principals are effective, and which
classroom practices are effective. To make these evaluations, the superintendents
must be able to use data effectively. As the level of accountability has increased for
superintendents, so has the level of data use by them. This study revealed what areas
of data use these superintendents were skilled in and where they learned those skills.
This study shows that current superintendents are still learning the vast majority of
their data using skills from on-the-job experiences rather than from formal training
or a university program.
Several studies have shown that successful superintendents are skilled in the use
of data and development of district wide assessments, among other actions. These
studies also revealed that assessing learner outcomes was one of their greatest

113

challenges, showing the importance of data-driven decision making and the
continuing challenge for superintendents to keep up with the increasing demands of
student accountability (Archer, 2005; Orr, 2006). One study in 2004 found little
evidence that accountability data were being used to change classroom practices
(Ingram et al., 2004), but this study showed that in the case of superintendents in
Southern California things have changed over the past six years.
Coburn and Talbert (2006) did research on the importance of using the right data
to determine the results that a person needs. They also discuss the fact that data
needs change by the position of the person and the type of problem that one is trying
to solve. Superintendents evaluating their principals might require different data than
teachers trying to determine the effectiveness of their classroom practices. The trust
level of data also changed related to the position of the person using it. The higher a
person’s position in the district, the more they trusted data. Superintendents trusted
student achievement data more than teachers did. The skill level of the person using
the data effects the trust level, with the more skilled data users trusting data more
than novice users. (Kerr et al., 2006) In this study superintendents unanimously
indicated that they referred to multiple sources and types of data in their decision
making process and rarely relied on just one type of data, but did trust data as a
reliable contributor of the information they were seeking.
One of the challenges superintendents face in their effective use of data is
promoting a data using culture within their districts. For data to make an impact on
classroom instruction, all of the educators in that district must use data to inform
their instructional practices (Luo, 2008). Superintendents in this study disclosed the

114

obstacles they face in creating this type of culture and how they have overcome
them, or attempted to overcome them. All of them spoke about the use of a
computerized data program, which they have provided to all of their teachers, that
allows them access to student data at any time. Luo (2008) reported in his study that
data that are difficult to access and understand is a major obstacle to creating a data
using culture in a school or district.
This study was conducted to discover which skills, knowledge, and practices the
23 participating superintendents possess or engage in that the literature said were
effective in making data-driven decisions that impacted classroom instruction both
directly and indirectly. The research questions were purposefully designed using the
literature to solicit those responses from them.
Using a mixed-methodologies approach consisting of a quantitative survey
questionnaire distributed to the 23 urban superintendents in California who led their
districts through at least the 2007-2008 to 2009-2010 school years, and qualitative
interviews with five of the successful urban superintendents in California, the current
study addressed four research questions:

1. What competencies do superintendents need to effectively use student and
staff data in making the decisions that will effect student achievement?
2. With many different sources and types of student data, what sources of data
do urban superintendents use and how do they determine the value of that
data?

115

3. What specific strategies or policies do urban superintendents use to increase
the acceptance of using student data by all educators in their district?
4. How do superintendents use data to determine the effectiveness of their
educators, administrators, and programs?
Significant Findings

The current study used a researched based survey and response to personal
interviews to collect pertinent data to answer the four research questions. The study
revealed several major findings about the competencies and knowledge
superintendents needed and possessed, the types of data that were valued and used,
and the strategies and practices used by the responding urban superintendents in
California to improve student achievement in their districts. The significant findings
of this study are reported below according to the four research questions.

The Competencies Superintendents Need and Apply Effectively
in the Use of Student and Staff Data in Making the Decisions
That Will Effect Student Achievement

A section of the research-based survey was developed by the researcher to
determine the specific competencies superintendents believe were necessary in their
position, which of these competencies they currently possessed, and how those
competencies were acquired by superintendents in California. The survey results
were triangulated with responses to the same type of questions from personal
interviews with 5 superintendents and from data listed on district and state websites.
The data from these 3 sources were analyzed to reveal trends and effective practices.

116

Competencies Required for
Effective Data-Driven Decision Making

In their responses during the personal interviews all five superintendents indicated
that they considered it necessary to be able to accurately use technical skills to
analyze student data. This competency was identified by them as the ability to look
at student data and see trends and changes in student achievement by school site,
teacher, or program. It also meant the ability to disaggregate data into sub-groups of
students in order to tell if one group of students was achieving at a higher level than
another. Another data analysis ability identified by them was being able to determine
the correlation between programs, practices, and resources and student achievement
data.
The majority of superintendents identified the ability to originate data-based
questions in a manner that would use data to formulate the answer as a necessary
skill. This inquiry method promotes the use of data in making decisions rather than
relying on past practice or anecdotal information to do so (Kerr et al., 2006)
According to these superintendents, engaging in an inquiry mode of data use is
something that not only all superintendents should engage in but also their principals
and teachers.
The ability to enable others to use data, communicate using data, and use data to
evaluate principals and programs, were also skills that all of the superintendents
identified as being important to various degrees. Enabling others to use data
included providing professional development in the use of data, providing an easy-
to-use computerized data program, and allowing school sites to make their own

117

decisions based on data. Communicating using data meant using data when reporting
district progress, decisions, and programs to the board of education, teachers, or the
community. This also meant engaging in dialog with educators, the board of
education, and parents using data and stating a clear expectation of data use by all in
the district. Evaluating principals and programs was revealed as meaning using
multiple sources of student data to evaluate the progress of a principal or program
over a period of time, even several years. All three of these abilities were critical in
creating a data-using culture in their districts and evaluating the progress the students
are making according to state and federal accountability measures (Luo, 2008;
Moore et al., 2005).
Although these superintendents felt that the ability to use a computerized data
system was valuable, they did not feel it was necessary for their position. They stated
that they as superintendents relied on others at the district level to provide the data
they needed based on the questions they asked. They reported that it was more
important to be able to ask the right questions. However, they all believe that the
ability to use a computerized data system was necessary for their principals and
teachers to be effective.
Effective Data-Driven Decision Making Competencies
Possessed by Superintendents in the Study
Using the results from the 23 superintendent surveys and the responses from the
five superintendents participating in the personal interviews, the levels of data-driven
decision making competencies were determined for these California urban
superintendents. The superintendents in the survey rated their skills in one of the

118

following five levels: Novice - still learning most components of this aspect of data
use; Developing - knowledgeable of many components of this aspect of data use, but
not all; Proficient - knowledgeable of most components of this aspect of data use;
Skilled - knowledgeable of all components of this aspect of data use; or Expert -
could train others on this aspect of data use.
The majority of participating superintendents rated themselves as Skilled or
Expert in all skill areas of data use except one. Superintendents rated themselves as
Skilled or Expert in their ability to analyze and disaggregate student data, with none
reporting that they had little or no skills in these areas. The use of data to evaluate
principals, teachers, and programs had the superintendents reporting that they were
Skilled or Expert in this area and no one reporting that they were Novice or
Developing in these skills. The skill area in which the most superintendents felt they
were most competent was the use of data to communicate the districts goals and
objectives to others, reporting that they were Skilled or Expert in this skill area. The
one they felt the least competent was in the ability to use a computer data system.
Male superintendents rated themselves slightly more competent than their female
counterparts in using data in three of the areas: their ability to analyze student data,
using data to communicate, and using data to evaluate principals or teachers, and
using data to determine a correlation between a district program and student
achievement. The females rated themselves slightly more competent than the male
superintendents in two areas: their ability to use a computerized data program and
disaggregating student data by sub-groups. The difference between the male and
female superintendents self-rated competency levels never exceeded 0.2 points on

119

the 1 through 5 scale, except in the two areas: the use of data to communicate and the
use of a computerized data program. Female superintendents rated themselves
slightly higher in using a computerized data system than did the male
superintendents. The male superintendents rated themselves slightly higher in their
ability to use data to communicate.
The superintendents participating in the personal interviews believed themselves
to be competent in all areas of data-driven decision making. They reported that they
knew how to analyze data, disaggregate data, communicate using data, use data to
evaluate their principals and programs, and to evaluate teachers when possible. As
with the survey results, these superintendents felt they had the competencies needed
for their position to use a computerized data system, but did not state that they felt
they were an expert in this area. One skill that three of the five identified that was not
part of the survey questions, but they believed they were competent in, was
developing data based questioning, for themselves and in others. They felt that they
knew how to ask the right kind of questions that would create data-driven decisions.
How Effective Data-Driven Decision
Making Competencies Were Acquired
The superintendents responding to the survey reported where they acquired their
levels of competencies. They indicated whether they learned their skills and
knowledge in the areas of data use from a formal university program or from on-the-
job experiences and training by rating the level of competency they reached in each
skill area.

120

In all skill areas of data use, superintendents reported that they acquired most of
their skills and knowledge from on-the-job experiences. The skills needed in using
data to communicate with others was the area that they reported they learned the
most from on-the-job experiences, but skills used in evaluating teachers and
principals, and determining the correlation between district programs and student
achievement were also identified as skill areas that were mostly acquired on the job.
There was one area with little difference between the level of knowledge and skills
learned from a university program and from on-the-job experiences, and that was the
ability to analyze student data. The superintendents reported that they learned nearly
the same level of skills from their university programs as they did from their job
experiences in this area.
The superintendents who participated in the personal interviews gave responses
that supported the findings from the surveys. All of these superintendents reported
that their university programs taught them how to analyze data and even
disaggregate data by sub-groups to some degree, but said that they learned the
majority of their data use skills from on-the-job experiences. They identified skills
learned in various positions they held prior to becoming a superintendent or people
they worked with during their careers as the main source of their knowledge and
skills. Some superintendents even named specific people or events that were major
contributors to their competency levels. Several of these superintendents described
the changes they have experienced in the use of data during their careers, saying that
the necessity for possessing data-using skills is much greater today than when they
first became an educational leader.

121

What Sources of Data Superintendents Use and
How They Determine the Value of That Data

This section of the study was developed to determine the types of data that
superintendents used and valued and how they determined which data to use for the
various types of decisions they need to make in their jobs. The results for this section
of the study came from responses on the survey that the 23 participating
superintendents provided and from responses to questions from personal interviews
with five superintendents.
Types of Data that Superintendents Use

The survey asked them to identify which types of data they used when making
various types of decisions. They could choose from the following types of data: CST,
benchmark, demographic, and program, or write in any type they used that was not
listed. They could choose more than one type of data and usually did. The
superintendents’ responses showed that they used multiple types of data in nearly all
of the decisions that they make.
California Standards Test (CST)
CST data was the most used data, used by the majority of the superintendents in
all of their data-driven decisions. All of the superintendents used CST data in
determining their district’s annual focus or goals, and most of them used it in making
decisions about district wide intervention programs. The five superintendents from
the interviews also reported that they used CST data, specifically API and AYP data,
in most of their data-driven decisions.

122

They reported that CST data provided them a broad look at how their schools,
students, and staff were doing and where they needed to look more closely. They
would then use other data to make that closer inspection and make more detailed
decisions. CST data was the most used in communicating the district’s visions and
goals, expressing expectations of data use, including principals’ evaluations.
Benchmark and Program Data
The two types of data that allowed them to take this more detailed look at their
students’ and schools’ progress were benchmark and program data. Benchmark data
and program data were also used by the majority of the superintendents in making
decisions, communicating with others, and evaluating principals and programs. The
superintendents participating in the interviews all stated that they valued benchmark
data, but two were not using it. Those two did not use it because their districts were
still developing the assessments that would generate the data. The other three
superintendents had developed benchmarks and were currently using the data from
them in their decisions. The superintendents that were using benchmark data stated
that they valued it above CST data in evaluating teacher performance.
Demographic Data
Demographic data was also used by all of the superintendents in various
decisions they made. The survey results indicated that the most likely area that a
superintendent used this type of data was when making decisions about intervention
programs. Demographic data, according to several responses from the
superintendents interviewed, ensures that those students needing the most help are
getting that help. Demographic data tells the superintendents who should be targeted,

123

who is being targeted, and what progress each group is making. Demographic data
was used the least in the evaluation of teachers and principals, but was still used by
nearly half of the superintendents in this decision-making area.
Three of the superintendents identified customer service survey results as data
they used in some decisions. Evaluation of principals and even the superintendent
was one area in which this type of data was used. The other was the effectiveness of
district and school site initiatives or goals.
The superintendents’ responses from the interviews revealed that they all used
CST data to take a big picture look at their district and schools, which often allowed
them to notice trends, areas showing good progress, and areas showing a lack of
progress. When areas of concern were discovered, then the superintendents would
look at benchmark and program data to determine the extent and cause of these
conditions. Superintendents would expect their principals to use CST, benchmark,
and demographic data to develop a plan to address the areas of need that were found,
and program data to determine the effectiveness of their plan.

The Specific Strategies and Policies Superintendents Use to Increase the Acceptance
of Using Student Data by Educators in Their District

The results from this section of the study revealed what actions, policies, and
resources the participating superintendents set in motion to promote the use of data
by their principals and teachers and to create a data-using culture in their districts.
Responses to survey and interview questions were analyzed to determine their
practices and the extent to which they engaged in these practices. These actions,

124

policies, and resources were identified as the following: providing an up-to-date
computerized data program, providing time for collaboration, providing professional
development in the use of data, conveying clear expectations of data use, providing
data and opportunities to use data, and promoting the use of a data-based inquiry
method.
Acquisition of a Computerized
Data Program and Technology
All of the superintendents participating in the study indicated that they believe an
effective, easy-to-use computerized data program was important. Nearly all of the
superintendents responded that they had provided one to the educators in their
district. The five superintendents participating in the personal interviews, also all
stated that they believed an up-to-date and effective computerized data program is
important in creating a data-using culture in their districts. All but one had recently
acquired and implemented a data program called Data Director, which provides
multiple sources of student data via an internet based data system. This system
allows teachers and principals to access the data from any computer connected to the
Internet any time they want. This program also is equipped with features that assist
the user in creating data reports. The fifth superintendent had recently purchased the
Data Director program, but had yet to implement it in her district at the time of this
study. All of the superintendents discussed the expectations they had for this
program and their teachers’ and principals’ use of it.

125

Providing Professional Development
in the Use of Data

The results of the study indicated that the superintendents in this study believe
that providing training for their teachers and principals is important.
The survey revealed that nearly all of the superintendents frequently or occasionally
provided this type of professional development to their staff. The superintendents in
the interviews discussed the professional development that they provided to the staff
of their districts. They identified the training connected to their new computerized
data programs as a type that had been recently provided. Over half of the
superintendents talked about professional learning communities at many of the
schools in their districts. This is a type of professional development in which
teachers work collaboratively to learn from one another and determine solutions to
achievement gaps in their schools. Teachers must be trained the process of working
collaboratively with data but then the learning is generated from the group. Several
of the superintendents also provided experts in the use of data and data-related
technology to provide training to individuals and school sites.
Communicating Clear Expectations
of Data Use by Educators
All of the superintendents responding to the surveys responded that they regularly
communicate clear expectations for data use by the educators in their districts. The
superintendents in the interviews explained that this communication came in several
forms, examples are the goals set between teachers and their principals, and between
principals and the assistant superintendents. Policies and objectives were created by

126

the board of education that focused on the use of data or had data use as a
component. Principals and, in several districts, teachers knew that their evaluations
would be based, in part, on their ability to effectively use student data.
Promoting the Use of Data and
Providing Opportunities to Use Data
In addition to the actions and policies discussed above, nearly all of the
superintendents in the survey frequently or occasionally provided time and
opportunities for their teachers and principals to use data. The five superintendents
from the interviews also reported that they provided as much time as they could to
their school sites for the analysis of data and collaboration around their findings. All
five were working actively to promote the use of student data by their teachers and
principals and engaging them in data-driven decision making at the school site level.
One superintendent reported that he was still negotiating with the teachers union to
establish collaboration time for teachers to look at student data and classroom
practices. The teachers union was opposed to setting time aside for this purpose, but
the superintendent felt that it was critical to moving his district forward.
How Superintendents Use Data to Determine the
Effectiveness of Their Educators, Administrators, and Programs

Several questions on the survey and during the interview addressed the types of
data superintendents used to evaluate their principals and teachers. These questions
also inquired about the method they used to evaluate their personnel using data.

127

Many superintendents responded that they were restricted by their collective
bargaining agreements on the types of evaluations that they could make of their
teachers and, in some cases, their administrators.
Use of Data to Evaluate Principals
The results from the surveys revealed that all respondents believed they were at
least Proficient in the use of data to evaluate their principals, with most reporting that
they were Skilled or Expert. All five of the superintendents in the personal
interviews felt they were Skilled in using data this way. All of them stated that they
acquired the majority of these skills from on-the-job experiences rather than their
university programs. Both sets of superintendents reported that they did actively use
student data as a component of their evaluations of principals. Several of the
superintendents from the interviews described the process in the following way. The
superintendent would review school site data for areas of student achievement
growth and areas where growth was slow or not occurring. He or she, or one of their
assistant superintendents, would then meet with the principal to discuss their plan to
improve student achievement in this area. This discussion would be data driven and
with measurable results. They reported that the evaluation would be based on the
principal’s ability to develop this plan using data and executing it. They said that low
student achievement data would not automatically trigger a poor performance
evaluation for a principal, but would create an area of focus that the principal was
expected to address.

128

Use of Data to Evaluate Teachers
Nearly all of the responding superintendents mentioned in some way the
challenges in using student data to evaluate teachers. When asked what types of data
they use in the evaluation of their teachers, the majority identified CST data, and the
same number identified both benchmark and program data, but several also wrote in
the margin of the survey form that their collective bargaining agreement kept them
from using it. All of the superintendents from the interviews stated that they would
have their principals use data in a similar manner as they used it with their principals,
but with a greater focus on the benchmark data. Three of the superintendents stated
that the principals did not use student data as part of their teachers’ evaluation
process because of restrictions on its use in their collective bargaining agreements.
Of these three, one superintendent reported that he did not use student data to
evaluate teachers, but did expect his principals to use it when determining teacher
assignments. Of the two that reported that they did use student achievement data in
teacher evaluations, one stated that his district only used student data in a teacher’s
evaluation if the teacher agreed to it. The other described his expectations of his
principals in the use of student data and teacher evaluations as similar to the process
used for his principals. A principal would look at student data to discover teachers
who appeared to be producing lower student achievement results than their
colleagues. The principal would then discuss and develop a plan with the teacher that
would use student data to determine the effectiveness of this plan. If the teacher was
not willing to participate in this process or could not implement the plan that was
developed, then they would receive a poor performance evaluation.

129

This superintendent also stated that any principal that was not willing to engage in
this process with their teachers would also be evaluated poorly.
Use of Data to Evaluate Programs
Results from the surveys indicated that the majority of superintendents rated
themselves as Skilled or Expert in the use of data to evaluate programs and that the
majority of them learned the competencies in this area from on-the-job training
rather than their university programs. The superintendents from the interviews all
stated that they also achieved their skill level from job experiences and that they felt
they were skilled in this area.
In evaluating programs, all of the superintendents used the data generated by, or
associated with, the program in determining its effectiveness. The majority also
relied on CST and benchmark data in the programs evaluation. Demographic data
was also used by most of the superintendents. The superintendents participating in
the interviews described how they evaluated their district wide programs. They used
demographic data in conjunction with CST and benchmark data to determine which
students were achieving at acceptable levels and which needed assistance. The type
of program that would be offered would depend on this data. For example, if the data
showed that 8
th
grade Hispanic students were underperforming in Algebra, then an
Algebra intervention program that targeted Hispanic 7
th
and/or 8
th
graders would be
designed and implemented. Benchmark, CST, and program data would then be used
to determine the effectiveness of the program. Programs that were found to be
ineffective would be re-designed or replaced with another program. Programs that

130

were found to be effective would be expanded as needed. All of the superintendents
in the interviews described a similar process.
Effective Data-Driven Decision Making Competencies
The results from this study supported the findings discussed in the review of the
literature in Chapter 2 regarding the changes taking place in educational
accountability and the impact this is having on data-driven decision making (Marks
& Nance, 2007; Sherman, 2008; Wolf, 2006). Superintendents, principals, and even
teachers are continually being held more accountable for progress in the academic
achievement of their students. This study found three data-driven decision making
competencies that are critical for superintendents to be skilled in performing. One is
the ability to communicate clearly their expectations of data use by their principals
and teachers. The other is the ability to analyze data effectively. The third is the
ability to use data effectively in communicating with their principals, teachers, the
community, and the board of education. The other skills were also felt to be
important, but these were the ones that determined the effectiveness of a
superintendent.
Superintendents must be able to communicate clear expectations of data-driven
decision making to all of their educators in order for data to have a real impact on the
classroom. Teachers and principals must engage in effective data-driven decision
making that drives the instruction in the classroom and focuses resources and
interventions when they are needed. Superintendents communicate this expectation
through policy, talks with staff, and the professional development that is provided.

131

Their most effective way of communicating this expectation was by making data-
driven decision making part of the principals’, and in some cases the teachers’,
evaluations. This was a clear way to let principals and teachers know that they were
expected to use data in the decisions they made about students.
Superintendents needed to be skilled in the analysis of data as well. Even
though they frequently relied on others to provide data and data analysis for them,
the ability to analyze the data themselves was important. They needed to be able to
look at data from their school sites or from sub-groups of students and determine
which schools or sub-groups of students were progressing adequately and which
were not. It was important that they could deduce the possible reasons for this lack of
progress and, in discussion with the principal or other administrator, determine a
valid solution to this achievement gap based on the data.
A large part of a superintendent’s responsibilities is communication. They are
constantly talking with their staff, principals, the public, teachers, and the board of
education. The ability of the superintendent to use data effectively in these
conversations was critical. Superintendents who used data to provide the foundation
for the decision they were making, or used data to report the progress of the district
or schools found themselves to be more effective in this area. The use of data in their
communications reduced the amount of resistance to their decisions and reduced any
feelings of arbitrary decision making on their part in the eyes of their teachers or
parents.

132

All of these competencies were learned through on-the-job experiences rather
than through a formal university program. The ability to analyze data was the only
competency that any of the superintendents reported learning to some degree through
their university program. They reported that they learned these competencies from
their experiences in different positions as they moved up through the ranks and from
various people that they worked with in these positions. They stated that each
positional level, including the superintendent position, had its own data-using
challenges and skills that they had to learn.
Practices Superintendents Use in Effective
Data-Driven Decision Making
Several practices that superintendents engaged in were identified as crucial in
making effective data-driven decisions. These were the use of multiple sources of
data, looking at data over a period of time and determining trends, and creating a
data-using culture in their districts.
Superintendents placed different value on the various types of data available to
them. Some data was preferred in certain types of decision but not used, or was less
valuable, in other types of decisions. The use of multiple types and sources of data in
making all important decisions was an effective practice described by Patton (2002).
When making decisions about student achievement, data related to student
achievement would be used but also data regarding the effectiveness of intervention
programs, demographic data, and financial data might also be used. This
combination of data sources enables the superintendents to determine both what
needs to be done and how it should be done.

133

Another important effective practice that superintendents engaged in was using
data to look at trends over time. Superintendents look at student achievement data
over several years before making major decisions about a program, principal, or
teacher. Data indicating lower achievement during just one school year would not
automatically trigger a decision to make changes by the superintendent. It would
prompt a closer look and the initiation of a conversation about what needs to be
done, but only after a longer period of time with less than desirable student
achievement would the superintendent make a decision to change the program or
remove the principal or teacher.
Engaging in practices that create a data-using culture in their districts was critical
in making changes in the classrooms throughout the district. Setting expectations of
data use, providing training in the use of data and computerized data systems,
providing time to analyze data and collaborate with colleagues regarding the data,
and providing an easy-to-use and access computerized data system were all practices
that superintendents engaged in that created a data using culture in their districts. An
effective practice that some of the superintendents engaged in was the propagation of
professional learning communities (PLCs) in their school sites. PLCs are groups of
teachers collaborating together around student data to make decisions about
classroom practices and resources. This practice is one of the effective methods used
by superintendents to promote the use of student data in increasing achievement and
providing time for teachers to use and collaborate around data.

134

Effective Use of Data in the Evaluation of Educators
Surprisingly, the effective way of using data to evaluate principals, and teachers,
was not to look at student achievement data each year and determine who the poor
performing principals were and who were the high performing ones. Instead, the
effective way for using student achievement data in evaluating principals was to
examine the data to discover trends and areas where there were achievement gaps.
Then discuss with the principal their plan to address the achievement gap(s) using
data for measurable outcomes. The evaluation of the principal comes from the
superintendents’ ability to use data effective to design this plan and implement it,
and respond to the outcomes. A poor performance evaluation would come from the
principal’s inability or willingness to use data in addressing this achievement gap
and their inability to effectively implement the plan. A similar process would be
followed for teachers addressing gaps in their students’ learning and their teaching
practices.
Implications for Practice

This study used mixed methodology of a quantitative survey and qualitative
personal interviews that gathered information from 23 superintendents who were
currently leading urban school districts in the southern portion of California.
Triangulating this data with the current research in the literature, the researcher
formulated the results that are presented. There were several significant findings
from this study that could have an impact on the effectiveness of a superintendent.

135

Formal Program for the Acquisition of
Data-Driven Decision Making Skills
The competencies needed by superintendents to effectively use data in the
decisions they make are largely acquired through on-the-job experiences. This
creates two dilemmas for aspiring superintendents. How can they ensure that the
experiences they encounter in lower positions will prepare them to be effective data
users at the superintendent level? And how can those who aspire to be a
superintendent but do not come with the traditional background in education gain
access to these experiences? Training in effective data-driven decision making at the
superintendent level should be included in university programs, Ed.D. programs, or
through the Superintendents Academy.
Changing Skill Requirements in the
Use of Data
As the accountability movement continues, and will continue for the foreseeable
future, data use by superintendents will also need to continue. Several of the
superintendents participating in this study discussed the fact that the necessity to use
data had changed significantly from the time they first became an administrator to
today. This change will continue as superintendents use data to meet the challenges
of increasing expectations for student achievement. Superintendents will need to
ensure that all educators in their districts are using data as effectively as possible.
Knowing how to use data themselves will be only one of the increasing skills but
ensuring their teachers and principals are using data effectively will be a skill they
also need.

136

Using Data to Evaluate Teachers
and Principals
In the past principals were evaluated on how well the parents liked them and how
well they managed their schools. Student achievement was not closely watched. This
changed with the passage of the NCLB legislature and since then principals, like
superintendents, have been held increasingly accountable for the achievement levels
of their students. Superintendents now need to be able to evaluate their principals
based on their ability to improve student achievement and meet the accountability
measures set by the state. They must be able to use an effective process for doing this
in order to ensure that each school site and principal is meeting the necessary
achievement levels. The next step in this process, which some superintendents have
already begun, is the evaluation of teachers based on student achievement. As
superintendents and principals are held accountable for student achievement, so will
teachers, and how this will be done will require a set of skills that superintendents
will need to develop.
Implications for Research
The findings of this research study made several contributions to the research
base regarding the knowledge, skills, and practices that superintendents possess to
engage in effective data-driven decision making.

137

Prior to this study there were research studies that provided knowledge as to what
skills and knowledge were required for effective data-driven decision making, but no
studies that looked at which of those skills and knowledge current superintendents
possessed, and how effectively they used those competencies in making decisions
about student achievement.
Although this study has made important contributions to the knowledge base on
the competencies and practices of urban superintendents in data-driven decision
making, several areas warrant further study. First, with the many types of data that
can be used to evaluate student achievement, which one can be used to show the
most direct correlation between the teaching practices of the teacher and the
students’ achievement and how can this be effectively and fairly used in the
evaluation of the teacher.
Another area for further research comes from the fact that this study looked at the
competencies and practices of urban superintendents in the southern section of
California and these superintendents were all considered traditional superintendents,
meaning they were promoted to their position from other educational leadership
positions within the public education system. An area for further research would be
to look at the competencies levels and practices of non-traditional superintendents in
the use of data in their decision making regarding student achievement.

138

REFERENCES

Acker, K. (2007). Data Driven Decision Making: From the Superintendent to the
Classroom. Wilmington College, DE. DAI, 68, no.01A, 31

Alsbury, T. L., & Whitaker, K. S. (2007). Superintendent Perspectives and Practice
of Accountability, Democratic Voice and Social Justice. Journal of
Educational Administration. 45(2), 154-174

American Association of School Administrators, (1993). Professional Standards for
the Superintendency, Arlington VA

American Association of School Administrators, (2002). Using Data to Improve
Schools: What’s Working., Arlington VA

Archer, J. (2005). Theory of action: the idea that schools can improve on their own
gives way to a focus on effective school leadership, Education Week,
September 14, S3-S5.

Bredeson, P. V. & Kose, B. W. (2007). Responding to the education reform agenda:
A study of school superintendents’ instructional leadership. Education Policy
Analysis Archives, 15(5). Retrieved [May 2009] from
http://epaa.asu.edu/epaa/v15n5/.

Cibulka, J. G. (2000). Organizational learning in educational policy systems. In K.
Leithwood (Ed.), Understanding schools as intelligent systems (pp. 185-210).
Greenwich, CT: JAI.

Coburn, C. E.& Talbert, J. E. (2006). Conception of Evidence Use in School
Districts: Mapping the Terrain. American Journal of Education. 112 August,
469-495.

Cromey, A., van der Ploeg, A., & Masini, B. (2000). The Call for Data-Driven
Decision Making in the Midwest’s Schools: NCREL’s Response, NCREL,
Oak Brook, IL.

Glass, T., Bjork, L., & Brunner, C. (2000). The Study of the American School
Superintendency. A Look at the Superintendent of Education in the New
Millennium. American Association of School Administrators, Arlington, VA

Halverson, R. (2003, October 10). Systems of practice: How leaders use artifacts to
create professional community in schools. Education Policy Analysis
Archives, 11(37). http://epaa.asu.edu/epaa/v11n37

139

Honig, M. I., & Coburn, C. (2008). Evidence-Based Decision Making in School
District Central Offices: Toward a Policy and Research Agenda. Educational
Policy. 22(4), 578-608.

Ingram, D., Louis, K. S., & Schroeder, R. G. (2004). Accountability Policies and
Teacher Decision Making: Barriers to the Use of Data to Improve Practice.
Teachers College Record, 106, 6, 1258–1287

Kerr, K. A., Marsh, J. A., Ikemoto, G. S., Darilek, H., & Barney, H. (2006).
Strategies to Promote Data Use for Instructional Improvement: Actions,
Outcomes, and Lessons from Three Urban Districts. American Journal of
Education, 112, p146.

Kroeger, M., Blaser, S., Raack, L., Cooper, E., & Kinder, A. (2000). How schools
use data to help students learn. NCREL’s Learning Point; 2(2), 13-15.

Lashaway, L., (2002). Data Analysis for School Improvement. National Association
of Elementary School Principals: Research Roundup. 19(2).

Leithwood, K., & Jantzi, D. (2008). Linking Leadership to Student Learning: The
Contributions of Leader Efficacy. Educational Administration Quarterly.
44(4), 496-528.

Luo, M. (2008). Structural Equation Modeling for High School Principals' Data-
Driven Decision Making: An Analysis of Information Use Environments.
Educational Administration Quarterly. 44(5), 603-634.

Marks, H. M., & Nance, J. P. (2007). Contexts of Accountability under Systemic
Reform: Implications for Principal Influence on Instruction and Supervision.
Educational Administration Quarterly. 43(1), 3-37.
Moore, A. D., Dexter, R. R., Berube, W. G., & Beck, C. H. (2005). Student
Assessment: What Do Superintendents Need to Know? Planning and
Changing, 36(1), 68-89.

Orr, M. T. (2006). Learning the Superintendency: Socialization, Negotiation, and
Determination. Teachers College Record, 108(7), 1362–1403.

Pan, D., Rudo, Z., & Smith-Hanson, L. (2002, November). Resource Allocation and
Student Performance Improvements in 12 Districts in the Southwest Region.
Paper presented at the annual meeting of the Mid-South Educational
Research Association, Chattanooga, TN.

Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.).
Thousand Oaks, CA: Sage.

140

Reichardt, R. (2000). The State’s Role in Supporting Data-Driven Decision Making:
A View of Wyoming. MCREL, Aurora, CO

Sherman, W. H. (2008). No Child Left Behind: A Legislative Catalyst for
Superintendent Action to Eliminate Test-Score Gaps? Educational Policy.
22(5), 675-704.

Virginia Department of Education, (2000). Guidelines for uniform performance
standards and evaluation criteria for teachers, administrators, and
superintendents. Division of Teacher Education and Licensure, Richmond,
VA.

Walpole, S., Justice, L., & Invernizzi, M. (2004). Closing the gap between research
and practice: Case study of school-wide literacy reform. Reading and Writing
Quarterly, 20(3) 261-283.

Wayman, Jeffery C., Stringfield, Sam; (2006) Technology-Supported Involvement of
Entire Faculties in Examination of Student Data for Instructional
Improvement. American Journal of Education, 112

West Ed, (2002). Improving Districts: Systems that Support Learning. Office of
Educational Research and Improvement, Washington D.C.

Wolf, M. (2006). Making Data Relevant: New Information Systems Are Helping
Boost Student Learning, Technological Horizons in Education Journal,
33(12), 32.

141

APPENDIX A

Interview Questions

1. a. What competencies (skills, knowledge, experiences) do you feel you need
or needed to effectively use student and staff data in making the decisions
that will effect student achievement?
b. How did you acquire these competencies? If through a university program
or other educational program what was the name/type of program?
c. Do you feel that you had the competencies needed for effective data-driven
decision making when you first started in the position as superintendent?

2. a. With many different sources and types of student data, what sources of
data do you use and how do you determine the value of that data?
b. What type of data do you ensure is available to teachers and principals that
you feel will aide them in making data-driven decisions?
c. Does your district have an existing data program/hardware to make data
use more effective and accessible for teachers, and if so what specific
attribute(s) of the program do you find important? Have you initiated a new
program or improved on the existing one, and if so how?

3. a. What specific strategies or policies do you use to increase the acceptance
of using student data by all educators in your district?
b. What obstacles to data use by teachers are present in your district and how
are you/did you overcome that obstacle?
c. What type(s) of professional development have you encouraged or
mandated for your educators related to effective data-driven decision
making?
e. In what ways do you use data to communicate with your Board of
Education?

4. a. What type of data do you use and how much does it weigh-in on your
evaluation of principal’s performance?
b. Do you require your principals/administrators to use student achievement
data when evaluating teachers or other educators?
c. How do you use data in the evaluation process? How do you ask your
administrators to use data when evaluating teachers?

142

APPENDIX B

Data- Driven Decision Making Survey Questions
Please check the box or boxes next to each question for the column that best matches
your response.
Indicate the level of proficiency you feel you currently have in the following
skills related to data use:
Novice = still learning most components of this aspect of data use. Developing = knowledgeable of many components of this
aspect of data use, but not all. Proficient = knowledgeable of most components of this aspect of data use. Skilled =
knowledgeable of all components of this aspect of data use. Expert = could train others on this aspect of data use.

Novice Developing Proficient Skilled Expert
1. Analyzing raw student achievement data
2. Disaggregating student data by sub-groups
3. Use of data warehousing computer software/hardware
4. Use of data to evaluate teacher/principal performance.
5. Ability to use data to communicate to parents/community
6. Use of data to determine a correlation between a district
program and changes in student achievement.

Indicate the level of proficiency you feel you have gained through formal
training, (university programs, Superintendents Academy, etc.) in the following
skills related to data use:
Novice Developing Proficient Skilled Expert
7. Analyzing raw student achievement data
8. Disaggregating student data by sub-groups
9. Use of data warehousing computer software/hardware
10. Use of data to evaluate teacher/principal performance.
11. Ability to use data to communicate to parents/community
12. Use of data to determine a correlation between a district
program and changes in student achievement.

Indicate the level of proficiency you feel you have gained through “on the job”
training, collaboration, or through self-teaching in the following skills related to
data use:
Novice Developing Proficient Skilled Expert
13. Analyzing raw student achievement data
14. Disaggregating student data by sub-groups
15. Use of data warehousing computer software/hardware
16. Use of data to evaluate teacher/principal performance.
17. Ability to communicate student data to parents/community
18. Use of data to determine a correlation between a district
program and changes in student achievement.

Indicate the types of data (you may choose more than one) that you use when
making the following decisions:
District tests/ Program Demo-
CST Benchmarks Data graphic Other
19. District level staff assignments ___
20. School site staff assignments ___
21. Budget allocations ___
22. Annual Focus or Priority ___
23. Evaluating teacher/principal performance ___
24. Purchase of programs, textbooks, consultants, etc. ___

143

APPENDIX B - continued

25. Types of interventions/remedial programs and materials ___
to be used district wide.

Indicate the types of data (you may choose more than one) that you use when
communicating the following with district and school site staff or the board of
education.
District tests/ Program Demo-
CST Benchmark Data graphic Other
26. District goals and objectives ___
27. District initiatives ___
28. Budget decisions ___
29. Student achievement ___
30. Challenges the district faces ___
31. Beginning /ending a program ___

Indicate the types of data (you may choose more than one) that you expect
district and school site staff to use when they are making the following
decisions.
District tests/ Program Demo-
CST Benchmark Data graphic Other

32. Types of interventions to implement ___
33. Placement / programming of students ___
34. Budget decisions ___
35. Hiring/placement of teachers/staff ___
36. Classroom instructional practices __
37. Evaluation of teachers/staff ___

What methods do you use to promote and support the use of data-driven
decision making in your schools.
How often do you engage in the following actions:
Never = have not done this. Sometimes = once or twice a year. Occasionally = more than twice a year but not at every
opportunity. Frequently = every opportunity.
Never Sometimes Occasionally Frequently
38. Up to date and effective data storage and analysis program
39. Time for teachers to analyze data and collaborate.
40. Professional development on data use.
41. Clear expectations of data use communicated.
42. Provides opportunities for teachers/principals to create useful data.

43. Data used/provided to address identified problems or focus areas.

44. Communicates clear link between data, student achievement, and
actions/decisions that need to take place

45. Uses inquiry method when communicating and working with staff to solve problems.

---------------------------------------------------------------------------------------------------------------------------
If you would be willing to participate in a short personal interview, either in person or by telephone,
to expand on the information you have provided above please indicate your availability below.
Yes No
I would be willing to participate in a 5 to 10 minute telephone interview.
I would be willing to participate in a 15-20 minute personal interview.

144

APPENDIX C

Research Introduction Letter

November 14, 2009

Dear Superintendent________,

I am currently conducting research regarding superintendents of urban school
districts and their use of student achievement data in decision making. It is expected
that the results of this study will provide guidance and insights in to effective data-
driven decision making by superintendents that could lead to increased student
achievement. The purpose of the study is to determine the knowledge, methods, and
skills that superintendents utilize to analyze data and make decisions based on that
analysis. I would like to invite you to participate in this study.

A survey is attached which is a measure of personal perception as to the skills and
methods you utilize for using data and the role that student data plays in instructional
practices within your district. There are no right or wrong answers. You are asked to
respond to each statement and choose the response that best fits your current opinion.
This survey should only take you 10 to 15 minutes to complete.

The focus of this study is not on the answers of individual superintendents
responding to this survey, but on the trends, which are evident through the analysis
of the responses of the entire group. For follow-up purposes only, I have added a
“check” box to contact those superintendents who select to continue with a 5-minute
telephone call or a 45-minute face-to-face interview at your convenience. You have
my assurance your responses will not be utilized in any manner other than to provide
confidential data for my study. Your participation in this study is completely
voluntary. I anticipate analyzing the data received early in November and would
sincerely appreciate your completing this survey as soon as possible. If possible,
please return this survey by September 30, 2009.

You may forward your responses by returning them in the self-addressed,
stamped envelope accompanying this letter. Should you have any questions relative
to this survey, please call me at (626) 394-0742 or contact through email at
lroot@usc.edu .

Sincerely,

Lonny Root
USC Doctoral Candidate

145

APPENDIX D

University of Southern California
Rossier School of Education

INFORMATION SHEET FOR NON-MEDICAL RESEARCH

SUPERINTENDENTS AND DATA-DRIVEN DECISION MAKING: HOW
URBAN SCHOOL SUPERINTENDENTS EFFECTIVELY USE DATA-DRIVEN
DECISION MAKING TO IMPROVE STUDENT ACHIEVEMENT

You are invited to participate in a research study conducted by Lonny Root and Dr.
Rudy Castruita at the University of Southern California because you are an urban
school superintendent in California. This research study will be the basis for a
dissertation done in completion of the Ed.D. program. A total of 29 subjects will be
selected from all urban superintendents in California to participate. Your
participation is voluntary. You should read the information below, and ask questions
about anything you do not understand, before deciding whether to participate.

PURPOSE OF THE STUDY
The purpose of this study is to examine how superintendents of urban school districts
use data to make decisions, especially those related to student achievement and
considered part of the NCLB accountability criteria. How they select sources of data
and determine them as reliable and useful from all of the data available to them, and
if the use of technology increased the usefulness of that data. What policies do they
put in place and what resources do they supply their educators to increase the
effective use of data among all administrators and teachers.

PROCEDURES
If you volunteer to participate in this study, you will be asked to complete a survey
that consists of items which requests that you rate the importance of each leadership
competency, strategy and practice as it relates to your data-driven decision making
and how they relate to improving student achievement, and five open-ended question
that asks you if there are any additional leadership strategies or practices related to
data-driven decision making that you have used to improve student achievement that
were not included in the survey.

You may also be asked to participate in a one hour interview at a time and place
convenient to you and the researcher. The interview will be audio-taped with your
permission and include questions about leadership competency, strategy and practice
as it relates to your data-driven decision making and how they relate to improving
student achievement. You can continue with your participation if you decline to be
audio-taped; hand-written notes will be taken.

146

APPENDIX D - continued

POTENTIAL RISKS AND DISCOMFORTS
There are no foreseeable risks to you for participating in this study. Any discomforts
that you may experience with questions may be managed by simply not answering
the question.

POTENTIAL BENEFITS TO SUBJECT AND/OR TO SOCIETY
You will not directly benefit from the study. However, your participation in this
study may add to the professional knowledge and understanding about the leadership
strategies and practices as it relates to your data-driven decision making and how
they relate to improving student achievement,. The findings will benefit other
superintendents who strive to improve teaching and learning in their districts.

PAYMENT FOR PARTICIPATION
You will not be paid for your participation in this study.

CONFIDENTIALITY
Any information that is obtained in connection with this study and that can be
identified to you will remain confidential and will be disclosed only with your
permission or as requested by law.

Only the researcher and the dissertation committee members will have access to the
data associated with this study. The data will be stored in the investigator’s office in
a locked file cabinet and a password protected computer.

The data will be stored for three years after the study has been completed and then
destroyed. When the results of the research are published or discussed in
conferences, no information will be included that would reveal your identity.

PARTICIPATION AND WITHDRAWAL
You can choose whether to be in this study or not. If you volunteer to be in this
study, you may withdraw at anytime without consequence of any kind. You may
also refuse to answer any questions you don’t want to answer and still remain in the
study.

RIGHTS OF RESEARCH SUBJECTS
You may withdraw your consent at anytime and discontinue participation without
penalty. You are not waiving any legal claims, rights, or remedies because of your
participation in this research study. If you have any questions regarding your rights
as a research subject, contact the University Park IRB, Office of the Vice Provost for
Research Advancement, Stonier Hall, Room 224A, Los Angeles, CA 90089-1146,
(213) 821-5272 or uprib@usc.edu.

IDENTIFICATION OF INVESTIGATORS
If you have any questions or concerns about the research, please feel free to contact
Lonny Root at lroot@usc.edu or Dr. Rudy Castruita, Faculty Supervisor, at
rcastrui@usc.edu.

147

Results from Superintendent Surveys - Data Use Competencies

Record of results from individual superintendent survey results by question. Novice(1) = still learning
most components of this aspect of data use. Developing(2) = knowledgeable of many components of
this aspect of data use, but not all. Proficient(3) = knowledgeable of most components of this aspect
of data use. Skilled(4) = knowledgeable of all components of this aspect of data use. Expert(5) =
could train others on this aspect of data use.

Survey Questions
Survey
Number
1 2 3 4 5 6 7 8
1001 3 3 3 4 3 3 3 3
1005 4 3 2 4 5 5 2 2
1007 5 5 3 4 5 5 3 3
1013 4 4 3 4 5 5 3 3
1018 4 4 4 4 5 4 3 3
1022 4 4 4 4 4 4 4 4
1024 4 4 3 5 4 4 4 4
1026 4 3 4 3 4 4 4 3
1027 4 4 3 4 5 4 3 3
1028 5 4 4 4 5 5 5 5
1032 4 4 3 4 4 4 4 4
1033 5 5 3 3 5 3 1 1
1034 3 3 2 3 4 3 2 3
1035 3 4 1 4 5 4 1 1
1036 5 5 5 5 5 3 5 5
1004 4 4 2 3 4 4 3 3
1006 4 4 3 3 5 4 3 3
1010 3 3 3 3 3 3 1 1
1015 4 4 4 4 4 4 2 3
1020 4 4 4 4 4 4 4 4
1023 3 4 4 4 4 4 3 4
1029 5 5 4 5 5 5 5 5
1037 4 4 4 4 4 3 3 3
Average 4 3.96 3.26 3.87 4.39 3.96 3.09 3.17
Novice 1
0 0 1 0 0 0 3 3
Developing 2
0 0 3 0 0 0 3 1
Proficient 3
5 5 9 6 2 6 10 12
Skilled 4
14 15 9 14 10 13 5 5
Expert 5
4 3 1 3 11 4 2 2
APPENDIX E

148

APPENDIX E – continued

Survey
Number 9 10 11 12 13 14 15 16 17
18
1001 3 3 3 4 3 3 3 3 3
3
1005 2 2 2 2 4 3 2 4 5
5
1007 3 3 3 3 5 5 3 4 5
5
1013 2 3 3 3 4 4 3 4 5
5
1018 3 3 2 3 4 4 4 4 5
4
1022 4 4 4 4 4 4 5 4 4
4
1024 3 5 5 4 5 5 4 5 5
5
1026 4 3 4 4 4 3 4 4 4
4
1027 2 3 3 3 4 4 3 4 5
4
1028 5 5 5 5 4 4 4 4 5
5
1032 3 4 4 4 4 4 3 4 4
4
1033 1 1 1 1 5 5 3 3 5
3
1034 2 4 4 4 3 3 3 4 5
3
1035 1 1 1 1 3 4 1 4 5
4
1036 5 5 5 3 5 5 5 5 5
5
1004 1 2 2 2 4 3 2 4 4
3
1006 3 2 3 3 4 4 3 4 5
4
1010 1 1 1 1 3 3 3 3 3
3
1015 2 2 2 3 4 4 4 4 4
4
1020 4 4 4 4 4 4 4 4 4
4
1023 4 4 4 4 3 4 4 4 4
4
1029 4 5 5 5 5 5 4 5 5
5
1037 3 3 3 2 4 4 4 4 4
4
Average 2.82 3.13 3.17 3.13 4 3.95 3.39 4 4.48 4.09
Novice 1
4 3 3 3 0 0 1 0 0
0
Developing 2
5 4 5 3 0 0 2 0 0
0
Proficient 3
8 8 6 8 7 6 9 3 2
5
Skilled 4
4 5 6 8 12 13 9 18 8
12
Expert 5
2 3 3 1 4 4 2 2 13
6

149

APPENDIX F

Results from Superintendent Surveys - Data That Superintendents Use and Value

Record of results from individual superintendent survey results by question. Types of data that
superintendents value and use. CST (1), Benchmark Exams (2), Program (3), Demographic (4), and
any other type of data (5).
Survey Questions
19 20 21 22 23 24 25 26 27 28
1001 1234 124 1234 124 124 124 1234 14 124 124
1005 123 5 123 1234 1234 123 1234 14 134
123
4
1007 134 34 134 134 34 1234 134 134
1013 12345 1235 3 12345 12345 12345
1234
5 12345 12345
123
5
1018 134 134 134 12 123 1234 123 1234 1234
123
4
1022 34 1234 1234 1234 134 1234 1234 1234 1234
123
4
1024 23 1234 1234 1234 1234 3 1234 123 1234
123
4
1026 23 12 23 1234 23 14 1234 1234 134 23
1027 14 24 14 1234 1234 12 23 1 134 14
1028 124 1234 124 1234 1234 1234 1234 14 124
123
4
1032 12 12 12 3 123 123 123
1033 1 13 12 1234 1234
123
4
1034 123 12 23 123 123
1035 5 5 5 13 13 1 1234 1234 1234 5
1036 12345
1234
5 12345 12345 12345 12345
1234
5 12345 12345
123
45
1004 13 134 134 1234 123 13 1234 123 1234
123
4
1006 12 12 134 1234 124 1234 1234 1234 1234 13
1010 1235 1235 12345 12345
123
45
1015 14 1234 34 1234 1234 1234 1234 134 34 13
1020 134 34 234 1234 123 1234 1234 1234 1234
123
4
1023 5 234 1234 1234 123 34 1234 1234 1234
123
4
1029 1234 1234 1234 12 1234 1234 1234 1234 1234
123
4
1037 1234 134 1234 134 1234 1234 134 134 1234
123
4
1=14 1=13 1=14 1=23 1=19 1=18 1=20 1=23 1=22
1=1
9

150

APPENDIX F - continued
2=10 2=12 2=12 2=19 2=17 2=14 2=21 2=16 2=18
2=1
7
3=13 3=13 3=15 3=18 3=17 3=16 3=22 3=19 3=21
3=1
8
4=12 4=14 4=14 4=16 4=13 4=14 4=17 4=18 4=21
4=1
4
5=4 5=4 5=2 5=3 5=2 5=1 5=3 5=3 5=3 5=4

Survey Questions

29 30 31 32 33 34 35 36 37
1001 14 14 234 1234 1234 1234 1234 1234 1234
1005 1234 1234 1234 1234 13 123 13 123 1234
1007 134 134 1234 134 134 1234 134
1013 12345 12345 12345 12345 12345 12345 12345 12345 12345
1018 1234 1234 1234 1234 1234 134 134 1234 23
1022 1234 1234 1234 1234 12 1234 134 123 12
1024 1234 1234 3 1234 23 1234 134 1234 12
1026 1234 14 1234 1234 234 23 3 2 23
1027 1234 1234 1234 1234 12 1234 14 234 13
1028 1234 1234 13 124 124 124 1234 124 1234
1032 123 3 123 3
1033 134 13 1234 1 12
1034 123 123 123 234 234 123 123
1035 1234 1234 123 123 123 13 13 123 123
1036 12345 12345 12345 12345 12345 12345 12345 12345 12345
1004 12 134 1234 1234 123 1234 12 2 12
1006 1234 14 1234 1234 123 14 134 124 123
1010 12345 12345 12345 12345 12345 12345 4 24 4
1015 123 123 1234 123 123 123 1234 123 123
1020 1234 1234 12345 1234 1234 1234 1234 1234 1234
1023 1234 1234 1234 1234 1234 1234 1234 1234 1234
1029 1234 1234 1234 1234 1234 1234 1234 1234 1234
1037 124 14 34 1234 124 1234 1234 23

1=23 1=21 1=19 1=21 1=19 1=18 1=16 1=18 1=17

2=20 2=15 2=18 2=22 2=19 2=16 2=9 2=22 2=18

3=20 3=17 3=23 3=22 3=17 3=18 3=15 3=16 3=18

4=19 4=19 4=17 4=20 4=13 4=15 4=14 4=14 4=10

5=2 5=3 5=3 5=3 5=3 5=3 5=2 5=2 5=2

15 13 15 17 8 12 8 10 8

151

APPENDIX G

Results from Superintendent Surveys –Frequency of Engaging in Data-Driven Decision Making
Activities
How often they engaged in the data-driven making actions: Never(1) = have not done this.
Sometimes(2) = once or twice a year. Occasionally(3) = more than twice a year but not at every
opportunity. Frequently (4) = every opportunity.
Survey Questions

38 39 40 41 42 43 44 45
1001 4 4 4 4 4 4 4 3
1005 4 3 3 3 4 3 4 2
1007 2 3 3 4 4 4 4 3
1013 4 4 4 4 4 4 4 4
1018 4 4 4 3 4 4 4 3
1022 4 4 4 3 4 4 4 3
1024 2 4 3 4 3 4 4 4
1026 3 4 4 4 4 4 4 4
1027 4 4 4 4 4 4 3 4
1028 4 4 4 4 4 4 4 4
1032 3 4 4 4 3 3 3 4
1033 1 2 2 3 3 3 3 4
1034 4 4 4 4 4 4 4 4
1035 3 3 3 3 3 3 3 2
1036 4 4 4 4 4 4 4 4
1004 4 4 4 4 3 2 4 3
1006 3 4 3 3 3 4 4 4
1010 4 4 4 4 4 4 4 4
1015 4 3 4 4 4 4 4 3
1020 4 4 4 4 4 4 4 3
1023 4 4 4 4 4 4 4 4
1029 4 4 4 4 4 4 4 2
1037 4 4 4 4 4 3 4 3
1s
1 0 0 0 0 0 0 0
2s
2 1 1 0 0 1 0 3
3s
4 4 5 6 6 6 4 8
4s
16 18 17 17 17 16 19 12

152

Frame Work for Effective Data-Driven Decision Making
Knowledge
/Practices
Ineffective
Somewhat
Effective
Very
Effective
Evaluatin
g Types
of Data
Uses the
same
set/type of
data for
all
decision
making.
Uses
several
types of
data to
triangulate
the
answer.
Poses a
question
then uses
the type of
data that
most
accurately
answers
that
question.
(1,4)
Data
Analysis
Skills
Able to
analyze raw
data and
ascertain gaps
in student
achievement
Able to
analyze raw
data and
correlate
results with
current
programs and
practices.
Able to
analyze raw
data and
correlate
results with
current
programs and
practices, and
connect data
down to the
students in
the
classroom.(2,
4,5)
Data-Decision
Making Process
Poses questions
and formulates an
answer then
connects data to
answer that
supports the
conclusion.
Analyzes data to
reveal
achievement gaps
or other issues
then formulates a
response or action
that addresses
those gaps or
issues.
Poses “research
type” questions
regarding student
achievement and
analyzes data for
appropriate
response and/or
action(s).( 1,2,5)
Technology
Skills
Able to read
data reports
sent to them
via
electronic
mail.
Able to
access the
data reports
they need
from district
or internet
data
programs. 2
Able to
create
specific data
reports they
need from
district or
internet data
programs. 2
Data
Communica
tion to
Board of Ed
Provides
written
reports,
charts,
graphs to
board with
written
interpretation
attached.
Presents data
to board that
are
connected to
goals and
objectives
the board has
or questions
the board
poses.
Works with
board to
develop the
ability to
hold
discussions
using data
and make
data driven
decisions.
(3,5)
Data
Communication to
Administrators
Issues periodic
general student data
to administrators in
connection with new
policy or initiative.
Communicates data
use expectations and
how specific students
data is connected to
district initiatives at
the start of each
school year.
Communicates data
use expectations and
administrative
responsibilities,
along with how
specific student data
is connected to
district initiatives,
and the connection to
classroom practices,
frequently
throughout the
school year. (2, 4,5)
Use of Data in
Evaluation of
Educators’
Performance
No use of data, or use
of one type of data
that is not directly
connected to actions
and policies of the
evaluatee(s).
Use of multiple types
of data to evaluate an
evaluatee’s past
actions, skills, or
policies, to provide
data on what was
effective and what
was not effective.
Use multiple types of
data to look at an
evaluatee’s past and
current actions, skills,
or policies and use
the data to make on-
going decisions about
current practice. The
ability of the
evaluatee to make
effective data driven
decisions is a key
factor in
evaluation.(1,4)
Creating Data
Using Culture
Expects teachers to
use data to guide
instruction, provides
raw data to teacher.
Sets expectations of
data use, provides
technology for data
use, and provides
raw data.
Requires the use of
data by teachers, and
includes data in
evaluations. Provides
resources, training,
and direction for data
use. Provides
opportunities for
educators to create
valid student data. (2,
3, 4, 5)
APPENDIX H

Abstract (if available)

Abstract With the passage of the No Child Left Behind (NCLB) Act of 2002, schools, districts, and therefore, superintendents have been held increasingly accountable for the achievement of the students. The states and federal governments have used student achievement data to measure the progress and success of schools and districts and have held districts accountable to this data. This study examined the use of data by superintendents in their decision making related to student achievement. The study looked at what competencies they possessed in the use of data and how they learned those competencies, and what types of data they used and valued. The study also identified what actions, policies, and communications superintendents used to promote data use among all of the educators in their districts. Superintendents’ use of data in the evaluation of principal, teachers, and educational programs was also included in this study.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

How successful urban superintendents in California improve student achievement

PDF

Superintendents and Latino student achievement: promising practices that superintendents use to influence the instruction and increase the achievement of Latino students in urban school districts

PDF

Sustaining student achievement: Six Sigma strategies and successful urban school district superintendents

PDF

School leaders' use of data-driven decision-making for school improvement: a study of promising practices in two California charter schools

PDF

Systemic change and the system leader: a case study of superintendent action to improve student achievement in a large urban school district

PDF

Data-driven decision-making practices that secondary principals use to improve student achievement

PDF

A school's use of data-driven decision making to affect gifted students' learning

PDF

The superintendent and reform: a case study of action by the system leader to improve student achievement in a large urban school district

PDF

Data use in middle schools: a multiple case study of three middle schools’ experiences with data-driven decision making

PDF

Superintendent's leverage: a case study of strategies utilized by an urban school district superintendent to improve student achievement

PDF

An examiniation of staff perceptions of a data driven decision making process used in a high performing title one urban elementary school

PDF

The sustainability of superintendent-led reforms to improve student achievement

PDF

Superintendents' viewpoint of the role stakeholders can play in improving student achievement

PDF

Strategies employed by successful urban superintendents responding to demands for student achievement reform

PDF

Superintendents increase student achievement by selecting effective principals

PDF

Successful communication strategies used by urban school district superintendents to build consensus in raising student achievement

PDF

Understanding the decision making process of California urban schools superintendents through Bolman and Deal's four leadership frames

PDF

The role of the superintendent in raising student achievement: a superintendent effecting change through the implementation of selected strategies

PDF

Defined autonomy: how superintendents work with principals to create the defined autonomy at schools necessary for improved student achievement

PDF

Effective strategies superintendents utilize in building political coalitions to increase student achievement

Asset Metadata

Creator Root, Lonny Gene (author)

Core Title How urban school superintendents effectively use data-driven decision making to improve student achievement

School Rossier School of Education

Degree Doctor of Education

Degree Program Education (Leadership)

Publication Date 05/10/2010

Defense Date 03/23/2010

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag analysis,data,data driven decision making,evaluation,OAI-PMH Harvest,student achievement,superintendent

Place Name California (states)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Castruita, Rudy M. (committee chair), Brewer, Dominic J. (committee member), Escalante, Michael F. (committee member)

Creator Email lroot@gusd.net,lroot@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-m3055

Unique identifier UC1214589

Identifier etd-Root-3635 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-341924 (legacy record id),usctheses-m3055 (legacy record id)

Legacy Identifier etd-Root-3635.pdf

Dmrecord 341924

Document Type Dissertation

Rights Root, Lonny Gene

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Repository Name Libraries, University of Southern California

Repository Location Los Angeles, California

Repository Email cisadmin@lib.usc.edu