University of Southern California Dissertations and Theses

Developing a culture of evidence: Using institutional data to identify inequitable educational outcomes

(USC Thesis Other)

Developing a culture of evidence: Using institutional data to identify inequitable educational outcomes

PDF

Download

Open document

Flip pages

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content DEVELOPING A CULTURE OF EVIDENCE: USING INSTITUTIONAL DATA
TO IDENTIFY INEQUITABLE EDUCATIONAL OUTCOMES
Copyright 2002
by
Georgia Louise Bauman
A Dissertation Presented to the
FACULTY OF THE GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(EDUCATION)
August 2002
Georgia Louise Bauman
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
UMI Number: 3094304
Copyright 2002 by
Bauman, Georgia Louise
All rights reserved.
®
UMI
UMI Microform 3094304
Copyright 2003 by ProQuest Information and Learning Company.
All rights reserved. This microform edition is protected against
unauthorized copying under Title 17, United States Code.
ProQuest Information and Learning Company
300 North Zeeb Road
P.O. Box 1346
Ann Arbor, Ml 48106-1346
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
UNIVERSITY OF SOUTHERN CALIFORNIA
THE GRADUATE SCHOOL
UNIVERSITY PARK
LOS ANGELES, CALIFORNIA 90089-1695
This dissertation, written by
GaorglcLJj.>_JBauman_____________________
under the direction of h e r dissertation committee, and
approved by all its members, has been presented to and
accepted by the Director of Graduate and Professional
Programs, in partial fulfillment of the requirements for the
degree of
DOCTOR OF PHILOSOPHY
Director
A u g u st 6 , 2002
Date_______________________
Comrdittee
Chair
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Dedication
To my parents, Wayne and Mary Louise Bauman
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Acknowledgments
During the beginning of my time as a graduate student at USC there was an
email discussion on the school list serve regarding educational research and its
usefulness, with comparisons being made to medical research. In recent years
authors have criticized educational research, and in particular research on higher
education, in one instance citing that “a primary factor impeding the advancement of
higher education is that the research to practice gap remains fairly unexplored, and
few suggestions exist to advance our thinking beyond blaming one side or the other”
(Kezar & Eckel, 2000, p. 2).
However, I have been fortunate to become a part of the Center for Urban
Education and have the opportunity to engage in research which I feel has effectively
bridged the gap between research and practice. In the research projects at CUE, we
work very closely with practitioners in institutions throughout the Los Angeles
region for the purpose of identifying racially stratified educational outcomes for
African American and Latino undergraduates. As a result of participating in this
work I see the possibilities for research to truly have an impact on improving
practice in our educational systems.
By being part of the Center for Urban Education I have also had a rare
educational opportunity as a graduate student to participate in a research group
which is collaborative, constructive, and creative. Each member of the Center has
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
contributed to the work reflected in this dissertation. My fellow Research Assistants
Michelle Gonzales Bleza, Paz Oliverez, Lan Hao, Melissa Contreras-McGavin; our
Research Associates Marta Soto and Marcy Drummond; and Professor Don
Polkinghome all read my work again and again and provided invaluable feedback
that has served to improve my work.
And I want to thank my chair, Estela Bensimon, who dedicated so much time
and interest and care to my work, even while on sabbatical and a Fulbright
Fellowship in Mexico. Estela has helped me to become a better researcher and writer
and to think of what is possible rather than what is traditional.
I want to thank The James Irvine Foundation, which provided support for this
study as part of a larger project called “Designing and Implementing a Diversity
Scorecard to Improve Institutional Effectiveness for Underserved Minority
Students.” The findings and opinions written here are solely mine and do not reflect
the position or priorities of the foundation.
Finally I would like to express my gratitude to my friends and family. The
support of my fellow graduate students, Julia Colyar and Felicia Lee, gave me the
confidence and wherewithal to get through this process (and to even have a few
laughs along the way). Ashli Cooper’s patience, generosity, and willingness to listen
were invaluable.
I would never have made it to this point without the love and support of my
parents as well as their dedication to my education and achievement.
iv
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table of Contents
List of Figures .................. IX
Abstract a a a « o e B « o » e B B « 0 a a « a » » » o » » « e e B B a a 0 a o B « a » o a o « o « a B a a a a e v a B a » a » a B « t « 0 a o o a a « o a B « B e e a o o B a a e o s o a B e « B a « e 0 0 a » s e xi
CHAPTER 1: INTRODUCTION ..... 1
Statement of the Problem ...... ......... ........
The Study ..... ................... » a e 0 o « s a » e e » e e e « D B B O » « « « « » « 0 » e « « o a » o a o a a « 7
Significance of the Study ................ 10
The Report .... 12
CHAPTER 2: REVIEW OF THE LITERATURE................ 13
A Problem in Organizational Performance............................. 13
Inequities in Educational Outcomes............. 14
Locating the Problem in the Institution...............................................................17
Organizational Learning ...... ••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••■ •••••a 23
Defining Organizational Learning .................................................. 23
Critique of the Literature on Organizational Learning....................................... 35
Facilitating Organizational Learning Through the Use of Data.........................40
Information and Knowledge ..... 44
Data and Information................... 44
Knowledge ............ 46
The Value of Groups
Data as a Trigger for Learning... 49
v
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
CHAPTER THREE: METHODOLOGY ..................................................... 52
The Project and the Text ....... © ® © e ® ® © ® f f l ® © ® « 0 e ® ® ® ® ® a e ® ® ® ® « ® ® 0 ® a © ® © ® © 0 » ® a ® e ® 0 ® a ® © ® 0 « © © e © ® ® ® ® 0 © ® ® © » e ® © © ® a s ® o 54
Creation of the Text ..... 57
The Advantages of these Data.......................................... 59
The Text versus Reality......................................... 63
Method................. 64
Ethical Considerations ...... 67
Categories for Text Analysis 0 ® ® ® © a ® 0 0 ® ® © © « e e © ® © © ® 0 0 0 0 * 0 0 0 ® ® ® 0 « « « « ® ® a •••••» •« © o 8 o ® a a © © ® © o B o e ® 0 o © © © ® © 0 © e B © ® s © « « © 69
Group Learning.................. 70
Data Focus.................................... 76
Experiential Knowledge ....................................................................... 79
Group Types............................................................... ....84
CHAPTER FOUR: RESULTS... e ® ® ® ® ® ® ® ® * ® * © © ® ® ® ® * ® © © ® ® © © ® * ® ® ® ® ® ® ® ® © ® ® ® * ® ® ® ® ® ® ® * ® * ® ® ® ® ® ® ® ® ® ® ® ® ® ® * ® © * ® ® ® ® ® ® * 87
Group Types ............ 9 6 ® ® ® ® ® ® ® ® ® ® © ® © ® ® ® ® ® ® ® ® ® ® ® ® ® ® * © ® ® ® ® © a ® ® » ® ® a « © ® ® ® ® a © « o © ® « © a © © ® 0 © ® o ® 9 © a ® 0 © ® o « © © © a a « s © » © ® © b ® 89
High Learning Groups..................................................... 91
Medium Learning Groups ..... 108
Low Learning Groups.......................................................... 118
Value of Groups................................................ 123
Conclusion............................................ 124
CHAPTER FIVE: DISCUSSION............... 127
Important Findings ..................... 127
Using Data to Develop New Recognition of Inequities in Student Outcomes:
Implications for Practice in Medium Learning Groups..................................... 131
1. Seek Evidence.............................. 131
2. Your Colleagues Are Not Always Right.......................................................132
3. Engage in a Second Order Level of Inquiry and Explore Educational
Processes......................................................... 132
4. Don’t Be Afraid of Data... ...........................................................133
5. Sustain Engagement with the Data ..............................................................134
6. Create Conditions for Group Learning...................................... 134
7. No One-Time Shots......................................... 136
Can Higher Education Institutions Learn? ..... . a ® ® ® * ® © ® ® ® * © © © © * ® © ® © ® ® * ® ® © © © ® © © © © © ® © © ® © ® ® ® » « » © » » 136
vi
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Indicators of Institutional Performance 140
Future Research ....... 142
Conclusion J44
REFERENCES 146
APPENDIX A 153
vii
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
List of Tables
Table 1: Percentage needing remediation in math and English, Fall 2000, by ethnicity
(California State University, Office of the Chancellor) 15
Table 2: Percentage of College Enrollment by Ethnicity, College-Aged Cohorts, 1976-
1995 (Digest of Education Statistics, 2001, p. 23) 16
Table 3: Percentage of bachelor’s degrees conferred by race/ethnicity, 1976-1995 (Digest
of Education Statistics, 2001,p. 131) 16
Table 4: The Percentage of 25-29 year olds who attained a bachelor’s degree or higher,
1971 versus 2000 (National Center for Education Statistics) 17
Table 5:
Table 6:
Table 6:
Table 7:
Three Perspectives on Organizational Learning 29
Differences between Ongoing and Episodic Organizational Learning 34
Levels of Operation along two dimensions 84
Group Type Classifications 84
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Figure 1 : Group Learning
Figure 2: Group Learning Index
List of Figures
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Abstract
In this study I investigated the ways in which institutional data might compel
institutional actors to develop new recognitions of the inequities in educational
outcomes for students from different ethnic/racial groups. Using the method of text
analysis I examined the field notes database associated with one year of meetings of
committees involved in a project focused upon revealing inequitable educational
outcomes through the examination of institutional data on indicators such as grade
point average (GPA), representation in majors, graduation rates, and representation
on the Dean’s List. 14 committees in 14 different higher education institutions in the
Southern California region participated in the project. I found that there were three
types of committees: High Learning groups, Medium Learning groups, and Low
Learning groups. The High Learning groups experienced the highest levels of group
learning and gave precedence to the information revealed through examination of the
data over their collective experiential knowledge of the institution and its students.
These committees were also in the best position to develop agendas for change at
their institutions because they had engaged in a deeper level of inquiry, looked at
indicators in the educational process for students rather than only indicators of
educational outcomes, and sustained their examination of the data outside the
boundaries of the project. As a result, these committees identified potential points
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
for intervention to make progress toward narrowing the gap in educational outcomes
for African American and Latino students.
xii
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Chapter 1: Introduction
Statement of the Problem
Despite almost 50 years of an agenda for access and equal educational
opportunity for traditionally underrepresented students, higher education has not
made sufficient progress toward those ends. While institutional actors in some
colleges and universities boast about their diverse student bodies and commitment to
diversity, they seem to be unaware that those students who make up this “diverse”
element do not enjoy equitable educational outcomes. Alternatively, institutional
actors who might be aware of the problem may not believe that it is something that
they can impact or change because they attribute unequal outcomes to student
characteristics, e.g., lack of motivation, cultural and educational background, limited
aspirations, and so on. While their own institutional data might reveal that African
American and Latino students are over-represented among those students in
academic jeopardy and underrepresented among those students who graduate,
institutional actors do not generally act with these facts in mind on a daily basis. In
not thinking of unequal educational outcomes as a problem of institutional
performance, members of the institution are likely to overlook the kinds of changes
in structures, policies, and practices that might make a difference in educational
outcomes. Thus teaching and learning processes as well as associated support
1
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
systems are likely to be overlooked, as potential causes of or solutions to the
problem. Institutional actors are certainly aware of and act in response to other data.
Among elite institutions, the average SAT scores for the incoming class is a common
indicator of gains or losses in quality; the annual budget is an indicator of financial
health; the kind of data that state authorities collect and report on indicate what
counts as evidence of institutional performance. But, it seems that no one asks for
data disaggregated by ethnicity that might reveal inequitable educational outcomes
for students of color, and as a result, this problem is not at the forefront of college
and university agendas.
Since the 1954 Brown versus Board of Education Supreme Court decision
that ended legally segregated public schooling, the gap in access rates between
minority students and White students has narrowed. However, the gaps in
educational outcomes have not decreased at the same pace. Adelman (1997) asserts
that “the message is unmistakable: if the gaps in access have narrowed, the gaps in
completion remain stubbornly wide” (p. 40). The critical issue accompanying the
increasing diversity of the student body in institutions of higher education is the fact
that traditionally underrepresented students experience differential educational
outcomes when compared to White students; “by all barometers used to measure
success in higher education, students of color, except for Asian Americans, are not
doing well” (Powell, 1998, p. 99).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The Department of Education followed a national cohort of students
beginning in their sophomore year of high school in 1980. By 1992, 23.1 percent of
the White students in this cohort had attained bachelors’ degrees, compared to 10.0
percent of the Black students and 9.0 percent of the Hispanic students in the cohort
who had attained bachelors’ degrees (Digest of Education Statistics 2001, p. 354).
The “college education gap,” which is the ratio of Whites 25 years of age and older
who have completed college versus African Americans and Hispanics, “remains
substantial” (Lee, 2002, p. 6). Nationally, in the year 2000 among those ages 25-29,
21 percent of African Americans and 15 percent of Hispanics had received at least a
bachelor’s degree, compared to 36 percent of non-Hispanic whites (Digest of
Education Statistics, 2001). While African Americans made up 7.5 percent of all
bachelor’s degrees conferred in 1996, they made up only 4.7 percent of all doctoral
degrees conferred. In contrast, Whites made up 78.8 percent of all bachelor’s degrees
conferred and 85.9 percent of all doctoral degrees conferred (Gandara & Maxwell-
Jolly, 1999).
In California’s state university system and community colleges, African
American and Latino students are over-represented1 among students in the lowest
remedial courses, students in academic jeopardy, and students who do not complete a
1 Over-represented means that the representation of this group of students in this particular area
exceeds the group’s representation in the overall student population.
3
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
bachelor’s degree within six years. They are underrepresented2 among students on
Dean’s Lists, students with high grade point averages (GPA) at graduation, students
who transfer from community colleges to four-year institutions, and students who
complete graduate programs (California State University, Office of the Chancellor;
California Community Colleges, Office of the Chancellor).
As the population of traditionally underrepresented students continues to rise,
these problems will become even more important. It has been predicted that by 2015,
African Americans will make up 14.5 percent of all 18-24 year-olds, but they are
projected to account for only 11.9 percent of enrolled college students. Hispanics
will account for 18.9 percent of all youth in the traditional college-age bracket, but
account for only 13.1 percent of undergraduates (Camevale & Fry, 2000). In fact, it
has been reported that nationally, “African American, Hispanic, and Asian/Pacific
Islander students will account for 80 percent of the increase in undergraduates by
2015, or about 2 million of the 2.6 million new students” (Camevale & Fry, p.23).
Not only is the problem of inequitable educational outcomes important for the
students, it is also important for the competitiveness and survival of the institutions.
Historically underrepresented students are going to become the principal source of
enrollments.
The typical approach taken in studies of minority student achievement in
higher education is to relate their retention and graduation rates to attitudes and
2 Underrepresented means that the representation of this group of students in this particular area is
proportionately lower than the group’s representation in the overall student population.
4
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
behaviors that are measures of their commitment to educational goals; their
perceptions of family expectations; their academic and social integration; and the
academic and cultural capital they possess. (See for example: Braxton, 2000;
Kraemer, 1997; Jun & Tiemey, 1999; Rendon & Valadez, 1993; Tinto, 1993.) This
study takes a different approach. I consider the continued existence of inequitable
educational outcomes as a problem in institutional performance. Rather than
focusing on the deficits of minority group students that prevent them from being
successful, I am interested in studying the use of routine data by institutional actors
that, when disaggregated on the basis of race and ethnicity, bring to light patterns of
unequal educational outcomes. My interest is in finding in what ways the inequities
revealed by examination of data might lead institutional actors to recognize that
inequities exist. The difference between studies that focus on students and this study
is the recognition that the factors that have been found to contribute to low
educational attainment for minority students are frequently beyond the control of
faculty and staff. For example, a common finding is that having taken two years of
advanced math courses in high school is one of the strongest predictors of bachelor’s
degree completion (Adelman, 1999). Once a student enrolls in college, nothing can
be done about what courses he/she took in high school. However, I believe that once
institutional actors understand the nature of the problem there is greater likelihood
that they will implement corrective actions and preventive measures that are within
their control.
5
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
In order to investigate this phenomenon I chose to analyze a database of field
notes associated with the first year of a project in which 14 committees in an equal
number of college campuses were charged with examining institutional data to
identify inequitable educational outcomes for African American and Latino students.
The premise of the project and this study was that institutional data could act as a
powerful trigger for group learning about inequities in educational outcomes. The
research question was: In what ways might institutional data compel institutional
actors to develop new recognitions of the inequities in educational outcomes? In
what ways do institutional actors react to evidence of inequities in educational
outcomes? Indeed, can examination of factual data on simple but specific indicators
of educational outcomes, such as choice of majors, grade point average (GPA), and
completion of remedial/developmental courses, raise awareness about the presence
of inequity?
Learning has been deemed a critical skill for organizations to adapt, remain
viable, and compete in a fast-paced, changing, and volatile environment.
Organizational learning involves a variety of activities including studying the
organization’s own history, studying the best practices of other organizations, and
systematic problem solving (Garvin, 1993). Three conditions highlighted in the
literature as those that can promote organizational learning are: (a) the presence of
new ideas, (b) the cultivation of doubt in existing knowledge and practices, and (c)
the transfer of knowledge among institutional actors. The examination of data related
6
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
to institutional performance on given indicators in novel ways, in this case
disaggregated by ethnicity, by working groups can provide all three of these
conditions, and, in turn, trigger episodes of group learning.
The Study
As explained above, I chose to use a database of field notes derived from the
meetings during the first year of a three-year project which focused on engaging
institutional actors in examination of the educational outcomes for African American
and Latino students in order to identify areas in which institutional data might reveal
inequities. The project addressed the effectiveness of institutions in serving the
educational needs of a changing student population—one that is becoming less and
less White. For example, the committees were encouraged to ask themselves
questions like: are African American and Latino students graduating at the same
rates as other students? Are they proportionately represented on the Dean’s List? Are
they over-represented in some majors while underrepresented in majors that lead to
careers that are in high demand?
The project involved a partnership of 14 institutions of higher education in
the Southern California region. Community colleges, public universities, and
independent colleges were represented among the 14 partner institutions. At each
institution the president appointed a small group to work together on this project,
including at least 3 members and no more than 7 members. The groups were
7
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
comprised of a mix of faculty and administrators who came from departments across
campus and held positions of varying levels within the institution, from transfer
counselors to the Executive Assistant to the President. The charge for these groups
was to identify racially stratified patterns of educational achievement and report back
to the president with recommendations about what areas the institution might address
to reduce these disparities—areas in which the institution might change. The project
staff members who worked with and facilitated these 14 groups strongly
recommended that each group use existing institutional data to identify these
problem areas. One of the operating assumptions of the project was that there were a
lot of data available in each of the participating institutions, but that much of it went
unused.
The features of the project under study facilitated the three conditions listed
above that are assumed to promote organizational learning: (a) the presence of new
ideas, (b) the cultivation of doubt in existing knowledge and practices, and (c) the
transfer of knowledge among institutional actors. Because institutional data were
examined disaggregated by ethnicity, in most cases for the first time, new
information and ideas were present. Seeing the disaggregated data caused the
committee members to doubt their own knowledge about the given indicators as well
as the performance of their institutions in serving the educational needs of African
American and Latino students. Finally because the examination of data happened in
the context of working committees, committee members had the opportunity to
8
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
interact and communicate about these findings, test their own understandings, and
discuss how their own understandings had been contradicted or confirmed. The
variety of interpretations and understandings of the data being examined juxtaposed
with the members’ experiences in the institution created learning opportunities.
The ways in which these committees were structured are also noteworthy.
For example, the fact that they were presidentially appointed gave them legitimacy
within the institution and reflected the way higher education organizations typically
operate. The project staff tried to avoid some typical shortfalls that reduce the
effectiveness of committees in higher education. By building in the expectation that
the committee would report back to the president within 18 months, acting as
facilitators and keeping the committees on task, bringing all project participants
together on a quarterly basis for mini-conferences on relevant issues, and
maintaining communication with the committee members over email, most of these
committees remained active and engaged in the project as well as productive. It was
also important that these committees included high-ranking administrators,
institutional researchers, as well as those “on the front lines” with the students like
faculty and student affairs staff. This combination of members provided for a variety
of viewpoints and experiences to be present in each committee. Finally, in many
committees the examination of data became more than a task—it was an exciting
intellectual exercise. Institutional researchers enjoyed a rare opportunity to share the
data they work so hard to collect, and faculty and staff learned more about the
9
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
students they seek to educate. For example, in one committee meeting the participant
observer reported:
[The Institutional Researcher on the committee] was very enthusiastic about
sharing the data. He said that he had never been asked to disaggregate their
student data by ethnicity and gender or by program and academic preparation
(Committee VII, 07-13-01).
Significance of the Study
The understanding and knowledge gained from this study is significant for
three reasons. First, it has become clear that business as usual in higher education is
not working for traditionally underrepresented students and changes must be made in
order to address this problem. While the proportion of African American and Latino
students continues to grow, the gap in educational outcomes persists. The implication
is that higher education institutions must change their practices in order to serve
African American and Latino students such that these students experience
educational outcomes equal to those experienced by White students. By examining
the ways in which these 14 committees engage in learning about inequitable
educational outcomes and the variance in the development of new recognitions of the
problem, we may gain practical knowledge about how awareness might be
effectively raised around this issue through the use of institutional data.
Second, this study is significant in that it locates the problem of inequitable
educational outcomes in the institution. I approach this as a problem in institutional
performance whereas it has typically been approached as a problem in student
10
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
preparation, achievement, or integration. By identifying this problem as one of
institutional performance, the onus is put on the institution to make improvements
and the problem is put on par with problems like fiscal management, productivity of
the faculty, and other indicators of performance for which colleges and universities
are held accountable.
Third, this study is based upon an educational research project that has
impacted practice. Some have lamented that educational research tends to be
irrelevant to educational practice, and that even when it is relevant, practitioners are
not privy to it. In an article titled “The Black Hole of Education Research,” the
author writes, “The research-to-practice pipeline, according to scholars and
educators, has sprung many leaks” (Miller, 1999). Because this was an action
research project in which the researchers (referred to here as “project staff’ ) and
practitioners (committee members) were engaged in the project activities side-by-
side, all involved were able to “learn by doing” (Pfeffer & Sutton, 2000, p.6). The
committee members were able to implement and leam from new practices as a result
of participating in this project in the moment, as it happened. As will be discussed in
chapters four and five, learning related to the examination of data in practice was a
particularly salient lesson for some of these committees.
1 1
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The Report
This report is divided into a total of five chapters. Immediately following this
introductory chapter I offer a review of the literature on organizational learning, the
use of information and knowledge in organizations, and the use of groups for
knowledge transfer and generation. In chapter three I explain the research design and
the choices I made for the purposes of this study. I discuss the project under study,
the method of text analysis, and the categories used to make sense of the field notes.
In chapter four I discuss the results of the study. In the last chapter I provide a
discussion of the conclusions I have drawn from this study and implications for
practice and future research. I begin with a review of the literature on organizational
learning.
12
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Chapter 2: Review of the Literature
This chapter begins with a discussion of inequitable educational outcomes as
a problem in institutional performance in higher education. This is followed by an
overview of the literature on organizational learning, a topic which is receiving more
attention in the literature on organizations as a strategy for developing solutions to
problems in organizational performance. As explained in Chapter 1,1 believe that the
features of the project under study and the processes involved in the examination of
data, collectively, by working groups can encourage three conditions considered to
facilitate organizational learning—(a) the presence of new ideas and (b) the
promotion of doubt in existing knowledge and practices, and (c) the exchange of
knowledge among institutional actors. This is addressed within the context of the
organizational learning literature. I then discuss the differences between information
and knowledge and the value of groups in the context of addressing problems in
institutional performance.
A Problem in Organizational Performance
Education is traditionally viewed as a democratizing force in the United
States. If one can attain an education new opportunities become available. And,
education should be equally available to all, regardless of socioeconomic status,
gender, or ethnicity/race. Attainment of a college degree has become increasingly
13
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
important in our new “knowledge economy” (Gumport & Spom, 1999), which
requires workers who are skilled in problem solving, knowledge generation, and
technological skills which tend to be associated with a college education. It has been
estimated that “22 percent of all new jobs in the year 2000 will require one to three
years of college, and another 30 percent will require a bachelor's degree or more”
(Miller, 1995, p. 30). However, a college degree and the associated educational
outcomes, despite almost 50 years of desegregation, are not equally available to all.
Inequities in Educational Outcomes
Achievement gaps exist across the educational system, from grade school
through college, as reflected by the National Assessment of Educational Progress
(NAEP), which is administered throughout the country annually (Lee, 2002).
Although the gaps have narrowed over time, they are still considerable; in fact since
the mid-late 1980’s the gaps have stabilized or widened slowly rather than narrowed.
Differences continue to exist and “recent progress in closing the remaining gaps has
been slow” (Miller, 1995, p. 26). Despite this, “Only passing concerns have been
raised about the growing racial and ethnic achievement gaps during the 1990’s, and
those have been accompanied by few empirical studies” (Lee, p. 3). Mathematics
education provides an illustrative example of the achievement gap. While the gap in
math proficiency exists at all levels, “the critical scores are at age 17, when students
are about to graduate” from high school (Schoenfeld, 2002, p. 15). By age 17 only 40
percent of the Latino students and less than 33 percent of the African American
14
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
students met benchmark performance levels in math, while two-thirds of the White
students met these levels (Schoenfeld).
Another example of the achievement gap is reflected in college remediation.
The percentage of students from different ethnic/racial groups who require
remediation in math and English upon entering college illustrates the gaps in
preparation among ethnic/racial groups. In the California State University system in
the fall of 2000, the state Chancellor’s office reported the following.
Table 1: Percentage needing remediation in math and English, Fall 2000, by
ethnicity (California State University, Office of the Chancellor)
Ethnicity/Race Math English
White 36.74% 27.77%
African American 74.51% 64.4%
Mexican American 64.79% 64.63%
Asian American 35.54% 63.49%
A comparison of the national college enrollment and graduation statistics
over time reflects inequity among ethnic/racial groups as well. (See Tables 2 and 3.)
While African American and Hispanic students have begun to account for slightly
larger proportions of the overall enrollment, their completion of bachelor’s degrees
(BA) has not increased at the same rates.
15
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 2: Percentage of College Enrollment by Ethnicity, College-Aged Cohorts,
1976-1995 (Digest of Education Statistics, 2001, p. 23)
1976 1980 1990 1995 % Change from 1976-1995
White 84.3 83.5 85.9 80.4 -9.6
Black 9.6 9.4 9.3 10.7 +1.7
Hispanic 3.6 4.0 5.8 7.9 +4.3
Asian/Pacific
Islander
1.8 2.4 4.3 5.8 +4.0
Table 3: Percentage of bachelor’s degrees conferred by race/ethnicity, 1976-1995
(Digest of Education Statistics, 2001,p. 131)
1976 1980 1990 1995 % Change from 1976-1995
White 89.5 88.5 85.9 80.4 -9.1
Black 6.5 6.7 6.0 7.8 +1.3
Hispanic 2.1 2.4 3.5 5.2 +3.1
Asian/Pacific
Islander
1.5 2.1 4.0 5.7 +6.2
The small change in the percentage receiving bachelors’ degrees is even more
striking when you consider the increases in enrollment. For example, in 1976
384,000 Hispanic students were enrolled in higher education. By 1995 this number
had jumped to 1.1 million. There was an increase of about 716,000 Hispanic
students, or 35 percent, in enrollment, yet the percentage receiving a degree
increased only 3.1 percent. This disparity in outcomes is also reflected in the
percentage of 25-29 year olds who have attained at least a bachelor’s degree.
16
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 4: The Percentage of 25-29 year olds who attained a bachelor’s degree or
higher, 1971 versus 2000 (National Center for Education Statistics)
Race/Ethnicity 1971 2000 % Change from
1971-2000
White 23% 36% +13%
Black 12% 21% +9%
Hispanic 11% 15% +4%
Locating the Problem in the Institution
Higher education’s focus has traditionally been on access to college, not on
educational outcomes associated with a college education, like attainment of a
bachelor’s degree. The policy of affirmative action was intended to increase access
to postsecondary education for ethnic/racial groups who had traditionally been
denied access. But, even among those students who gain access, the outcomes they
experience are not equitable. Thus it seems access to college is not the “silver bullet”
the higher education community had hoped or thought it would be.
Inequitable educational outcomes for students of different ethnic/racial
groups have not traditionally been recognized as a problem in institutional
performance. The problem of inequitable educational outcomes, particularly
retention and attainment of degrees, has usually been approached using the student as
the unit of analysis and the target of change rather than the institution. Differential
representation and achievement have been attributed to the individual student’s lack
17
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
of academic preparation, his/her integration into the institutional environment both
academically and socially (Tinto, 1993) and his/her lack of opportunities to develop
the cultural capital associated with successful transition to college (Jun & Tierney,
1999; Tierney 1999). The psychological traits of students who leave versus those
who do not leave college have also been studied (Bean & Eaton, 2000). The primary
concern is with how students adapt to the institution and which types of adaptation
lead to greater levels of success. Recommendations are made in such research
regarding what the institution might do to help students make the transition to
college and to promote student academic success, but the focus remains the same—
making up for what’s wrong with the students. For example, Smith and associates
(1997) developed 15 statements about “what works” according to the literature to
promote the educational achievement of all students. Most of these statements
included programs in which helping the students to adapt was the focal point, like
mentoring programs, increasing intergroup dialogue, and developing student
perception of broad campus commitment to diversity. There were fewer statements
that addressed institutional adaptation like changing the curriculum and teaching
methods to reflect the needs of a changing student population. There is now a new
body of literature emerging about college preparation programs which address what
constitutes an effective college prep program (Jun & Tierney). Again, it is the
students being prepared for college, not the colleges being prepared for students.
18
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
I will use a fictitious example to illustrate the problem to be discussed here.
Leland College is a private, four-year institution which boasts a diverse student
body. The enrollment is made up of 25 percent African American students, 35
percent Latino students, 20 percent Asian American students, and 20 percent White
students. Leland College has recently begun a self-study for the purposes of drafting
a strategic plan. The college community (faculty and administration) feels that it
excels in serving its diverse student body, graduating a high percentage of its
students who are qualified, by virtue of their high grade point averages (GPA), to go
on to the top graduate schools in the country. These are indicators of excellent
institutional performance for faculty and administrators alike. While engaging in this
self-study, a committee on ethnic/racial diversity is formed, charged with reporting
to the president how Leland College is doing with regard to promoting academic
excellence among students of all ethnic/racial backgrounds. Through their
investigation the committee members look at institutional data and discover that
African American and Latino students graduate at lower rates and have significantly
lower GPAs than their White peers. This finding is at odds with what the college
community has believed to be true. Moreover, it belies the values and mission that
the college wishes to be known for and what the college professes. One could view
this as a student achievement problem, or one could view it as a problem in
institutional performance. Clearly, the standpoint from which one views the problem
affects how the problem is framed and what solutions are considered. The question
19
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
to be asked here is: In what ways might institutional data compel institutional actors
to develop new recognitions of the inequities in educational outcomes for students
from different ethnic/racial groups? Can institutional data, as illustrated in the tables
provided above, generate the development of new recognitions of this problem?
One answer to ameliorating problems in institutional performance is provided
in the literature on organizational learning. Organizational learning has been
identified as a critical process by which organizations make sense of their
environment, understand their relationship with it, and adapt and adjust to changes in
the environment in order to remain competitive in the marketplace (Argyris &
Schon, 1978; Daft & Huber, 1987; Fiol & Lyles, 1985; Garvin, 1993; Harvey &
Denton, 1999; Huber, 1991; Levitt & March, 1988; Marsick & Watkins, 1999;
Popper & Lipshitz, 1998; Weick & Westley, 1996). Organizational learning has been
defined and conceptualized in various ways, but essentially it represents change in
awareness and/or practice among organizational actors for the purpose of improving
organizational performance (Daft & Huber, 1987; Fiol & Lyles, 1985; Huber, 1991).
It involves promoting the variety of ideas present in the organization by studying
lessons learned from past experiences, seeking out best practices of other
organizations, engaging in experimentation, and exploring for new ideas (Garvin,
1993; Huber, 1991; Levitt & March, 1988; March 1991a). Authors assert (Dill, 1999;
Garvin, 1993; Weick, 1979; Weick & Westley, 1996) that the presence of new ideas
and the promotion of doubt in an organization’s existing knowledge and practices as
20
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
well as transfer of knowledge among institutional actors facilitate organizational
learning. The examination of institutional data in novel ways within the context of
working groups can create these conditions. For example, in the context of the
project under study, even if the collection and use of particular institutional data, like
passing rates in mathematics courses, are considered routine, new ideas can be
generated by asking new questions of this data or examining it in new ways, such as
disaggregated by ethnicity. Also, by disaggregating data by ethnicity, an institutional
actor may be given cause to doubt that the institution is serving all students equally
well—if African Americans are failing math at higher rates than any other group,
this is evidence that may raise questions in peoples’ minds. By examining data in the
context of working groups, transfer of knowledge is immediate. Actors interact and
communicate, sharing their understandings of, thoughts about, and reactions to the
evidence they study.
There is a body of literature which addresses the use and transfer of
information and knowledge within organizations in order to maximize institutional
performance (Cutcher-Gershenfeld et al., 1998; Huber, 1991; Levitt & March, 1988;
Sormunen-Jones, Chalupa, & Charles, 2000; Wenger & Snyder, 2000). And recently
working groups (also referred to as teams, communities of practice, committees, etc.)
have begun to gain recognition as constituting an important forum for knowledge
sharing and knowledge generation to take place.
21
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Colleges and universities have been used as an example of a type of
organization that does not engage in organizational learning effectively (Dill, 1999;
Garvin, 1993). While learning is the central work of colleges and universities, the
institutions themselves are believed to lack the attributes needed for organizational
learning. David Garvin (1993) asserts that to be a learning organization the entity
must acquire new ideas that trigger improvements in the way the organization does
business, and “many universities fail to qualify [because]... these organizations have
been effective at creating or acquiring new knowledge but notably less successful in
applying that knowledge to their own activities” (p. 80). However, as explained
above, I think that the use of institutional data by committees in colleges and
universities can provide new ideas that might trigger improvements in organizational
problems. While the involvement of outsiders in the work of these campus
committees makes them atypical of the way institutional committees normally
function, they provide a situation that is ideal for an empirical examination of
Garvin’s assumption that institutions of higher education are unable to effectively
engage in organizational learning.
In the following review of the literature I begin by giving an overview of the
literature on organizational learning. I also discuss the conditions that are thought to
promote organizational learning and address the notion that data can act as a trigger
and powerful force for group learning. This is followed by a discussion of the
differences between data, information, and knowledge. I then address the importance
22
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
of groups within organizations and their role in the development and transfer of
knowledge.
Organizational Learning
The concepts of organizational learning have developed prominence in the
study of organizations and particularly in the business literature as a critical
organizational skill in a highly competitive and complex environment. Leaders and
managers of business firms worldwide have “seized upon organizational learning as
a coherent response to pressures emanating from significant changes in the business
environment” (Harvey and Denton, 1999). As a result, organizational learning is
considered an essential process for organizations to remain innovative and
competitive. Many scholars of organizations (Cook & Yanow, 1996; Easterby-
Smith, 1997; Fiol & Lyles, 1985; Huber, 1991; Levitt & March, 1988; Popper &
Lipshitz, 1998; Weick & Westley, 1996) lament that although the idea of
organizational learning itself is well received and broadly acknowledged, “no theory
or model of organizational learning is widely accepted” (Fiol & Lyles, p. 803).
Defining Organizational Learning
There are ongoing debates in the literature on important aspects of
organizational learning, however Fiol and Lyles (1985) found two key premises in
their review of the literature on organizational learning: (a) the assumption that
learning will lead to improvement in organizational performance, and (b) that
23
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
alignment between the organization and the external environment in which the
organization operates is essential for competitiveness and long-term survival.
Because improvement in performance and survival are desirable outcomes, there is
an implicit assumption throughout the organizational literature that learning is good
and desirable. In general it seems that authors refer to unlearning poor practices and
learning new, better practices. A critical factor in defining organizational learning is
that it must be recognized as a process and not an end in and of itself.
Authors define and classify organizational learning in various ways. For
example, Easterby-Smith (1997) examines the various approaches to organizational
learning through six different academic disciplines which have contributed to the
literature. Each discipline presents a different way of defining the central concern of
organizational learning: psychology and organizational development are concerned
with human development; management science focuses on information processing;
sociology is concerned with social structures; production management’s focus is
competitiveness and efficiency; and cultural anthropology focuses on meaning
systems in organizations.
Shrivastava (1983) offers four approaches to organizational learning: (a) as
adaptation, (b) as assumption sharing, (c) as developing knowledge of action-
outcome relationships, and (d) as institutionalized experience (p. 9). Organizational
learning as adaptation refers to organizations adapting to changes in the environment
through incremental adjustment of goals, search, and decision-making. Learning
24
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
processes that are described as assumption sharing result in changes in the
commonly held organizational theories-in-use—those theories that guide
organizational action (Argyris & Schon, 1978). Development of knowledge
regarding action-outcome relationships requires the organization to investigate and
acquire new knowledge about how the organization relates to the environment.
Learning as institutionalized experience is “an accumulation of efficiencies through
experience and tradition” (Daft & Huber, 1987, p. 4).
Another common notion in the literature is that there are different levels of
organizational learning. Fiol and Lyles (1985) refer to higher level versus lower-
level learning; Argyris and Schon (1978, 1996) developed the notion of single versus
double loop learning. The lower level or single loop learning processes are ones that
detect errors in the alignment between the organization and the environment and find
ways of correcting the misalignment but do so without affecting the norms or values
of the organization. A good example of this kind of learning is improving access to
higher education for minority group students without taking into account the racial
climate of the institution. Improving access represents a correction but does not take
into account the nature of the institution’s racial climate. Without considering
whether the institution maintains a welcoming and supportive climate for a diverse
student body, the correction might have only short-term effects. In contrast, higher
level or double loop learning are considered to be more cognitively complex
processes that focus the attention of organizational actors on the root causes of a
25
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
problem and the changes that need to be made in organizational structure, or values,
or goals to bring about a more long lasting change. Needless to say this kind of
learning is more difficult to bring about in organizations that have been described as
organized anarchies (Cohen & March, 1974; Bimbaum, 1988) and loosely coupled
systems (Weick, 1976). However, even though learning in its most ideal form may
not be attainable, what is important is that the members of an institution be conscious
of the two forms of learning and that they strive to ask the kinds of questions that are
most likely to lead to double-loop learning.
Three Perspectives on Organizational Learning
All organizations leam, for better or for worse (Huber, 1991; Marsick &
Watkins, 1999), however, there are several perspectives from which organizational
learning has been conceptualized. Organizational learning is viewed through the
functionalist, cognitive, and interpretive perspectives quite differently. This is
illustrated by the conflicting views regarding who in the organization is actually
doing the learning.
The functionalist or systems-structural perspective on organizational learning
views the organization as a type of information processing machine (Daft & Huber,
1987). The organization learns by acquiring data about the environment. By both
scanning and probing the environment, the organization acquires information (Levitt
& March, 1988). The view of the environment is that it is objective and can be
understood and managed if enough data are collected. The organization is then
26
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
concerned with the dissemination of information in clear, unequivocal, and efficient
ways. This perspective on organizational learning is based on rational images of
organizations, like the organization as machine (Morgan, 1997), and is reflected in
the literature on the management of information via technological innovations
(Kock, 1999).
As might be expected, the cognitive perspective has been used quite
extensively by organizational scientists (Argyris, 1996; Easterby-Smith, 1997; Fiol
& Lyles, 1985; Huber, 1991; Levitt & March, 1988; March & Olsen, 1975; Morgan,
1997). From a cognitive perspective organizational learning is treated either as
learning by individuals in organizational settings or by applying individual learning
theories to organizations (Cook & Yanow, 1996; Popper & Lipshitz, 1998). For
example, cognitive capabilities such as memory are attributed to the organization in
this perspective. Henriksson (1999) describes the cognitive approach as “delving into
the ‘black box’ of the mind” (p. 23) of an organization. Popper & Lipshitz operate
from this perspective and distinguish between learning in organizations (LIO) and
learning by organizations (LBO). LIO signifies “learning processes that occur inside
individuals’ heads, albeit in organizational contexts” (p. 166), thus the individual
represents the learner. LBO represents “various learning processes that occur outside
individuals’ heads, which thus justify designating the organization as the learning
agent” (p. 166). In the latter type the organization is deemed the learner because the
learning is dependent upon the collective rather than the individual.
27
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
In the interpretive perspective organizations are not viewed as machine-like
information processors or as brain-like entities but as systems that give meaning to
data via interpretation, and the focus is not upon individuals but on the practices of
groups (Weick & Westley, 1996). The organization must give meaning to and make
sense of data (Weick, 1979) derived from an environment which is not clear or easily
understood (Daft & Huber, 1987) and learns through discussing and developing
shared interpretation of events. This shared interpretation is more closely associated
with knowledge, which is more person-oriented than data or information which can
simply be stored in a database. Knowledge and interpretation require knowers and
interpreters. This notion will be discussed further below. For an organization to
engage in learning, equivocality, which is defined as the presence of multiple and
conflicting interpretations of a phenomenon, must be reduced to an acceptable level.
Indeed “the essence of organizational learning is the reduction of equivocality, not
data gathering” (Daft & Huber, p. 9). This process is dependent upon communication
and interaction among institutional actors, which is promoted within working groups
(Weick, 1979).
The interpretive perspective upon organizations views them as cultural
entities. Organizational culture is cumulatively built up knowledge of the norms,
values, routines, and rules of the organization, which guides future actions. An
organization develops collective “know how” which is unique to one organization
(Cook & Yanow, 1996). There is mutual learning that occurs between the individual
28
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
organizational member and the organization itself (Cook & Yanow, 1996; March,
1991a). When a new person enters an organization, the organization must learn and
adjust somewhat in order to make the new person’s actions and ideas “compatible
with the actions (and underlying meanings) of other members of its culture and to do
so in a way that fosters its own continuity, flourishing and survival” (Cook &
Yanow, p. 454). At the same time the new person is socialized into the existing
culture of the organization. For example, Cook & Yanow highlight a flute making
company where each organizational member has a particular and integral job in the
making of a single flute; the “know how” needed by the company to make flutes is
not lost when an individual leaves because it is embedded in the organization—in the
ways work is structured, what each worker must contribute in order to produce a
good flute, and what constitutes a flute from this particular company as opposed to
flutes from other companies.
Table 5: Three Perspectives on Organizational Learning
Perspective Defining characteristics
Functionalist • Organization learns via acquiring data.
• Image of organization as machine.
Cognitive • Organization learns as an individual would learn, or
organizational learning is the sum of individual learning
in the organization.
• Image of organization as a brain.
Interpretive • Organization learns by giving meaning to data through
collective interpretation.
• Image of organization as cultural entity.
29
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Activities of Organizational Learning
Because organizational learning is viewed as an ongoing process, it can also
be defined by considering the activities associated with organizational learning.
There are generally three steps involved: the acquisition of new knowledge, the
transfer of knowledge to others in the organization, and storing what has been
learned in organizational memory.
The first step is to acquire new knowledge and ideas which then generate
learning (Garvin, 1993; Huber, 1991). New ideas are essential for learning to take
place, but they do not guarantee learning. Acquisition of new ideas can be
accomplished through learning from one’s own experience, learning from the
experiences and best practices of other organizations, and experimentation (Garvin,
1993; Huber, 1991; Levitt & March, 1988). The second step is to disseminate the
new knowledge to the whole organization in a way that will be uniformly interpreted
by organizational members (Huber, 1991; Levitt & March, 1988). The outcomes or
products of organizational learning include organizational maps or schemas that act
as mental representations of the relationships and processes within the organization
(Argyris & Schon, 1996; Weick & Westley, 1996). Systematic problem solving
(Garvin, 1993), a proven and consistent method of approaching all organizational
problems, is also an example of a potential product of organizational learning
activities. During the third step the lessons learned and knowledge gained must be
stored in organizational memory for future use. Depending on the type of
30
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
knowledge, it could be stored in an operation manual, in a database, in a mission
statement, or become part of the social norms in the form of unspoken expectations
of employee behavior, etc. This is not a linear process, particularly when the
knowledge to be stored is not as “clean” and straightforward as data to be stored in a
computer database. Without effective storage of new knowledge, the knowledge may
be lost to the organization when the individual learner leaves (Garvin, 1993; Huber
1991; Levitt & March, 1988).
There is some conflict in the literature as to whether organizational learning
leads to outcomes of any sort. Some authors maintain that learning does not
necessarily lead to change (Huber, 1991; March, 1991a; Weick & Westley, 1996).
Huber (1991) explains that learning does not need to be conscious or intentional, nor
does it necessarily lead to improvements in effectiveness or any observable changes
in behavior. He suggests that “an entity learns if, through its processing of
information, the range of potential behaviors is changed” (p. 89, emphasis added). In
other words, because of the learning that has taken place there is only the potential
for change in organizational behavior, no change is guaranteed. On the contrary,
Garvin (1993) developed a now classic definition which requires that there be
observable change to prove that learning occurred: “A learning organization is an
organization skilled at creating, acquiring, and transferring knowledge and at
modifying its behavior to reflect new knowledge and insights” (p. 80). The learning
that takes place becomes embedded in the organization’s knowledge, which, in turn,
31
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
guides change in organizational routines, behavior, and actions which are presumed
to lead to improvement in performance. The changes in behavior must be observable
so that learning is measurable. The groups involved in this study might not meet
Garvin’s test for learning because during the first year of their activities, there were
no measurable changes in the educational outcomes for African American and Latino
students recorded.
Ongoing versus Episodic Organizational Learning
The types of organizational learning referred to in the literature can be
divided into two, broad categories: ongoing learning and episodic learning. Ongoing
learning represents the daily learning activities of an organization. Slow and
incremental change over time, such as becoming 5 percent more efficient in
production every year, might reflect ongoing learning. A decrease of 5 percent in
efficiency would reflect ongoing learning as well, but the organization might be
considered to have learned the wrong things. The bulk of the literature refers to
ongoing learning in an organization. For example, Levitt and March (1988) assert
that: (a) behavior in organizations is based on routines; (b) organizational actions are
history-dependent; and (c) organizations are oriented to targets. Within this
framework “.. .organizations are seen as learning by encoding inferences from
history into routines that guide behavior” (p. 320). This type of organizational
learning reflects small and consistent adaptations to changes in the environment that
become embedded in the organization’s collective knowledge, which, in turn, guide
32
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
change in organizational routines, behavior, and actions which are presumed to lead
to improvement in performance. Organizational actors, and therefore the
organization, do learn every day. However, change does not necessarily result from
this learning, and, therefore, it is not necessarily measurable. Huber (1991) contends
that only the potential range of behaviors is changed from such learning. An example
of ongoing learning would be if the admissions office of a college gradually
increased the efficiency of finalizing and sending out financial aid packages to
admitted students over a five-year period. The office’s target was to improve
customer service, which might, in turn, improve yield rates such that a higher
percentage of the admitted students enroll.
In contrast, episodic learning involves intentional and purposeful learning
activities focused upon improving a particular organizational problem. These are
instances in which organizational actors identify a problem and actively engage in
learning activities for the purpose of improving organizational performance in the
identified area. The expectation in this type of learning episode is that positive
change will take place and outcomes will be apparent and measurable. Garvin (1993)
asserts that if you can’t measure it, you can’t manage it. Garvin seeks to provide a
“framework for action” and “the gritty details of practice” as opposed to the
traditional focus on “high philosophy and grand themes, sweeping metaphors” (p.
79). This orientation reflects organizational learning episodes that occur because a
problem or area of potential improvement has been identified. For example he cites
33
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
training and job share programs which have the explicit intention of helping
employees to share knowledge in companies where the leadership has recognized
that employees do not share best practices which impacts efficiency and quality of
production. Episodic learning is targeted and reactive to a particular change in the
environment or problem rather than continuous and ongoing adaptation. Using the
example of the admissions office again, an example of episodic learning would be if
the director of admissions called together an ad hoc committee to develop methods
for decreasing the time it takes to finalize and send out financial aid packages to
admitted students. She expects that the packaging time should be cut in half within
one year as a result of the committee’s recommendations.
Table 6: Differences between Ongoing and Episodic Organizational Learning
Ongoing Organizational
Learning
Episodic Organizational Learning
Encode inferences from history,
continuous and incremental
adaptation
In response to an identified problem
in performance or environmental
change
Part of daily operations Finite episode linked to a problem
Not necessarily intentional;
potential range of behaviors
changed, not necessarily visible
outcomes
Person/group charged with
rectifying the problem; measurable
outcomes expected
Tacit Explicit
Higher education typically engages in ongoing learning, making ongoing and
incremental adaptations to changes in the environment (Cameron, 1984). However,
34
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
the type of organizational learning of interest for this study is episodic organizational
learning. Fourteen committees of higher education involved in the project under
study were formed for the purpose of working on an identified problem in
organizational performance—inequity in educational outcomes for African American
and Latino students. The expectation was that as a result of engaging in these
committees, group learning would lead to organizational change in practices that
impact educational outcomes. It was also expected that these changes would be
measurable because institutional data would be used to monitor changes in
outcomes.
Critique of the Literature on Organizational Learning
While the organizational learning literature provides a useful point for
departure, there are problems in the literature and in its application to institutions of
higher education. Four problems in the literature will be discussed here. First,
theories of organizational learning are abstract concepts that have remained at a
conceptual and theoretical level. Second, the literature does not address what exactly
is to be learned by an organization and therefore seems to treat learning for any
objective or occasion as the same. Third, the literature applies learning theories to the
organization as a whole and tends to ignore subunits within the organization as
potential learning units. Finally, the organizational learning literature seems oriented
to leaders and managers rather than being a tool available to all within an
organization.
35
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The writing about organizational learning tends to be descriptive about what
happens when organizations learn or fail to learn rather than studies of organizational
learning in action for a defined purpose. Reports on organizational learning tend to
be based upon experience as consultants for organizations (Argyris 1996; Garvin
1993; March 1991b) or based upon simulations of organizations in the laboratory
(March, 1991a; Putnam & Sorenson, 1982).
While there are certainly exceptions (Argyris 1993; Henriksson, 1999), the
literature often describes ways of operating, such as cultivating reflective openness
among workers (Senge, 1990), and avenues for learning, like rooting out the
underlying causes of problems (Argyris & Schon, 1978) or “storing” newly learned
information in organizational memory (Huber, 1991; Levitt & March, 1988) without
providing concrete methods for doing so. The research related to organizational
learning has “traditionally been rich in conceptual frameworks, simulation studies
and behavioral modeling” (Henriksson, 1999, p. 23) and focused upon fit between
the organization and its environment for strategic advantage. The authors of this
research seem quick to generalize their findings and lessons learned to organizations
in general. For example, Garvin (1993) advocates that his readers adopt a systematic
problem-solving scheme, as Xerox did, to engage in learning problem by problem.
By remaining at a generalized and abstract level, the concepts of organizational
learning seem quite flexible and applicable to a range of organizations. If all
organizations have to solve problems, why can’t all engage in systematic problem
36
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
solving as Garvin suggests? However, just as the business model of organizing has
been found to be incompatible with the characteristics of institutions of higher
education (Bimbaum, 2000), I believe that the same applies to organizational
learning. The way Xerox learns is not likely to be the way that Harvard University
learns. Business organizations and institutions of higher education have different
concerns, objectives, measures of performance, leadership structures, stakeholders,
and contexts. Harvard cannot solve all of its organizational problems using the same
problem solving method. A problem of academic dishonesty requires a different
method of problem solving than does a problem of developing curricular changes.
Second, theories of organizational learning do not speak to what is to be
learned. There are general statements made that the organization seeks to improve
performance via learning. However, one would expect that learning to improve
efficiency in packaging financial aid might be quite different than learning new
pedagogical tools to teach students calculus. These different learning objectives may
call for different learning strategies to be in place. In fact, in some cases authors
advocate forgetting past practices (March 1991a; Weick & Westley, 1996), while in
other cases authors suggest learning from past practices (Garvin 1993; Huber, 1991;
Levitt & March, 1988). Again, because the literature remains very theoretical, it is
flexible and can theoretically be applied to many situations. But, does this reflect the
reality of organizations? Can organizational actors learn equally well using different
tools and processes regardless of the object of learning?
37
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Third, organizational learning theories are applied to the organization as a
whole—the total organization engages in learning. Many organizations, including
higher education, do not function that way. The importance of subunits within an
organization becomes apparent when the topic of knowledge transfer comes up in the
literature. The literature suggests that the knowledge gained in one part of the
organization must be transferred to the rest of the organization in a way that will be
uniformly understood (Huber, 1991). Knowledge transfer is highlighted as part of the
process of storing new knowledge in organizational memory such that when the
individual learner leaves, the new learning is not lost (Levitt & March, 1988). But,
there are also implications in the knowledge management literature that because
different subunits have different activities, priorities, and goals, they do not actually
need to know all of the same things (Dixon, 2000). Colleges and universities are
complex, loosely coupled organizations that operate via smaller, departments and
committees with members who share common goals, priorities, and tasks that may
engage in organizational learning even though the institution, itself, may not. When
the financial aid department learns about new application forms for federal funding,
this knowledge does not need to be transferred to the committee on promotion and
tenure, which is concerned with evaluating faculty performance.
Finally, while the literature seems to address learning for the organization as
a whole, it also seems to be aimed at the leaders and managers of organizations,
assuming a top-down orientation to learning and change. Henriksson (1999) writes
38
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
that the literature portrays “organizational learning as a tool available to managers
seeking organizational improvements” (p. 27) rather than as a tool available to all.
The work and decision-making in higher education is accomplished through the use
of small, working groups, like the committee on promotion and tenure. These groups
and committees shape the curriculum, what is required to be considered a graduate of
a particular department, how resources will be allocated, which students will be
admitted, which faculty will be hired, which buildings will be built, and how the
university will react to crises ranging from a tragedy like the brutal beating and
murder of Matthew Shepard, a gay undergraduate at the University of Wyoming, to a
budget shortfall. By treating organizational learning as the tool of organizational
leaders, a large segment of potential learners are neglected.
In her critique of the literature on organizational learning, Henriksson (1999)
points out that by focusing upon the manager the literature leads “practitioners away
from the very context in which organizational learning is likely to occur, i.e. in the
everyday organizing and implementing of ... activities which people do in social
interaction with one another rather than individually” (p. 42). Organizational
learning is dependent upon the actors within the organization, their interactions with
one another, and the knowledge that is shared and developed via those interactions. I
have chosen to study a specific problem in institutional performance in which
particular tools are put to use by groups of institutional actors (rather than only
39
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
leaders) who are members of the same type of organization—postsecondary
institutions.
Facilitating Organizational Learning Through the Use of Data
Organizational learning is facilitated by (a) the presence of new ideas
(Garvin, 1993) and (b) the cultivation of doubt in existing knowledge and practices
(Weick, 1979; Weick and Westley, 1996) and (c) the transfer of knowledge among
institutional actors in groups (Dixon, 2000; Wenger & Snyder, 2000). I contend that
use of institutional data can provide these conditions in higher education institutions.
New Ideas
Examining existing institutional data in novel ways and asking new questions
of routine data can provide new ideas and information. Using the example of Leland
College from the beginning of this chapter, the committee gained new insight
regarding graduation rates by looking at the data disaggregated by ethnicity for the
first time—that in fact African American and Latino students graduate at lower rates
than students from other groups. By looking at the data in this new way, new
information became visible. The same would be true for examining GPA
disaggregated by ethnicity. This also represents asking new questions of the data.
Rather than only asking, “What is the GPA of our graduating class,” institutional
actors asked, “What is the GPA of our graduating African Americans?” Garvin
(1993) asserts that colleges and universities do not apply the research skills used for
other purposes to institutional self-study and improvement. I think this is true to the
40
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
extent that institutional actors do not typically ask questions of institutional data that
might provide new ideas as they might for research projects in other areas. The
Knight Higher Education Collaborative (2000), made up of educational leaders and
researchers, asserts that many questions can be answered through the use of data,
such as “Who starts but does not finish and why? What is being learned, and for
what purpose” (p. 8)? Answers to such questions through the exploration of
institutional data would provide new ideas and knowledge about institutional
effectiveness and performance.
Promoting Doubt in Existing Knowledge and Practices
Weick (1979) developed a model of how groups (or organizations) reduce the
multiple meanings that members may hold about a phenomenon to a shared,
common understanding such that the group could act on the understanding in the
future. He did not label this learning; he called it the “organizing” process. This
process is dependent upon communication and interaction among group members.
Weick theorized that something in the environment triggers the organizing process-
some event or piece of information catches the group’s attention. The group then
engages in cycles of communication, restricted by rules and norms embedded in the
culture of the group (or the organization within which they operate), and eventually
they reach some sort of common understanding which is then stored in some manner
for future use by the group/organization. For example, the institutional actors of
Leland College have traditionally believed that they serve all members of their
41
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
diverse student community equally well. That common understanding has been
“stored” in a sense in the culture of the college.
Weick (1979), like other organizational theorists (Argyris & Schon, 1978;
March 1991a), suggests that organizations leam when the stored understandings and
information are called into question. When organizational actors doubt what they
have traditionally believed, an opportunity for learning arises. To leam is to doubt
and discredit the retained understanding which resulted from the organizing process.
Organizations should treat the “past as a pest” (Weick, p. 221) and question the
retained information such as routines, norms, rales, and other elements of the
organization’s culture and operations, which are often considered sacred and
unchangeable in organizations. Unfortunately, “the thick layering of routines in most
organizations, coupled with the fact that departures from routine increase
vulnerability, mean that discrediting is rare” (Weick p. 225).
Argyris and Schon (1978) provide an illustrative example of how the
promotion of doubt and questioning can promote organizational learning in their
model of single versus double loop learning. As explained above, single loop
learning occurs when an error is detected in the environment and the organization
corrects that error without disturbing or questioning the norms and values under
which the organization operates. Double loop learning requires questioning of those
norms and values when seeking to correct the error. Argyris and Schon (1978) use
the image of a thermostat to illustrate these concepts. A single loop learning
42
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
thermostat may detect that the temperature in a room has fallen below a particular
target and will turn on the heater to regain the correct temperature. A double loop
learning thermostat would have the capacity to question whether the target
temperature was, in fact, the most desirable temperature and if turning on the heat
was the most efficient way of reaching the target.
Double loop learning can be triggered when the retained information of an
organization is called into question. One of the indicators of institutional
performance at Leland College is the fact that their students, regardless of race,
graduate at high rates, and the members of the college administration feel they are
doing well on this indicator. To engage in double loop learning the institutional
actors would have to doubt the college’s performance on that indicator for some
reason, or the validity of the indicator itself, opening the door for other possibilities,
such as examination of the graduation rates by ethnicity. When the administration
cultivated doubt in this indicator by asking new questions and exploring data in
novel ways—and looked at the students who graduate by ethnicity—a problem in
institutional performance was revealed. A moment of double loop learning is a
moment in which order and disorder are juxtaposed. Engaging in only single loop
learning, maintaining order to the exclusion of disorder, inhibits productive
organizational learning because patterns of thought, organizational routines, values,
and norms are maintained regardless of their relevance or usefulness.
43
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Information and Knowledge
Both information, or data, and knowledge play important roles in
organizational learning. Daft and Huber (1987) see organizational learning occurring
along two dimensions: the systems-structural dimension which focuses upon the
acquisition and distribution of information, and the interpretive dimension which
involves the interpretation of that information. Interpretation and understanding of
information by institutional actors is associated with knowledge. Daft and Huber
assert that “organizations undertake both types of activity” (p. 10), and that both
activities contribute to organizational learning.
Data and Information
Davenport and Prusak (1998) provide definitions for data and information.
Data are “a set of discrete, objective facts about events” (p. 2). Kock (1999) refers to
data as “carriers” of information and knowledge, and that data are distinct from
information because “an ocean of data may contain only a small amount of
information that is of any value to us” (p. 30). Information is described as a message
that intends to change the way in which its receiver perceives something, “to make
some difference in his outlook or insight” (Davenport & Prusak, p.3). Brown and
Duguid (2000) distinguish data and information as distinct and different from
knowledge because they are independent of people, existing as self-contained in
documents or databases. And having data or information does not imply
understanding or knowledge; they are repositories of unprocessed facts available for
44
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
interpretation. In most institutions of higher education the various information
repositories on students, faculty, programs, and finances offer a wellspring of
knowledge that, for the most part, is untapped.
The organizational learning literature does not explicitly address the use of
data and information to generate learning but tends to focus upon the transfer of
knowledge. Knowledge will be defined and explored below. With the advent of new
technologies, a whole new industry of “information technology,” or IT, sprung up
(Davenport and Prusak, 1998; Kock, 1999). The central concern of IT is the
management of data and information through technology such that it can be accessed
and used effectively by an organization. For example, at colleges and universities
information about students from the time they are admitted, through every course
they take, to their GPA at graduation are now contained in complex information
systems. These systems require significant investment by the college and training of
personnel to be able to use them. Yet even with these investments in data and
information management, the data are not used to the extent possible to inform the
work and decisions of the institution.
Higher education institutions routinely use data in a variety of ways for a
variety of purposes. Data about inputs like the characteristics and qualifications of
incoming students are used by institutions (and US News and World Report) to
illustrate high quality. Data are also used for budgeting and planning on an annual
basis. But, there are few data that reflect the outcomes of teaching and learning, core
45
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
processes of colleges and universities. And, such data are certainly not touted
publicly to the extent of the SAT scores of the freshmen class. The Knight Higher
Education Collaborative (2000) reflected upon the use of data in higher education.
While colleges and universities, as communities of scholars, value the use of data in
the pursuit of new knowledge, “most institutions have yet to leam how to use data
strategically” (Knight Higher Education Collaborative, p. 1). There is a fundamental
aversion to data because they may threaten the “existing order by identifying and
averting potential problems” (p. 1). The members of the collaborative advocate the
development of a genuine culture of data in which data are used to define the
direction of the institution and make internal decisions.
Knowledge
Knowledge is “broader, deeper, and richer than data or information”
(Davenport & Prusak, 1998, p. 5). Knowledge is contained within the minds of
knowers and results from an amalgamation of experiences, personal values, personal
characteristics, and interactions with others and is used to interpret, evaluate, and
incorporate new experiences and interactions. Kock (1999) labels knowledge
“associative.” He states that knowledge “allows us to ‘associate’ different world
states and respective mental representations, which are typically linked to or
described by means of pieces of information” (p. 35). In organizations knowledge
often becomes embedded in “organizational routines, processes, practices, and
norms” (Davenport & Prusak, p. 5). Knowledge has been delineated into various
46
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
types. For example, Dixon (2000) defines “common knowledge” as “the knowledge
that employees learn from doing the organization’s tasks” (p. 11). Davenport and
Prusak define this same type of knowledge as “working knowledge” that is put into
action within the organizational context.
Because knowledge is dependent upon knowers, the exchange and creation of
knowledge take place within and between humans. The field of knowledge
management is concerned with managing, transferring, and maximizing the
knowledge held in organizational actors’ minds for improvement of the organization
as a whole. Many organizations lament that they don’t know what the sum of their
members know; Hewlett-Packard’s (HP) former chairman, Lew Platt, has been
quoted as saying “if only we knew what we know at HP” (Brown & Duguid, 2000, p.
123). To find out what members know, organizations set up interactions such as job
transfers where employees are moved from department to department with the
intention of picking up best practices and other important knowledge along the way.
In other cases experts float from department to department sharing their knowledge
and spreading it throughout the organization. Each of these examples is structured to
promote social interaction between actors for the purposes of knowledge sharing and
creation. Authors who study organizations consider this transfer of knowledge one of
the essential ingredients for organizational learning (Daft & Huber 1987; Garvin
1993; Huber 1991; Levitt & March 1988).
47
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The Value of Groups
Groups are becoming a highly valued structure within organizations for the
purposes of promoting knowledge sharing and creation. In fact, “companies in every
industry are striving to create team-based work systems” (Cutcher-Gershenfeld et al.,
1998, p. 59). There is a boom of literature focused upon the characteristics and
qualities that lead to effective teams in organizations: “books and articles have been
written about how to design empowered or self-directed work teams... executive
teams... and team-based organizations” (Cohen & Bailey, 1997, p. 116). For
example, Bensimon and Neumann (1993) differentiated between “real” and
“illusory” college executive teams on the basis of the groups’ functions and the
diversity of thinking displayed by the members of the groups.
Cutcher-Gershenfeld et al. (1998) define team based work systems as
“complex amalgams of tangible practices and intangible elements such as
interpersonal interactions” which promote “the creation of knowledge within the
firm” (p. 59). This emphasis on teams and teamwork in business has been described
as “parallel to the emphasis on cooperative learning in schools, colleges, and
continuing education... through which students work together to maximize their own
and each other’s learning” (Sormunen-Jones, Chalupa, & Charles, 2000, p. 154).
An innovative and newly recognized organizational strategy that is gaining
some attention is the “community of practice.” Communities of practice are typically
small groups within larger organizations who congregate due to “expertise and
48
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
passion” (Wenger and Snyder, 2000, p. 139) in a particular area, such as issues
related to retention and transfer of students, and meet on a regular basis over an
extended period of time. Those who engage in communities of practice “share their
experiences and knowledge in free-flowing, creative ways that foster new
approaches to problems” (p. 140).
Higher education operates in large part through small working groups
typically referred to as committees. As explained above, many of the most critical
decisions are made within such committees such as curriculum committees within
departments, promotion and tenure committees, planning and enrollment
management committees, and any number of ad hoc committees. It seems logical to
assume that these committees represent the work groups most likely to benefit from
the untapped knowledge that lies in the institutions’ numerous repositories of
information. In view of the rich source of data at their disposal, these questions
come to mind: Can committees work in ways that enable them to turn these data into
knowledge? Does knowledge produce a different or new understanding of a problem
among committee members? Does knowledge produce new thinking about potential
solutions to the problem?
Data as a Trigger for Learning
In this review of the literature three activities have been identified as
potential triggers for the learning process. Among them are: (a) acquiring new ideas,
49
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
(b) cultivating doubt in an organization’s stored knowledge, practices, and routines,
and (c) transferring knowledge between actors through communication and
interaction. I propose that the examination of data and information within the context
of working groups can accomplish all of these in institutions of higher education. If,
as Daft and Huber (1987) suggest, organizations operate on two dimensions when
engaging in learning—the acquisition and distribution of data and information and
the interpretation of that information—it seems that data could be used strategically
to provide new ideas, create occasion for communication among actors, and promote
doubt and curiosity of institutional actors.
As the Knight Higher Education Collaborative (2000) advocates, colleges
and universities must use data strategically “as a gauge of capacity and prospects” (p.
2). Data can help institutional actors to gain understanding of where the college has
been as well as the potential for the ways in which it might grow and improve. But,
actors within higher education resist using data in order to protect the status quo and
prevent airing of dirty laundry. The Collaborative explains that “data are not
collected for two reasons: ‘We really don’t want to know because it will make us
change our minds’ or ‘We don’t want someone else to know...’” (p. 3). These
orientations toward the use of data are unproductive and lead to single loop learning
to the detriment of double loop learning.
Data that have not previously been collected or examined can provide new
ideas and understandings about the institution, its members, its operations, and the
50
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
associated educational outcomes. Committees could be formed to further investigate
what the data reveal and raise fundamental questions like: What have we learned
from these data? Is this knowledge consistent with how we think things are? How
does this new information fit with the mission, vision, and goals for the institution?
What should be our collective response to this new information?
By providing this new information, the data also promote doubt and
questioning of retained organizational understandings, practices, and norms. Before
examining the data, Leland College may have prided itself on the diversity of its
student body and the achievements of all students, operating under the assumption
that all students, regardless of ethnicity/race, experienced the same educational
processes and outcomes. Viewing data that show that African Americans and Latinos
graduate at significantly lower rates and with significantly lower GPAs than their
White peers would certainly call these operating assumptions into doubt. This doubt
has the potential to trigger both the organizing process and an episode of
organizational learning.
In the next chapter I describe the methods I used to study this phenomenon
and the data I chose to use as the subject of study.
51
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Chapter Three: Methodology
In this chapter I discuss the project which served as the subject for this study
and the methods used to analyze and make sense of the field notes database. The
project was made up of 14 presidentially appointed committees at 14 different higher
education institutions. The purpose of these committees was to identify areas in
which African American and Latino students experienced inequitable educational
outcomes when compared to their White and Asian American peers and/or when
compared to their own representation in the overall student body. For example, if
African Americans made up 20 percent of the student enrollment but only 8 percent
of the students majoring in engineering while White students made up 25 percent of
the student enrollment and 50 percent of the students majoring in engineering, that
would signify an inequitable educational outcome for African Americans. The
operating premise for this project was that information, such as that used in the
example, could be a powerful trigger for learning.
The text from this project resulted from nine months of participant
observation of 14 committees in higher education institutions, which suited my
interest in developing understanding of how such groups engage in group learning,
particularly through the use of institutional data. This database was particularly well
suited for this study because the fieldnotes document the interactive processes that
are critical to the understanding of how learning happens in a group and because
52
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
these processes are focused on learning about a specific problem. Thus the method
used in this study overcomes two problems in the study of organizational learning:
(1) the lack of empirical documentation of how learning happens in a group and (2)
the tendency to discuss organizational learning at an abstract level rather than in
relation to a particular problem.
I used iterative text analysis in making sense of the field notes, returning to
the text multiple times with new questions that were derived from previous analysis.
As the result of analysis, I placed the 14 committees into three groups along a
continuum from those that experienced high levels of group learning, medium levels
of group learning, to low levels of group learning. I returned to the text determine
the differences between the committees along this scale and identified two important
dimensions that were operating in each of the 14 committees—data focus and
experiential knowledge.
The committees focused on data in their work to varying extents. While some
committees used data to drive their work, others seemed to value and occasionally
consult data, and still others did not utilize data at all. The committees also varied on
the degree to which they relied upon or used their experiential knowledge of the
institution. There were committees that prioritized their own experiential knowledge
over other sources of information, like institutional data. Other committees shared
their experiential knowledge and used it to define their inquiry and examination of
institutional data.
53
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The variation along these two dimensions paralleled the different levels at
which the committees experienced development of new recognitions regarding
inequities in student outcomes, which served as evidence of group learning. For
example, those committees that experienced high levels of group learning also
practiced high data focus and medium reliance on experiential knowledge.
I will begin by describing the project under study and the creation of the field
notes database, which served as the subject of this study. I will then describe the
analysis of the text in three areas—group learning, data focus, and experiential
knowledge. Finally, I will discuss the development of group types along the
continuum of group learning.
The Project and the Text
The project’s primary objective was to engage institutional actors in
examining data related to the educational outcomes of African American and Latino
students and identify areas in which inequities were revealed. The premise or
operating assumption of the project, as stated earlier, was that information could be a
powerful change agent and lead to group learning about inequitable student
outcomes. The institutional data used by groups included quantitative data that were
derived from the institutions’ student information databases, admissions records,
human resources information databases, and in some cases academic department
information databases. The results from institutional surveys such as a periodic
54
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
campus climate surveys and national surveys such as the Cooperative Institution
Research Program American Freshman survey were also used as data in this project
(Higher Education Research Institute, University of California, Los Angeles).
The project involved a partnership of 14 institutional committees. These 14
committees represent institutions from three segments of higher education—
community colleges, public four-year institutions, and independent four-year
institutions. All of these institutions are located in the Southern California area. All
of the public institutions and most of the independent institutions enroll either 25
percent of a particular minority ethnic/racial group and/or 50 percent of traditionally
underrepresented students overall . For this reason these institutions were dubbed
“opportunity colleges,” as they provide access and opportunity to traditionally
underrepresented students, which is evident in their enrollment figures.
The committees were appointed by the presidents of the participating
institutions. The committees were made up of three to seven members, with most
having three or four members. The individuals who made up the committees
represented a cross-section of institutional actors. There were transfer counselors (at
community college partner institutions), faculty members, deans, members of the
Academic Senate, institutional researchers, and an Executive Assistant to the
president represented in these committees. The fewest number of committee
meetings held by any one committee was four, and the greatest number of meetings
3 These were the criteria for institutions to be included in the project under study.
55
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
for any committee was ten. The members of the project staff, which was made up of
faculty, researchers, and graduate students (all of whom were affiliated with the
Center for Urban Education4 at the University of Southern California), met on a
regular basis over the course of the year to review data on educational outcomes
disaggregated by race and ethnicity and identify areas where there was an equity gap
in the outcomes for African American and Latino students. To avoid compromising
the pledge to not reveal the names of the participating institutions and to protect the
anonymity of the members of the committees, I cannot describe each committee and
its members in detail.
The development of the project for each committee generally involved
identifying and examining areas in which African American and Latino students
experience differential educational outcomes when compared (a) to their White and
Asian American counterparts, and/or (b) to their representation on campus. For
example, if Latinos make up 25 percent of the student body but only 8 percent of the
Dean’s List, that is considered to be an inequitable outcome5. The committee
members were asked to use existing institutional data to identify these problem areas.
After identifying the areas, the group members would construct goal statements
4 The Center for Urban Education is a research and action center whose mission is to conduct research
that will result in the creation of enabling institutional environments for children, youth, and adults
from socially and economically disenfranchised groups residing in urban settings.
5 Equity is defined in this project as the point at which a particular ethnic group’s representation
across all majors, programs, honors, etc. at the institution is equal to the group’s representation in the
student body. Therefore, if Latino students make up 25 percent of the student body, they should also
make up 25 percent of the Dean’s List.
56
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
which indicated in what ways they would like to see the situation change. For
example, a goal might read, “To increase the representation of Latino students on the
Dean’s List.” Each goal was operationalized by a corresponding measure and by a
standard (or benchmark) of equity that would need to be reached in order to declare
the equity gap closed. Thus, for this goal the corresponding measure would be the
number and percentage of Latinos currently in the Dean’s List, and the standard of
equity establishes what it should be in relation to the representation of Latinos in the
student body. In this example, since Latinos represent 25 percent of the student body,
the equity standard would be met if 25 percent of the students who are in the Dean’s
List were Latino.
Between March 2001 and December 31, 2001, 69 meetings of these groups
were held which generated over 400 pages of field notes. The field notes were
produced through participant observation of every committee meeting. I will discuss
the development of the field notes text below.
Creation of the Text
At each of the 69 meetings there were typically two members of the project
staff present. One would facilitate and contribute to the meeting while the other acted
primarily as note taker. When only one project staff member was present, she
maintained the primary role of note taker, but she also contributed to the meeting to a
greater degree than when she only held the role of note taker. The meetings were not
audio taped and transcribed. The project staff decided that it was more important to
57
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
create a context for free and spontaneous conversation and that audio taping, while
ensuring greater accuracy, would probably increase self-consciousness and self-
censoring. The concern for maintaining a “safe” context was particularly critical
for this project because its central topic—race and inequity— is value laden and
makes people uncomfortable, a tape recorder might be more of a hindrance than an
aid. Thus the note taker, to the extent possible, acted as a human recorder, recording
as much that went on in the meeting as she could. Initially the field notes were
constructed to answer seven questions which reflected theories of organizational
learning, the committee’s familiarity and comfort with working together, and the
committee’s progress. After a few meetings, the project staff decided that these
questions were no longer relevant and chronologically structured field notes were
taken. These notes reflected descriptive observations in which the individual tried to
record as much as possible in order to answer the general question, “What’s going on
here” (Spradley, 1980)?
When two project staff members were present, they would engage in a 15-30
minute “de-briefing” session immediately following the committee meeting which
was audio taped. In this session the two staff members had the opportunity to ensure
that the recorder included the observations of the other person, to check
understanding, and to discuss the progress of the committee. Each de-briefing
session was included at the end of the field notes for that meeting. The note taker
would immediately go to her computer and type up the field notes in order to guard
58
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
against the inevitable loss of memory. All field notes were typed up, at a minimum,
within 24 hours of the meeting. The project staff met for two hours twice per week
throughout the duration of the project. These meetings served as opportunities to
discuss the field notes, note taking methods, and particular topic areas to attend to for
the purposes of research. Organizational learning was one such topic area throughout
the project. The 400-page database was stored on a common computer drive,
accessible only to project staff and researchers. This database was converted for use
with Atlas.ti qualitative software (Muhr, 1997).
The Advantages of these Data
This project provided an ideal source of data for my study in four ways: (a)
the field notes reflect the activities of 14 working committees in a variety of higher
education settings; (b) the field notes are longitudinal; (c) the field notes were
recorded via participant observation in real time rather than through reflection after
the fact; and (d) the situation under study by these committees, inequitable
educational outcomes, constitutes a problem in institutional performance that calls
for organizational learning.
These committees, which were my unit of analysis, are typical of groups
which play such a critical and vital role in higher education; it is through such groups
that the work of colleges and universities such as curriculum development, policy
implementation, program planning, evaluation of personnel, etc. is accomplished. In
the case of this project, the members of the committees were asked to examine
59
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
existing institutional data on student outcomes that had been disaggregated by race
and ethnicity. The field notes documented how each committee went about its task, I
was able to note those sections in the field notes in which, upon being presented with
factual data, disaggregated by race and ethnicity, on a variety of educational outcome
indicators such as students’ choice of majors, grade point average (GPA), completion
of remedial/developmental courses, the reactions of the committee members showed
that they recognized or failed to recognize the presence of inequity. The following is
an excerpt from the text which shows the types of interactions and detail that were
captured by the observer:
The committee member pulled out his copy of the data and shared it with the
observer.
He said, “We have not collected any more data that we had at the last
meeting.”
The observer asked, “What does this data tell you? What did you
leam from doing this?”
The committee member answered, “There were some places where
we noticed underrepresentation. For example, in Communication we are
doing well, but in Environmental Studies there are very few Latinos”
(Committee XIII, 11-28-01).
In being able to study the detailed notes that have been kept for each
committee’s meetings over time, from the start of the project in the spring of 2001 to
the end of the year, I had the rare opportunity of studying how the members of
committees on college campuses work together and get a task done over time. I say
that this was a rare opportunity because most studies of decision-making,
governance, leadership, and curricular change in higher education are based on post-
60
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
facto interviews. Having access to data that documented the natural routines of the
committees and the evolution of their work over the course of several months
allowed me as intimate a look into the life of these committees as one can get
without actually being there.
Participant observation is the process in which “an investigator establishes
and sustains a many-sided and relatively long-term relationship with a human
association in its natural setting for the purpose of developing a scientific
understanding of that setting” (Lofland & Lofland, 1995, p. 18). And, this method
has been identified as a valuable research method for the study of organizations
because “it problematizes the ways that individuals and groups constitute and
interpret organizations... on a daily interactional basis” (Schwartzman, 1993, p. 3).
In this case, I consider the “organization” to be the committee itself. The field notes
were prepared by an observer who wrote down what transpired during the
committees’ meetings. The notes are very detailed and capture dialogue among
members of committees. In many cases the observer who took down the notes also
inserted clarifying remarks about what was said, who said it, and what, if anything,
prompted particular remarks or exchanges among the members of a committee, as
reflected in the passage quoted above. The text that I worked with chronicles
everyday routines of committees on college campuses, most of which are often
“taken for granted or dismissed as unimportant” (Schwartzman, p. 38). However, it
is through the ordinary and mundane activities of a group that the researcher can gain
61
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
insights into the social structure, culture, and operations of organizations
(Schwartzman). And, by conducting participant observation on a repeated and
longitudinal basis, the observers could consider the committees as active and
ongoing entities rather than considering the committee at a single moment in time.
Participant observation is “not restricted to static cross-sectional data but allows real
study of social processes and complex interdependencies in social systems”
(McCall& Simmons, 1969, p. 2).
Finally, these 14 committees were dealing with a problem in institutional
performance. This problem emanated from a major change in the environment—the
changing demographics of the college-going population. The literature on
organizations asserts that a situation such as this calls for organizational learning
(Fiol & Lyles, 1985; Huber, 1991; Levitt & March, 1988). Inequitable educational
outcomes for African American and Latino students become more imperative as the
number of these students participating in higher education continue to increase. This
is both a moral issue and an issue of institutional effectiveness to which higher
education must attend. As noted in chapter 1, historically underrepresented students
will become the principal source of new students in the next decade (Camevale and
Fry, 2000). In order to remain competitive and survive, postsecondary institutions
will have to find ways to adapt to this new student population.
62
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The Text versus Reality
There is always a gap between what the field notes (the text) reflect and the
reality of a situation under study. It is the job of qualitative researchers to minimize
that gap to the extent that is possible and reasonable. In this case there were several
decisions made that impacted the gap between the text and reality.
Lofland and Lofland (1995) assert that field notes are more accurately
described as a filter rather than a mirror because the observer’s own socially
constructed reality is at play. It is impossible to act as a completely objective
observer since all researchers operate from their own cultural and personal context.
Multiple members of the project staff generated the text. Using multiple authors
helped to reduce the impact of any one note taker’s bias and interpretation.
Inevitably each note taker may attend to different things in the meetings and interpret
events in different ways. Having multiple note takers guards against one person
attending to the same things in every meeting, neglecting other areas completely. In
addition, the de-briefing sessions served to include a second observer’s reflections
and observations on the meeting, helping to capture as much of the activity as
possible from two different perspectives.
Another advantage to the structure of this project was that the notes were
taken in real time, as the meeting happened, rather than only observing and then
relying entirely upon recall to construct the text. Of course it is impossible to catch
everything that occurs and, therefore, some of what happened in the meetings was
63
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
missed. One would presume that the notes were more complete when there were two
project staff members present so that the note taker could focus exclusively on taking
notes.
Finally, the fact that the text was generated from repeated observations over a
period of nine months helped to minimize the gap between the text and reality. The
note takers developed familiarity with the groups, their members, and the topic areas,
which aided in capturing the essence of what was said and how it was dealt with by
group members. They also improved their skills as note takers, becoming more able
to capture what went on and verbatim notes.
Ultimately the gap between the text and the reality of the given situation can
never be closed. Angrosino and Mays De Perez (2000) invoke Stephen lay Gould
who said, “We can only see what fits into our mental space, and all description
includes interpretation as well as sensory reporting” (p. 696). But the project staff
was conscious of this pitfall and made efforts to minimize the gap.
Method
Because I decided to use a previously collected database of field notes, which
I will refer to as the text, the most appropriate method for analysis was text analysis.
I used an iterative process to analyze the text and answer the research question, “In
what ways might institutional data compel institutional actors to develop new
recognitions of the inequities in educational outcomes?”
64
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Categorizing, or coding, is a strategy for data analysis in which meaningful
labels are assigned to sections of data which illustrate themes, recurrent practices, or
theoretical concepts. Boyatzis (1998) refers to the use of thematic analysis and code
development to describe the process of analyzing qualitative information. The
researcher must be able to: (a) sense themes in the text, (b) train and discipline
herself to recognize instances of these themes consistently and reliably and
distinguish between what fits and does not fit with the theme, (c) develop a code or
category to capture the essence of her analysis, and (d) interpret the themes in a way
that contributes to knowledge (Boyatzis). In analyzing the text I looked for “social
practices, recurrent categories of talk or action” that may hold analytic significance
(Lofland & Lofland, 1995, p. 103).
The researcher must remain open-minded and flexible to new ideas the data
may teach her, and she must be reflexive and cognizant of how her own values may
influence what she sees in the data (Nason & Golding, 1998). The end goal is to
provide an analytic description of a complex social organization (McCall &
Simmons, 1969), in this case the 14 committees associated with the project.
According to McCall & Simmons, an analytic description: (a) employs concepts and
propositions of a body of scientific theory as the guides for analyzing and reporting,
(b) utilizes systematic collection, classification, and reporting, and (c) generates new
empirical generalizations, concepts, or propositions based on the data (p. 3).
Thematic codes or categories can be considered inductive and data driven, driven by
65
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
prior research, or theory driven. My categories are theory driven because they are
rooted in the organizational learning literature.
The most significant challenge I faced when I began the study lay in its ex
post facto design. Because I chose to use a finite, previously collected database of
field notes, I did not have the luxury of designing the database to fit my research
questions. The text itself imposed particular limits upon the questions I could ask. I
had to design research questions that could be answered by the text in its current
form. For example, I found myself curious about related questions that would require
verbatim, transcribed notes as opposed to observation notes in order to be answered.
While I was not able to return to the scene of the research to ask group members new
questions, I was able to engage in an iterative process with the text, itself. Returning
to the text multiple times, I developed new and more refined questions on an ongoing
basis.
As mentioned above, I used qualitative software, Atlas .ti (Muhr, 1997), to
engage in analysis of the text. This software enables the researcher to categorize the
data, define the categories through “memo-ing,” and adjust and refine the categories
as the analysis proceeds. I utilized the software for a previous research project and
found that it was a flexible yet precise tool that aided in categorizing, making sense
of, and learning from the data.
I analyzed the text by considering one committee at a time, going through all
of the meetings associated with a committee in chronological order. This allowed me
66
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
to see the development of a committee and its practices over time. When reporting
the findings and using quotes from the text, I use double quotation marks to indicate
the observer’s words and single quotation marks within the double quotation marks
to mark a direct quote of a participant as recorded by the observer. I cite each excerpt
from the text by referencing the committee and the date of the meeting for that
portion of the text. Each committee was designated by a Roman numeral between I
and XIV.
Ethical Considerations
As in all research studies, there were ethical choices to be considered.
Regardless of how unobtrusive and safe a study may seem, research always “pries
into the lives of informants” (Spradley, 1980, p. 22). Spradley suggests that
researchers comply with a set of guidelines such as “Principles of Professional
Responsibility” published by professional groups such as the American
Anthropologic Association. These particular guidelines include ethical principles that
are also put forth in the literature: do no harm; safeguard informant’s rights and
interests; communicate the research objectives; and protect the privacy of
informants. I worked to uphold each of these principles.
As part of the project, each committee member received a document entitled
“Roles and Responsibilities” (see Appendix A). This document was in each
member’s project binder so that he/she always had access to it. The document was
reviewed and discussed at the first meeting with each of the 14 committees. The
67
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
document delineated that the anonymity of all participants as well as the institutions
involved would be maintained; that participants would have the opportunity to
review and comment on pre-publication drafts associated with the project; and any
person and/or institution could leave the project at any time. The president of each
institution appointed the institutional committees and, therefore, the identities of the
committee members were well known which reduced the risk that their employment
would be put in jeopardy as a result of their participation.
Thus far no participant has asked to gain access to the field notes for the
project; however, they have received minutes of meetings periodically throughout
the project. If a committee member asked to see field notes related to the project,
he/she would only be given access to field notes generated during meetings in which
he/she was in attendance. Access to the field notes by anyone who is not a committee
member, a member of the project staff, or a member of my dissertation committee
would be prohibited. Although this study did not aim to analyze or evaluate value
statements made during these meetings, such statements may be present in the field
notes. The identities of the institutions involved in the project are kept anonymous by
referring to them by Roman numeral. The committees are referred to in the same
manner. All identifying references to individual members of the committees have
been removed. To refer to individuals I call them members. When there are two
members involved in a block of text, I distinguish between them using references
68
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
like “one member” versus “another member.” I follow the same pattern for the
observers of the groups.
Categories for Text Analysis
In analyzing the text I used an iterative process to develop an understanding
of the ways in which these committees engaged in group learning through the use of
data. There were three stages of my analysis. I began by looking for occurrences of
“new recognition,” moments of group learning, in which it was recorded in the text
that group members developed a new recognition of inequity in student outcomes as
a result of their work. I placed the committees along a continuum on a Group
Learning Index at three general levels—high, medium, and low (see Figure 2). At
that point it became clear that there were two important dimensions at work in the
groups—data focus and experiential knowledge. Thus, in my second stage of
analysis, I analyzed the text along these two dimensions, determining the levels at
which the committees in each group type operated with respect to data focus and
experiential knowledge. In the third stage of analysis, I developed defining
characteristics of the three group types. Please note that one committee was excluded
from the study because during the period covered by this study the committee
members had no access to institutional data. The institution’s data system was being
converted to a new system, and no data were available for examination. Therefore, I
69
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
could not determine to what extent the committee was focused upon data because
they had no choice but to proceed without examining data.
Group Learning
Using the organizational learning literature as a point for departure, I began
the first stage of my analysis of the text by looking for instances in which it was
recorded that a group member acknowledged that he/she had learned, or newly
recognized, a piece of information that he/she had either not known before or that
he/she had suspected to be true but had now been confirmed by the institutional data
under examination. In the literature such a learning moment may be categorized
under the umbrella term of “organizational learning.” For the purposes of this study I
refer to what may typically be labeled “organizational learning” as group learning
since the groups, not the organizations of which they are a part, are the unit of
analysis. Group learning is defined as the new recognition that results from the
introduction of new information. It is the change in understanding that takes place
between the understanding held by the committee before the introduction of new
information (pre-condition) and the understanding held by the committee after the
introduction of new information (post-condition). (Please see Figure 1.) For example,
a committee may have held a common understanding that African American women
graduated at the lowest rates of all student groups in their institution. However, upon
examining the data, the committee learned that Latinos actually graduated at the
lowest rate of all groups for the past three years and that African American women’s
70
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
graduation rates have been rising steadily for the past five years. As a result of
examining the data, the committee’s understanding of graduation rates among ethnic
groups was changed.
The group learning, or new recognition of inequities in student outcomes,
that results from examining data may be characterized in two ways: (a) a new
understanding is developed, as in the example given above, or (b) the committee’s
previously held understanding (pre-condition) is confirmed by the new information
that is introduced. In fact, the committee may have discovered that the data verified
that African American women graduated at the lowest rates of all ethnic groups.
Both of these types of learning were labeled “new recognition” for the purposes of
analysis.
Learning has not taken place when the previously held understanding is
maintained by committee members despite the introduction of new information
which may contradict or disconfirm it, or introduce a potentially new understanding
to the committee. If the committee described above ignored or justified the data that
contradicted their understanding showing that Latinos graduated at lower rates than
African American women and resisted changing their understanding, no learning
would have taken place.
71
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Figure 1 : Group Learning
The change
between the
pre-and post
conditions is
group
learning.
Introduction o f new information
Precondition—previously held understanding
Post-condition —
a. new recognition
b. previous understanding confirmed by
new information
c. no learning—maintain previous
understanding despite contradicting
information
The committees experienced group learning at three levels—high, medium,
and low (please see Figure 2). In order to determine which committees experienced
each of these levels, I used a combination of three general criteria: (a) the amount of
output from Atlas.ti (Muhr, 1997) categorized as new recognitions for each
committee, (b) acknowledgement of and reflection on new recognitions by the
committee recorded in the text, and (c) committee response to new information.
First, I began by printing the text that I had categorized as an episode which reflected
new recognition using the qualitative software tool. I did not count the frequency of
occurrences of new recognition because I was not working with transcripts, and the
72
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
note takers could have easily missed occurrences and/or not recorded repeated new
recognitions. But, by examining the outputs from the Atlas.ti database (Muhr), I
could make an informed judgment regarding the overall amount of new recognitions
recorded in the text for each committee. Some committees had only two pages of
output that reflected new recognitions of inequity while others had up to five pages.
For those committees that experienced the highest levels of group learning, the
output contained more episodes of new recognition. The following excerpt from the
field notes provides an example of an episode that reflects group learning.
In brainstorming goals, Member A said that overall the university has a high
retention rate for Latinas, but a dismal retention rate for African-American
women.
“African- American women are the most at-risk at [our institution], I
want to know why there is this unexpected difference.”
Member B disagreed, “Actually African-American males are doing
worse.”
Member A was surprised and remarked, “Then I've been hearing
incorrect information” (Committee VII, 03-20-01).
Second, I looked for statements in the text which illustrated that committee
members reflected on the group learning which would indicate the development of a
new recognition or change in understanding. Thus, it was not only my interpretation
that the member had learned something new; a statement was recorded in the text in
which the member, him/herself, acknowledged that he/she had learned something
new. This is illustrated in the following excerpt from the text:
[The member] said he did not know about the fact that his institution serves
the ‘academic average’ until he looked at these survey results (Committee
XIV, 10-03-01).
73
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Third, I looked for what committee members did with this new information.
Did they accept the new recognition, or did they avoid or rationalize it in order to
maintain their own previous understanding? Did the new information lead to new
inquiries or deeper levels of inquiry into the data? The examination of new
information via institutional data resulted in the following response in one
committee, which indicated an acceptance of new information presented:
In looking at financial aid data, [the member] said, “I have never seen data
like this before... This project has given us opportunities to do new kinds of
thinking” (Committee XIV, 08-23-01).
74
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Figure 2: Group Learning Index
The Committees
High Level of Group
Learning about inequities
in educational outcomes
Medium Level of Group
Learning about
inequitable educational
outcomes
Low Level of Group
Learning about inequitable
educational outcomes
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
So what were the practices or characteristics of the committees at these three
levels of group learning and how did they differ? What were the characteristics and
practices of those committees that experienced high levels of group learning about
inequity in educational outcomes versus those that did not? I returned to the text and
went through it again, committee by committee, in chronological order looking for
important differences between the committees as related to group learning. There
were many characteristics and practices that did not seem to make a difference.
However, there were two dimensions which did seem important—data focus and
reliance on experiential knowledge.
Data Focus
The introduction of new information, in this case institutional data, was the
catalyst to bring about new recognition (see Figure 1 above). The committees were
encouraged by the project staff to utilize existing institutional data and examine them
in new ways—disaggregated by ethnic/racial groups—in order to learn about their
institution’s performance as illustrated by their students’ educational outcomes. For
example, institutional actors may traditionally study the annual graduation rates of
their students. For the purposes of this project, the committee would look at the
graduation rates of students who are African American, Latino, Asian American,
White, international, etc. rather than the aggregate number. Committees differed
76
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
along this dimension, acting with more or less focus upon the institutional data in
their work processes.
The ways in which the committee worked with and valued new information,
in this case institutional data, impacted the development or lack of development of
new recognitions about inequitable educational outcomes. One of the operating
assumptions of the project under study was that evidence could act as a powerful
tool. However, a committee’s processes and the extent to which it focused upon and
valued the data could enhance or diminish the power of evidence as a productive
tool. While one committee may have used institutional data to inform its work and
decisions, another committee may not have used data or ignored what the data may
have revealed. What were the characteristics and practices of those committees who
utilized and valued the use of institutional data versus those committees who did not?
In this second stage of analysis, I delineated three levels of data focus in the text—
high, medium, and low. A committee that operated with high levels of data focus
was defined as data driven.
High Data Focus
There were instances recorded in the text when committees used institutional
data to drive their work process. The data charts were the focal point of their
meetings, discussion hinged upon what the data revealed, and the data defined the
committee’s project focus. There were instances in which the data confirmed what
the committee suspected about particular phenomenon, and there were instances in
77
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
which the data disconfirmed their suspicions. For committees with high data focus
when suspicions were contradicted, a new recognition was acknowledged and the
data were accepted. These episodes served as further evidence that data were being
used and that data “trumped” pre-conceived understandings. In committees with high
levels of data focus, the examinations of the data also lead to the development of
new questions. Often these committees would follow up on these questions by
collecting and examining new data at a deeper level in search of answers.
Medium Data Focus
Groups that operated with medium levels of data focus were not categorized
as data driven. The text did reflect that these committees valued data and consulted
data during their work processes; however, the data were not the focus of the
committee’s work nor did the data define the direction of the work. These
committees appreciated data and recognized its value, yet did not function only with
respect to what the data revealed. In this category there were some instances of data
confirming or opposing previous understandings, but the data did not necessarily
trump the previous understanding. There were some new questions as a result of
examining data, but these were fewer in number and new questions would not
necessarily be pursued through further data examination.
Low Data Focus
Committees that operated with low levels of data focus were marked by the
lack of instances in which data were examined. More often there were examples of
78
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
data being deemed inaccessible or unreliable. And there were instances in which data
were not examined by the committee.
During this second stage of analysis, as I analyzed the text along the data
focus dimension, it became clear that there was another dimension at work in these
groups. None of the groups operated based upon data information alone. The groups,
to varying extents, brought their own experiential knowledge of the institution and its
operations to the table. This experiential knowledge also shaped the direction and
priorities of the groups.
Experiential Knowledge
I defined experiential knowledge as knowledge that committee members have
as a result of their personal experiences in the institution as well as what they have
accepted as true based on other institutional actors’ opinions, experiences, and
assertions. Knowledge that is considered to be common to institutional actors would
also be included. For example, it may be widely known and accepted on a university
campus that students who enter the institution as freshmen perform better
academically than students who transfer into the institution from community colleges
as juniors. This is a common assumption under which faculty and advisors operate.
It’s just something that people on campus “know” to be true. Many institutional
actors have personal stories that verify the validity of this common assumption,
reflecting upon transfer students who had really struggled through courses or
adjusting to the new campus.
79
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
This type of knowledge has been labeled and described in different ways by
various authors. Some authors refer to this type of knowledge as tacit knowledge,
which operates almost unconsciously. Lindblom and Cohen (1979) talk about
“ordinary knowledge” as opposed to knowledge derived from professional social
inquiry (research). Ordinary knowledge “does not owe its origin, testing, degree of
verification, truth status or currency to distinctive [professional social inquiry]
.. .techniques but rather to common sense, casual empiricism, or thoughtful
speculation and analysis” (p. 12). Whether such knowledge is true or false is
dependent upon whether others accept is as a basis for action. Polkinghorne (2000)
discusses the importance of “background knowledge” and practical knowledge in
psychological inquiry. He writes, “We ordinarily act based on our practical or know
how knowledge, and this knowledge usually functions tacitly in the background and
out of conscious awareness” (p. 460). Background knowledge and practical
knowledge are transmitted through one’s culture. Every organization has its own
culture and transmits an accumulation of customs, know how, and other background
knowledge to its members. The notion that transfer students do not perform as well
as “native,” freshmen students may be transmitted to institutional actors via the
culture of that university. I have chosen to use the term experiential knowledge for
this study in order to reflect that (a) the knowledge is clearly expressed to other
group members and is present in the text and is therefore made explicit rather than
tacit, and (b) the knowledge expressed is typically linked to the committee member’s
80
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
experiences and positions at the institution. Background knowledge is certainly at
work as well and is included within my notion of experiential knowledge.
There are also distinctions made in the literature between information and
knowledge, as noted in chapter two. Brown and Duguid (2000) make three
distinctions between the two. First, knowledge requires a knower. Information can be
treated as independent of a knower; it can be contained in a book or a database rather
than an individual’s mind. Knowledge, however, is bound to a person and his/her
understanding. Second, due to this connection to a knower, knowledge is difficult to
detach from the knower. While information is a “self contained substance” (p. 120)
in and of itself, knowledge is difficult to transfer from one knower to another. Third,
knowledge must be digested and absorbed. To have knowledge of something implies
that one also has understanding of it. One can have information without
understanding it. “Knowledge” in the term experiential knowledge refers to that
which is known and understood by group members about the institution. It is more
difficult to transmit and share among members than the institutional data—
information—that they examine. Sharing knowledge and group learning are social
and interpersonal acts; they require more than passing along a chart of data. They
require dialogue, explanation, testing of understanding, and interdependence.
All of the committee members associated with the project brought their own
experiential knowledge to the table. This reflected their understanding of the
institution, its priorities, the students, the institutional culture, the ways in which
81
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
people operate at the institution, etc. Committees relied upon the collective
experiential knowledge at different levels, which I defined again as high, medium
and low.
High Experiential Knowledge
Committees that used experiential knowledge to a high degree prioritized this
knowledge over other sources of information, namely the institutional data. A
committee member’s assertion or opinion about why Latinos don’t successfully
transfer to four-year colleges would be prioritized over data on Latino performance
in courses required for transfer. Experiential knowledge was used to justify or
explain what the data revealed by these groups. For example, a committee member
may assert that Latinos don’t want to transfer because their families don’t encourage
it and want students to stay close to home. When experiential knowledge is
prioritized, such a statement made by a committee member would be accepted as fact
and go unchallenged by other committee members. This inhibits further inquiry into
this area; the committee does not pursue other information, and, therefore, inquiry
stops. In these committees, experiential knowledge trumped the data.
Medium Experiential Knowledge
Committees that operated with experiential knowledge at a medium level
were defined as valuing and considering experiential knowledge, but they did not
prioritize this knowledge over other sources of information. Experiential knowledge
was considered a source of information among many. In these committees
82
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
experiential knowledge created inquiry; it acted as a facilitator for inquiry, drawing
the committee’s attention to particular areas over others. For example, a committee
member who works as a transfer counselor may contribute to the committee that in
her experience Latinos do not use the transfer center and personnel to the same
extent as students of other ethnic/racial groups. As a result of her sharing this
experiential knowledge, the committee decides to look at data which reflect the
transfer rates of Latino students versus students of other ethnic/racial groups. Thus,
her experiential knowledge led to a path of inquiry. This reflects that the committee
values experiential knowledge but then may rely upon other sources of information
to learn more. The committee does not rely solely upon the counselor’s ideas
regarding why this is the case but seeks out information that may reveal more about
the phenomenon.
Low Experiential Knowledge
Low levels of experiential knowledge would be reflected in the text by the
lack of instances in which experiential knowledge was shared among committee
members. These committees would not reflect upon or exchange ideas based upon
their own experiences in the institution or what they had learned about the
institution. These committees would rely upon other sources of information to
inform their work and shape the committee’s direction. Not surprisingly, as will be
discussed in chapter four, there were no committees that operated with low levels of
experiential knowledge.
83
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 7: Levels of Operation along two dimensions
Level of
Operation
Data focus Dimension Experiential Knowledge
Dimension
High Data-driven Experiential knowledge
prioritized; inhibits other
inquiry
Medium Data valued and consulted Experiential knowledge
valued; creates inquiry
Low Lack of data use Lack of experiential
knowledge shared
Group Types
In the third stage of analysis, after going through the text in its entirety using
the two dimensions, I went back to the category of new recognition and did an
additional level of analysis using the text categorized as new recognition, data focus,
and experiential knowledge for each committee, again in chronological order. Thus, I
could examine more closely the ways in which those committees with high levels of
new recognition of inequity in student outcomes focused upon data and used
experiential knowledge. As a result of this analysis, I defined the three levels of
group learning as three distinct group types, each of which practiced data focus and
used experiential knowledge in common ways, as shown in Table 8.
Table 8: Group Type Classifications
Committee Type Dimension Classification:
Data focus Exp Knowl
Defining
Characteristics
84
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
High Learning
High Medium Data driven;
experiential
knowledge creates
inquiry.
Medium
Learning
Medium High Data consulted;
experiential
knowledge
prioritized, inhibits
other inquiry
Medium Medium Data valued;
experiential
knowledge valued,
creates inquiry
Low Learning Low High Lack of data use;
experiential
knowledge
prioritized, inhibits
other inquiry
There were other possible group types not present in the text— for example,
high/high, low/medium, and low/low. I would not expect to find these types because
it did not seem logical that a committee would operate such that both data and
experiential knowledge were prioritized. The low/medium type was also not
expected because it did not seem possible for a committee to use experiential
knowledge to create inquiry but also be characterized by a lack of data use. How
might that inquiry be carried out if no data were accessible, reliable, or consulted?
Finally, low/low seemed unlikely because it was difficult to conceive of a committee
that operated without data and without any experiential knowledge shared among
members. A committee assembled for the purpose of identifying patterns of racially
stratified educational achievement would be compelled to seek out some sort of
85
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
information. While one could create a fictional description of such committees, they
did not seem realistic. Thus such illogical combinations were not expected nor were
they found.
By categorizing these 13 committees into three group types, I am imposing a
false order upon a disordered reality in order to make sense of it. In reality there are
13 group types. Within each of the three group types there is variation—some High
Learning groups were more “high learning” than others within that type. Some
committees really occupied the borderline between two of these group types. For
example, in one committee classified as a Medium Learning group there was one
member whom I would define as driven by the data while the other two members
relied primarily on their experiential knowledge. In the end these orientations
complemented one another such that the committee as a whole worked as a one
which consulted data and experienced medium group learning.
In the next chapter I describe the defining characteristics of each of the group
types as well as what was learned by each.
86
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Chapter Four: Results
This chapter presents the findings of the study and describes each of the three
group types identified in the text: (a) High Learning, (b) Medium Learning, and (c)
Low Learning groups. Within each of these types, the role of data focus, the role of
experiential knowledge, and what was learned by groups of that type are explained.
Within the role of data focus, the group type’s level of inquiry, orientation toward
the data, and engagement with the data are explored and described. I determined that
High Learning committees were those that experienced the highest levels of group
learning and that their prioritization of the data was a critical factor.
The inequity in educational outcomes for African American and Latino
students, as illustrated by gaps in graduation rates and similar indicators, represents a
problem in organizational performance for institutions of higher education. As
explained in chapter two, the problem of inequities in educational outcomes has not
traditionally been approached as a problem in institutional performance. Typically
research on minority student participation and achievement in higher education treats
the student as the unit of analysis—the students are the subject of study and
investigation and the targets for change, not the institution (Braxton, 2000). In this
study I approached the inequities in educational outcomes among students of
different ethnic/racial groups as a problem in institutional performance.
87
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
In view of the great inequities in educational outcomes that characterize the
collegiate experience of African American and Latino students, the question I
explored in this study was the following: In what ways might institutional data
compel institutional actors to develop new recognitions of the inequities in
educational outcomes? Simply put, I was interested in finding out in what ways the
examination of factual data on simple but specific indicators of educational
outcomes, such as choice of majors, grade point average (GPA), and completion of
remedial/developmental courses, could raise awareness about the presence of
inequity.
The notion that data might trigger the recognition that a problem in
organizational performance exists and that a response is required is based on theories
of organizational learning, which indicate that the presence of new ideas and
cultivating doubt in or questioning of current knowledge and practices can promote
learning. Knowledge sharing and knowledge generation in groups has also been
identified in the literature as a way in which institutional performance can be
improved and enhanced. Might institutional data prompt both new ideas and doubt in
current knowledge in these committees?
I found that data did, in fact, generate the development of new recognitions of
inequities in student outcomes for these committees. Data were the key in promoting
group learning by (a) generating new ideas among committee members and by (b)
creating doubt among committee members about their own experiential knowledge
88
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
of what is true about educational outcomes at their institution. The level of inquiry, a
focus on indicators of educational process instead of only on indicators of
educational outcomes, and the intensity of the engagement with the data contributed
to the level of group learning which took place, the content of what was actually
learned by the group, and the potential for developing an agenda for change.
However, experiential knowledge also played a role. There were no committees that
operated without experiential knowledge.
In the pages that follow I explicate each of the three group types identified in
the text: (a) High Learning, (b) Medium Learning, and (c) Low Learning. Please note
that one committee (Committee X) was excluded because this committee did not
have access to institutional data during the first year of the project, and therefore
only 13 of the original 14 committees are reflected in these findings.
Group Types
Using the dimensions of experiential knowledge and data focus, I identified
important distinguishing characteristics and practices among the committees who
experienced different levels of group learning and delineated them into three group
types: (a) High Learning, (b) Medium Learning, and (c) Low Learning. The
committees of the High Learning type experienced the highest level of group
learning, they considered their own experiential knowledge as hypotheses to be
proven, and they made judgments about inequities based on data. The data took
89
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
precedence over experiential knowledge. The important feature of what was learned
by the High Learning groups was that they identified potential points for intervention
to impact the identified inequitable educational outcomes. Those committees that
were of the Medium Learning type consulted institutional data but relied more
heavily upon their experiential knowledge. Medium Learning groups identified areas
in which there were inequitable educational outcomes, but unlike the High Learning
groups, they did not identify potential points for intervention. The committees
classified as Low Learning groups seemed to be spinning their wheels and
experienced very little, if any, group learning about inequities in educational
outcomes. These committees operated without examining data and relied primarily
on experiential knowledge.
Below I describe each of the three group types by explaining the role of data
focus, the role of experiential knowledge, and the group learning that took place—
what committee members learned about inequities in student outcomes. In discussing
the role of the data I will address the level of inquiry, whether indicators of
educational outcomes or processes were used, and the extent of the committee’s
engagement with the data. Indicators of educational outcomes are endpoints of a
student’s college career, such as graduation rates or GPA at the point of graduation
or the major he/she graduated in. Indicators of educational processes are those that
are captured at points within a student’s college career such as success in courses
90
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
that are required for particular majors or the act of switching majors or his/her GPA
at the end of each semester.
High Learning Groups
I classified committees that experienced high levels of group learning in
which the role of the data was dominant and experiential knowledge facilitated the
committee’s inquiry as High Learning groups. There were several characteristics
which defined the five committees of this type. First, for the most part these
committees seemed to begin with a sense of the problem of inequities in educational
outcomes for African American and Latino students, and many also had a sense of
the institution’s responsibility in resolving this problem. This seemed to already be
part of their experiential knowledge and beliefs about the institution, and, as a result,
there seemed to be little resistance to the idea of inequities in educational outcomes.
Of course, this varied across the five committees. The committees used their
experiential knowledge of the institution, its students, and its priorities to facilitate
their inquiry into the data. Committee members seemed to have a fairly good sense
of where inequities existed in the institution and were not generally shocked by what
the data revealed, although they did learn new things by examining the data. These
committees prioritized the data over other sources of information, like their own
experiential knowledge, and used the data as the foci of their meetings and work
together.
91
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
New ideas were generated from examining and talking about the data. And,
in some cases, doubt about existing experiential knowledge and the practices of the
institution was raised as a result of examining data. For example, mythologies, the
campus lore that was considered common campus knowledge, were contradicted or
called into question. What is noteworthy about this in High Learning groups was that
the committee members were willing to question such campus mythologies and give
precedence to the data over their experiential knowledge, even when that knowledge
was contradicted.
The High Learning groups experienced more group learning than committees
of other types. However, what was important about the learning of these groups was
not necessarily that they focused on more important or unique areas of inequity—
committees in both the High Learning and Medium Learning groups identified
quantitative reasoning and gateway courses6 as problem areas, for example. High
Learning groups learned about areas of inequitable educational outcomes at a
different level by approaching the data with a second order level of inquiry, digging
deeper into the data rather than maintaining a telescopic perspective, and attending to
indicators within the educational process rather than only indicators of outcomes. As
6 A gateway course is one that acts as a prerequisite for particular majors or programs or a generally
required course for graduation, and a student’s success or failure in such a course might limit his/her
options and/or the ability to graduate. For example, calculus is a pre-requisite for engineering. If a
student does not pass calculus, he/she cannot declare engineering as his/her major.
92
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
a result, High Learning groups identified potential points for intervention rather than
only indicators of inequitable educational outcomes.
To illustrate the characteristics and practices of High Learning groups,
described briefly above, I will use the text associated with one of the five committees
of this type, Committee HI. This committee is representative of the experiences of all
five committees and will help to paint a picture of how High Learning groups
operated. I will begin by describing the role of data focus in High Learning groups.
The Role of Data Focus
The institutional data played a dominant role in this committee and were the
focus of this committee from the beginning of the project. As early as their second
meeting, Committee III dedicated its meeting time to discussing pages and pages of
data provided by one of their colleagues. These data were used to determine
inequities in the areas the committee was concerned about. Data were provided for
each of the goal statements that the committee had written and included the
following, all of which were disaggregated by ethnicity: comparison of the applicant
pool to those admitted into an impacted program; the passing rates in the top 100
enrolled courses; the ethnicity of the faculty by college; and the ethnicity of students
who leave in good standing versus those who leave in academic jeopardy over time.
By its third meeting this committee added two new members whose job it was to
work with the data exclusively. Each committee meeting revolved around the data.
93
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The committee member who was from the Institutional Research office (IR member)
would bring copies of these many data sets. She would give brief explanations of
what was included and not included in each data set. Then the committee spent a few
minutes in silence, looking over the data. Occasionally clarifying questions to the IR
member would break the silence. When all present were satisfied with their
individual understanding of the data, they engaged in discussion about it—what the
data said to each of them, what the data might mean to the institutional community,
how the data should shape their work and goal statements for the project, etc.
In examining the data, this committee engaged in a second order level of
inquiry, developing microscopic measures. They were not satisfied with broad,
telescopic indicators, and they dug deeper into the data. For example, the institution
had previously generated data on an annual basis which showed the top 100 enrolled
courses (those courses with the highest number of students enrolled) and the passing
rates for each of the courses. However, these data did not identify differences in
achievement between students of different ethnic/racial groups nor where African
American and Latino students faced the greatest challenges. Therefore, using this
data set, the IR member of the committee and her assistant created a table which
showed the courses among the top 100 which had overall passing rates below 75
percent. They then disaggregated these passing rates by ethnicity and listed those
courses in which students experienced the highest failure rates. Ethnic/racial groups
who achieved passing rates above the average for the courses of that type appeared
94
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
in blue. And those ethnic/racial groups who achieved passing rates below the average
appeared in red. The observer recorded her reflection that this seemingly simple table
told a shocking story. It became clear that in almost all of the courses, which were
primarily mathematics related, African American and Latino students appeared in
red—they achieved passing rates below the average—while Asian American and
White students appeared almost exclusively in blue—above the average passing
rates. The observer reported:
The color code paints a very grim picture for African American and Latino
students (Committee III, 05-30-01).
The data table described above illustrates the second order level of inquiry
engaged in by High Learning groups. This committee identified individual courses,
as well as a theme in the discipline reflected in these courses (mathematics), where
inequities lay. They now know much more about the state of inequity than if they
had maintained a more telescopic perspective through which they may have only
learned that 50 of the top 100 enrolled courses had high failure rates.
The institution also routinely collected data about the diversity of its faculty.
This would be considered a broad or telescopic measure because it gave an
overarching picture of faculty diversity at the institution. However, this committee
again mined the data at a deeper level and compared the diversity of the faculty in
each college with diversity of the students in that college which paints a more
complete picture of faculty diversity at the institution and the extent to which it
95
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
reflects the diversity of the student body. The committee felt that looking at the data
in this way helped to define their own ideas with regard to the project and “drove
home” what they wanted to accomplish through the project. The implication was that
this might not have happened had they not looked at the data in this manner.
[The committee member] handed out new data about faculty diversity by
college juxtaposed with student diversity by college. He provided a little
explanation about this. Everyone looked at the data in silence for a few
minutes.
[Another member] said, “This drives home what we’re trying to do.”
We all agreed and looked at the data a bit more (Committee III, 05-30-01).
Another important characteristic of the data examination by High Learning
groups that was exemplified by this committee was that they tended to look at
indicators which reflected points in the educational process rather than only
indicators of inequitable outcomes. For example, this committee examined data
which showed that students who were accepted into an impacted program7 on their
campus were predominantly White and Asian American. This disparity in enrollment
indicates an inequitable educational outcome. However, this committee then
examined the data related to the applicant pool for the program and realized that
students of different ethnic/racial groups were, in fact, equitably represented in the
applicant pool. Therefore, they concluded that there was a diverse pool of students to
draw from, and they were able to identify more specifically what might contribute to
the lack of diversity in the program. They considered what it takes to be admitted
7 An impacted program is one which is considered to be elite and is so over-subscribed that students
must apply to be admitted rather than just declaring it as their major program of study.
96
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
into the program and in what areas African American and Latino students were not
performing as well as Whites and Asian Americans.
In this case it is not only whether students pass the prerequisite courses, but
whether they achieve within, perhaps, the top 10% of the class... one
measure might be to look at the top 10% of achievers in the feeder courses
into [the program], disaggregated by ethnicity. Or, they could look at the
grade distribution in general in the prerequisite courses. This may serve as an
intermediary measure that could reveal an earlier point for intervention
(Committee III, 04-19-01).
The committee mapped backward from the point of admission to the
processes that preceded and tried to identify intermediary measures that might
indicate early on whether a student was on a path for admission or not. As a result of
examining the data in this manner—at a second order level and identifying process
indicators rather than only outcomes indicators—new questions came up. The
committee asked themselves questions they had not asked before like: “What does it
take to be admitted to this program? Why is there not proportional representation of
the applicant pool among those admitted? At what point might we intervene that
would make a difference for African American and Latino students?” The committee
generated new ideas about the program. For instance, the committee thought about
other criteria that could be used for admission to the impacted program that may
serve to show the strengths of students beyond their GPA. And, in looking at the top
enrolled courses with high failure rates disaggregated by ethnicity, the following idea
was generated:
97
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Two of the courses in the list of high fail courses for African American and
Latino students were Econ 101 and 102. These would serve as gatekeepers
for students who wanted to go into the school of business. They are
prerequisites for any business major.
[The observer] asked if there were any math prerequisites for these
econ classes. The group said they didn’t think so. Then [one member] looked
in the course catalogue to make sure and it was confirmed that there were no
math prerequisites (Committee III, 04-19-01).
The implication was that without math prerequisites for the course, students might be
enrolling in courses that they are unprepared for.
One member, impressed with what they had found by looking at the top
enrolled courses in this manner, suggested examining other important courses in the
same way.
[The member] suggested that perhaps they should analyze the required
General Education courses with the highest enrollments in the same way that
these “gateway” courses had been analyzed. Everyone agreed that would be a
good idea (Committee III, 06-18-01).
It seemed that this committee began the project with an orientation towards
doubting the common campus knowledge and the institution’s practices with regard
to African American and Latino students. One committee member was reported as
having said the following in the first meeting of the committee:
[The member] said that the institution is made up of 80% non-White students.
She wanted to explore what that means at their institution.
“Yes, we are very diverse, but what does that mean to us as an
institution? I want to be able to say, ‘Yes, we have [these diverse] students
and we’re the best at educating and graduating these students.’ But I can’t say
that now... What value do we add to the students while they are here? What
are they prepared to do when they leave” (Committee III, 04-02-01)?
98
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
This member did not take for granted that they do, in fact, add value to students
while they are at the institution. She doubted that they serve their students as well as
people on campus seem to think that they do.
This same committee member’s doubt was newly raised when the committee
examined data which showed that a larger than expected number of students left the
institution in good academic standing, and that these students were predominantly
White and Asian American. This was in contrast to those who left in academic
jeopardy who were primarily African American and Latino.
The [leaving] rates were disaggregated into 4... categories: (1) those who
persisted in good standing, (2) those who persisted but were on academic
probation, (3) those who did not persist who were in good standing, and (4)
those who did not persist who had been on academic probation. What she
was surprised by and pointed out was that there was a significant number
who left who were in good standing, even among transfer students, and they
tended to be White or Asian American (Committee III, 04-19-01).
Previous to examining these data, the committee members believed the common
campus understanding that students who left the institution did so primarily because
they were in academic jeopardy, primarily African American and Latino students,
and that those in good academic standing would persist. By examining the data the
committee now has reason to doubt this experiential knowledge and consider
alternative understandings—the institution loses all types of students and African
American and Latino students in academic jeopardy are not the only reason
persistence rates are low. The data also revealed that African American and Latino
students were more likely to be in academic jeopardy.
99
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Finally, like the other High Learning groups, this committee sustained their
engagement with the data beyond their committee meetings. Between the committee
meetings the IR member met with her two assistants and directed them as to how
they could further refine and define the data such that it could reflect the
phenomenon under study most accurately. For example, she explained to the rest of
the committee how they would refine the data on the gateway courses with high
failure rates:
[My assistant] is going to rework the data on the gateway courses so that only
those students who make it to the end of the course are included rather than
including those who withdrew from the class (Committee HI, 06-18-01).
Spending time refining the data set in this way is evident of sustained engagement
with the data. They continued to think about the data and how to make the data set
more reflective of the problem they were seeking to define. The committee also
sought out other institutional actors to share and discuss their work with such as the
provost, the Executive Council, the president, and the Dean of Graduate Studies.
They even invited the Dean of Graduate Studies to join in one of their meetings to
discuss data that they had examined that were relevant to his work.
The committee members also used the knowledge and information they had
gained by being part of this committee in their work for other committees such as
program review and the strategic planning committee. This was done, in part, to help
position themselves to bring about results from their involvement in this project.
However, there was also natural spillover as a result of becoming so engaged with
100
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
the institutional data. For example, one committee member reflected on his
participation in the strategic planning committee when they were discussing faculty
diversity.
In a planning discussion faculty discussed the ‘mix of faculty’ as full-time
versus part-time. I said, hey, wait a minute, faculty ethnic diversity is one of
our weaknesses, let’s talk about that, and the rest of the planning group was
silent (Committee III, 06-08-01).
He wanted to discuss the ethnic/racial diversity of the faculty, which had been
identified as a problem area through examination of the data and challenged the rest
of the committee to consider it. This example illustrates that this committee did not
confine what they learned through examination of the data to the work of this project
alone. The data raised their awareness about particular areas, and they reflected on
this, further defined the areas, and shared their new recognitions with others outside
the committee meetings.
The Role of Experiential Knowledge
'Experiential knowledge also played a role in the development of new
recognitions. Committees of this type were characterized by using their experiential
knowledge of known problem areas and institutional priorities in order to facilitate
and shape their examination of the data. This knowledge helped them to choose
which data to examine out of the abundance of institutional data available. Being
cognizant of institutional interests and priorities helped to prevent the committees
from pursuing what may be deemed lost causes. For example, a committee might
101
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
intentionally study an area of interest to the president and strategic planning
committee in order to maximize the likelihood that they will see results from their
hard work.
Committee HI, used to illustrate the role of data in High Learning groups,
will also be used to illustrate the role of experiential knowledge in these groups. This
committee had fairly certain ideas about where the problem areas lay for inequitable
educational outcomes as well as which of these areas others in the institution would
be interested in as a result of their own experiences at the institution. The committee
members knew, for example, at the outset of the project that there were two issues at
the forefront of the president’s and other important institutional actors’ minds—
disenrollment8 of students who do not succeed in the remedial course sequence
within one year, and the success of students in mathematics-related courses. This
knowledge helped them to determine which areas of data to examine, one of which
was the failure rates in the top 100 enrolled courses, knowing that 80 percent of first
time freshmen require remediation. After examining the failure rates for the top 100
courses disaggregated by ethnicity, they decided to focus on those courses that were
related to quantitative reasoning, including remedial math sequences, which was in
line with the interests of the president and others.
8 “Disenrollment” occurs as a result of an Executive Order from the Board of Regents which states
that any student who does not successfully remediate within one academic year—has not completed
all remedial coursework required such that he/she can enroll in college-level courses—will be
academically dismissed from the institution.
102
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The experiential knowledge related to institutional priorities was important to
the committee because they felt that if they aligned their work with these priorities,
then their work might have a longer lasting effect. Thus, they used their knowledge
of the strategic planning process, which was currently underway, to help shape their
inquiry as well. The committee did not pursue any data which could not logically fit
into one of the strategic planning initiatives or objectives. By doing so this
committee aimed to place the issue of inequitable educational outcomes at the
forefront of the university’s agenda rather than allowing it to remain an “ad hoc” or
committee issue. If this were to become part of the strategic plan, it would become a
university issue. The following excerpt from the text reflects an exchange about
aligning this project with the strategic plan.
[The member] said that he wanted to be sure to think consciously about
whether there are any other of their current project goals that needed to get
worked into the strategic plan.
[Another member] said maybe the investigation into math could be
worked in because it fits under a strategic planning goal regarding
strengthening curriculum and instruction. She said there will be an overall
retention objective which includes both persistence and graduation rates.
Tactics for the objective may include reducing time to degree. “Our work on
this project falls under that.” The strategic planning committee wants to see
an increase in the average unit load, which is one of the current project goals.
[The member] said, “I want to get integrated into strategic planning as
much as possible” (Committee HI, 07-12-01).
The committee went through their own goal areas for this project one by one, testing
to see whether there was a natural fit with the strategic plan goals and objectives.
They looked for ways in which their own goals could be “worked in” to the strategic
103
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
plan. In the passage quoted above they found that their math goal fit with a strategic
planning goal around strengthening the curriculum, and their goal to reduce time-to-
degree fit well with the strategic planning goal regarding increasing the average unit
load for students per semester. Thus, they retained these goals.
The committee examined the diversity of the impacted program described
earlier because it was going through program review, when examination of the
program and suggestions for improvement were happening anyway. In this way the
program was ripe for investigation. The committee’s recommendations could simply
be added to what was already happening.
The committee members also had their own ideas and interests at the start of
the project. At the first meeting each member contributed what he/she had found
intriguing about the project description that each had received.
[One member] said he was drawn to the idea presented in the project
description of ‘producing leaders not just survivors’ among diverse students.
He wants to do more than just get students through, and he wants to create
ways to do it on a systematic basis. He also liked the project’s definition of
access in the description, particularly the idea of access to majors. He said
that some of their majors are almost homogeneous ethnically and there is
something fundamentally wrong with that (Committee III, 04-02-01).
The interests of this member, as well as the interests of the other members, shaped
and facilitated their inquiry into the data. For example, the member who was
interested in producing “leaders not just survivors” suggested that the committee
look at the ethnic/racial composition of the 23 honor societies on campus to find out
if African American and Latino students were proportionately represented. He and
104
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
another member reflected upon their own experiential knowledge related to honor
societies and their understanding that African American and Latino students don’t
avail themselves of this opportunity even when it’s presented to them.
[The member] pointed out that students are naive about why it is important to
be in an honor society, at which point [another member who is Latina herself]
said, “I can tell you that is exactly true.” And she went on to tell the group
about having herself been eligible for membership in an honor society.
She said, “I got the letter and I just tossed it.” She went on to say that
later, as a result of her stepmother, she became a member of it and “Phi Beta
Kappa has been a major part of who I am. But at the time it meant nothing to
me. Lots of our students don’t realize the long term implications”
(Committee III, 05-30-01).
As a result of their personal interests in promoting excellence and leadership
opportunities among African American and Latino students, and their personal
experiences in honor societies, the committee decided to pursue data which reflected
the ethnicity of those who were members of honor societies as well as the
requirements for becoming a member. This facilitated and enhanced their exploration
into institutional data for the purposes of discovering areas of inequitable educational
outcomes.
What Was Learned
The High Learning groups experienced more group learning as well as a
different quality of learning than the other two group types. Again, the High
Learning groups did not necessarily identify a greater number of areas, different
areas, or more critical areas of inequitable educational outcomes than those of the
Medium Learning groups. In this section I will use all five High Learning groups to
105
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
describe what was learned. There were common themes regarding what was learned
among many of the 13 committees being considered here like high failure rates in
mathematics-related courses for African American and Latino students. What was
remarkable about what was learned by the High Learning groups was that by
engaging in a second order level of inquiry and attending to indicators of educational
process rather than only outcomes, these groups identified potential points for
intervention that could impact the inequity in educational outcomes. These groups
went beyond identifying problem afeas and identified specific problems like
individual courses with high failure rates.
For example, one of the committees was interested in access to different
majors and wanted to know whether African American and Latino students were
proportionately represented in those majors that lead to careers that are in high
demand, like engineering and computer science. Or, are these students over
represented in particular majors? Initially the committee looked at graduation rates
by major, disaggregated by ethnicity. However, the committee learned more by
tracking cohorts of students from their original major to the major they graduated in.
The IR member of this committee proposed the following:
He said that they “can track from entry major to graduating major.” This
might show if students intended on majoring in one major then changed their
mind later on. If many students sign up for more economically advantageous
majors, like engineering, but then graduate with majors in the humanities,
this might give the committee an idea about access to certain majors for
African American and Latino students (Committee VII, 03-20-01).
106
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
By investigating the movement of African American and Latino students who
intended to major in engineering to majors like communications, this committee
might identify particular courses or prerequisites that act as hurdles for students and
prevent them from pursuing engineering. This committee learned more by engaging
in a second order level of inquiry and looking at indicators that reflect educational
process rather than educational outcomes—they peeled away the layers, learning
more with each layer. They could learn at what point African American and Latino
students leave particular majors and intervene rather than only learning that these
students are underrepresented among the graduates of the majors, when it is too late
to intervene.
By looking at the success rates of students in General Education (GE) math
who had completed remedial math, another committee learned that African
American and Latino students fail GE math at high rates despite the preparation
remedial math was supposed to have provided. The majority of students in remedial
math were African American and Latino. This caused the committee members to ask
themselves the following:
Why isn't there a smooth transition from remedial math to GE math? Is there
a curriculum disconnect? A conceptual disconnect? We've got the basic data.
It may be worth looking at more closely (Committee II, 09-19-01).
This committee learned that the remedial course sequence did not prepare African
American and Latino students properly such that they could succeed in GE math.
Again, they learned more by following these indicators from one step to the next in
107
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
the math educational process instead of only looking at success in remedial math or
only success in GE math. Making the connection between the two implied a point for
intervention—aligning the curricula of remedial and GE math such that students
receive the necessary preparation. As a result, this committee knew more about this
area of inequity than other groups that did not take such a microscopic perspective
on the institutional data.
Medium Learning Groups
In contrast to High Learning groups, the Medium Learning groups
experienced medium levels of learning, and data focus and experiential knowledge
played different roles. The data were not considered preeminent sources of
information. Experiential knowledge was relied upon to a greater extent than data.
Medium Learning groups consulted data, but they seemed to operate under the
assumption that the data would confirm their experiential knowledge—as though
they already knew what the state of inequity was at their institutions. When they did
examine data, these groups stayed at a fairly broad level, used telescopic measures,
tended to look primarily at outcomes indicators, and maintained limited engagement
with the data. Because these groups did not delve deeply into the data and tended to
look at more traditional, broad measures, they seemed to maintain the status quo and
learned less about inequitable educational outcomes than High Learning groups. As a
result of the first year of the project they might have learned that African Americans
108
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
had lower GPAs than any other group at the point of graduation, but they didn’t
know more than that. For example, they didn’t know if the GPA gap was worst than
when the students started college or at what point the GPA gap began to widen in a
student’s career. There was no implied point for intervention without digging deeper
into the data.
Experiential knowledge was prioritized over the data. These groups treated
statements made about inequitable outcomes based on personal experiences as fact,
as if this knowledge were data. The members of these committees held onto their
own assumptions and explanations of inequitable educational outcomes despite a
lack of supporting evidence and even when there was disconfirming evidence
presented. While the High Learning groups seemed to begin the project with a sense
of the inequitable educational outcomes for African American and Latino students
and seemed to accept, at least to some extent, that the institution should take some
responsibility for resolving these inequities, the Medium Learning groups seemed to
resist the notion of institutional responsibility. They did not necessarily resist the
idea that inequitable educational outcomes existed, but when explaining and
rationalizing the inequities found in the data, the members of Medium Learning
groups tended to attribute the problem to students and their lack of motivation, poor
academic preparation in the K-12 education system, competing demands from family
and job responsibilities, and family structures that tended to inhibit them
academically—all of which the college or university had no control over, and
109
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
therefore no responsibility for. Members of these committees also wanted to avoid
examining or sharing data that might make their institution “look bad.”
There were fewer new ideas presented and less cultivation of doubt because
these groups tended to rely on what they already “knew” to be true about the
institution and its students. Relying so heavily upon existing experiential knowledge
actually gave more credence to this knowledge, rather than calling it into question or
creating doubt in it. The Medium Learning groups based their work primarily on
experiential knowledge, which is the very knowledge that High Learning groups
tended to doubt.
In the following sections I will describe the role of the data focus and the role
of experiential knowledge in Medium Learning groups. I have chosen to use one
committee to represent the role of data focus and a second committee to represent
the role of experiential knowledge because I think that is the way in which I can best
illustrate the characteristics and practices of all six committees that were determined
to be of this type.
The Role of Data Focus
Committee IX did not examine data as a group until the second to last
meeting of the first year of the project. A good deal of their meeting time over the
course of the year was devoted to discussing why they should adopt the goal areas of
another diversity-related project. They also spent their time discussing what other
administrators on campus would think of this diversity project as well as other
110
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
diversity-related issues. After reading the text in its entirety, it almost seemed as
though there was some resistance to looking at data because (a) the data might show
that they should concentrate on areas other than the ones they had already chosen,
and/or (b) because the types of data that the project staff suggested that they examine
(graduation rates and GPA disaggregated by ethnicity) would not be of interest to
administrators on campus. For example, the IR member of this committee explained
that GPA issues must not be important because he had never been asked about them.
Issues like high GPA are not important. No one has ever said, ‘Go look at
GPAs’ (Committee IX, 10-10-01).
This committee decided on its area of focus before ever examining data—an
elite academic program in which African American and Latino students were
underrepresented and which was the subject of another grant.
[The member] said that they know minority participation is low in this elite
academic program because it is part of the other grant (Committee IX, 06-07-
01).
Even during the meeting in which they examined other data, which revealed
alternative areas of concern, the committee maintained its focus on this elite
academic program.
One committee member reiterated to the others, “Whatever we are doing for
the other grant, we will do for this project” (Committee IX, 10-10-01)
When discussing this elite academic program, the committee focused on
participation rates, which they already knew as a result of the other grant, and they
did not dig deeper into the data nor did they look at indicators that would lead to a
111
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
student participating or not participating in this program—GPA, for example. They
did not ask, “What does it take to participate in this program? Why are Asian
American and White students participating at such high rates?” So, even though they
were so focused on this particular program, they did not learn as much about it
because they did not mine the data at a second order level of inquiry; they only
talked about participation rates. And they did not examine the educational processes
that might lead to this outcome—low participation among African American and
Latino students.
When they examined institutional data more broadly at the suggestion of the
project staff, the committee discovered problems with differential GPA achievement
among different ethnic/racial groups and inequitable representation of Latino and
African American students among those students with high GPAs in particular. It
was even implied through the conversation that these problem areas might be
considered important by the institution. However, at the first meeting they indicated
they would align their goals for this project with those of another diversity project on
campus, which did not focus on equity in GPA achievement or majors.
[The observer] asked [the member] if they had looked at GPAs before. He
said that there are GPA gaps between ethnic groups, particularly for African
Americans who have lower GPAs and also have a lower graduation rate. He
went on to explain that Latinos have a smaller GPA gap and they have very
good graduation rates. The committee decided not to include graduation rates
in their project because it is not part of their grant initiatives (Committee IX,
07-09-01).
112
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Even after looking at the data, they did not waver from this position. One of the
reasons cited for sticking with their original goals was that the administrative
support, and therefore financial support, was already in place. It was reported that,
[The member] wanted the items on the scorecard to reflect their [funded]
projects. The two goals they selected were directly related to other grants that
they would need to assess. He added that these goals would be supported by
the administration financially and otherwise (Committee IX, 06-07-01).
In the High Learning groups, committee members used their knowledge of what
might be supported by the administration to facilitate their inquiry—giving them
some direction as to which data might be deemed important. In this case, the
knowledge of what might be supported by the administration superseded what was
found in the data.
This committee also maintained limited engagement with the data. Unlike the
High Learning groups, this Medium Learning group did not refer to instances in
which they examined the data outside the committee’s meetings. In fact, the IR
member of the committee had given all of the members a packet of data to examine.
The project staff assumed that since everyone had possession of the packet for at
least a week, they must have looked at it prior to the meeting. This was not the case.
One of the committee members commented that he had received the packet in
another meeting, but that they had never gone over it.
We never went over the figures in the meeting. It was just handed out. The
IR person generated it for a meeting of the campus-wide diversity committee,
but we did not look at it (Committee IX, 10-10-01).
113
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Not only was there not sustained engagement with the data, spending time outside
the committee meetings examining them and reflecting upon them, but when the data
were presented to another diversity-related committee, the committee did not look at
them. It was as though handing out the data was a formality. This served as another
indication that the data were not considered an important source of data—they did
not even take the time to look at them.
There were few new ideas present in this committee’s work, and there were
few, if any, questions raised regarding the experiential knowledge of the committee
members or the practices of the institution as a result of examining data. Looking at
graduation rates and GPA of graduating seniors disaggregated by ethnicity did seem
to spark some interest and provided new information, but this came very late in the
first year of the project and did not seem to detract from the original focus of the
project which they never doubted—the participation rates of African American and
Latino students in an elite academic program.
The Role of Experiential Knowledge
To illustrate the role of experiential knowledge in Medium Learning groups I
will use the text associated with another committee, Committee VI. Of all the
Medium Learning groups, this committee was the one that attended most closely to
the data. However, even in this committee experiential knowledge overpowered the
data and inhibited the inquiry of the committee.
114
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The committee looked at data related to different issues that seemed to
differentially impact African American and Latino students who wanted to transfer
from this community college to a four-year institution. These issues included
financial aid, English and math courses required for transfer, and support services
available to students that might aid them in the transfer process. It seemed that often
an explanation or rationalization was provided regarding what the data revealed, and
the reason given for the inequitable educational outcome had to do with student
characteristics rather than the institution. For example, the data showed that Latinos
transferred at lower rates than White students, and even when they did transfer it
tended to be to the lower tier state university nearby rather than a more elite state
institution further away. The following explanation was provided:
In our discussion of access to four-year institutions, [the member] stated that
there might be a number of reasons that more of their students transfer to the
CSU rather than the UC system. She explained that location might be one of
them. Possibly because CSU X is closer to [their geographic area] than UC
Y, students might want to transfer to some place closer to home.
[Another member] explained that this may be a bigger issue for
Latino students because of pressure from family to remain close to home
(Committee VI, 03-27-01).
The low transfer rates of Latino students to University B were attributed to family
pressure. And, this statement went unchallenged by the other committee members
and seemed to be accepted as fact. There were no arguments made nor any
alternative explanations provided.
115
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Another instance in which individual committee member’s assertions seemed
to be accepted as fact occurred when one committee member explained why some
African American and Latino students don’t apply for financial aid:
Students have a ‘gimme’ attitude. They want free money [financial aid] but
they do not want to pay it back.
[The member] also stated that he works with a lot of athletes and
many of them do not want to go through the process of applying for financial
aid. “My biggest headache is the athletes. They don’t want to do anything”
(Committee VI, 07-10-01).
When such assertions go unchallenged and are accepted as fact, inquiry into these
areas is inhibited. Why would the committee continue to pursue an inquiry into why
particular groups of students don’t apply for financial aid when committee members
have already provided answers that are accepted? Two members of this committee
volunteered to gather some information about financial aid, but this did not happen
during the first year of the project.
No new ideas were generated because the committee was holding fast to
existing ideas based on experiential knowledge. And, the committee did not raise any
doubts about the assertions described above that might call experiential knowledge
into question. The practices of the institution were certainly not called into question
or doubted by this group. Attributions for student failure were made to the students,
themselves. For example, that the students don’t even use the support services the
institution provides to them, as was recorded in the following passage.
[The member] stated that they do offer various programs such as tutoring,
counseling, etc. but that many students did not take advantage of these
116
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
services. She explained that this might be due to the fact that many students
are not aware that these services exist.
[Another member] stated that the fact that underrepresented students
do not fully utilize these services could also be due to the negative attitude
that many of these students have toward school in general. He explained that
many of the students he has worked with do not value education intrinsically
but rather as a means to acquire a good-paying job. He added that some
students are embarrassed to use these services while others do not see the
relevance of such services to their educational success (Committee VI, 03-27-
01).
The reason that students don’t use support services is not due to the institution and its
practices, like poor advertising and lack of programs that target students in academic
trouble. According to this committee member, students don’t use these services
because of their negative attitudes. By relying so much on experiential knowledge
and allowing it to take precedence over what the data might show, this knowledge is
given greater credibility and authority and deters the committee from learning new
things from the data.
What Was Learned
Ultimately, committees of this type functioned to maintain the status quo by
relying upon existing experiential knowledge and explaining away or rationalizing
data that might introduce new ideas or cast doubt upon experiential knowledge and
the practices of the institution. Again I will use all groups of this type to discuss what
they learned.
Themes of the group learning which occurred in Medium Learning groups
included recognition that GPA was differentiated by ethnic/racial group with African
117
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
American and Latino students achieving lower GPAs than all other groups; previous
institutional studies had shown financial aid became a barrier for African American
and Latino students to persist and attain a four-year degree; African American and
Latino students participate in an elite academic program at disproportionately low
rates; Latinos are not equitably represented across all majors; and remedial course
sequences are a barrier for African American and Latino students because they
experience measurably lower success rates than their White and Asian American
peers.
While these are important indicators to become aware of, in contrast to the
High Learning groups, the learning stopped there. The Medium Learning groups
identified important problem areas, but because they did not examine the data at a
more microscopic level, focused on outcome indicators, and maintained limited
engagement with the data, they did not identify potential intervention points that
might impact these inequitable educational outcomes. Because they relied more
heavily upon their experiential knowledge than institutional data, their inquiry was
inhibited and less likely to open new areas of information and knowledge to the
committee.
Low Learning Groups
Low Learning groups were distinguished by the fact that they experienced
little, if any, group learning about inequitable educational outcomes at their
118
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
institutions. Unlike High and Medium Learning groups, Low Learning groups never
examined data as a group during the first year of the project. Experiential knowledge
far outweighed data focus because it was really the only source of information that
they utilized when discussing inequitable educational outcomes. And, what is
noteworthy is that even the experiential knowledge that was shared among
committee members did not have any real focus or direction.
In reading the text associated with one of the two committees of this type, it
seemed as though the same meeting occurred again and again. The committee
consistently revisited the same topics, describing and discussing other diversity-
related programs that were being developed on campus. No connections were made
to their work on this project; they just seemed to be spinning their wheels. This same
pattern was followed by the other committee, discussing whatever problems came to
mind, whether they were diversity-related or not.
Below I describe the role of data focus and the role of experiential knowledge
in Low Learning groups. I will use a different committee to illustrate each of these
and the characteristics and practices of the two committees that I classified as this
type of learning group.
The Role of Data Focus
The role of data in Low Learning groups was almost non-existent. These
committees were characterized by a lack of data examination. Due to this lack of
data examination I cannot speak to their level of inquiry, whether they examined
119
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
indicators of educational process or outcome, or their engagement with the data
because none of these were present in the text.
In Committee I, ironically, there were multiple members of the committee
who represented the Institutional Research office. This committee relied upon the
data knowledge held in the IR members’ heads, but they never examined data as a
committee. They used the IR members’ knowledge of data for means of discussion,
not true data analysis. New questions came up in reaction to this data knowledge that
was shared, but they seem to have been treated as rhetorical questions because they
were never addressed and the committee members never sought out answers.
Committee members brought up problems in the advising system, tracking students
who attended tutoring, and migration from math-oriented majors to social science
majors as problems that could be addressed in the institution’s project goals. But
none of these areas was ever pursued. They never examined or even asked for data in
these areas. The IR members wrote off several data sources under discussion, like
information about particular majors and advising, because they were “bad,” and they
talked about new data that did not yet exist. The following passage from the text
illustrates the undirected brainstorming that tended to go on in this group in meeting
after meeting, as well as the problems they referred to when the project staff
encouraged the committee to collect data in the problem areas they had expressed
interest in pursuing.
120
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
[One committee member] said, “So what I hear you saying is that we need to
develop a student survey (to determine student goals).”
[The observer] asked if they could get information that they
want/need about student goals and intentions from any other existing body of
data. There was some discussion around a particular survey, the application
to the school, and putting questions on the touch-tone registration system, but
all of these were years in the future.
[The observer], trying to bring the conversation back to the project,
mentioned that in the last meeting the committee had discussed the problems
with academic advising.
[One of the IR members] said that he wanted to find out the different
advising methods and requirements for each department. The problem is that
no one keeps analyzable records of advising. They make the assumption that
all of their students are getting advising. “We may not be able to get anything
meaningful out of it (their data), based on my experience” (Committee I, 06-
06-01).
By the end of the first year of the project, this committee had never examined
data and had not yet determined any areas of inequitable educational outcomes they
wanted to address for the purposes of the project. There were no new ideas present in
the group nor was doubt cultivated because there was no data to instill such ideas or
create doubt.
The Role of Experiential Knowledge
Experiential knowledge played the dominant role in groups of this type.
Committee VIII, like Committee I, relied exclusively on experiential knowledge as
the only source of information or knowledge used in the group. Like the Medium
Learning groups, the members of this committee made assertions based on their
experiential knowledge that were accepted as fact and, therefore, inhibited further
121
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
inquiry. For example, when discussing proportional representation of African
American and Latino students, a committee member asserted,
Many students major in Liberal Education. Those that don’t end up in
Education get into Child Development. Mine move into Human Resources.
They get lower-paying jobs... (Committee VIII, 03-19-01).
In another instance a group member explained,
Many Latinas go into Education. Maybe because they don’t have any
encounters with women in technical majors... Perhaps we perpetuate that
because Latinas want to give back (Committee VIII, 03-19-01).
It seems that these committee members felt that they already knew that Latinas tend
to pursue those majors that are likely to lead to lower paying jobs due to a cultural
value to “give back.” Not only did such assertions go unchallenged by other
committee members, they were not challenged by data either since experiential
knowledge was the only source of knowledge or information used by the committee.
What Was Learned
There were no recorded instances in which members of Low Learning groups
developed new recognitions of inequitable educational outcomes for African
American and Latino students at their institutions. The committee members had
ideas about where problems might exist, based on their experiential knowledge, but
these areas were never explored or confirmed by examining institutional data. While
these groups may have learned about other things through their conversations and
122
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
interactions with one another, they did not engage in group learning about
inequitable educational outcomes. They may have learned what one another thought
about the gaps in educational outcomes and where they might lie, but they never
used data to confirm or disconfirm these notions.
Value of Groups
In chapter two I discussed the value of groups in facilitating organizational
learning by transferring knowledge between institutional actors and in generating
knowledge through communication and interaction. Brown and Duguid (2000)
indicate that through discussion and interaction information, which is captured in
institutional data, can take on a “social life” and become knowledge. It seemed in
reading the text associated with all of the committees involved in this project that the
conversation among members was important in creating new ideas, new approaches
to old problems, and new questions. And, through discussion committee members
did seem to adopt information as knowledge. However, all of the committees
engaged in such group interaction. There were not any committees that operated with
only one or even two people. Therefore, I have no way of knowing whether the
group structure, itself, promoted learning. There are no counterexamples present in
this text against which I could compare the group interaction. The differences in
group learning among the committees in this study seemed to be more closely
associated with the content of the groups’ discussions—whether it was an intense
123
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
and close analysis of the data or whether it was reiteration of existing experiential
knowledge which provided the subject matter for discussion.
Conclusion
The treatment of or approach to the data was the critical factor in promoting
group learning. The High Learning groups treated the data as preeminent and
approached them with a second order level of inquiry, examined indicators of
educational process not just educational outcomes, and sustained their engagement
with the data even outside the work of their committees. As a result, the High
Learning groups learned more than where inequities in educational outcomes exist.
They learned about the building blocks within their institutions that might
cumulatively contribute to inequitable educational outcomes. For example, they did
not learn only that African American and Latino students were underrepresented in
math-related majors but that these students had high failure rates in courses that lead
to such majors.
The experiential knowledge of High Learning groups acted as a guide for
inquiry. The groups resolved their own questions and curiosities by examining data
rather than relying primarily on their own past experiences and understandings.
When their own experiential knowledge was disconfirmed by the data, they were
willing to trust and believe what the data revealed. Because of this, the data were
124
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
able to provide new ideas and cast doubt on existing knowledge and institutional
practices.
This stands in contrast to the practices of Medium Learning groups and Low
Learning groups in which experiential knowledge was dominant. Medium Learning
groups held fast to their own understandings and found ways to rationalize or justify
data that disconfirmed or challenged their own views. They learned less because they
relied so heavily upon already existing knowledge. Low Learning groups learned the
least of all of the committees in the study which seems related to the fact that they
never examined data during the first year of the project. Thus, it seems that instances
of new recognition were most likely to occur when experiential knowledge was the
ground and data became the figure (Weick, 1979)— experiential knowledge served as
the context or background, but the data were what the committee really paid
attention to. These two were not held in balance; a productive tension between the
two was developed and maintained in which data was prioritized but experiential
knowledge served to guide the inquiry.
In the end I believe that the High Learning groups were in a better position to
develop agendas for change at their institutions. They had identified potential
intervention points in the educational processes of their students which preceded
inequitable educational outcomes, and therefore could have an impact on changing
those outcomes. For example, by identifying which courses contribute to the
tendency for African Americans and Latinos to switch from majors that lead to
125
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
careers in high demand to majors that do not, the committee has identified where
extra support might be needed for those students. These committees have learned
more about these problems to a finer level of detail and can communicate to campus
community members where their attention, resources, and energy should be
allocated.
126
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Chapter Five: Discussion
In this chapter I begin by discussing the important findings from this study. I
then describe the implications for practice and what it might take to convert Medium
Learning groups to High Learning groups. I address Garvin’s notion that higher
education institutions do not learn effectively. Following that discussion I reflect on
inequities in educational outcomes as indicators for institutional performance. The
chapter ends with recommendations for future research and concluding statements.
Important Findings
The primary finding of this study was that when data were dominant in a
participating committee’s work and prioritized over the committee’s experiential
knowledge, higher levels of group learning about inequities in educational outcomes
took place. The following characteristics and practices of High Learning groups were
also important findings:
1. These committees based their judgments on inequitable educational
outcomes on institutional data.
2. These committees treated their own experiential knowledge as hypotheses
to be confirmed or disconfirmed by the data.
3. These committees were willing to abandon their own experiential
knowledge when it was disconfirmed by data.
127
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4. These committees moved past the task of the project, which was to
identify areas of inequitable educational outcomes, and, by mapping
backwards from the outcome, identified potential points for intervention
that might impact these inequities. This seemed to be related to the
presence of new ideas and doubt in existing knowledge and practice,
which were functions of examining data.
5. These committees resisted the idea of inequitable educational outcomes
as a problem in institutional performance less than committees that were
Medium or Low Learning groups. It seemed that the High Learning
groups may have started this project with some sense of inequities in
educational outcomes as a problem in institutional performance while
Medium Learning groups, as illustrated by their rationalization of data
which revealed inequities, seemed more resistant to this idea and tended
to locate the problem in the student. This may be an indicator of
“readiness” for a project of this type. High Learning groups may have
been ripe for the project because they already agreed to a greater extent
with the premise. This is not something that the researchers/project staff
could instill or control. This initial attitude or orientation may have
rendered the task less difficult for High Learning groups.
128
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
All of these characteristics and practices are interconnected. For example, because
these committees treated their own experiential knowledge as hypotheses to be
proven, perhaps they were more willing to abandon them when they were
contradicted by the data.
The type of learning that went on in the High Learning groups was important
because these groups were able to identify potential intervention points. Gateway
courses, recruitment and retention in particular majors, and points of academic
jeopardy like midterm warnings all represent critical junctures in a student’s
educational process where an intervention could possibly make a difference. One
committee identified that Latino and African American students tend to place in the
lowest levels of remedial math and then do not succeed in those courses. Even when
they do succeed in remedial math, they fail General Education (GE) math at higher
rates than students of other ethnic/racial groups. A clear point for intervention is
remedial math. These committees were able to identify these potential points for
intervention by focusing upon the data and by looking at the indicators of
educational processes from a microscopic perspective. In contrast, Medium Learning
groups tended to identify inequitable educational outcomes like graduation rates, and
gaps in GPA achievement at the point of graduation, when it would be too late to
intervene. High Learning groups also learned more about the inequities that they
identified than if they had not attended so closely to the data or had not investigated
the indicators which may lead to inequitable outcomes. For example, they no longer
129
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
presume that remedial math prepares all students equally for GE math. Medium
Learning groups did not learn as much about a given problem area because they only
attended to the first layer of the data—the outcome, itself.
As mentioned above, High Learning groups seemed more likely to already
have or to adopt an orientation towards the problem of inequitable educational
outcomes as a problem of institutional performance. This not only gives the
institution some responsibility in resolving these problems, but it also gives the
institution some control over these problems because these committees identified
factors that the institution could influence and change—curriculum in gateway
courses, support and tutoring in particular math courses, pedagogy, etc. If African
American students who initially intended to be engineers tend to switch to social
science majors in large numbers after taking Calculus 103, the institution can do
something to explore this. The institution could provide tutoring services, create a
two-quarter course that covers the same material in greater depth as an option for
students, and other such interventions. Medium Learning groups were more likely to
conceptualize inequities as a problem related to student characteristics, background,
attitude, and preparation—all of which lie outside the control of postsecondary
institutions.
130
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Using Data to Develop New Recognition of Inequities in Student Outcomes:
Implications for Practice in Medium Learning Groups
In the spirit of High Learning groups, I mapped backwards from the
outcome—different levels of group learning between Medium and High Learning
groups—and asked myself, “What would it take for a Medium Learning group to
become a High Learning group?” I found five characteristics and practices in the
High Learning committees which served to promote group learning about this
problem in institutional performance. The implications for practice in Medium
Learning groups are explained below. The key is that the information that is revealed
through data exploration must be prioritized over the experiential knowledge of
committee members. In addition I explain two practices I feel would contribute to
ongoing learning in the area of inequitable educational outcomes for all groups.
1. Seek Evidence.
When committee members can trust the source of the data, that the
information system is accurate, and that the Institutional Research member who
produced the data knows what he/she is doing, committee members must be willing
to trust that the data reflect reality in the institution, even when the data contradict
their own understanding. Committee members must be willing to prioritize what the
data reveal as opposed to the common knowledge and mythologies that are part of
131
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
the institutional culture and lore. The committee must assume the place of student
and allow the data to become the teachers/experts.
2. Your Colleagues Are Not Always Right.
While experiential knowledge can serve to shape a committee’s inquiry in
productive ways, it is important to keep reliance upon this knowledge in check.
When experiential knowledge is prioritized and the committee members treat this
knowledge as if it were factual data, inquiry is inhibited. One must challenge the
assertions made by fellow committee members that are not backed up by the data.
When individuals attempt to generalize from their own experiences with particular
student groups, for example, other committee members should pursue data that can
confirm or contradict the given statement.
3. Engage in a Second Order Level of Inquiry and Explore Educational Processes.
While engaging in data examination, as new questions develop or the data
reveal something surprising or important, keep going. Keep mining the data at more
microscopic and deeper levels in order to develop understanding beyond the
traditional, telescopic indicators. If the data show, for example, that African
Americans are underrepresented in the engineering major, it is important to look for
other indicators that may lead to this underrepresentation. Explore the educational
processes that may be related to this inequitable outcome. What does it take to
become an engineering major? What might be the critical course(s) to enter the
132
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
major? What does it take to succeed as an engineering major? At what point do
African Americans tend to switch majors? Are there patterns in the majors that they
switch to? Without following up on new questions and new findings, the inquiry, and
therefore the group learning process stops.
4 . Don’t Be Afraid of Data.
As the Knight Higher Education Collaborative (2000) pointed out, there is an
aversion to using data in higher education because data can bring bad news, make the
institution “look bad,” show that interventions currently in use are not bringing about
expected results, etc. Committee members must be willing to be the bearers of bad
news and to have their own experiential knowledge contradicted. If committee
members are not willing to accept that by examining data the institution may look
bad and it may be revealed that they are wrong, then all is lost. The generation of
new ideas and practices and the cultivation of doubt in current knowledge and
practices are less likely to occur if committee members only examine data which
show what the institution is doing well and concur with the committee’s existing
understanding. Preservation of an institution’s reputation and one’s own views leads
to the maintenance of the status quo and acts as a barrier to the development of new
recognitions of inequities in student outcomes. If a committee does not at least know
where the institution currently operates on any given indicator, then they cannot even
conceptualize the problem let alone the potential remedies.
133
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5. Sustain Engagement with the Data
In order to become high learning, the Medium Learning groups should
sustain their engagement with the data under exploration beyond the boundaries of
the committee’s work. By simply spending more time reflecting on the data,
discussing it with others, and allowing it to inform other areas of one’s work, it
seems that more learning might take place. This also allows for cross-fertilization
between committees on campus. Other committees might benefit from the
knowledge one member has gained from examining data on educational outcomes by
ethnicity/race.
The following two practices are not only applicable in the promotion of
Medium Learning groups to High Learning groups. These implications for practice
are applicable to High Learning groups as well.
6. Create Conditions for Group Learning.
As organizational theorists have advocated (Garvin, 1993; Senge, 1990), the
institution must create time, space, and resources for committees to engage in data
exploration in order for it to lead to group learning. And, the committees must be
rewarded for their data exploration. There should be at the very least no fear of
reprisal if the committee delivers bad news as explained above. In the case of the
project under study here, an outside group intervened at the institution and provided
conditions in which time and space and personnel resources were dedicated by the
president to the committee’s exploration of data related to inequities in educational
134
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
outcomes for African American and Latino students. These outsiders were also in a
position to ask the critical and uncomfortable questions that raised doubts and
generated new ideas, which an “insider” might not be able to ask. Outsiders can ask
seemingly ignorant questions or risky questions which the cultural norms of the
institution might ordinarily guard against. An outsider can ask questions that lead to
double loop learning while insiders tend to ask questions that lead to single loop
learning because the tough questions are never asked (Argyris, 1994). Argyris
explains that CEO’s of companies rarely make a complete diagnosis of
organizational problems because they don’t dig deep enough in order to avoid
negatively impacting morale and “open[ing] Pandora’s box” (p. 92). This is a
defensive strategy used to “avoid vulnerability, risk, embarrassment, and the
appearance of incompetence,” and it is a “recipe for ineffective learning” (p. 95).
In order for double loop learning to happen without the intervention of
outsiders, there must be an internal call for group learning about problems in
organizational performance and institutional actors who are willing and able to ask
the tough questions. Both institutional leaders and other members of the institution
“must all begin struggling with a new level of self-awareness, candor, and
responsibility” (Argyris, 1994, p. 109). Also, in the lives of administrators and
faculty members, time is in short supply, and therefore this type of activity should
become an integral part of their jobs rather than another add-on. In this sense again,
the data must be prioritized.
135
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7. No One-Time Shots.
Group learning via data exploration such as that illustrated in this study
should not be considered a one-time shot, ad hoc committee that will soon disappear.
This is true for both High and Medium Learning groups. The hope is that the
committees associated with this project will continue to monitor the data they
explored for the purposes of this project on an annual basis in order to determine if
educational outcomes are becoming more or less equitable over time. The committee
should engage in annual “check-ups” on these indicators. Also, there are plenty of
other indicators to be explored—not all indicators of inequities in educational
outcomes were revealed through this project. If the committee, or the institution,
loses sight of the data, then mythologies will abound again.
Can Higher Education Institutions Learn?
Garvin (1993) uses higher education institutions as an example of a type of
organization that does not learn effectively because colleges and universities, which
are effective at creating and acquiring new knowledge, do not apply this knowledge
to their own activities and improvement. Garvin also explains that there must be
observable change in the behavior of the organization to show that learning has taken
place. A critical problem with Garvin’s assessment of higher education is that he
refers to the organization as a whole rather than subunits within the organization. As
136
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
explained earlier, this does not fit with the ways in which institutions of higher
education operate. If we substitute the committees involved in this project for the
“organization,” I think that the High Learning groups in this study provide a
counterexample to Garvin’s assertion.
The members of the High Learning groups, and to some extent the Medium
Learning groups, did apply knowledge and practices from their own work as faculty
or institutional researchers or administrators to benefit the activities of these
committees. The institutional researchers who were members of these committees
certainly applied what they knew about collecting, refining, and examining data to
the activities of these committees, which resulted in the generation and acquisition of
new knowledge about inequities in educational outcomes. And all of the committee
members used their own experiential knowledge of the institution to focus their
inquiry into the data.
While there may not have been measurable change in the educational
outcomes of African American and Latino students after the first year of the project,
there were certainly observable changes in the practices and awareness of the
committees, particularly those that were classified as High Learning groups. The
committee members learned about the effectiveness of examining institutional data
in these new ways, and their reflections on the power of using data seemed to be
even more enthusiastic than their reflections on inequities in educational outcomes.
137
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Even the institutional researcher of one committee learned about new methods for
using institutional data:
Doing this project I’ ve found many ways of thinking about data. I’ ve even
learned new techniques as an Institutional Research person (Committee XIV,
11-28-01).
A member of another committee reflected on what he had learned about using
institutional data as a result of this project—the new possibilities that had opened to
him.
We have a chance to look at where we are. We can make arguments
supported with the numbers. Maybe we could even ask some new questions.
For instance, I never knew to ask our IR office to disaggregate the data for
[my] department (Committee VII, 07-13-01).
These reflections recorded in the text serve as evidence that the committee members
did engage in learning which impacted their behavior. So, although there were not
measurable improvements in the educational outcomes for African American and
Latino students, important group learning—both about inequitable educational
outcomes and about the power of using data for institutional improvement—took
place in some of these committees.
I think that the High Learning groups provide an example of
“organizational,” or more accurately, group learning in institutions of higher
education. While I disagree with Garvin (1993) and his contention that higher
education institutions do not engage in learning, I agree with his assertion that
138
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
colleges and universities perhaps do not learn as effectively as they could.
Institutional actors should apply their practices as communities of researchers to
studies of the institution, itself, for the purposes of improvement. Yes, faculty who
excel in research in other fields such as economics, sociology, business, the study of
organizations, and many others should apply those research methods to the
assessment and improvement of their own institutions. Committees made up of such
researchers should examine data that are relevant to the committee’s charge, as they
would examine data for a funded research project. The study and improvement of
our own institutions should be as rigorous as our study of other types of institutions
and social structures.
Peter Ewell (1997) shares the notion that data are critical to institutional
learning and improvement, and he suggests that becoming what is referred to in the
literature as a “learning organization,” involves “creating institutional capacities for
gathering and interpreting data at all levels” (p. 6). This implies that an organization
should not only gather and store data but also engage in the interpretation of that
data. Institutional research should not be an activity reserved solely for
administrative and reporting functions. At the lower levels of the organization
“concrete mechanisms for gathering data, and the incentives to use them are equally
important” (p. 6). These suggestions illustrate other ways in which postsecondary
institutions might enhance their capacity for ongoing learning and self-study for the
139
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
purposes of institutional improvement. It seems that the potential for effective
learning is there, but it is up to institutional actors to put this learning into action.
Indicators of Institutional Performance
I find when I step back and look at this project as a whole, I am struck by the
fact that these institutions had not previously examined many of the indicators they
used for the purpose of this project. The data examined by the committees were not
remarkable—graduation rates, passing rates in key courses, representation in majors,
etc. It seems that these would be indicators that an institution would naturally attend
to since they reflect two of the most important activities of higher education—
teaching and learning. One might expect that the accrediting body for all of these
institutions would have asked for these types of information. But in fact, this
accrediting body is currently in heated debate as to whether it is necessary or even
appropriate to ask for institutional data disaggregated by ethnicity. Why have such
data been largely overlooked or ignored? Without such data, how might one identify
inequitable educational outcomes? How might one identify this problem in
institutional performance? Can we close the gaps in achievement without even
understanding where the gaps are?
There are so many data currently available in the majority of institutions of
higher education. Colleges and universities have made sizeable investments in
technology and training to develop the capacity to collect all sorts of information
140
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
about students—from their incoming SAT scores through every course they take to
their graduating GPA. In addition, higher education now finds itself in an “age of
accountability” (Sewall, 1996; Radner, 1996; Ohmann, 1999; Gumport & Spom,
1999; Alexander; 2000), which requires providing all sorts of data to outside
agencies. While colleges and universities have not exactly welcomed this scrutiny
from the outside, I think that higher education can turn this task which most
begrudge into a positive and productive group learning exercise. Accountability
reports, which are based primarily on institutional data that reflect indicators of
institutional performance, could actually become more than an empty, annual
exercise seen as only a tool to appease outsiders. The Knight Higher Educational
Collaborative (2000) asserts that,
Today, universities and colleges expend more time, effort, and money than ever
before in gathering data... For all that, higher education institutions still have not
learned to organize and use data effectively for internal decisions or public
accountability (p.5).
O’Neil, Bensimon, Diamond, and Moore (1999) found that when they approached an
accountability initiative as an opportunity for institutional self-assessment and
improvement that it had “latent benefits that contribute to organizational well-being”
(p. 40). In this study the High Learning groups experienced the highest incidence of
group learning about inequities in educational outcomes among students of different
ethnic groups. This occurred because of their attention to and thorough examination
141
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
of data. Accountability reports may provide an excellent starting point for
exploration into problems in institutional performance.
Future Research
There are certainly other factors that may impact group learning about
inequities in educational outcomes beyond the use of data examination and reliance
upon experiential knowledge. Institutional culture, stability in the committee
membership, the influence of individuals on the committee, group dynamics, and the
individual group members’ philosophical orientations toward the notion of equitable
educational outcomes all may have played a role. Also, the fact that this project was
initiated by outsiders may have influenced the development of new recognitions of
inequities in educational outcomes. To explore some of these other factors it would
be important to have verbatim transcripts of the meetings, and the researcher, herself,
would have to attend all meetings. Without having transcripts and being in
attendance, it would be difficult to glean, for example, what one committee
member’s influence over others in the group might have been.
A follow-up study to this one investigating whether, in fact, this project does
lead to any change would be valuable. In the end, do the High Learning groups’
institutions actually experience more change than those of the other types? Did their
group learning ever have any impact outside the committees, themselves? One way
142
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
in which this could be determined would be by comparing data collected by the
committee with data on the same indicators several years later. Budget allocations
may also serve as evidence of change. For example, if remedial math was an
identified problem area, were more financial resources allocated to the math
department?
The researcher might also conduct interviews to determine whether the
individual committee members applied what they learned to their own practices, both
with regard to the use of data and what they discovered about inequitable educational
outcomes. Do the committee members continue to examine data in their work? Do
they advocate the examination of data in committees and other groups of which they
are a part? Do the committee members consider in what ways they might contribute
to inequities in educational outcomes and/or contribute to minimizing the gap?
Finally, the conclusions of this study should be tested by repeating it using a
case other than inequities in educational outcomes. There are other problems in the
institutional performance of colleges and universities. Do data act as a trigger for
group learning when a committee’s focus is something other than inequities in
educational outcomes? Is it as critical that data be prioritized over experiential
knowledge when investigating other problems?
143
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Conclusion
We, in higher education, cannot go on in ignorant bliss, patting ourselves on
the collective back for having more diverse student bodies in our colleges and
universities. We must attend to indicators of educational processes and outcomes
which reveal how we’re doing with regard to serving the educational needs of all
students, regardless of ethnicity/race. We can’t write off the disparities in outcomes
to poor K-12 education or student characteristics, neither of which institutional
actors in colleges and universities can control. That won’t make a difference. We
should ask ourselves, “Given that students of different ethnic/racial backgrounds
enter postsecondary education with these gaps in achievement, what can we do to
minimize this gap during their time in college?” We should live up to our own
ideals—equal opportunity through education.
Members of the educational research community have an important role to
play here. In recent years authors have criticized educational research, and in
particular research on higher education citing that “a primary factor impeding the
advancement of higher education is that the research-practice gap remains fairly
unexplored, and few suggestions exist to advance our thinking beyond blaming one
side or the other” (Kezar & Eckel, 2000, p. 2). This study represents an educational
research project that actually made a difference on practitioners and effectively
bridged the research to practice gap. It is clear that almost all of the practitioners
(committee members) involved in the project under study, and particularly those in
144
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
High Learning groups, learned about inequities in educational outcomes for African
American and Latino students in their institutions. Educational researchers could
make a great contribution to minimizing the gap in educational achievement among
students from different ethnic/racial groups as well as the gap between research and
practice by working with practitioners to develop efficient, effective, and reliable
methods for identifying where inequities exist and investigating practices that might
reduce the gap, as shown in the project under study here.
145
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
References
Adelman, C. (1997). Diversity: Walk the walk and drop the talk.
C/iange(July/August), 34-45.
Adelman, C. (1999). Answers in the tool box: Academic intensity, attendance
patterns, and bachelor’ s degree attainment. U.S. Department of Education.
Alexander, K. (1999). The changing face of accountability. The Journal of Higher
Education, 77(4), 411-431.
Angrosino, M. V., & Mays de Perez, K. A. (2000). Rethinking observation: From
method to context. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of
qualitative research (second ed., pp. 673-702). Thousand Oaks, CA: Sage
Publications, Inc.
Argyris, C. (1993). Knowledge for action: A guide to overcoming barriers to
organizational change. San Francisco: Jossey-Bass Publishers.
Argyris, C. (1994). Good communication that blocks learning. In On organizational
learning. Boston: Harvard Business Review Press.
Argyris, C., & Schon, D. A. (1978). Organizational learning. In D. S. Pugh (Ed.),
Organization theory. New York: Penguin Books.
Argyris, C., & Schon, D. A. (1996). Part I: An introduction to organizational
learning, Organizational learning II (pp. 3-72). Reading, MA: Addison-
Wesley Publishing Company, Inc.
Bean, J. P., & Eaton, S. B. (2000). A psychological model of college student
retention. In J. M. Braxton (Ed.), Reworking the student departure puzzle.
Nashville: Vanderbilt University Press.
Bensimon, E. M., & Neumann, A. (1993). Redesigning collegiate leadership: Teams
and teamwork in higher education. Baltimore: The Johns Hopkins University
Press.
Bimbaum, R. (1988). How colleges work: The cybernetics of academic organization
and leadership. San Francisco: Jossey-Bass Publishers.
146
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Birnbaum, R. (2000). Management fads in higher education: Where they come from,
what they do, why they fail. San Francisco: Jossey-Bass Publishers.
Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis and
code development. Thousand Oaks, CA: Sage Publications, Inc.
Braxton, J. M. (2000). Reworking the student departure puzzle. Nashville: Vanderbilt
University Press.
Brown, J. S., & Duguid, P. (2000). The social life of information. Boston: Harvard
Business School Press.
California Community Colleges, Office of the Chancellor (2000). Available:
http://www.cccco.edu/reports/reports.htm.
California State University, Office of the Chancellor. (2000). Available:
www.calstate.edu.
Cameron, K. S. (1984). Organizational adaptation and higher education. Journal of
Higher Education, 55(2), 121-144.
Camevale, A. P. and Fry, R. A. (2000). Crossing the Great Divide: Can We Achieve
Equity When Generation YGoes to College? Princeton: Educational Testing
Service.
Cohen, S. G., & Bailey, D. E. (1997). What makes teams work: Group effectiveness
research from the shop floor to the executive suite. Journal of Management,
25(3), 239-290.
Cohen, M. D., & March, J. G. (1974). Leadership and ambiguity: The American
college president. New York: McGraw-Hill Book Company.
Cook, S. D. N., & Yanow, D. (1996). Culture and organizational learning. In M. D.
Cohen & L. S. Sproull (Eds.), Organizational learning. Thousand Oaks, CA:
SAGE Publications.
Cutscher-Gershenfeld, J. et. al. (1998). Knowledge-driven work: Unexpected lessons
from Japanese and United States work practices. Oxford: Oxford University
Press.
147
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Daft, R. L., & Huber, G. P. (1987). How organizations learn: A communication
framework. Research in the Sociology of Organizations, 5, 1-36.
Davenport, T. H., & Prasak, L. (1998). Working knowledge: How organizations
manage what they know. Boston: Harvard Business School Press.
Dill, D. D. (1999). Academic accountability and university adaptation: The
architecture of an academic learning organization. Higher Education, 38,
127-154.
Dixon, N. M. (2000). Common knowledge: How companies thrive by sharing what
they know. Boston: Harvard Business School Press.
Easterby-Smith, M. (1997). Disciplines of organizational learning: Contributions and
critiques. Human Relations, 50(9), 1085-1113.
Ewell, P. T. (1997). Organizing for learning: A new imperative. AAHE Bulletin.
December, 3-6.
Fiol, C. M., & Lyles, M. A. (1985). Organizational learning. Academy of
Management Review, 10(4), 803-813.
Gandara, P.; Maxwell-Jolly, J. (1999) Priming the pump: Strategies for increasing
the achievement of underrepresented minority graduates. The College Board.
Available: www.collegeboard.org.
Garvin, D. A. (1993). Building a learning organization. Harvard Business Review,
July-August, 78-90.
Gumport, P. J., & Spom, B. (1999). Institutional adaptation: Demands for
management reform and university administration. In J. C. Smart & W. G.
Tiemey (Eds.), Higher Education: Handbook of Theory and Research (Vol.
14, pp. 103-145). New York: Agathon Press.
Harvey, C., & Denton, J. (1999). To come of age: The antecedents of organizational
learning. Journal of Management Studies, 36(7), 897-918.
Henriksson, K. (1999). The collective dynamics of organizational learning: On
plurality and multi-social structuring. Lund, Sweden: Lund University Press.
148
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Higher Education Research Institute, University of California at Los Angeles.
Cooperative Institution Research Program, American Freshmen Survey.
Huber, G. P. (1991). Organizational learning: The contributing processes and the
literatures. Organization Science, 2(1), 88-115.
Jun, A., & Tierney, W. G. (1999). At-risk urban students and college success: A
framework for effective preparation. Metropolitan Universities, 9(4), 49-59.
Kezar, A. & Eckel, P. (Eds.). Moving Beyond the Gap Between Research and
Practice in Higher Education. New Directions for Higher Education, no. 110.
San Francisco: Jossey-Bass.
The Knight Higher Education Collaborative. (2000). The data made me do it. Policy
Perspectives, 9(2), 1-12.
Kock, N. (1999). Process improvement and organizational learning: The role of
collaboration technologies. Hershey, PA: Idea Group Publishing.
Kraemer, B. A. (1997). The academic and social integration of Hispanic students
into college. The Review of Higher Education, 20(2), 163-179.
Lee, J. (2002). Racial and ethnic achievement gap trends: Reversing the progress
toward equity? Educational Researcher, 31(1), 3-13.
Levitt, B., & March, J. G. (1988). Organizational learning. Annual Review of
Sociology, 14, 319-340.
Lindblom, C. E., & Cohen, D. K. (1979). Usable knowledge: Social science and
social problem solving. New Haven: Yale University Press.
Lofland, J., & Lofland, L. H. (1995). Analyzing social settings (third ed.). Belmont,
CA: Wadsworth Publishing Company.
March, J. G. (1991a). Exploration and exploitation in organizational learning.
Organization Science, 2(1), 71-87.
March, J. G. (1991b). Organizational consultants and organizational research.
Journal of APPLIED COMMUNICATION Research, 19, 20-31.
149
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Marsick, V. J., & Watkins, K. E. (1999). Facilitating learning organizations. Making
learning count. Vermont: Ashgate Publishing Co.
Me Call, G. J., & Simmons, J. L. (Eds.). (1969). Issues in participant observation: A
text and reader. Reading, MA: Addison-Wesley Publishing Co., Inc.
Miller, D. W. (August 6, 1999). The black hole of education research. Why do
academic studies play such a minimal role in efforts to improve schools? The
Chronicle of Higher Education, pp. A17.
Morgan, G. (1997). Learning and self-organization: Organizations as brains. In
Images of Organizations (pp. 73-118). Thousand Oaks, CA: Sage
Publications, Inc.
Muhr, T. (1997). Atlas.ti The knowledge workbench (Version 4.1 for Windows 95
and Windows NT) [Computer Software]. Berlin, Germany: Scientific
Software Development.
Nason, J., & Golding, D. (1998). Approaching observation. In G. Symon & C.
Cassell (Eds.), Qualitative methods and analysis in organizational research
(pp. 234-249). Thousand Oaks, CA: Sage Publications, Inc.
National Center for Education Statistics. U.S. Department of Education. Available:
www.nces.ed.gov.
National Center for Education Statistics. (2001). Digest of Education Statistics.
Washington, D.C.: Department of Education.
Ohmann, R. M. (1999). Historical reflections on accountability. Radical Teacher, 57,
2-7.
ONeil, H. F., Bensimon, E. M., & Diamond, M. A. (1999). Designing and
implementing an academic scorecard. Change, 31(6), 32-40.
Pfeffer, J., & Sutton, R. I. (2000). Knowing "what" to do is not enough. In The
knowing-doing gap: How smart companies turn knowledge into action (pp. 1-
28). Boston: Harvard Business School Press.
Polkinghome, D. E. (2000). Psychological inquiry and the pragmatic and
hermeneutic traditions. Theory and psychology, 10(4), 453-479.
150
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Popper, M., & Lipshitz, R. (1998). Organizational learning mechanisms: A structural
and cultural approach to organizational learning. Journal of Applied
Behavioral Science, 34(2), 161-179.
Powell, M. H. (1998). Campus climate and students of color. In L. A. Valverde & L.
A. J. Castenell (Eds.), The multicultural campus (pp. 95-118). Walnut Creek,
CA: AltaMira Press.
Radner, S. G. (1996). Accountability, evaluation of tenured faculty, and program
reviews. Radical Teacher, 48, 11-12.
Rendon, L. I., & Valadez, J. R. (1993). Qualitative indicators of Hispanic student
transfer. Community College Review, 20(4), 27-37
Schoenfeld, A. H. (2002). Making mathematics work for all children: Issues of
standards, testing, and equity. Educational Researcher, 31(1), 13-25.
Schwartzman, H. B. (1993). Ethnography in organizations (Vol. 27). Newbury Park,
CA: Sage Publications, Inc.
Senge, P. M. (1990). The fifth discipline. New York: Doubleday.
Sewall, A. (1996). From the importance of education in the 80's to accountability in
the 90's. Education, 116(3), 325-332.
Shrivastava, P. (1983). A typology of organizational learning systems. Journal of
Management Studies(20), 7-28.
Smith, D. G. A. (1997). Diversity works: The emerging picture of how students
benefit. Washington D. C.: Association of American Colleges and
Universities.
Sormunen-Jones, C., Chalupa, M. R., & Charles, T. A. (2000). The dynamics of
gender impact on group achievement. Delta Pi Epsilon Journal, 42(3), 154-
170.
Spradley, J. P. (1980). Participant observation. New York: Holt, Rinehart and
Winston, Inc.
151
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Tiemey, W. G. (1999). Models of minority college-going and retention: Cultural
integrity versus cultural suicide. The Journal of Negro Education, 68(1), 80-
91.
Tinto, V. (1993). Leaving College (2nd ed.). Chicago, IL: The University of Chicago
Press.
Vemez, G., Krop, R. A., & Rydell, C. P. (1999). Closing the education gap: Benefits
and costs. RAND.
Weick, K. E. (1976). Educational organizations as loosely coupled systems.
Administrative Science Quarterly, 21, 1-19.
Weick, K. E. (1979). The social psychology of organizing (second ed.). New York:
McGraw-Hill, Inc.
Weick, K. E., & Westley, F. (1996). Organizational learning: Affirming an
oxymoron. In S.R. Clegg, C. Hardy, and W.R. Nord (Eds.), Managing
organizations. Thousand Oaks: Sage Publications.
Wenger, E., & Snyder, W. M. (2000). Communities of practice: An organizational
frontier. Harvard Business Review, 78(1), 139-145.
152
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Appendix A
Designing and Implementing a Diversity Scorecard
to Improve Institutional Effectiveness fo r Underserved Minority
Students
The Center for Urban Education (CUE) at the University of Southern
California’s Rossier School of Education has invited «Name_of_Institution» to
participate as a partner institution in the “Diversity Scorecard” project. The
duration of this project is from 1/01-12/31/03. This document delineates the
roles and responsibilities of «Name_of_Institution» and the partners from the
Center for Urban Education.
«Name of Institution
1. The college will appoint a team of people to work in collaboration with CUE
partners.
2. The college’s team will work with the CUE partners to: 1) develop a
Diversity Scorecard by December 2001; 2) implement the Diversity
Scorecard from January, 2002-December 31, 2003.
3. The Campus Team Leader will convene the site-based meetings, to be held at
the institution, and work with the CUE partners in developing the work plan.
As provided below, the CUE partners will be present at three site-based
meetings and four all-partners meetings, to be held at USC, between
«Kick_off_event_date» and December, 2001. It is anticipated that in addition
to these regularly scheduled meetings each institution is likely to hold
additional campus self-initiated meetings to get the work done.
Unfortunately, due to the large number of institutions in the project, it will
not be possible for CUE’s Research Associates to attend additional meetings
that the college schedules. However, the Research Assistant may be able to
attend some of these meetings. Even though CUE partners may not be present
at all the site-based meetings, we recommend that the campus inform them of
when additional meetings are scheduled.
4. The college’s identity as well as the identities of the individuals involved in
the project will be kept confidential in all publications from this project
unless the college wishes to be identified.
153
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5. The college’s team will have the opportunity to read and comment on pre
publication drafts authored by CUE partners as well as those authored jointly
with members of the institutional teams.
6. The college has the option of withdrawing from the project at any time.
7. At this time, the college will not receive any compensation for its
participation in this project.
154
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The Center for Urban Education (CUE) Partners
1. A CUE Research Associate and Research Assistant will work in
collaboration with the college’s team to develop the Diversity Scorecard.
2. The role of CUE is to provide a framework (the Diversity Scorecard) that is
intended to make “invisible” information “visible.” The Diversity Scorecard
is like a “mirror” in which institutions can see new information that tells them
how they are doing in their selected goals and measures. The Scorecard itself
is not a “solution.” If the image that the Diversity Scorecard reflects back to
the institution reveals problems, the solutions to those problems will be
developed by the participating institutions.
3. The Research Associate and Research Assistant will attend three site-based
meetings that will be scheduled by the Campus Team Leader in consultation
with CUE partners, and the four all-partners meetings scheduled to take place
on the USC campus, starting with the kick off meeting on
«Kick_off_event_date».
4. The Research Assistant, an advanced PhD candidate, will be an observer of
the process and take notes, which will be used as data for her dissertation.
The Research Assistant will provide the college’s team with a description of
her dissertation project once it has been fully conceptualized.
5. The CUE partners will provide resources that could be useful in the
development of the Scorecard (e.g., syntheses of research that are relevant to
the development of measures).
6. CUE partners will involve the college’s team in the review of products
resulting from this project.
8. CUE partners will maintain the confidentiality of the identity of the college
and the members of the work team in publications unless the college wishes
to be identified.
Format for work group meetings
1. Between now and December 2001, there will be 7 or more meetings of the
work group, three of which will be site-based meetings at the campus.
155
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2. The Campus Team Leader will convene the site-based meetings and will
work with the CUE partners to develop meeting agendas.
3. The Research Assistant will observe and take notes.
4. At the end of each meeting, the Research Assistant will summarize the
meeting and remind the group of assignments made, etc.
5. About 20 minutes before the meeting ends, the Research Associate will have
a “time out session” which will give the team members an opportunity to
review how the group is doing, concerns, etc. The time-out will give all of us
a “reading” of how we are doing.
6. This part of the meeting will be tape-recorded so that we can have a full
record of the development process.
156
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Assessing equitable postsecondary educational outcomes for Hispanics in California and Texas

PDF

Improving the transfer rates of minority students: A case study

PDF

Chicana poetics: Performing desire, recasting archetypes

PDF

Asian American leaders in higher education: Career aspirations in student affairs

PDF

Anorexia nervosa: A developmental perspective

PDF

Considering the impact of culture on the therapeutic relationship: A look at Latino school-age children in a special education setting

PDF

Integration of science and practice: A collective case study of scientist -practitioner programs in counseling psychology

PDF

An investigation of a new diagnostic sub-type: Post traumatic stress disorder with psychotic features

PDF

Design and synthesis of foscarnet -peptide conjugates, halogenated bisphosphonates and phosphonocarboxylates and biochemical sensing using film -bulk -acoustic -resonators

PDF

Identity in midlife lesbians: A kaleidoscopic view

PDF

A phenomenological inquiry into the essential meanings of the most intense experience of religiosity and spirituality

PDF

Delving into the center: Women's lived experience of spirituality through occupation

PDF

Signing identity: Rethinking U.S. poetry, acts of translating American Sign Language, African American, and Chicano poetry and the language of silence

PDF

Constructing identities in social worlds: Stories of four adults with autism

PDF

An assessment of general education requirements in Taiwanese higher education

PDF

"I'm not much different": Occupation, identity, and spinal cord injury in America

PDF

Race, ethnicity, and the ideology of marginality

PDF

Internationalization of higher education: A case study of a private United States research university

PDF

Collectibles, Fetishes, And Hybrid Objects: Object Discourses And Syncretic Female Identity In Recent Cross-Racial North American Women'S Representation

PDF

Experimental evaluation of a new intervention designed to promote dementia caregivers' acceptance and empathic responding towards care -recipients

Asset Metadata

Creator Bauman, Georgia Louise (author)

Core Title Developing a culture of evidence: Using institutional data to identify inequitable educational outcomes

Contributor Digitized by ProQuest (provenance)

School Graduate School

Degree Doctor of Philosophy

Degree Program Education

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag education, higher,OAI-PMH Harvest

Language English

Advisor Bensimon, Estela Mara (committee chair), McKenna, Teresa (committee member), Polkinghorne, Donald (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c16-274100

Unique identifier UC11334741

Identifier 3094304.pdf (filename),usctheses-c16-274100 (legacy record id)

Legacy Identifier 3094304.pdf

Dmrecord 274100

Document Type Dissertation

Rights Bauman, Georgia Louise

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA