University of Southern California Dissertations and Theses

Beyond access: an evaluation of attitudes and learning towards achieving equitable educational outcomes in higher education

(USC Thesis Other)

Beyond access: an evaluation of attitudes and learning towards achieving equitable educational outcomes in higher education

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

BEYOND ACCESS: AN EVALUATION OF ATTITUDES AND LEARNING
TOWARDS ACHIEVING EQUITABLE EDUCATIONAL OUTCOMES
IN HIGHER EDUCATION

by

Madeleine D. Bruning

A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION

December 2006

Copyright 2006 Madeleine D. Bruning

ii

DEDICATION

To my students and colleagues serving in the United States Armed Forces who
have deferred their education, goals and lives so that I may have the continued
freedom to pursue and complete this project. To these individuals and all who serve
with them, I am sincerely and eternally grateful.

iii

ACKNOWLEDGMENTS

I wish to acknowledge and thank the individuals who have contributed their
time, thoughts, energy and support during the completion of this project.
My sincere gratitude and appreciation to my family and friends for
persevering with me, for their patience and good humor. They have consistently
helped me keep perspective.
To the many students and patients I have had the privilege to teach or
provide care who have taught me the most important lessons about learning and
teaching.
Special gratitude is extended to the committee members, Dr. Adriana Kezar
and Dr. Georgia Bauman who have generously given their time and expertise to
better my work and for their good- natured support and contributions.
A heartfelt thank you to Ms. Arlease Woods for all of the “behind the scenes”
efforts and kindness she extends to all students.
To my colleagues at Mount St. Mary’s College, the Keck School of
Medicine at the University of Southern California for all of the prayers and
“cheers” of encouragement.
To“educate” emanates from the Latin word “educare” which means “to lead
forth” so finally and most importantly, I am grateful for the opportunity of working
with my dissertation chair, Dr. Estela Mara Bensimon. This project would not have
been possible without her time, energy, faith and guidance. Her generous spirit and

iv
passion for learning sustained my efforts and she constantly inspired me to find
“my voice”.

v
TABLE OF CONTENTS

DEDICATION ........................................................................................... ii

ACKNOWLEDGMENTS ......................................................................... iii

LIST OF TABLES ..................................................................................... vi

LIST OF FIGURES ................................................................................... vii

ABSTRACT ............................................................................................... viii

CHAPTER 1- Introduction .......................................................................... 1

CHAPTER 2-Review of the Literature ....................................................... 17

CHAPTER 3-Methodology ......................................................................... 56

CHAPTER 4- Results ................................................................................. 70

CHAPTER 5-Discussion.............................................................................. 126

REFERENCES ............................................................................................ 145

APPENDICES
Appendix A .......................................................................................... 152
Appendix B ........................................................................................... 153
Appendix C .......................................................................................... 154

vi
LIST OF TABLES
Table 1: Sources of Evidence used in this case study ................ 64

Table 2: Table 2. Sample Coding Scheme ............................................... 67

Table 3: Table 3- Riverton Lake College –Enrollment by Ethnicity
and Gender ................................................................................................ 93

Table 4: Table 4. Riverton Lake College – Freshmen Retention
by Ethnicity ................................................................................................. 96

Table 5: Table 5- Riverton Lake College –Student and Faculty
Ethnicity, Fall 2001 .................................................................................... 100

Table 6: Table 6- Students with Departmental Distinction in the

Major, by Gender 00-01 and 010-02 .......................................................... 102

Table 7: Table 7- Admittance and Yield Rates by Ethnicity ...................... 107

Table 8: Table 8- Number of Student Who Do Not Pass by Ethnicity ....... 114

Table 9: Table 9 Freshman “Gatekeeper” Courses by Ethnicity ................ 115

vii
LIST OF FIGURES
Figure 1: Diversity Scorecard Framework (University of
Southern California, 2004) ...................................................................... 6

Figure 2: Figure 2: Positive Constructivist Models of Learning .............. 13

Figure 3: Figure 3: Characteristics of single case study designs ............ 59

viii

ABSTRACT
This study is an evaluation of learning in individual institutional actors
during their participation in a project that involved them 14 campus teams whose
responsibility was to examine data on students’ educational outcomes
disaggregated by race and ethnicity. The purpose of the project was to assist
campus teams in the construction of a diversity scorecard to continuously monitor
the progress of minority students toward the attainment of equitable educational
outcomes. The premise of this project, known as the “Diversity Scorecard”, was
that, by becoming aware that inequities exist among historically underrepresented
minority students, institutional actors would then be compelled to take
responsibility for addressing them.
I evaluated what the members of one team learned by analyzing the field
notes that document the work of the three member team of an independent college
over a period of 18 months. Learning was defined using Mezirow’s conceptual
model of transformative learning, which includes the three stages of critical
reflection: content, process, and premise reflection. The text comprising the field
notes for 31 team meetings were coded using Mezirow’s these three stages of
critical reflection. Through an in depth analysis and coding of the text, I examined
whether there was evidence of learning and subsequent change in the attitudes,
values, and practices of institutional actors as a result of their participation in this
project. A guiding principle of this study is that learning and change are socially

ix
constructed and facilitated by engagement of joint productivity. As a result of their
participation on the committee, the team members learned new ways to look at the
institutional data and practices, enabling them to define problems leading to
inequities in the educational outcomes among Hispanic and African American
students

1

CHAPTER 1
THE PROBLEM AND UNDERLYING FRAMEWORK
Introduction
During the past 50 years of affirmative action, institutions of higher
education have primarily focused upon access and enrollment, producing little
evidence of progress towards equitable educational outcomes for the historically
underrepresented minority students. Educational reform has not prompted
institutional actors to become aware that although the student body has become
multicultural, equitable educational outcomes for underrepresented minority
students has not been accomplished. Failure to acknowledge and respond to
existing inequities is a result of inadequate learning and institutional performance.
Traditional strategies to measure institutional performance, such as retention, do
not place an onus of accountability upon institutional actors and is still seen as a
function of student performance, rather than institutional performance. These
strategies do not promote insight or knowledge into equitable outcomes, largely
due to the fact that the lens has not shifted in how these outcomes are analyzed,
interpreted, and reported. The University of Southern California’s Diversity
Scorecard Project was designed to assist institutions of higher education in
assessing their effectiveness in achieving equitable outcomes in underrepresented
minority students. A key feature of the project was to use existing data to construct
new knowledge and insight into the attitudes and values that influenced the

2
institution’s ability to hinder or enhance equitable educational outcomes in
historically underrepresented minority students.
The following section, background of the problem, will include two aspects
that are critical in understanding how individual institutional actors engaged in the
project and the subsequent dynamics. Through that engagement, institutional actors
would create new knowledge that would lead to an enriched, deeper comprehension
of existing inequities and subsequently change their attitudes and actions to rectify
these inequities. The first part of the background of the problem is an overview of
the historical events and relevant current research related to affirmative action and
educational outcomes in higher education among the underrepresented minority
student populations. In the second part of the section, I will discuss the framework
and implementation design of the Diversity Scorecard Project which was developed
at Center for Urban Education at the University of Southern California.
Background of the Problem
Historical background of the problem
In 1954, the United States Supreme Court determined that the principle of
“separate but equal” was unconstitutional. The precedent setting case, Brown
versus the Board of Education lead to the desegregation of public schools in the
United States. The Civil Rights Act of 1964, affirmed that the inherent right of
access and opportunity is also correspondingly a commitment to results: meaning
that, having access to higher education is also the right to be able to realize the
skills and benefits of that education. In his 1965 Commencement Address at

3
Howard University, President Lyndon B. Johnson observed that despite the steps
taken in desegregating public education, access had only marginally increased and,
in fact, obscured the greater, more insidious impasse to attaining equality: “To this
end equal opportunity is essential, but not enough, not enough. Men and women of
all races are born with the same range of abilities. But ability is not just the product
of birth. Ability is stretched or stunted by the family that you live with, and the
neighborhood you live in--by the school you go to and the poverty or the richness
of your surroundings…Despite the court orders and the laws, despite the legislative
victories and the speeches, for them the walls are rising and the gulf is widening”
(Johnson, 1965).
National statistics in higher education reflect an increase in access for
minority students but the number of awarded degrees in the African American and
Hispanic populations fall short when compared to the white population (U.S.
Department of Education, National Center for Education Statistics, Enrollment in
Higher Education, 2003). To reiterate this fact, McGlynn (2005) cited that within
six years, four year colleges and universities lost 37 percent of their freshmen. For
minority students and students with low incomes, the retention numbers were even
worse, with 54 percent of African American and 53 percent of Hispanic, first time,
fulltime freshman fail to earn a college degree in six years. Although there has
been a concerted effort to increase opportunities for minority students to access
higher education, this step has not produced the anticipated positive outcomes such
as increased retention or degree attainment in groups of underrepresented minority

4
students. As President Johnson stated, it has not been enough to “open the gates of
opportunity” (Johnson, 1965).
The 2002 Census Bureau statistics demonstrated that over a lifetime, a
baccalaureate degree enables a person to earn almost twice as much as a high
school graduate (United States Census Bureau, 2002). Inequitable educational
outcomes for historically underrepresented minority students has been sustained as
the “status quo” and has remained as one of the most deeply entrenched problems
in institutions of higher education. The ability to be a viable contender in a
competitive market is profoundly disabled without an equitable education.
Bensimon, Polkinghorne, Bauman & Vallejo (2004) illustrates this consequence by
referring to impact of demographic changes upon education and the economy in
California, “Increasingly, the scenario being written for California is one of
economic polarization because the growing sector of the population, primarily
Latino, is not reaching the educational levels that will be a prerequisite for the jobs
in the future” (p.110-111).
Background of the Diversity Scorecard Project
In 2000, the Center for Urban Education (CUE) at the Rossier School of
Education at the University of Southern California was awarded a grant from the
James Irvine Foundation to assist institutions of higher education in monitoring the
status of educational outcomes among historically underrepresented minority
students, specifically, the Hispanic and African American populations. The
Diversity Scorecard Project was developed, designed and implemented as a method

5
by which institutions of higher education could look at existing data from an equity
perspective and develop indicators to assess the institution’s improvement towards
that end. The goal of the project was twofold: to identify areas of inequity and then
to create “benchmarks” or indicators to measure the improvement or decline of
equity. The Diversity Scorecard Project participants consisted of 14 campus
partners which were defined by the researchers as “opportunity colleges” or
colleges that had already achieved equity in access and enrollment. Each institution
appointed an “evidence team” comprised of college wide and departmental
administrators, faculty, and staff from multiple disciplines. A staff member from
the Office of Institutional Research was a required participant and acted as a key
resource in accessing and disaggregating the data.
Traditionally, institutional performance and educational outcomes are
measured using aggregated data that do not capture educational outcomes for
specific student populations (Bensimon, 2004). Disaggregating the data by race
and ethnicity would allow institutions to re-frame existing facts and figures from an
equity perspective. By using this approach, the “missing link between goals of
diversity and the evidence of results” would be made visible (Bensimon, 2000).
The evidence team’s charge was to not only collect and report the status of the
current equity from the data base, but to use the Diversity Scorecard as a
framework or tool to analyze and interpret the findings. The conceptual framework
of the Diversity Scorecard consists of four perspectives: access, retention,
institutional receptivity, and excellence. These four perspectives provide a “lens”

6
through which an institution of higher education can view educational outcomes as
an indicator of institutional performance rather than student performance. It is
important to appreciate this critical distinction since it shifts accountability from the
student to the institution and the institutional actors, requiring institutional actors to
look at data in novel and unfamiliar ways. Using these four perspectives as a
framework, the evidence team in each institution presented their findings and
recommended indicators and criteria to systematically measure and monitor the
progress in achieving educational outcomes in the underrepresented minority
populations.

Figure 1: Diversity Scorecard Framework (University of Southern California,
2004)
One of the desired dynamics was that team members would engage in
reflective discourse, and thus reshape their attitudes and understanding through
evidence that demonstrated inequities in educational outcomes for Hispanic and
African American students. The expectation was that by developing new ways of

7
thinking (cognitive frames) individuals would learn to use evidence from an equity
perspective (Bensimon, 2005).
Research staff and facilitators from the Diversity Scorecard Project were
assigned to each of the 14 evidence teams to provide guidance in the process and to
keep the institutions’ progress focused with quarterly meetings, conferences and
frequent communiqués.
An additional characteristic of this project was the divergence from the
conventional approach in conducting institutional research where “outsiders”
collect, analyze and report data back to the institution. The indicators to monitor
equitable outcomes would be determined by the institutional actors and focus upon
the values and attitudes of the colleges themselves, thereby contextualizing the
proposed interventions or recommendations. Bensimon, Polkinghorne, Bauman
and Vallejo (2004) present the “practitioner-as-researcher” model as a strategy to
encourage practitioners to assume the role of the researchers. Another hopeful
outcome would be that institutions would not only recognize the state of
inequalities on campus, but become well-versed experts who were empowered to
make meaningful changes in their environment. There is considerable complexity
inherent to this approach and delves into the realm of meta-cognitive processes,
involving the integration of cognitive and affective learning domains and
conceptual adult learning models such as transformative learning (Kegan, 2000).

8
Statement of the Problem
The most fundamental purpose of a university and its faculty is to create a
scholarly environment that is committed to teaching and learning. Equity is often
philosophically embraced as a “core value” in mission statements and strategic
plans. Inasmuch as institutions and faculty claim to have knowledge and
understanding of diversity and incorporate this and the needs of underrepresented
minority students, institutions and its actors do not demonstrate evidence of
learning related to recognition of the inequitable educational outcomes shouldered
by underrepresented minority students. Assessment and interventions to promote
retention and persistence of underrepresented minority students have been
primarily superficial and palliative- an example of “single loop
learning”(Argyris,1991) which does not encourage an intense, more in depth
investigation of the problem. It requires more than asking the customary questions
when reviewing the annual statistics. In order to have a clear and valid
understanding of the factors that create the problem, institutional actors must learn
how to ask questions and analyze data in a manner that raises consciousness to
probe further into causation. Organizations often “fix the wrong problem” because
the intervention does not focus on identifying and manipulating causative factors.
In this way, institutions do not reap the benefits of meaningful learning, such as a
positive change in practices and continue to perpetuate vested attitudes and
practices (Argyris,1977).

9
Many private and public institutions of higher education have been awarded
grants or accreditations, such as the Hispanic Serving Institutions, for their
achievements in designing programs that are intended to attract, admit, enroll or
retain culturally diverse students or target Hispanic and African American students.
Although these achievements should not be minimized or discounted as progress, it
represents only a limited advancement towards genuine equity when translated to
the educational outcomes of underrepresented minority students. This type of
recognition can lead to misguided perceptions by institutional actors that “equity”
is inherent to accomplishing diversity and that their “work is done.” This
perception is supported by the fact that individual and institutional performance are
not traditionally measured in terms of educational outcomes by race and ethnicity.
Therefore, inequity in educational outcomes may not be unearthed as a problem at
all or if identified, may not be viewed as a problem related to institutional learning
or performance.
Customary methods of analyzing institutional data perpetuates existing
cognitive frames and therefore can only reveal what is already apparent,
metaphorically similar to only seeing the “tip of an iceberg.” It is this continued
lack of intense questioning, a resistance to conducting a deeper investigation or a
more complete awareness of the problem that has perpetuated the widening gap of
inequities in educational outcomes for over fifty years of attempted educational
reforms. Bensimon (2005) explains that: “Diversity-minded individuals are attuned
to demographic differences, e.g., they will comment on how diverse the student

10
population is or how it lacks diversity, but, more likely than not, they will be blind
to the fact that the very students whose presence makes campus diversity possible
are themselves experiencing unequal educational outcomes”(p.102). Without an
awareness, recognition or impetus to look beyond what is apparent, it is unlikely
that institutional actors would be motivated to learn, develop or execute “equity
minded” strategies to improve institutional performance.
The Purpose of the Study
The purpose of this study was twofold. The first purpose was to explore the
presence of transformative learning by the individuals of one evidence team
participating in the Diversity Scorecard Project through Mezirow’s transformative
learning framework, specifically through the stages of critical reflection, content,
process and premise reflection. The second motive was to gain an insight into the
elements that encouraged or inhibited individual learning among the team
members. To achieve this, I have read, coded, analyzed and interpreted the
conversational texts from the meetings of one evidence team composed of
institutional actors from one of the 14 participating colleges in the University of
Southern California, Center for Urban Education-Diversity Scorecard Project. In
addition to the field notes, I explored the institution’s official website to gain some
familiarity with various public resources such as the college catalogue, mission
statement, campus cultural events, student affair activities and academic policies.
Central to this study was the intention to determine if institutional actors,
once aware of inequities, would be able to learn how to use this newly acquired

11
evidence to further investigate and define problems through the DSP’s equity
perspectives of access, retention, receptivity, and excellence.
The Significance of the Study
There is dual significance to this study. Since institutions are composed of
individuals and groups, it is important to gain insight into how these individual
approached information and the processes that encouraged learning. Additionally
germane to this evaluation of learning, was to identify, through the DPS transcripts,
how this learning manifested through the words and actions of the individual
institutional actors.
The database used in this study was from the collected field notes of one
institution during its participation in the Diversity Scorecard Project (DSP). To
fully grasp the work of the evidence team and the CUE researchers, it is important
to provide an overview of the project’s theoretical framework, premises and
guiding principles.
Research Questions
The research questions in this study emanate from an overarching principle
that is socially constructed and facilitated by engagement in joint productive
activity. The research questions for this case study were:
1. During their participation on the DSP, how did the individual evidence
team members become aware, reflect and learn about inequities in
educational outcomes for minority students?
2. How did the individual evidence team members’ use of content, process

12
and premise reflection to learn and what was the impact or change upon
their interpretations or assumptions about inequities in educational
outcomes for minority students change over the course of the project?
3. What were the factors that enhanced or inhibited the individual evidence
team members’ reflection and learning? What can be said about learning
and change in relation to equity, what facilitated or inhibited the subjects’
learning and change in relation to equity and how did group dynamics,
reflective discourse, and assumptions influence this process?
I chose Mezirow’s transformative learning theory and content, process and
premise reflection as the conceptual framework because it provided a method to
organize and interpret information, keeping a focused approach in interpreting the
text within the parameters of the research questions. It provided a structure for
reviewing information and artifacts.
However, since the construction of knowledge by individuals and groups is
achieved through a matrix of physiological, psychological, and sociological
processes, knowledge can be highly contextual and is shaped through experiences
that are interpreted through both the cognitive and affective domains. Figure 2 is a
diagram representing the salient premises of a positive constructivist model as it
relates to Mezirow’s transformative theory and how critical reflection transpires.

13

Figure 2: Positive Constructivist Models of Learning

Methodology
This study employs a single, revelatory case method to explore the
comments, actions and behaviors of individuals participating in the Diversity
Scorecard project in one of the 14 participating colleges The unit of analysis is the
individual, specifically the learning that transpired in each individual through the
team’s dynamics and processes. The single, revelatory case method allows the
researcher to explore and describe emerging relationships and themes and the
relationships between or among evolving phenomena that was previously
inaccessible to scientific investigation (Yin, 2003, p.43).
Limitations of the study
This study was limited to the review of field notes from one team of
institutional actors recorded during their participation as evidence team members in
the Diversity Scorecard Project. The intent of this study was to assess whether
individual learning had taken place through participation and what were the effects
of this learning, if any, upon the attitudes and actions of the individuals. Great
attention was given to examining the text for depth of awareness and understanding
rather than generalizability. The field notes, though comprehensive, served as the

14
main resource to evaluate whether learning had occurred as a result of participating
on the team. This is also a retrospective study that analyzed field notes over an 18
month period of time, from the initial implementation period through the
presentation of the Scorecard in the President’s Report. Therefore, I can not
validate that learning was sustained by the individual institutional actors or that
their knowledge was translated into organizational learning. Additionally, since the
text was recorded by the CUE facilitators, the interpretation of the teams’
discussions is subject to the unavoidable perceptions or biases inherent to
observation.
Definition of Terms
Access Perspective: Access indicators enable institutional leaders to become more
fully informed about the extent to which underrepresented students gain access to
the institution’s programs and resources (Bensimon, 2004).
Retention Perspective: Indicators constructed from the retention perspective
address questions related to retention rates, disproportionate withdrawal from
courses, completion rates, etc in underrepresented minority students (Bensimon,
2004).
Institutional Receptivity Perspective: Examines dimensions of institutional support
that can help create a more accommodating and responsive campus environment
for students drawn from underrepresented groups (Bensimon, 2004).
Excellence Perspective: Refers to indicators of exceptional achievement or provide
evidence of students’ achievement. (Bensimon, 2004).

15
Equitable: having or exhibiting equity, dealing fairly and equally with all
concerned.
Evidence Team: Composed of "evidence monitors," this team presents compelling
data to others in the institution which show the status quo as well as the desired
status. The team continues to monitor progress toward the achievement of the
equity benchmarks (Bensimon, 2004).
Stage I Diversity: Refers to equity in representation. Historically underserved
students make up significant portions of the student body. Many ethnic/racial
groups are represented on campus as the result of initiatives aimed at achieving
equal access to higher education. (Bensimon, 2004).
Stage II Diversity: Refers to equity in outcomes. Initiatives are aimed at achieving
equitable educational outcomes for students. Equal representation of historically
underserved students not only in the student body, but also in graduation rates, on
the Dean's list, across all majors, etc.
Transformative learning: Refers to a rational process of learning within awareness
as a meta-cognitive application of critical thinking that transforms an acquired
frame of reference- a mindset or worldview of orienting assumptions and
expectations involving values, beliefs, and concepts-by assessing its epistemic
assumptions (Mezirow, 2006).
Organization of the study
A description of this study, including the review of the literature,
methodology, results and concluding remarks are discussed throughout five

16
chapters. The first chapter lays the foundation and background for the study as a
method for evaluating individual learning as a result of participating in the
Diversity Scorecard Project. Chapter 2 presents a review of relevant literature and
research describing the conceptual frameworks and theories used in assessing
individual and group learning and its relationship to organizational learning. The
methodology of this study is the focus of Chapter 3 which explains the research
design, conceptual framework, and the techniques used to organize, analyze and
interpret the texts which comprise the data base for this study. In Chapter 4, the
results of the analyzed text are presented and finally, Chapter 5, discusses
conclusions drawn from the study as well as recommendations for future research.

17
CHAPTER TWO
REVIEW OF THE LITERATURE
For centuries, scholars and researchers from vast and diverse disciplines
have struggled to understand the complexity and dynamics of learning. Enlightened
by Merriam’s comment, “No problem in education exists in isolation from other
areas of human behavior” (Merriam, 1998, p. 50-51), this review of the literature is
a discussion of developmental cognitive learning theories, research and conceptual
frameworks that are relevant to the topic of individual learning and how learning
affects or changes an individual’s behaviors and attitudes. However, it is imperative
for individuals to assess their own values, perceptions and knowledge about
learning before they can modify their thoughts and practices. This requires a
deliberate effort to integrate the new knowledge through the affective and cognitive
learning domains. When learning about problems that impact inequities and social
change, one must also become awakened to a “social consciousness” as critical part
in the process of learning.
As described in Chapter One, the purpose of this case study was to
determine whether or not there was evidence of learning in individual team
members, specifically, if learning occurred, how did this newly acquired
knowledge influence, change or transform the attitudes and actions of the
individual. An underlying assumption of the Diversity Scorecard Project was that
once the team members were exposed to the newly constructed knowledge

18
produced from the disaggregated data or evidence, they would also gain insight into
how their attitudes, feelings and practices influenced or promoted
inequities in the educational outcomes of historically underrepresented minority
students. An ultimate goal of this individual and group learning would then be able
to be translated into organizational learning.
Organization of the literature review
Since there is a vast expanse of literature addressing learning theories, this
specific review of the literature will address three areas of adult learning. The first
section provides an overview and discussion of cognitive learning through the lens
of two modern constructivists, Jean Piaget and Lev Vygotsky followed by a
discussion of transformative learning theory and the interconnectedness of
cognitive developmental theories to Mezirow’s conceptual processes that culminate
in what is referred to as transformative learning. The final section presents
relevant current research describing the integration and application of the principles
of transformative learning theory as it relates to individuals and groups in various
settings including learning communities, organizations and institutions of higher
education.
Constructivist cognitive theories
Principles of constructivism
A central tenet to constructivist thought is that individuals actively construct
meaning, understanding and knowledge through their experiences and reflections

19
of the world. In this way, individuals create mental models from previous
experiences to help them make sense of new encounters.
Constructivism is a complex, tightly woven paradigm whose roots are
deeply embedded in philosophy, psychology and education. It is an epistemology
that delves into the thoughts, experiences, knowledge and skills of an individual.
The complexity is further influenced by the social context and developmental stage
of the individual. Although individuals learn through cognitive and socio-affective
domains they must ultimately be able to incorporate conceptual, experiential, and
emotional knowledge into their actions and interactions with others.
Modern Constructivists
In general, modern constructivists believe that people generate meaning
using existing or previously acquired knowledge to understand or make sense of
new information and situations. The key principles related to constructivist thought
are: knowledge is socially constructed, learning is an active process, knowledge is
constructed from experiences and that learning and development are self regulated
(Byrnes, 1996). Two of the leading pioneers who produced research-based
evidence in relation to cognitive development and learning are Jean Piaget and Lev
Vygotsky. In the following section, I will present an overview of the cognitive
models developed by these two theorists which will provide a theoretical
foundation for approaching transformative learning models.

20
Piaget’s perspectives on cognitive development
Jean Piaget, a developmental psychologist, described how children
construct knowledge in terms of schemata, concepts and structures (Byrnes,1996).
As with many stage theories, mastery is progressive and somewhat dependent upon
successful achievement of previous steps. Byrnes clarifies the distinction Piaget
makes between schemata and concepts by pointing out that concepts are ways of
understanding relationships or constructing meaning whereas schemata are goal
directed in terms of learning a skill or procedure (Byrnes, 1996). One of the key
elements of Piaget’s developmental cognitive framework is that learning is not
solely internally motivated or externally directed, but a complex association of
experiences and activities by which the individual constructs meaning. For
example, an individual will take a new experience and try to place it within the
realm of existing knowledge. This process is referred to as assimilation, whereas
accommodation is used for those experiences that are so novel that individuals
must somehow manipulate the concept or experience to make sense of it in the
context of their lives (Piaget, 1952).
According to Piaget, cognitive development, from infancy through
adulthood, is an orderly and sequential process that is predictable. Schemata are
structures created by the child’s mind which are used to help the child adapt to or
organize the environment (Piaget, 1969). To have sustained learning, Piaget posits
that the acquisition of problem solving skills, actions or schemata need to be
enduring and could be generalized to the greater population (Piaget, 1952). This

21
process of intellectual development continues into adulthood through the
individual’s ability to adapt, construct and reconstruct complex cognitive
structures.
Piaget considered learning to be a dynamic process that is stimulated by events
that trigger a disruption to the individual’s equilibrium. Constructivism has at
its core a belief that people are intuitively aware when their experiences and
expectations are incongruent. Pulling from basic biological principles, Piaget
likened human learning behavior to that of organisms that are always striving to
maintain homeostasis. The impetus that motivates individuals to adapt or
modify their behavior are prompted by situations that challenge homeostasis or
their equilibrium, a discomfort that creates a state of “disequilibrium”. This
state of psychic discomfort acts as a catalyst for learning and is referred to as a
“dynamic disequilibrium”. Therefore, in order to self regulate, the individual
must be able to perceive cognitive tension. The goal of the individual, at any
age, is to regain balance or re-equilibrate using the processes of accommodation
and assimilation. Through these processes, knowledge is re-organized and new
schemata are generated.
Formal operations or abstract thought is evident when the individual is able to
construct meaning from new knowledge and formulate hypotheses. With
abstract thinking, the individual is able to solve complex problems and can
extract the logic of the problem devoid of content – the problem solving has
evolved to the point whereby the individual can apply previously learned

22
constructs to a myriad of daily encounters where facts and details constantly
change. The role of language development was seen by Piaget as part of this
general cognitive development and he believed that language acquisition
allowed the individual to use symbols, that is, words or gestures to
communicate concepts.
Piaget’s theory of cognitive development is highly contingent upon the child’s
chronological age and does not take into consideration that wide ranges of
variations exist which are dependent upon one’s physical, psychological and
sociocultural status.
Chronological age cannot be the sole determinant of how one’s cognitive maps
are formulated (Byrnes, 1996).
Constructivism according to Vygotsky
While the individual is the pivotal component of Piaget’s cognitive theory,
Vygotsky envisioned learning as an integrated experience that relied heavily on the
individual’s socio-historical and cultural experiences. A child’s learning is then
highly dependent upon the social environment and particular language of the social
unit. Unlike Piaget, where learning occurs because of preexisting, innate
determinants, Vygotsky felt that socio-cultural-historical interactions were the chief
factors influencing a child’s ability to learn.
A distinctive concept in Vygotsky’s work is the identification of the “zone of
proximal development” which is the distance between the actual developmental

23
level (the learner’s ability to problem solve independently) and the level of
potential development when provided with coaching, mentoring, modeling or
assistance (Byrnes, 1996).
The steps that are initiated to assist the learner by mentors, coaches, educators
or peers is referred to as “scaffolding”- much in the same way that construction
is supported by intermediate structures. Vygotsky contends that providing
opportunities that motivate an individual to extend their learning beyond their
current level of skill or competence is critical. Learning takes place within the
context of the individual’s milieu and it is this context with which an individual
approaches lived experiences and interactions.
This concept of “context “is pervasive throughout one’s life when approaching
new situations and knowledge and is infused in one’s cognition, values and
actions. To Vygotsky, language was acquired and used as a tool to understand
the relationships among thoughts, experiences and social interactions.
Individuals use words and language as a way of mediating and navigating their
thoughts and experiences. This notion of “word” as an avenue to communicate
thoughts and feelings plays a critical role in Vygotsky’s approach to cognitive
development as well as in transformative learning theory.
To summarize, constructivism places a heavy emphasis on the individual’s
ability to construct meaning from their experiences. However, constructivists must
take into consideration that knowledge is not acquired in isolation, but is socially

24
constructed. The construction of meaning is highly dependent upon the individual’s
social interactions, personal perceptions, cognitive ability and unique interpretation
of the experiences. Although stage theories do provide some normative
expectations at particular points in the trajectory of maturation, they are often
inflexible and do not take into account the variables that enhance or inhibit
cognitive development. The ability to synthesize interactions with the physical and
social environment are highly variable and contingent upon one’s opportunities to
encounter and make meaning from these experiences. Therefore, more importantly
than chronological age or predetermined stages, individuals learn when there is
value and meaning in the learning, when there are repeated opportunities to practice
or apply what is learned and when they are able to contextualize new cognitive
frames or mental models.
Transformative learning
Principles of transformative learning as a dimension of adult learning
The epistemological “heart” of transformative learning theory, lies in one’s
ability to engage in critical assessment and re-assessment of the values, thoughts,
experiences, knowledge and skills that construct current perspectives and drive his
or her actions. As Merriam (2004) points out, higher levels of cognitive ability
must be present for critical reflection and rational discourse. In other words,
transformative learning requires the process of metacognitive reasoning
(Mezirow,2003). Mezirow asserts that transformative learning is “learning that
transforms problematic frames of reference-sets of fixed assumptions and

25
expectations, (habits of mind, meaning perspectives, mindsets)-to make them more
inclusive, discriminating, open, reflective, and emotionally able to change
(Mezirow, 2003, p. 58).
Jack Mezirow and Paolo Freire shared a fundamental philosophy and belief
that emerging educational theories needed to encompass an element of critical
reflection to enable an increased awareness of one’s thoughts, feelings, values and
actions. While Mezirow focused upon the personal transformation in individuals,
Freire turned his energy towards social transformation and believed that the
transformation of a society could not occur in isolation or be separated from
personal transformation. Both theorists acknowledge the importance of developing
a critical consciousness, challenging an individual or a society to unearth the
assumptions and actions that undermine or promote oppression, marginalization or
social inequity. Inherent to this belief, is the notion that to create deep and enduring
change, it is necessary for individuals and societies to discard distorted realities that
undermine growth and equity and make meaning and sense of new perceptions.
Freirean approach to transformative learning
Many of the foundational constructs of Paolo Freire’s critical pedagogy are
consistent with a constructivist approach to learning. Similar to Vygotsky, Friere
believed that if individuals are to learn, they need to actively engage with others
and their environment. Dialogue is an imperative aspect of active social learning
and the construction of knowledge through meaningful contexts.

26

Through the process of “critical transitive thought” or critical social
consciousness, individuals are able to develop an awareness and act upon injustice
(Freire, 1970). When addressing issues of conscientizacao
1
, Freire compels
individuals to actively and consciously change the attitudes and practices that
perpetuate and sustain inequities (p. 20).
The traditional educational model as described by Freire was a process in
which students were viewed as empty “receptacles” that could be filled with
knowledge that was exclusively held by educators. “Banking” is the term Freire
used for a bureaucratic educational system that promoted an attitude that
knowledge was bestowed upon or deposited into the empty learner from a
benevolent and paternalistic expert (p.58). This tactic promotes an educational caste
system that perpetuates discrimination of the “haves” from the “have-nots” and
denounces the value of an individual’s or group of individuals’ unique and personal
attributes (p. 44).
The course of action forged by Freire was to reinvent an educational
structure that empowered marginalized groups to find their voice. Word and
dialogue take on a deeper and richer interpretation than what was previously
described by Piaget and Vygotsky. The power and action of words penetrate
beyond the cognitive and social domains of language or conversation. The “word”
is more than an instrument to converse, it has the dimensions of reflection and

1
The term conscientizacao refers to learning to perceive social, political, and economic
contradictions, and to take action against the oppressive elements of reality (Freire, 1970).

27
action. “There is no true word that is not at the same time a praxis. Thus, to speak
a true word is to transform the world”(p.75).
As conjectured by Freire, dialogue must be hopeful in that those who
mediate must be critical thinkers in search of resolving contradiction. To
authentically change a deeply entrenched culture of oppression, the oppressed and
the oppressors would need to confront and reflect upon the realities of social
inequity and willingly dialogue to mediate change. This transformation occurs
when education is embraced as a process that requires “acts of cognition” rather
than a transference of information (p. 67).
Mezirow’s Transformative Learning Theory
Jack Mezirow (2000) strives to remind us that learning is primal,
fundamental to our humanity, and presents itself with an urgency that drives us to
make order and sense of our lived experiences. He views transformative learning as
the process by which we transform our taken-for-granted frames of reference
(meaning perspectives, habits of mind, mind sets) to make them more inclusive,
discriminating, open, and emotionally capable of change. Reflection is also
necessary to generate new beliefs and opinions that would prove to guide one’s
actions with greater truth or justification. Taylor (2001) concurs with Mezirow that
the process of “making meaning is shaped and circumscribed by meaning
structures”(p.220). In considering adult learning, Mezirow places considerable
emphasis on understanding how the adult makes meaning and sense from their
contextualized experiences. He surmises that the adult must be able to engage in

28
dialogue or discourse that leads to an introspective, probing assessment of one’s
beliefs and assumptions, that is, critical reflection(Mezirow, 2000).
The dialogue or reflective discourse that Mezirow refers to is an
interpersonal process that occurs when beliefs and values are the topics of the
discussion. The discourse does not focus on debating or arguing the facts and
figures of an issue, rather, it challenges and obliges the individual to become open
to acknowledging the existence of other assumptions, truths, and perspectives
beyond their existing frames of references and habits of mind. He explains that
critical dialectical discourse involves listening with empathy and an open-
mindedness that shuns premature conclusions while searching for common ground
(Mezirow, 2003, p. 60). Analogous to Freire’s foundational concept of
conscientizacao, individuals who are marginalized from society through
homelessness, illness and other social conditions that result in destitution are
excluded from this discourse. Mezirow declares there is an epistemological
justification for individuals to engage in social actions that overcome exclusionary
practices (Mezirow, 2003).
Communicative and instrumental learning. According to Mezirow, several
processes are necessary to understand the learning domains of transformative
theory which include communicative learning and instrumental learning.
Drawing from the work of Habermas, Mezirow emphasizes that a key feature is
one’s ability to understand what others are attempting to convey through their
words, that is, the feelings, intentions, perceptions, values and beliefs that are

29
embedded in the communicated word (Mezirow, 2000. p. 8). Knowledge derived
from communicative learning is interpretative and is contextualized by our
interpersonal knowledge of our culture or society and is conveyed through
language. With communicative learning, individuals are encouraged to pose
questions to deeply investigate the truthfulness of their assumptions. Therefore, this
type of learning is highly dependent and bound by one’s culture and contextual
understanding of the words that are used to communicate the information. In
addition to understanding others, one must learn how to make oneself understood
(Williams, 2000). Mezirow suggests that our empirical knowledge is tested
through an instrumental approach and our justification of this knowledge and
assumptions is processed through communicative learning (Mezirow, 2000, p.10).
Through communicative learning, individuals dialogue or engage in
discourse that prompts a critical appraisal of one’s assumptions and beliefs through
self-reflection. Whereas, communicative learning requires an interpretation of
another’s assumptions, objectives, meanings, values and unarticulated innuendo of
dialogue, instrumental learning refers to manipulation and control of the
environment as a means to learn tasks and gain proficiency. The content and
knowledge is concrete, objective, empirical and constrained by principles and laws.
The key distinction made by Mezirow is that instrumental learning involves
hypothetical deduction in contrast to communicative learning which requires a
developmental logic that encompasses “analogic-abductive” suppositions
(Mezirow, 2003). Therefore a combination of both types of learning is necessary if

30
transformation is to occur. One must not only understand epistemic meaning of
knowledge (instrumental learning), but must also be able to engage in a thoughtful
self analysis to understand, interpret and validate rightness and authenticity of the
one’s assumptions and claims of truth (communicative learning), thus prompting
the individual to become critically reflective (Mezirow, 1998).
In discussing how adults learn and transform knowledge into action,
Mezirow (2000) stresses the importance of acknowledging “habits of mind” (p.17).
A “habit of mind” is a broad, generalized assumption that guides our interpretation
of an experience and is expressed as a point of view (p.18). Habits of mind are
contextually constructed, well ingrained, automatic cognitive and affective
responses to new situations. It is a way of thinking that is reflexive, devoid of
thoughtful analysis and outside of our awareness.
A frame of reference, as defined by Mezirow (2000) has two features,
habits of mind and the resulting points of view (p. 17). It is from these frames of
reference or meaning perspectives that we filter our experiences, and become
grounded, oriented and can assign meaning and sense to our lived experiences in
the context of the world around us. Individuals become dependent upon these core
values and beliefs to navigate through their world of interactions, thoughts, feeling
and actions. According to Mezirow, our frames of reference are so closely aligned
with “self”-what you believe is who you are: “We may transform our habit of the
mind by becoming critically reflective of our premises in defining the problem,
such as by questioning the validity of our assumptions” (p.20). Therefore,

31
individuals and groups must be willing to re-form the vision, mission, systems and
practices to enhance performance. As with all stage theories, the exacting moments
that illustrate the progression of an individual’s journey towards transformation are
illusive, fluid and difficult to encapsulate at a specific juncture or moment in time.
Similar to Piaget’s “dynamic disequilibrium”, Mezirow’s transformative
learning theory employs a similar catalyst, in the form of a ”disorienting
dilemma” which challenges one to re- frame our “meaning perspectives” or set
of expectations based on past experiences (Mezirow, 1998). This disorienting
dilemma provokes one to engage in critical reflection in an attempt to make
meaning and sense from an experience that created state of psychological or
social incongruence. Through critical reflection, can new meaning perspectives
can be generated which allows one to re-equilibrate and re-establish sense of
balance through new understanding.
Mezirow’s critical reflection
The ultimate goal of transformative learning is to enable the individual to
critically reflect upon social experiences and assign a new interpretation and
meaning to these experiences. Critical reflection is the process through which
transformative learning occurs, therefore it not only pivotal but imperative that
an adult is able to engage in this particular type of thinking. Critical reflection is
the thought process used to revise meaning structures and has three phases:
content reflection, process reflection and premise reflection.

32
When individuals consider a problem or experience, they learn about the
problem by analyzing the content. This encompasses what we perceive the
problem to be and how we think, feel or act upon it (Mezirow, 2000). It is an
examination of the problem’s subject matter and often asks the question
“what”. For example, when interpreting retention data in higher education, one
would not only look at the percentages of students retained or lost, but would
also investigate deeper into other factors that may impact or influence the
problem such as gender, age and ethnic background. This type of inquiry leads
the individual to content reflection and enables a better understanding of the
issues and consider different perspectives.
The next phase is process reflection, which involves the development of
strategies or plans to solve the problem. It is primarily concerned with the
manner in which we think about or manage the problems and includes the
exploration of additional facts to understand how current practices impact the
problem which can aid in developing new strategies and approaches in
managing the dilemma. Process reflection compels us to ask the question
“how” and is a method to garner additional facts or content (Williams, 2001).
However, when discussing critical reflection, Mezirow suggests that
transformative learning occurs at the level of premise reflection, whereby the
individual questions their own assumptions and basis for the existence of the
problem. At this juncture, the individual asks “why” and examines the validity
of their assumptions, perceptions and values and how these influence their

33
actions, strategies and decision making. Mezirow, like Freire, holds steadfast in
the belief that transformation occurs when there is a fundamental re-ordering,
reframing, or questioning of pre-existing assumptions or “habits of the mind”
(p. 21). Both theorists are emphatic in placing reflective discourse or dialogue
as an essential constituent in the process of critical reflection and finding one’s
voice.
Perspective transformation
In order to make sense of our experiences, we must be able to organize,
interpret and assimilate or assign meaning to our thoughts, actions and perceptions.
There are life events, however, that may be overwhelmingly disruptive to an
individual’s core beliefs and values and threaten one’s psychological, social, and
spiritual existence. These can be personal events such as loss of a parent, loss of
health, or something catastrophic that unnerves the very essence of daily routine
and stability such as the threat of terrorism. These life changing events transform
the lens and subjectivity through which our lives are experienced, and consequently
we reframe how life is experienced. There is a critical distinction between a change
in perspective (how one views life) and perspective transformation (how one lives
life) in that, perspective transformation requires the individual to reintegrate the
newly acquired knowledge, skills and conditions to make their lives congruent with
the new perspective (Taylor, 1993). Mezirow contends that individuals must move
through several phases to experience perspective transformation. These phases are
largely dependent upon one’s ability to engage in self reflection and a willingness

34
to be open to understanding and integrating new knowledge. Mezirow structures
these phases into 10 progressive steps: 1) a disorienting dilemma; 2) self
examination with feelings of guilt or shame; 3) a critical assessment of one's
epistemic, socio-cultural, or psychic assumptions; and 4) recognition that one's
discontent and the process of transformation are shared and that others have
negotiated a similar change, 5) exploration of options to form new roles,
relationships, and actions 6) planning a course of action; 7) acquisition of
knowledge and skills for implementing one's plans; 8) provisional trying of new
roles; and 9) building of competence and self-confidence in new roles and
relationships and lastly, 10) a reintegration into one's life on the basis of conditions
dictated by one's new perspective. (Mezirow,1991).
An alternative perspective to Mezirow’s transformative learning theory was
explored by Robert Boyd. According to Boyd, transformation is a process and
journey of individuation where the individual discovers one’s inner self; the
emotional, spiritual and affective dimensions of self, that ultimately form one’s
personality (Boyd, 1991). Whereas the dominant view focuses on the cognitive
rationality of critical reflection, Boyd’s vision is grounded in psychoanalytical
constructs inspired by Carl Jung. The most significant distinction between these
two models of transformative learning is that while Mezirow’s “conflict”
emanates from an individual’s struggle when meaning perspectives become
incongruent with the external forces such as social and cultural experiences,
Boyd’s focus is upon the internal conflicts that lead one to transformation.

35
Transformative learning as adult learning theory
Transformative learning attempts to describe how our cultural and social
assumptions navigate our expectations, meaning perspectives and actions.
However, as with all stage theories, there are commonly held concerns about
the empirical rigidity of these conceptual models when applied to “real life”
human behavior. The processes are not necessarily accomplished in a linear,
progressive and step-by-step fashion. Conceptual frameworks and models
attempt to generalize human responses to interactions and situations and may,
to some degree, allow us to anticipate commonly observed or anticipated
behaviors. Yet, because of the complexity of human experience, there may be
wide ranges of normal variants and individuals may move through the stages
out of sequence or even regress. Mezirow conjectures that meaning
perspectives are developed unconsciously throughout one’s childhood and it is
not until adulthood that one can critically assess or reflect upon these deeply
entrenched beliefs, values and actions. The assumption, then, is that at a certain
age, one is able to engage in collaborative inquiry and reflective discourse.
Similarly, Piaget’s tenet that adolescents should be transitioning to formal
operations or abstract thinking has been challenged by empirical studies over
the decades. Chronological maturation alone does appear to have as much
significance as the composition of physical, social and cultural experiences
upon meta-cognition. Merriam (2004) maintains that to “engage in reflective
discourse, with others assumes the ability to examine alternative perspectives,

36
withhold premature judgment, and basically think dialectically, a characteristic
of mature cognitive development” (p.61).
A major criticism of transformative learning theory is that Mezirow relies too
heavily on the “rationality” of critical reflection (Taylor, 2001, p. 220). In an
effort to substantiate the emotional character of rationality and other “ways of
knowing”, Taylor (2001) contends that the disproportionate emphasis on
rationality and critical reflection serves to negate the synergy between feelings
and thoughts, while acknowledging that such an interdependence would
encourage transformative learning. In considering the complexity of human
cognitive processes, Taylor examined seminal studies that explored the
neurobiological perspectives that exist among rationality, implicit memory,
habits, emotions, feelings and attitudes in transformative learning theory. If we
were to consider that man is a bio-psycho-social-spiritual being with the
capacity to learn and integrate knowledge from a multitude of venues. It is
apparent that individuals construct meaning through cognitive-somatic domains
as well as intuitive-affective realms. Therefore, the adult learner must be able
to approach life experiences and social interactions with a level of
developmental and emotional maturity to engage in critical reflection and
reflective discourse requiring meta-cognitive skills.

37
Application of Transformative Learning Theory
Individual learning
The ability to critically reflect and engage in rational discourse is vital if
transformational learning is to occur (Merriam, 2004). Inherent to these
metacognitive functions lie an individual’s psychological, emotional, social and
spiritual maturity. An introspective analysis of one’s values, beliefs, attitudes
and actions is daunting, especially for those who realize the ramifications of
becoming aware of “self”. One must be emotionally and cognitively capable of
not only reflecting upon personal perspectives, but must then seek to
understand the contextualized perspectives of others. Merriam posits that these
higher levels of cognitive functioning would reflect Piaget’s fourth stage of
formal operations
(p. 63).
Learning involves complex interactions that intersect the community and
environment which provide rich and multidimensional experiences leading to a
“plane of consciousness” (Tharp & Gallimore, 1988). This concept of socialization
and acculturation is similar to Mezirow’s “frames of reference” in which the
individual uses previously acquired interpretations to navigate through new
conditions. Like Vygotsky, Tharp and Gallimore (1988) conducted a study that
assisted performance in relation to individual learning and revealed that learning is
not a passive process in which individuals are “empty receptacles” awaiting the

38
deposition of knowledge. The notion of “empty receptacles” reiterates one of
Freire’s most fundamental premises for the education reformation which he defined
as “banking”(Freire, 1970). This concept is critical in an analysis of individual or
group learning and the impact upon organizational change since it shifts the energy
of teaching from passive “downloading” into active inquiry.
Kim (1993) similarly defines learning in terms of operational and
conceptual skills. Kim describes that operational learning requires the ability to
access and gain knowledge (know-how) while conceptual learning involves the
ability to understand and use the knowledge (know-why) (p. 38). Therefore, it
would not be sufficient to have institutional actors simply gather, interpret and
report recruitment and retention numbers. This is an example of instrumental
learning in that the practice and manipulation of data is a technical skill. Current
methods of reporting higher education statistics in areas of retention and
completion are instrumental and often, unidimensional when reporting outcomes.
Much of the knowledge constructed by individuals is contextually grounded
in experience. Belenky and Stanton (2000) acknowledge the importance of
Mezirow’s transformation theory and reiterate that reflective discourse is an
important communication asset. However, they also believe that this skill implies a
sophisticated approach and is not easily achieved by everyone. The capacity to
construct knowledge and assign meaning to knowledge is a lifelong, developmental
process. The process requires opportunities for growth and coaching. This study
emphasizes that all individuals who engage in reflective discourse must be equals

39
on a leveled playing field (p.73). Culture and poverty impact an individual’s
perceptions, behaviors, and how they learn. Those who are marginalized by
homelessness, hunger, sickness and violence are unlikely to engage in reflective
discourse when priorities are focused on food and shelter. Relationships of power
and equity are often “asymmetrical”. As I have described earlier, emancipatory
transformation summons discourse communities to the duty and responsibility to
include all groups and give a “voice” to those who have been relegated by age or
social position to silence. Learning communities and organizations must prepare
and equip themselves to deal with the complexity of asymmetrical relationships.
When applying these principles in educational settings where disparity exists
among groups of students, faculty must shed their reluctance to critically examine
and re-frame perspectives. The concept of “cultural capital’ or the lack of it,
implores institutional actors to recognize and take action when inequities or
injustices exist.
One of the most challenging areas of in any area of education, is the evaluative
process used to determine if learning has occurred in the individual and have
the educational goals been met. Evaluation methods are often driven by the
curriculum, content and learning objectives. There are numerous strategies to
assess critical thinking skills and the students’ ability to synthesize and apply
information in a variety of settings.
Perhaps the most daunting appraisal is an evaluation of learning within the
realm of the affective domain specifically in the areas such as reflective thinking or

40
practice. Several professional programs have struggled to develop methods to
evaluate students’ attitudes, feelings and thoughts. Based on Mezirow’s theory of
reflective thinking, Kember (Kember,1999) created a coding scheme to evaluate the
levels of reflective thought from the written journals of undergraduate health
science students. The purpose of the study was to use this scheme to assess the
quality of the students’ reflective thinking as well as an evaluation of the tool itself
in terms of reliability. Using specific coding categories (content and process
reflection, premise reflection, introspection, thoughtful action, habitual action) for
reflective thinking, eight assessors were asked to read and interpret the journal
entries using the scheme. The authors concluded that the coding category
definitions “are unambiguous and usable in practice” for evaluating the levels of
reflective thinking form journal writing. Although the tool was tested for internal
rater reliability, the authors recommend extensive testing in other contexts to
validate whether the category descriptors are readily interpretable by others
external to the development of the tool.
Reflective or transformative learning has been widely researched as a
conceptual framework in other disciplines within higher education such as nursing
and is now being considered as a valid element in medical education. In order for
medical students to become reflective practitioners, Plack and Greenberg (2005)
assert that opportunities for reflection must be incorporated into the core
curriculum. The authors’ believe that although there are several scenarios in
medical school that offer reflective moments, faculty seldom, if ever, use these

41
opportunities to encourage or mentor reflection. Journal writing and verbal
reflection are two strategies that may be employed to focus students and provide
feedback. Since the traditional medical model and curriculum is conceptually based
in the physical and biological sciences, Plack and Greenberg emphasized the need
to address the pyscho-socio-cultural elements of providing care. They reiterate that
“stopping to think” about an issue based upon what is already known, is not
reflection and does not encourage an inquiry into deeper understanding. This would
essentially be defining the content of the problem without an appraisal of the factor
that influence or perpetuate the issue. They propose a curricular thread, beginning
with the theoretical foundations in the initial years with application in patient care
settings during the clinical years. Although this study emanates from literature in
medical education, the principles may be applied to other disciplines within higher
education.
Individual learning and change
Vygotsky’s theory of scaffolding is applied in Franke’s (2000) study of
generative change which examines the outcomes of an intervention program to
assist teachers in understanding the development of mathematical thinking in
children. A fundamental premise described by the author is the ability to
differentiate learning as simply an acquisition of knowledge and learning with
understanding or what is described as generative learning. Generative learning
reflects an individual’s ability to build upon previously acquired knowledge bases
and skill sets. Learning with understanding enables the learner to transfer, modify

42
or adapt problem solving principles when faced with novel situations. Franke also
emphasizes that generative learning is dependent upon self motivated inquiry and a
foundation rooted in “rich structure and connections”(p. 656).
Individuals must be willing and ably empowered to participate in activities that
expand their knowledge and lens beyond the instrumental activities of data
gathering and analysis. It is at this individual level that collective, generative
learning can begin to occur in organizations.
Approaching education with respect for an individual’s diversity of
background, language, culture and way of life is a core value of how one executes
programs or curricula and engages with students. It is the capacity to intervene
and educate across cultural and language barriers (Boyd, 1996). Cultural
consciousness is an awareness and acknowledgement that one's own worldview
may be profoundly different than another's personal view (Bennett, 1995). This
insight is increasingly recognized as an important factor in the student-teacher
relationship and in institutions that desire retention of students with multicultural
roots. With the increased diversity and demographic shifts in the United States, the
need to provide culturally sensitive instruction at all academic levels is imperative
for quality education and equitable outcomes.
Knowledge of cultural practices influence and impact the ability of the
educator to accurately assess student learning as well as a curriculum that will
promote student participation and active learning. Though it may not be possible to
have “competence” in another’s culture, educators need to have a working

43
knowledge of cultural practices and the contextual nature of an individual’s
learning.
“Frames of reference “ as described by Mezirow affect all aspects of human
experience. Since learning is socially constructed, the classroom plays a pivotal
role in the contextualization of knowledge. Paccione’s work with teachers and
multicultural education culminated in identifying a contextually –based,
developmental sequence of life experiences that promoted or inhibited one’s
perception and understanding of racial and cultural diversity (Paccione, 2000).
Eleven themes emerged from one hundred written surveys and 45 transcribed and
coded taped interviews of multicultural educators and advocates. From these
themes, Paccione constructed a progressive- stage model that illustrated the process
and trajectory of developing a commitment (p.987). The four stages are 1)
contextual awareness, 2) emergent awareness, 3) transformational awareness, and
4) committed action in advocacy for diversity/multicultural education. Her
research reflects the difficult process of developing a practical application of
complex conceptual frameworks such as Freire’s critical transitive thought and
critical reflection in Mezirow’s transformative learning.
This study reveals that emerging awareness is significantly influenced by
transforming life experiences and the recognition that racial- cultural equity must
be embraced as a core value if one is to achieve any level of cultural competence in
working towards equitable outcomes as a measure of commitment and
performance. Although all sequential, stage theories are subject to a myriad of

44
factors that influence how and when individuals proceed through a linear process,
Paccione’s model provides a lens and framework to initiate the process of
transformation which can lead to changes in attitudes and practices in the
classroom and the organization as a whole.
Several strategies are mentioned by Paccione to support this process and
emphasizes that didactic coursework as well as immersion experiences are
necessary for transformative awareness (p. 998). However, Howard (1999)
cautions that changing teacher preparation and curriculum or providing in-service is
not adequate as an intervention to advance transformative learning and cultural
awareness. Howard points out, most educators are white, monolingual and middle
class and experiential knowledge in diversity is widely variable and limited,
therefore supporting Paccione’s recommendation of immersion experiences.
In a study conducted by Hallett (Hallet, et al., 2003), a research based,
professional development program was implemented to improve teaching practices
through reflection. The program was based on Schon’s model of “educator as
reflective practitioner” and was offered as an approach for professionals to reflect,
stimulate and change disparate practices that existed at both an individual and
institutional level. Using Mezirow’s conceptual framework of critical reflection as
the foundation of their research design, participants were asked to focus their
assessments of various teaching practices through the lens of content, process and
premise reflection. The project was conducted for one year and employed face- to-
face and online formats to stimulate the reflective process.

45
Hallett reports two major limitations to this process. One of the
impediments was the inability of some participants to find adequate literature that
related to their discipline to build a foundation which could be used as a basis for
reflective practice and the second barrier was the lack of peer review and
collaborative discourse to encourage deeper inquiry.
Individual and group learning
An initial assessment of a group’s learning may appear to be
indistinguishable from individual learning therefore making it difficult to clearly
differentiate the theoretical processes or models.
Kim (1993) contends that although the definition of learning is
fundamentally the same, the process by which organizations learn is significantly
different. Organizations are comprised of individuals and are components of small
group infrastructures. In this manner, organizations learn though the dynamics of
its groups and transcends the learning processes of individuals within the group.
Kasl, Masick, & Denchant (1997) define team or group learning “as a
process through which a group creates knowledge for its members, for itself as a
system, and for others”(p.227). Therefore, the team or group is able to harness and
channel new knowledge throughout the organization. Collaborative problem
solving challenges groups to actively address a project or problem and requires
“action”. Action learning (AL) and collaborative inquiry (CI) are learning strategies
that cultivate a cycle of action and reflection (Yorks & Marsick, 2000). In this
way, action learning is an intervention that can support organizational learning by

46
acting as a conduit for the acquisition of knowledge (p.255). Small groups reflect
upon the actions and learning that has occurred with an insight that empowers the
group to use and apply what they have learned from the initial action. Project
teams are often comprised of individuals with divergent interests and have
representation from a broad cross-section of disciplines within the organization.
Individuals are confronted with reframing existing knowledge and opinions as part
of learning how to become a valuable contributor to the team.
According to Yorks & Marsick, collaborative inquiry is “the process
consisting of reflection and action through which a group of peers strives to answer
a question of importance to them”(p.266). The group is comprised of peers and
each member of the group maintains accountability for planning, evaluating and
communicating the experience and outcomes. Two distinguishing characteristics
of CI are that groups are comprised of volunteers and are self-contained. The self-
directed groups are afforded the ability to freely participate and grow in the depth
and breadth of shared experiences through reflective discourse (p. 267).
Argyris (1977) asserts that groups must be able to move beyond superficial
assessments of problems if valuable, sustainable learning strategies are to be
attained. In describing his conceptual framework, Argyris emphasizes that when
groups engage in “single loop learning” to solve problems, fundamental beliefs,
values, learning, and behaviors may temporarily change. However, single loop
learning does not encourage a deeper assessment into how the organizations’
policies, procedures and culture may have created or perpetuated a problem.

47
Argyris likens this to a thermostat that can regulate hot and cold, but is still unable
to question itself in terms of relative degrees.
“Double loop learning” involves recognition of the problem as well as an
awareness of the underlying organizational systems and structures. Argyris and
Schon (1996) describe “double-loop” learning as “learning that results in a change
in the values of the theories in use, as well as its strategies and assumptions” (p.
21). Organizations are often vested in preserving norms of behavior that protect
customs and practices and therefore, the “culture” may inhibit “double loop
learning when the “status quo” is threatened. There may also be a reticence to
engage in the level of inquiry that is necessary to understand the etiology of the
problem. Double-loop learning entails questioning the basic assumptions behind
ideas, policies and underlying goals. Confronting ingrained learning systems
demands a deep investigation of the problem which could lead to insight and
action. This type of reflection is more than fundamental inquiry and could be
equated to what Mezirow would consider premise reflection, which is the precursor
to transformative learning. “Single loop learning” is the familiar method,
unperturbed method of problem solving and therefore does not interfere with
cultural norms and “games” may be perpetuated (p.117). This cycle exhausts
groups within the organization resulting in diminished motivation to produce
sustained learning.
Solomon (1995) concluded that an institution’s readiness and willingness
are critical if faculty are to effect a change in values and norms towards culturally

48
diverse students. This implies that when individuals and groups simply have
didactic knowledge of cultural differences and practices, it does not necessarily
lead to a change in their values, attitudes and practices. In-service teacher
education is not meant to marginalize the concept of antiracism but should provide
institutional actors with the knowledge, strategies and power to ensure equity for all
students. If an educational program expects an internalized change in values, then
faculty must be provided with opportunities to integrate new practices in ways that
are meaningful and central to their daily activities. It is less a matter of instrumental
learning activities as it is a process of critical reflection and identifying the attitudes
and feelings that sustain the marginalization of select groups of students.
In a study conducted by Bauman (2002), a comparative analysis of 14
college committees revealed characteristics of what Bauman identified as “High
Learning”, “Medium Learning” and “Low Learning” groups. Group learning was
classified by two dimensions, data focus and experiential knowledge. Though
experiential knowledge played a role in all group learning, the results indicated that
High Learning groups used data to drive their learning and that a “data focus”
triggered the generation of new knowledge and were more likely to engage in
sustained, deeper inquiry.
Since organizations are comprised of individuals, principles of
organizational learning provide additional insight and perspective. Individuals and
groups have mental models that shape perceptions and are deeply rooted in an
organization’s culture, values and attitudes (Boyce, 2003). Boyce explains that

49
organizational members probe into existing assumptions and strategies through
dialogue and inquiry but embedded perceptions and practices are difficult to
recognize and therefore, difficult to change Unfortunately, organizational changes
are often short term and virtually not discernable because learning has been
superficial and reactionary.
Individuals and Organizational Learning
Just as individuals and groups learn through cognitive frames and mental
maps, organizations have a general schemata for learning. Organizational learning
(OL) can be viewed as an organization’s ability to acquire, understand, and use
knowledge in its planning, processes, products or practices. However, there is no
single framework and little empirical evidence or consensus of what and how (OL)
is achieved and evaluated.
When discussing organizational learning, it is important to make the
distinction between organizational learning and learning organizations. The terms
are often used synonymously, however several researchers differentiate the
characteristics.
Esterby-Smith (1997) distinguishes organizational learning from learning
organizations in that organizational learning is anchored in theoretical models and
structured settings. Conversely, learning organizations are milieus that experience
the “lived” transformation. The conduits for learning organizations are the
individuals, teams and groups that are the fundamental units of learning (Yorks &
Marsick, 2000, p.254).

50
Bensimon (2005) broadens the definition of organizational learning as a
method or strategy to gain insight into practices and opportunities for cognitive
change and explains that organizational learning takes place when individuals begin
to see equity as a cognitive frame, becoming more conscious of racial and ethnic
inequalities in educational outcomes; thus increasing the likelihood that they will
assume personal and collective responsibility for their eradication. It implies a
willingness to explore and act upon the evidence for the common good, in essence,
a call to action.
As Easterby-Smith (1997) advises, there is no single cultural, social or
cognitive framework that can meet the dynamic complexities of organizations at all
times. A review of the literature related to organizational development and learning
was conducted by Easterby-Smith and revealed that each of the six disciplines
(psychology and OD, management science, strategy, production management,
sociology and cultural anthropology) has a specific ontology with dichotomous
strengths and weaknesses. This review suggests that organizational learning needs
to be recognized, defined and evaluated as a multidisciplinary field (p.1086).
Four approaches to organizational learning described by Shrivastava (1983)
are: adaptive learning, assumption sharing, development of knowledge and
institutionalized experiences. Adaptive learning refers to the ability of an
organization to adjust expectations and goals as a response to changes in the
environment. This adaptive ability may be necessary for survival and organizations
learn to adjust to achieve sustainability in competitive markets. Organizations may

51
also attest to certain or specific values and assumptions that are shared by
organizational actors. Learning organizations must then consciously choose to
evaluate if all groups share the assumptions and are these assumption valid and
valued by stakeholders. The development of knowledge is the relationship between
actions of an organization and outcomes in the organizational environment. The
process of learning with an awareness of the concomitant changes in organizational
behavior, perceptions, policies and practices can be summed up as institutionalized
experiences (p.4). Daft & Huber (1987) explain that the systems-structural
perspective of organizational learning focuses upon acquiring and disseminating
information. Organizations learn about the internal climate in addition to external
forces that impact upon the life of the organization. Departments within the
organization are termed as “macro” units and the individuals “micro” units (p.4).
Information is also acquired through the actions of monitoring and probing.
Monitoring strategies may include the systematic analysis and interpretation of
both internal and external marketing reports or surveys, while probing implies a
deeper assessment and investigation of data. Probing focuses on details with an
intent to gain insight.
Boyce (2003) presents organizational learning from the perspective of
successful change. Organizations must learn “how” to learn, specifically, how is
learning internally sustained and promoted in order to meet the increasing external
demands.

52
Learning organizations are continually challenged to assess and evaluate the
impact of programs and efforts that lead to organizational change. Reform and
changes in practice must be contextually meaningful and relevant to the
organization if learning is to continue to be generated. As Boyce (2003)
emphasizes, learning must be “incorporated into the on-going life of an institution
”as well as the importance of ongoing, authentic conversation among institutional
actors (Boyce, 2003, p.132).
Boyce describes the contrast between Argyris’ first and second- order change
and notes that first-order change is prevalent. Although “double-loop” learning is
present in both first and second-order change, first-order change is superficial with
evidence of change in the nuts and bolts of actions and performance. This kind of
change is transient and maintains the status quo. Deep, irreversible,
transformational learning is a product of second-order change. Weik & Westley
(1996) support the philosophical foundations described by Argyris’ theory of single
and double loop learning with a discussion of “learning moments” as opportunities
for re-punctuating the process and what has been achieved. In order for individuals
to learn through this process there must be an organizational climate that is
receptive to doubt with a vested interest in inquiry versus concealing. These
“learning moments” can be likened to Vygotsky’s “scaffolding” as incremental
steps towards a deepening awareness and interpretation in the process of sense-
making.

53
Fullen (1993), offers the idea that crucial factors in learning and change are
those interventions that make an impact on the cultural norms of organizations.
The need to investigate what it is about the culture, policies, and practices of an
organization that will support a given reform, and what will impede progress
become important variables in establishing reform ownership.
A study of institutional transformation presented by Kezar and Eckel (Kezar
& Eckel, 2002) reaffirms the importance of linking sensemaking and specific core
strategies to effect organizational transformation. (p.317). “Sensemaking” is
referred to as a reciprocal process in which individuals gather and assign meaning
to information and that this contextual reconstruction of knowledge is essential for
transformation (p.314). Although it has been generally acknowledged that strong
leadership plays a pivotal role in cultivating organizational change (Kezar & Eckel,
2002) institutional actors benefit from leaders that can lend insight and
contextualize organizational sensemaking.
Another approach in looking at transformation is described by Senge
(1999). Profound change is used to describe the scope and quality of the change.
Senge professes that profound change incorporates both an internal shift in people's
values, aspirations, and behaviors, and external changes in the fundamental
thinking patterns of organizations. This is consistent with Marsick’s (2000) belief
that as individuals expand and grow in knowledge, transformative learning can be
transferred to other groups and ultimately to the organization as a whole. While
many authors and researchers agree that leadership and culture are pivotal

54
components for revitalization and transformation, the variables that lead to
sustained change continues to elude us. Pascale (Pascale, Millemann & Gioja,
1997) studied three major organizations (Sears, Roebuck and Company, Royal
Dutch Shell, and the United States Army) and recognized elements that impacted
the ability of these organizations to experience transformational change. The
authors discovered that these organizations employed common approaches to
restructuring such as eliciting employees’ participation in the process of dealing
with the challenges, leading and teaching the organization to constructively deal
with stress and “instilling” new habits of mind to sustain their new behavior
(Pascale, 1997).
This revelation is important because it provides insight into strategies that
can be replicated in other settings. Although this work studied massive
organizations, the increment of change is still dependent upon the people who
comprise the culture.
Chapter Summary
The review of the literature discussed foundational learning theories in both
the cognitive and affective domains as they relate to individual learning. Relevant
research studies addressed the “how, what, when and why” individuals learn and
how learning can be translated to others in groups or organizations. Constructivist
cognitive theories were introduced to provide a base of knowledge to segue into an
examination of socio-cultural learning models. Transformational learning theories
and transformative learning were presented as a lens through which we can

55
understand and evaluate adult learning. In Chapter 3, I describe the methodology
and analytical strategies that were used to address the research questions guiding
this study. The field notes were examined for evidence of learning in individual
institutional actors through the lens of critical reflection from Mezirow’s
transformational learning model.

56
CHAPTER 3
METHODOLOGY
Description and scope of project
The qualitative method used in this study was the single, revelatory case
method (Yin, 2003, p. 43). The purpose of this study was not to “solve problems”
but, rather, to explore the dialogue for learning of individual institutional actors as
members of the evidence team. In keeping with the purpose of this study, I
explored the text from the field notes for the presence of learning by the individual
evidence team members through the lens of Mezirow’s transformative learning
framework, specifically through the stages of critical reflection, content, process
and premise reflection. Subsequently, I also wanted to gain an insight into the
elements that may have encouraged or inhibited individual learning and reflection
among the team members.
In this chapter, I will discuss the methodology used in this case study to
examine whether individuals learned through the content, process and premise
reflection. The team members’ statements were taken from the text of the compiled
field notes and were coded, categorized and analyzed according to the stages of
content, process and premise reflection.
When retrospectively analyzed, field notes act as a historical chronicle of
the setting, actors and interactions through the lens of the observer. The
“omniscient” observer is uniquely positioned to write thick descriptions about the
reactions and insights of the group and the individual members. There is great

57
value in collecting detailed documents in a longitudinal study as they allow us to
have a glimpse of the “lived meaning” of the phenomena through the actions and
experiences of the institutional actors. This type of knowledge reflects the “depth,
complexity and contextuality” of the individual’s experience and may in fact
enhance information attained through quantitative methods (Polkinghorne, 1997,
p.13).
The case study as a method of evaluation
Yin explains that the case study as a method of evaluation may be used to
explain presumed effects, describe an intervention, illustrate salient topics, explore
situations that need clarity, and finally, may be used as a meta-evaluation as a study
of an evaluation study (Yin, 2003, p.15). As Patton points out, the qualitative case
study can provide a contextually deep and holistic evaluation of a program. This
type of evaluation does not focus upon generalizability as much as it probes to
discover unique and diverse differences among the individuals, groups,
organizations or programs (Patton, 2002, p.55). The heuristic nature of a case study
encourages the researcher to delve into the “how”, “why” and “what” questions that
drive the evaluation process (Merriam, 1998, p.30-31). As a method of evaluation,
the case study is an investigation that pursues the discovery of phenomena rather
than the validation of assumptions or hypotheses.
Case study design
The case study method is an approach that can be referred to as a process of
analysis or the product of an analysis (Patton, 2002). According to Bromley, the

58
case study strives to describe the phenomena of interest, enabling the richness of
the data to be explored ((Bromley, 1990, p. 302). Interviews, direct observations,
field notes, archival documents and observations may be used as evidence in a case
study. In addition to the exploration of discovered phenomena, relationships that
develop between and among phenomena may also be revealed.
Single-case studies. Single case studies are especially useful when the
researcher wants to challenge a theory or is able to gain access to and evidence for
a particular phenomenon (Yin, 1994). Whereas multiple-case designs may be
replicated, a criticism of the single–case design is that external validity or the
ability to generalize the results beyond the scope of the study is limited. However,
Yin (2003, p.40) upholds that single case designs can be applied with
methodological rigor. He delineates five potential applications of single case
designs. Single-case studies may be holistic, examining the overall nature of a case
under study, in contrast to an embedded design which encompasses multiple or
subunits of analysis within the case. There are limitations and advantages in both
approaches. The holistic design is useful when no other subunits exist, however as
the study progresses it may be difficult to avoid digressing into other emerging
themes. The embedded design has the potential and intention of evaluating a large
unit of analysis consisting of subunits. Yin cautions that one of the pitfalls of an
embedded design is becoming too entrenched in the subunits (Yin, 2003. p.45).
The single-case, revelatory case study method may be chosen when the
investigator has an opportunity to observe phenomena that may have been

59
previously inaccessible (Yin, 2003, p. 42). This type of in depth methodology,
although limited to one case may give rise to other studies when commonalities are
recognized by other researchers. This approach was particularly useful in this study
since the intent was to examine the field notes for the presence the of learning
through Mezirow’s framework using content, process and premise reflection.
In the longitudinal case study, the researcher studies a single case over specific
points in time. An advantage to using this method is the ability to analyze the
Potential Single-Case Studies
Experiment
Justify a single experiment
Critical case
test a well formulated theory
Extreme or unique case
so rare that it merits
analysis
Representative or typical
of average experiences
validates commonalities
Produces revelation of
of observed phenomenon
Rationales
Single-Case (holistic)
Single-Case (embedded)

Figure 3: Characteristics of single case study designs
effects of conditions overtime upon the unit of analysis.
The “bounded” case study is confined by place, time and contextual events
of the case (Creswell, 1998, p. 61). It is important to define the boundaries of the
case study in terms of time frames, pertinent events and processes used to gather
evidence. Stake (1995) and Yin (1994) have included documents, archival records,
physical artifacts, participant and direct observation and interviews as sources for
evidence. Merriam’s differentiates this bounded system from other qualitative
research methods in that the case study provides an intensive analysis of a single
unit such as a program, event, intervention, community or individual (Merriam,
1998, p.18).

60
This case study contains the characteristics of both longitudinal and
bounded studies in that the field notes were collected and archived over a specific
timeframe and was “bounded” in that the place and members did not change during
that 18 month period of time. The field notes were written and transcribed in a
sequential manner and therefore allowed me to examine the team’s statements for
reflection and learning from their initial meeting.
Objective and Research Questions
The focus of this study was to examine whether there was evidence of
learning by individual institutional actors who participated as team members, and if
learning did take place, how did individuals learn, through content, process and
premise reflection? What can be said about their learning in relation to their
assumptions and interpretations of equity? What facilitated or inhibited the
subjects’ learning in relation to equity and how did group dynamics, reflective
discourse, assumptions influence this process?
The research questions for this study are:
• During their participation on the DSP, how did the individual evidence team
members become aware, reflect and learn about inequities in educational
outcomes for minority students?
• How did the individual evidence team members’ use of content, process
and premise reflection to learn and what was the impact or change upon
their interpretations or assumptions about inequities in educational
outcomes for minority students change over the course of the project?

61
• What were the factors that enhanced or inhibited the individual evidence
team members’ reflection and learning? What can be said about learning
and change in relation to equity, what facilitated or inhibited the subjects’
learning and change in relation to equity and how did group dynamics,
reflective discourse, and assumptions influence this process?
This study was retrospective and the text was limited to the field notes of one
institution during the implementation phase of the Diversity Scorecard Project.
The units of analysis for this study were individual members of the evidence team
at one institution participating in the Diversity Scorecard Project.
Unit of analysis. According to Patton, “cases are units of analysis”(Patton,
2002. p. 447). The unit of analysis is a critical factor and may be a system of action
or a group of individuals. In this way, the insights of participants or “actors” are
incorporated into the case being studied. The unit of analysis in evaluating the
Diversity Scorecard were three individuals who participated on one evidence team.
Sources of data LeCompte’s definition of analysis is “transforming data
into research results” and reminds us that in order to make sense of “what
happened” data must be turned into results (LeCompte, 2000, p.6). Yin (2003),
Merriam, (1998) and other qualitative research experts agree that several sources of
qualitative data exist and the challenge is in choosing documents that are germane
to the study. Multiple sources facilitate corroboration and strengthen the internal
validity and reliability of the case study. Yin explains that there are three useful
principles to maximize data: 1) using multiple sources, 2) create a case study data

62
base and 3) maintain a chain of evidence (Yin, 2003, p. 97-107). The six sources
of evidence for case studies are documents, archived records, interviews, direct
observation, participant observation and physical artifacts (Yin, 2003, p.85).
Documents may include memoranda, agendas, minutes, policies, administrative
reports, newspaper or journal articles, surveys and field notes or interview
transcripts.
Once the evidence has been collected, Miles and Huberman (1994) suggest
strategies for organizing data to help us think “what goes with what”, “what’s
there”, how do these “sharpen our understanding”, how do we “see things and their
relationships” and how does this interpretation become a “coherent understanding
of the data” (p.245-246). Patton’s process is similar in that a case record is
constructed from assembled data and then is written as a case study narrative
(Patton, 2003, p. 450) providing not only facts, but also an account of the
contextual nuances and detailed dynamics of the group. This type of analysis
concentrates on the reconstruction of the interactions and views of institutional
actors, that is, the members of the evidence team. An analysis of the field notes
allows the researcher to explore and interpret the complex dimensions of the
groups’ infrastructure and subsequent impact upon the individual, the group or the
institution at large. According to Patton, a “thick” or a complete, literal, detailed
description, enables the reader to enter the setting in an effort to gain a deeper
understanding of the phenomenon (Patton, 2002, p.437-439). It is through a thick
description of the studied phenomena that we are able to interpret and attribute

63
meaning to the actions and outcomes of the particular experience. The field notes
for this study were collected and transcribed by one of CUE research staff assigned
to the institution. For this study, one set of field notes was examined and coded to
categorize statements that demonstrated evidence of awareness or learning through
the lens of critical reflection, that is, content, process and premise reflection.
During the implementation phase of the Diversity Scorecard, extensive,
detailed field notes were taken at each site. These notes were then meticulously
transcribed, providing “thick” descriptions of the institutional actors and their role
as participants on the evidence team, the setting, the agendas, resources, and
verbatim conversations. The fact that the CUE research assistant was specifically
assigned to this institution lends consistency to the collection and interpretation of
the notes.
Documents. In addition to field notes, secondary sources of evidence such
as, meeting minutes also provided insight into the institutional climate and culture.
Website. Although much depends on the individual institutions’ sophistication,
funding and support, the college website can provide an expanse of current and
archived information including college wide publications, course catalogues and
schedules, student services and links to other resources. Faculty and organization
web pages are of particular interest in attaining a “360” degree view of the college
as a learning community. The institution used in this study developed a website
that was accessible through the World Wide Web. The website was updated in
January, 2005 and allows access links to archived catalogue publications,

64
announcements, historical information, reports, photographs, meeting minutes,
academic policies, offerings and presentations such as the “President’s Address.”
Faculty and student web pages were also present.
Table 1. Sources of Evidence used in this case study

Source of Evidence Specific Evidence Rationale
Documents
• Archival Records

• Field Notes
• Evidence Team
documents
• Meeting Minutes

1. Website/college
publications/Awar
ds
Grants
2. Stable—can be reviewed
repeatedly
3. Unobtrusive
4. Detailed—contains exact
names, references, and
details of an event,
historical perspective
5. Broad —long span of
time, many events, and
many settings
6. Insight into cultural
features
4. Insight into technical
operations
In order to meet the objective of this study, I reviewed several archived
historical documents that were accessible through the institution’s website.
Additionally, I read the field notes and debriefing summaries which chronicled the
conversations, interactions, statements and outcomes, both perceived and actual, of
the evidence team participants.
Database. The database used in this study originated from the text of
comprehensive meeting minutes and field notes from one institution participating
in the Diversity Scorecard Project. The meeting minutes and field notes were
recorded at each meeting and archived onto a compact disc in chronological order.

65
Eighteen months of these documents were compiled including a power point
presentation. The compact disc was the sole source used to transcribe the notes. At
each meeting, the CUE research project staff included: a diagram of the seating
arrangement, attendees, location of the meeting (Appendix A).
Analytical techniques. To begin the analysis of the documents, I read all of
the minutes and field notes for each meeting as they appeared in chronological
sequence . For each set of minutes, I wrote a memo that answered specific guiding
questions that that related to back to the research questions. This enabled me to
remain focused during the process of coding the individual statements throughout
the text. This initial step was also my “introduction” to the institutional actors and
the CUE facilitators. It allowed me to develop a sense of the team’s dynamics and
interfacing with the CUE staff prior to attempting to code the dialogue for
statements that could be considered or interpreted as content, process or premise
reflection. A table of the meetings by year was maintained with general summaries
and the agenda items to provide a “quick reference” of the major themes or
discussions that occurred with key statements from the team members or the CUE
facilitators.
Coding. According to Merriam (1998) coding is a “sort of short hand
designation” that allows specific parts of data to be organized and retrieved (p.164).
The coding designations can be single words, phrases, numbers or letters and can
be simple or complex. Coding is a way of categorizing information and coding
schemes occur at two levels- the first level relates to information that emerges from

66
the data while the second level (Merriam, 1998. p.164). The formulation of a pre-
determined set of guidelines or concepts may also be required prior to beginning
the process of coding which creates a “mindset” or “lens” to approach the text. In
using this approach, themes emerge that contribute to establishing a body of
knowledge and evidence to assess whether individuals have experienced an
increased awareness or that learning has occurred.
After I read the minutes and wrote a memo on each of the meetings, I examined
each member’s statements and “role” as it emerged from the teams’ discourse and
dynamics with the CUE facilitators. In order to categorize and code the dialogue
from the field notes, I extracted direct quotes and excerpts from the text and
arranged each member’s statement according to the predetermined coding scheme
of content, process or premise reflection. Once the statements were assigned to one
of the three codes, I looked for emerging themes or patterns of learning or new
awareness. A fictitious name was assigned to each team member to retain their
anonymity and maintain confidentiality of the sources and the institution. The
citations and referencing of the field notes were organized and managed manually
and then electronically through word processing. The database and coding
information was stored and “filed” electronically with secured access.

67

Table 2. Sample Coding Scheme
TEAM REFLECTION CONTENT PROCESS PREMISE
KARL

Statements re:
Learning/positive
attitudes

I’m not sure if this is
where you would put it,
but I want it understood
that at first I was very
skeptical about this
project. However I have
found it be really useful

Karl explained the
process for determining
the “ “based on an
equation which factors in
SAT scores, HS GPA,
and other factors.]

[Karl said,
“Disaggregating by
discipline doesn’t make
sense for us because most
students do not decide on
a major until later on …”

We’ve been looking
specifically at first
generation Math and
Science students in order
to get a ( )grant.

I think we
could tag the
incoming
students and
check this
out further.”
(re: student
athletes)

But the question to ask,
can we refine our
understanding of what
makes them
retain/succeed?(Re: not
highly prepared
students)

Validity
LeCompte (2000) asserts that “validity is critical to the ‘goodness’ of the
analyzed data, because no matter how elegant a researcher’s own model building is,
results lack credibility, utility, or validity if the cultural whole presented by the
researcher makes no sense to the persons or groups whose cultural whole, is in fact,
being portrayed”
(LeCompte, 2000, p.152). Construct validity in case study research has been
problematic and criticized for potential subjectivity of the investigator (Yin, 2003).
In qualitative research, the tactics to establish internal and external validity may be
diverse.
While internal validity with “how research findings match
reality”(Merriam, 1998, p.201), external validity is concerned with the

68
generalizability of the research to other situations. Creswell and Miller (2000)
summarize the pros and cons of nine different validity methods using
constructivist, post positivist and critical paradigms and conclude that any approach
is highly dependent on the nature of the research, the investigators and the
participants in the study. Since this study relied exclusively on field notes, the
ability to validate elements in the text is limited. The field notes were written and
transcribed by the CUE research staff assigned to the institution used in this study.
Although caution is taken to avoid subjectivity and bias, there are unavoidable,
inherent aspects to human nature that allow us to view or listen to situations from
experiential frames of reference. To counter this limitation, the project director
acted as the sole source for internal validation and also as a content expert.
Ethical Considerations and Confidentiality
This case study was technically unobtrusive since previously collected field
notes were the source of text, however, the dignity and privacy of the research
subjects was maintained and protected from vulnerability and harm resulting from
any breech of confidentiality. Discussions and conversations that were transcribed
by the CUE research staff contain highly sensitive text dealing with issues of race,
color, and ethnicity. Prior to obtaining access to the institution’s field notes, the
dissertation chair who also holds the position as the project director, obtained a
confidentiality agreement from me to maintain confidentiality of all records,
transcripts and verbal conversations related to this study. The notes have been

69
electronically filed on a personal computer with a secured username password and
access code.
Anonymity of the institution, participants and analysis of the text were maintained
by the assignment of fictional names and these names will be used when the study
must reference the institution or the participating institutional actors.
Chapter summary
This chapter provided an overview of the case study method of evaluation
as an approach in a qualitative research study. The conceptual framework of case
study methodology was discussed as well as the rationale for choosing this method.
The database sources and analytical strategies were outlined and included some
potential limitations that may arise during the course of analysis. Since this study
involves in depth detailed coding of conversations and interactions of individuals,
methods to secure institutional and individual confidentiality was described and
maintained throughout the course of the case study.

70
CHAPTER 4-RESULTS
The purpose of this case study was to explore the presence of
transformative learning by the individuals who participated as evidence team
members in the Diversity Scorecard Project at Riverton Lake College and to gain
insight into the elements that encouraged or impeded the development of awareness
and subsequent learning in these individuals. In order to assess the presence of
transformative learning among the team members, I reviewed the conversational
text documented in meeting minutes, debriefing notes and the final diversity
scorecard report to the president. From December, 2000 through March, 2004, the
evidence team held 31 meetings with detailed field notes and minutes recorded at
each meeting by the two CUE facilitators assigned to the project. These sources
provided a chronological documentary of the team members’ conversations during
the meetings and the CUE facilitators observations and interpretations of the team’s
dynamics. The date, location, attendees, seating diagram and names of the CUE
facilitators were noted in the field notes for each meeting. The field notes were in a
narrative format that combined slices of dialogue, observations made by the note
taker and CUE research associate. A sample of the meeting minutes and diagram is
illustrated in Appendix A.
Analysis of Learning in Members of the Evidence Team
The framework used to analyze the learning that occurred among the
members of this evidence team was derived from Jack Mezirow’s conceptual
framework of adult learning theory, that is transformative learning. To summarize

71
the core concepts presented in Chapter 2, Mezirow’s transformative theory is
grounded in cognitive and developmental models of learning and expands the
vision of adult education to include how values and perceptions guide actions. It is
a socio-affective-cognitive process by which individuals critically and rationally
reflect upon their knowledge, perceptions, assumptions, beliefs and values of the
world and in relation to others. Critical reflection is a key element in Mezirow’s
framework, enabling one to abandon or reformulate new meaning perspectives and
integrate new knowledge and experiences into their relationships. This process of
attaining insight and knowledge is not constructed in isolation but is acquired in
relation to others through discourse or interactions. According to Mezirow, for
transformative learning to occur, there must be a catalyst, trigger or “disorienting
dilemma” to stimulate critical reflection. In this case study, the existing data on
educational outcomes disaggregated by race and ethnicity was expected to be a
“catalyst” for critical reflection, particularly as individuals became aware of racial
patterns that were unfamiliar to them.
In reviewing the text from the meetings, I identified statements made by
each of the three team members and observations by the CUE facilitators that
reflected attributes of Mezirow’s stages of critical reflection. This method of
analysis helped me organize the members’ statements into three categories.
According to Mezirow transformative learning can be understood as having three
stages of reflection, which he labeled (1) content, (2) process, and (3) premise.
Content reflection refers to the ways in which individuals identify and respond to

72
the substance or facts of the problem. It includes gathering fact, statistics, and other
sources of information that helps one define the problem. Having gathered the
facts, process reflection refers to the steps that individuals take towards defining the
problem in greater detail or refining the problem. This stage of reflection compels
one to think about other facets of the problem that need to known or uncovered
prior to addressing any solutions. And finally, premise reflection refers to an
individual’s self-awareness of how one views the problem, particularly the values,
personal biases, and knowledge that inform an understanding of the problem.
Premise reflection is evidenced when individuals question their existing meaning
perspectives and come to the realization that their underlying assumptions may be
unfounded. It is within the realm of this deeper questioning that one might become
receptive to constructing new meaning and perspectives. Although the DSP
creators did not specifically refer to these three stages of reflection they fit well
with the aims of the project. The primary activity of the DSP was to disaggregate
routine data by race and ethnicity in order to develop the participants’ awareness of
racial patterns in educational outcomes or in Mezirow’s terms, “content reflection.”
The DSP framework was exceptionally useful in facilitating content reflection
because it drew the attention of participants to the four perspectives from which the
content of problems would emerge: access, retention, excellence, and institutional
receptivity. In sum, this new awareness was expected to elicit new questions to
gain a deeper understanding of the problem and test assumptions. This aspect of
the project corresponds to “process reflection” in that it resulted in the creation of

73
specific measures or fine grained measures to monitor improvements for groups
who were experiencing unequal outcomes. The creators of the DSP were
cognizant that even though team members might be successful in developing a new
awareness of race-specific patterns of unequal outcomes and create a scorecard to
monitor changes, they would not necessarily engage in “premise reflection”, that is,
be able to identify the underlying assumptions of the problems as one of
institutional accountability and performance.
It is important to note, that although the activity of critical reflection is, in
theory, a sequential, linear course of thought, the ideas and statements of the team
were often interrelated and intertwined across the three processes of content,
process and premise reflection. For example, if we consider a statement made by
Laura about Native American Indians we can see aspects of content and premise
reflection: “We know that Native Americans usually decline loans. We found out
that their tribes encourage them to not take out loans. To see the differences on
paper was especially helpful. We have never tried to understand these students
beyond the numbers”. This statement could be interpreted as adding substance or
content to the problem (content reflection), that is, now the team simply learned
that Native American Indians decline loans. Taking this fact into consideration at
face value could lead the team to a very simplistic intervention, such as providing
access to the organizations or forms to apply for loans. The team could take it
another step and provide counseling and act as advocates for the students to apply
for the loans. These interventions were logical but would not “fix” the problem.

74
When Laura relays the fact that the “tribes encourage them not to take out loans”
this information takes on new meaning. Laura makes sense of the data by
acknowledging that it not an issue of access for students but an issue of a cultural
core value. The underlying motive for Native American students to decline loans
was driven by the cultural beliefs and practices of the tribe. Laura’s ability to “look
beyond the numbers” and assumptions is how premise reflection was apparent in
her thinking of the data. Therefore, when the team was presented with the data,
their thoughts, discussions and reflection evolved and could not always be “slotted”
into a particular “stage”, the data could be interpreted through two different facets
of critical reflection and that reflection was a dynamic process.
The remainder of this chapter is organized as follows, first I provide a brief
portrait of Riverton Lake College in order to anchor the analysis of learning among
the three participants within the institutional context. Next is the heart of the
chapter which provides an analysis of the three participants’ learning organized
according to Mezirow’s three stages of critical reflection. I conclude the chapter
with a summary of the key findings. In the last chapter I discuss the findings in
relation to the research questions that I posed in chapter one.
Background of Riverton Lake College
Established in the early 1900’s, Riverton Lake College is an independent,
four-year private, liberal arts college with a founding commitment to provide
equitable access to a liberal arts education. It is financially dependent upon tuition,
special programs and grants from various sources. Geographically located in a

75
culturally diverse, urban setting in a large metropolitan city, the college has been
recognized for the ethnic diversity of the student body. Originally founded by a
religious organization, the mission and philosophy of the college places a distinct
emphasis on community, social awareness, value of the individual, and pursuit of
learning. The college offers 26 majors in 23 disciplines with terminal degrees at
the baccalaureate level and selected advanced degrees.
During the timeframe of the project, the majority of students enrolled were
White, totaling 690 or 54% of enrolled students. Hispanic students were the next
largest ethnic student group comprising 26% of enrolled students. Asian American,
African American, and Native American students account for the remaining
students, with the number and percentage of enrolled students 8%, 5%, and 2%,
respectively. For all ethnic groups the number of female students exceeded the
number of male students; 56% of students enrolled in Fall 2001 were female and
44% were male. Riverton was also recognized for its diverse student population
and achieved the designation of being a Hispanic Serving Institution (HSI).
There were 97 full time faculty at the college and a small percentage as
administrators. Approximately 59 percent of the faculty were men and 41 percent
were women, while 20 percent belong to minority and international ethnic groups.
The student -faculty ratio was about 13 to 1. The CUE researchers remarked that
high quality teaching was a strong determinant for promotion, “Emphasis regarding
the tenure process is truly based on teaching. Weak teaching is the primary factor

76
for losing faculty”. A distinguishing point was made that all courses were taught by
faculty rather than teaching assistants.
Historically, faculty governance and decision making was through
consensus. In discussing Riverton Lake’s culture, the debriefing notes quotes one
of the team members as saying, “We don’t have a faculty senate. We just have
faculty meetings and everyone goes to these.” This posed a problem in one team
member’s opinion, “He [Karl] shared, “We’re thinking about not using consensus.
For example we never vote and because of that there is lots of discussion. Problems
are exacerbated at meetings of the entire faculty. When I first arrived we used to
vote. However, for close votes we had a discussion. In the past two decades, we
moved to complete consensus. ” I mention this characteristic of Riverton’s faculty
governance to provide some insight into how this team works together and their
subsequent conversations among the members when they are discussing how to
attain “buy in” when presenting the scorecard.
Before moving to the findings about how the team members approached the
data to learn and reflect about inequities, I will provide a brief biographical
introduction to each of the members. While this case study focuses upon the critical
reflection and learning experienced by the individual members of the team, it is
important to have an understanding of these actors and their attributes in the
context of their roles and longevity at the college. The following section will
present the members of the evidence team describing their appointments,
institutional roles and backgrounds.

77
The DSP Evidence Team at Riverton Lake College
Other than the CUE research staff’s request for a representative from the
office institutional research, membership on the team was left up to the discretion
of the president. Riverton’s team was small, in addition to the institutional
researcher, it included two mid-level administrators. The three administrators had
concurrent teaching responsibilities, giving them a bi-focal perspective, as faculty
members as well as administrators. One of the team members explained, ” that in a
small college, like Riverton Lake, the faculty “wear many hats.” Each member
brought extensive institutional knowledge and distinctive skill sets “to the table”.
“Sean”, the team leader, was a professor in humanities. He also identified himself
as Hispanic as referenced in the field notes. In the field notes created by the CUE
researcher, Sean is depicted as very committed to the equity goals of the project
and equally as enthusiastic about the organizational learning theory and methods
used in the project. At the first meeting, Sean commented, “I’m interested
inherently [in the DSP] because the topic is equity, but also I’m interested in
organizational learning.” At the start of the project, all of the participating
campuses had a training session that included a well-known speaker on
organizational learning and Sean conveyed his interest by saying to the CUE
facilitators. “I want to tie in the organizational learning seminar that we had at the
beginning of the project.” The minutes from the first campus meeting summarize
Sean’s demeanor and characterizes him as a strong proponent of diversity issues,
“Sean brings an understanding and appreciation for organizational learning. He also

78
has an interest in how students of color navigate their way through the university.
Sean is interested in institutional change.”
The appointed member from the Office of Institutional Research (IR) was
“Karl”, the Director of IR, an associate dean and a professor in physical science.
He was identified by the other members of the evidence team as the “fact finder”
and “king of data”. When Karl described himself to the CUE facilitators, he said, “I
am Institutional Research…. I was tapped because I know math and these sort of
things. The CUE facilitators clarified his statement in the debriefing note, “IR is a
department of one member, that is, Karl, and because of his longevity of twenty
years Sean also referred to him as ‘the institution’s memory’”. One of the
concluding remarks found in the CUE facilitator’s debriefing notes clarified Karl’s
stance on the DSP by commenting, “Karl is Riverton’s institutional researcher. He
shares an interest with Laura and Sean in improving Riverton College.”
Laura, the final member of the evidence team, was the only woman
appointed to participate in the project. She was the Director of Cultural Affairs as
well as an associate dean in the office of student services. The CUE facilitators
described Laura as Laura having a “student-centered” attitude, “She brings a
passion for students of color and understands what students of color at Riverton are
doing. As the Director of Cultural Affairs, she was in charge of the Minority
Cultural Center on campus. There was also an indication in the field notes that
Laura may have reported to Sean at some point in time. With the exception of

79
Sean’s self disclosure of his cultural background, the field notes did not make any
references to the race or ethnicity of the team members.
Recalling from Chapter 1, Stage I Diversity was defined as “equity in
representation of historically underserved minority students, specifically, the
Hispanic and African American populations, which make up significant portions of
the student body (Bensimon, 2004). From this standpoint, there was a shared belief
among the team members that Riverton Lake College was successful in achieving
the tenets of Stage I Diversity and that the institution was now positioned to move
into what the CUE researchers referred to as Stage II (achieving equity in
educational outcomes in minority student groups). The team members were aware
of Riverton’s designation as an HSI and mentioned Riverton’s unique attributes
several times in the course of the initial meetings which were recorded by the CUE
facilitators, “Several times during the meeting(s), they made comments like,
“Riverton is unique, Riverton is unlike most other colleges,” and “Our student
population is much different than other small liberal arts colleges.” Sean stated,
“We are one of the most diverse colleges of our kind,” or “As part of our founding
heritage, Riverton has embraced diversity in a very global sense, and has achieved
great successes in enrolling one of the most diverse student bodies.”
Throughout the field notes, both the participants’ own comments and the
observations made by the CUE researchers made it clear that this team exhibited a
very positive attitude toward the DSP. Even Karl, the Institutional Researcher, who
towards the end of the project admitted having been skeptical, was still very

80
cooperative and worked in harmony with his colleagues. In the field notes for the
first meeting, the CUE facilitators observed, “this group seemed receptive to the
DS project from our first contact with them [referring to their demeanor at the all-
campus orientation session] This meeting [referring to the first campus meeting]
seemed to solidify how the project will fold into some of the things they are already
doing. For instance, at the end of the meeting Sean said, “A lot of our info is
WASC-related and related to retention. Also, the Cultural Focus project is funded
by a large grant.” She [CUE facilitator] said this meeting “validated” the
assumptions of the DS project. “Everyone came up with ideas. There wasn’t one
person who did come up with great ideas when we were brainstorming.”
The evidence team often met monthly during the first year of the project
with structured agendas prepared by Sean. All of the team members were
consistently present and any absence was recorded with an explanation. According
to the CUE facilitator’s notes and seating diagram, Sean customarily positioned
himself at the head of the table, “although Sean commanded respect and sat at the
head of the table, they all contributed to the conversation. Sean had a very strong
leadership persona and much charisma. He asserted his position, but encouraged
conversation. The others were not intimidated by him and did not hesitate to ask
questions or participate in the conversation”.
The general climate of the group was congenial and collaborative and they
seemed to know each other’s work. This team also had a” history of working
together” on various initiatives and knew each others’ strengths and expertise. This

81
was mentioned several times during the duration of the project in the CUE
facilitator’s debriefing notes, “The group really listened to each other, they had
much respect for each other, were aware of each other’s work, and were very kind.
Everyone seemed comfortable in the room.” This is an important observation made
by the CUE facilitators since the development of the scorecard crossed boundaries
of race, ethnicity, gender and disciplines. Therefore, having a climate that
encouraged open, non-judgmental discussions may have made it easier for this
team to ask “tough” questions regarding equity. There were many instances where
this sense of collegiality appeared in the notes, but Laura’s comment emphasizes
the importance she places on the team’s ability to work together towards a common
goal, [during a discussion of the Practice of Collegiality]
“Laura stopped him [Sean] at this point. She said, ‘I want to make it clear that the
practice of collegiality is not just because of this project. It’s true that we have a
real vested interest in diversity issues but collegiality is a common practice at this
institution.’” As seen previously, the DSP design advantageously merges with this
team’s established philosophy and dynamic which may help strengthen the team’s
work and influence their learning.
As mentioned earlier, the logic behind the Diversity Scorecard project was
that the involvement of individuals in the examination of routine data disaggregated
by race and ethnicity would prompt attention to an existing, albeit, unrecognized
problem (racial patterns of inequality) and would increase the likelihood that these
individuals would want to learn more about the problem and take action to

82
eliminate it. The use of disaggregated data could be a catalyst to spur content
reflection.
In the following section I will discuss how critical reflection evolved among
the evidence team, starting with content reflection as a way to make sense of the
data in identifying areas of inequities in student outcomes. I will also discuss how
the process of content reflection was supported by the CUE facilitators and
experienced by the individual evidence team members.
Content Reflection
There were several steps that made content reflection possible for the
evidence team. Parenthetically, one of the most important exercises in the DSP,
disaggregating the data, was also one of the most important steps for content
reflection as it was the information or content that triggered the team members to
think about what the data meant in terms of equity. The team then started to look at
the fundamental measures of student performance, called “vital signs”. Examples of
“vital sign” measures included enrollment in the institution, enrollment in each
academic major, year-to-year persistence rates, and graduation rates. This data was
collected, disaggregated and analyzed on a routine basis, but now the team would
look at the same data, or “vital signs” through the lens of DSP’s four perspectives
of access, retention, institutional receptivity, and excellence.
Identifying the Content

The team’s first action was to request the data they would need to examine
from Karl. Since Karl was the resource for IR, he had access to the databases. He

83
was then asked to disaggregate the data by race and ethnicity for each meeting. In
the DSP framework, certain databases were designated as vital signs which would
give the team a starting point in their analysis of the data. These indicators were:
enrollment by race/ethnicity and by major or department, freshman to sophomore
year retention by race/ethnicity, graduation by race/ethnicity, and the number of
faculty by race/ethnicity bring the existing data to the meetings.
Once the team was able to review the data, they needed to “make sense” of it, that
is, they needed to understand what the “numbers” meant in terms of equity
outcomes. Since this was a “new” way of thinking about the data, the CUE
facilitators were a valuable resource and highly instrumental in guiding the team
members. This was expressed in one of the debriefing notes, “Sean really wanted to
know if they were going about creating their scorecard the ’right way’. He needed
assurance from us. He was pleased when we told them to keep doing what they’re
doing.”
To facilitate this process of identifying areas of inequities, the CUE
facilitators posed questions that would motivate the team members to consider
other aspects or approaches to interpreting the disaggregated data. In promoting
purposeful inquiry, the CUE facilitators supported two tenets of critical reflection:
1) there must be a catalyst for critical reflection and 2) the problem content must be
identified before one can respond or reflect upon the problem. Simply stated, before
one can respond to a problem, one must know what comprises the problem and
have an understanding of the content. In addition to posing questions, other

84
strategies were used by CUE facilitators to “spark” the teams’ interest when data
revealed a particular finding.
For example, “she [the CUE facilitator] also explained the idea of target groups
and showed them [the team] that not only could they look at data by ethnicity, but
they could also look at cohorts of students and how that might differ depending on
the measure they are looking at.” Because the team had no previous experience in
working with the DSP’s perspectives, they were less apt to create alternate ways of
looking at the data. The CUE facilitators were especially helpful in this respect.
The team members’ content reflection was fueled by the questions posed by the
CUE facilitators and often proved to be a challenge for the team members. For
instance, the CUE facilitators pointed out that Riverton had a large academic
middle, which was not readily apparent to the evidence team prior to dissagregating
the data, “You have: a Large Academic Middle, were you aware of it? Karl said he
did not know about the Academic Middle problem…. Sean said, “I knew about this
issue but I did not think about the implications until we started working on this
[Diversity Scorecard] project. Most importantly, it characterizes who we are as an
institution. Some of our colleagues might not be open to this characterization.
Laura commented, Laura said that as an institution, “We have not thought seriously
about the academic middle.’ The CUE Project Director, said that for the Diversity
Scorecard, ‘You don’t always have to focus on the negative. You can look at
resiliency. More specifically, you can try to answer the question, who are these
academic middle students? Perhaps you could build on the academic middle? Sean,

85
‘Something we’ve learned from this project is to ask questions about how we can
do things better.’” Having this new awareness challenged the team members to
consider the “academic middle” and reflect how to respond to this issue within the
context of the Scorecard. When one of the CUE facilitators asked “What is your
concern towards the academic middle?” Karl responded with “How can we serve
them better?” His response reflects a positive attitude and could be interpreted as
“receptive” in that he asks how “we”, implying the institution, could intervene. In
this manner, Karl looked not only at the numbers of “academic middle” but began
to address how to best meet their needs. Since Laura and Sean were somewhat
cognizant of the issue, they shared anecdotal information that would help Karl mine
additional data, “Karl followed up grad survey and thus discovered the existence of
the academic middle.”
As mentioned previously, Karl’s “job” was to gather and distribute statistical
information with the team at each meeting. His approach to analyzing the data
tended towards conventional approaches and he initially did not see value in the
process of disaggregating the data. He did not necessarily refuse to separate the
data, but could not fathom the rationale in doing so, “ Karl said, “Disaggregating by
discipline doesn’t make sense for us because most students do not decide on a
major until later on in their career.…I still want to break this down by gender.”
However, part of the rationale in disaggregating by discipline was to determine if
the enrollment reflected “clusters” of minority students which Karl did not
consider. This is another illustration of how the CUE facilitators steered the process

86
to illuminate the findings through a DSP lens, encouraging Karl to ponder or
reflect on this content as an issue of equity.
Another content area emerged for the team members when the CUE facilitator
asked the team “How do you define gateway courses?” This was a critical question
to ask the team since it would help them think about which students enrolled in
these classes and more importantly, which students successfully completed the
courses. Gateway courses were defined as those courses that are commonly taken
by all students (e.g., first-year course sequence) and are required in the course
sequence for a major (e.g., introductory courses to a major). When queried by the
DSP facilitators, “Karl said, ‘We asked ourselves, what classes do first year
students take?’ Once we knew what courses they take, we looked at Calculus &
Chemistry. Laura stated firmly, “we define gateway courses as portals into a major.
Unlike large research universities, access to gateway courses were not prohibitive.”
The next step was to disaggregate the courses by ethnicity and in doing so, the
team found that the percentage of minority students who successfully (grade of “C”
or higher) completed the course was disproportionate to the targeted groups’
representation on campus. When “seeing” this content emerge from the numbers,
Karl interjected,” I had never thought about gateway courses — ever.’” Karl’s
content reflection, triggered by the data, demonstrated how this inequity was not
about student access, but rather student success and in this case, an inequitable
outcome. Karl could have hindered content reflection because he did not seem to
see the value of disaggregating the data. There are a number of ways he could have

87
inhibited the team members’ focus and content reflection since he was the
appointed member from IR and was the most familiar with accessing the
institution’s databases. His assumptions about how to analyze the data could have
undermined the deeper questioning that lead to new knowledge. During one of the
interim meetings, the project director broached the topic of mathematics courses, in
terms of placement and attrition in the Hispanic and African American freshman.
For example, when asked about a placement test for incoming freshmen, Karl
explained that although Riverton had a placement test, it was primarily geared
towards students who would need math in their proposed major, so that for some
students, math courses acted as a “gateway”, “Yes, primarily intended for those
who will need math in their majors, but everyone takes it. Riverton is responsible
for creating this placement test. Part of the [purpose of the] placement test is to use
the results to help push those who are prepared into calculus…. math entrance
requirements are non-existent. As far as the math they have [from high school] we
have not had a chance to explore this problem…. there is a special class for those
who are not ready for regular freshman writing. Provisional status (equivalent to
remediation) at we are looking for an improved version of a remediation course for
math. We have one (a liberal arts math), we can’t eliminate it due to the nature of
our students ‘math for what you might need if you don’t plan on going into a
technical field’. It was not until this discussion that Karl could see that while math
courses acted “gateways” for some students, and a “gatekeepers” for others.

88
Subsequently, this discussion lead to a more intense level of inquiry into the
attrition rate in mathematics courses where Karl’s initial response was that there
was no way of monitoring this component. When the project director asked, “Do
you monitor attrition in the Math/Science/Etc. majors?’ Karl: We don’t monitor
this. There is no real way of gauging this.” However, one of the CUE facilitators
offered the team a suggestion, “The Dean’s list data [referring to the information of
students on the Dean’s List is very compelling as well as the data on math, QR
(Quantitative Reasoning), and Chemistry.” After a few ideas were more ideas were
presented, Karl agreed “that it could be done.” The outcome of this dialogue was
that mathematics courses, quantitative reasoning and chemistry courses were
incorporated into the analysis of the gateway/gatekeeper classes, Sean shared his
perspective of the impact that this knowledge may have on curriculum
development, “We’re talking about curriculum. I’ve been wondering how can this
be a class? I’m thinking that the instructor I am paired with for my seminar class
and I could come up with a class that integrates math and sociology. We could have
a whole class of institutional researchers. There are many pedagogical possibilities.
I think that undergraduates would like this stuff.”
When another preliminary finding demonstrated that the retention of White
males was the lowest of all populations, the CUE facilitators expressed concern that
Karl “kept returning to the issue of White Males” even though it was not the focus
of the DSP.” Even though Karl’s attention seemed to drift from the aims of the
DSP, the other members of the team were able to create the environment for

89
content reflection on inequalities. At several intervals, the CUE facilitators
stimulated content reflection by discussing articles or research models to augment
the teams knowledge, such as the “Toolbox” [Adelman’s] or the work of other
expert researchers in organizational learning. As the CUE facilitators pointed out
that many campuses had problems with the gateway courses, one of the CUE
facilitators observed, “Karl sat back in his chair, crossed his right foot over his left
knee and put his right hand on his beard. He nodded in understanding [referring to
Adelman’s work].” The CUE facilitators’ debriefing notes make a point of
mentioning this since Karl’s “body language” could be interpreted as receptive or
open to contemplating a new way to regard gateway courses and their impact upon
equitable outcomes.
Unlike Karl, when Laura would work with the data, she would write narratives
to illustrate the numbers that sent up “red flags.” For example, Laura wanted to see
‘if the [minority] students were clustering’ in any major’. While Laura wanted the
group to consider the racial and ethnic makeup of the student body by academic
department, Karl still seemed more interested in the gender distribution. While Karl
kept returning to gender, Laura continued to pursue her notion that “the most
important figures [numbers] for Hispanics are in Social Work and Child
Development”, therefore supporting the importance of looking at race and ethnicity
by department and major. The field notes do not specifically interpret Karl’s
comments, but he may have wanted to look at gender again to surmise whether
there was any relationship among gender, race and ethnicity in these traditionally

90
female-dominated majors. One of the tactics that Laura and Sean utilized to get
Karl to re-focus on racial patterns was by relating the aims of the DSP to other
initiatives that were more familiar to Karl.
Laura’s capacity to relate the DSP to existing institutional initiatives was
another strength in helping the other members see how the project could augment
their efforts, “Laura discussed how the DSP fit into some of the work she’s already
doing. For instance, she asked if in looking at the excellence perspective we could
disaggregate the academic awards by ethnicity such as department citations,
leadership awards, and service awards…..another example, we track financial aid,
but we don’t usually disaggregate it by ethnicity and types of award”. As
referenced in the field notes, when Laura uses examples that relate to one of the
DSP perspectives, it helps Karl contextualize the DSP within his current frames of
reference, perhaps helping him view the data through different lens and
encouraging him to engage in content reflection, “He could have just dismissed her
since she’s not an IR person. Instead, he listened to her and even complemented her
on how she presented the data.”
As the project progressed, Karl continued to bring the data as requested and, as
the CUE facilitators interpreted, became more open to new ways of looking at it
from the focus of the DSP, “Karl said no problem; he could generate the data in the
way and the field that [Laura] wanted.” Laura enjoyed working with the data as
much as Karl, referring to herself as a “data junkie” but she was also aware of
having to look beyond the numbers. Laura’s ability to “tell a story from the data”

91
enabled Karl to view the data through a broader lens which was mentioned in the
field notes on several occasions, “She [Laura] is not only passively waiting to see
the data, but she takes the initiative to think about what data she needs to inform
their work.” When Laura requested the data and reorganized into one of her
narrative summaries, Karl could have reacted defensively or he could have been
reticent in sharing it with her. Rather than being territorial about his role on the
team, the CUE facilitators interpreted his attitude as appreciative.
Laura’s “stories” facilitated the team’s content reflection by lending her
interpretation of the numbers in the context of Riverton and Sean’s comment
expresses how this influenced the we haven’t looked at it before… He explained
that Karl helped him learn the statistics he never learned in college. Laura helped
make meaning of the data and tell the story.” When the team began looking at the
disaggregated data by major, Karl reiterated, “But a lot of what we go on are myths.
i.e. Business is a Hispanic major- but now, we see that’s not the case.”
In summary, the analytical strategy used in the DSP (analysis of
disaggregated data) was expected to act as the impetus to prompt critical reflection.
During the initial phase of working with the DSP, the team gathered and
disaggregated data which led to the identification of problems from the lens of the
four perspectives related to equity (content). As the content to the problems
emerged, the team members needed to query additional information and respond to
the evidence that illuminated areas of inequitable outcomes for underserved
minority students.

92
It is in the team members’ response to the content where critical reflection is
revealed through the text of their conversations.
In the next segment, I will discuss how the team members with the guidance
of the CUE facilitators engaged in content reflection and “sense making” activities
to develop Riverton’s Scorecard.
Problems identified through Content Reflection.
According to Kezar and Eckel (2002), sense-making is a ”collective process
of structuring meaningful sense out of uncertain and ambiguous organizational
situations”(p.314). Therefore, the DSP created a purpose and setting to assist the
team members in understanding and crafting new perceptions. To illustrate how the
team used the process of content reflection to define the problems and gain a richer
understanding of the complexities of the inequities, I will direct the following
discussion on how the team identified problems through the lens of the DSP
perspectives using critical reflection.
Inequities in Minority Access and Enrollment. Through the lens of the
DSP’s access perspective, the team evaluated the extent to which underrepresented
minority students could gain access to courses, programs and resources (Bensimon,
2004). By considering the evidence from the disaggregated data, the team found
that the undergraduate student enrollment at Riverton Lake College reflected a
White majority, a change in the ethnicity of students that was due to the steady
decline in the percentage of African American, Asian American, and Hispanic
students. The percentage of African American, Asian American, and Hispanic

93
students enrollment had declined steadily from Fall 1998 to Fall 2001. Conversely,
during the same time frame, the number of Native American and White students
had increased from 13 to 19 and 635 to 690 respectively.
The team’s deeper inquiry into the data revealed a three percentage point
decline in Hispanic students during a four-year period. Upon examining this data,
the team members asked Karl to breakdown the data by major and gender to
ascertain whether patterns or particular variables emerged. If the team merely
wanted to identify patterns of enrollment, they would not have been intrigued or
motivated to pinpoint the origin of the problem. When Karl brought the data to the
next meeting, the team found that for all ethnic groups the number of female
students exceeded the number of male students. Table 3 illustrates the team’s
findings:
Table 3- Riverton Lake College –Enrollment by Ethnicity and Gender

Fall Enrollm e n t '98 '99 '00 '01
T otal Undergrads 1269 1291 1294 1278
Male 558 555 582 562
Female 711 736 712 716
International 63 52 65 72
Native Am. 13 13 26 19
African Am. 76 77 78 66
Asian American 127 103 91 104
Latino 368 349 349 328
White 635 697 673 690
Figure 3.

Each time Karl presented the team with more data, the members generated
more questions or requested a different “look” at the information. For example,
once the distribution of students was analyzed by ethnicity and gender, Laura

94
wanted to look at the distribution within the various majors, “I did not find that
certain groups were not excluded from particular majors” This was a surprise to
Laura showing how through further investigation Laura dispelled her assumption
that perhaps some majors, such as the physical sciences may have excluded
underrepresented minority students.
By offering suggestions that could broaden the lens of analysis, the CUE
facilitators continued to foster the teams’ content reflection in relation to
enrollment inequities. For example, when discussing admission criteria and
enrollment, the project director introduced the idea of looking at the tiered divided
into the five admission categories by ethnicity. From here the team members began
to look at admission SAT scores, high school GPA’s and the number of minority
students who may have been accepted and enrolled.” The techniques employed by
the DSP worked advantageously for this team since it prompted deeper reflection
into problems while helping them to become aware that some of their underlying
assumptions or perceptions may not be valid.
As Karl presented the data to clarify the actual minority enrollment,
additional facts came to light and there was a realization that many long standing
assumptions were going to be challenged. When referencing the underrepresented
minority students, the team members universally felt that these students were
characteristically first generation. After disaggregating the data, Karl pointed out
“there are only about 18-20% first generation students on campus. That is still
significant but definitely not what we thought.…I’ve been looking specifically at

95
first generation Math and Science students in order to get a this grant. This
requires that 2/3rds of the students be first generation or low-income….discovered
that their numbers of these students are quite low. Quite contrary to what we
expected to find, or thought they had.” The data served to invalidate their
perceptions that most of their underrepresented minority students were first
generation. This is an important example of how evidence can be used to dispel
myths and assumptions. With this team, the perception “first generation”carried a
constellation of other misgivings regarding the minority students, for example, the
dearth of cultural capital which may greatly influence a student’s confidence and
ability to understand the complexities and innuendos
of successfully acclimating to the collegiate milieu.
Inequities in Minority Retention. Retention was another DSP perspective
that revealed unexpected inequities. The retention perspective compelled the team
to look at comparative retention rates, completion of courses and the completion of
degrees, credentials or certifications (Bensimon, 2004). While the team attempted
to understand the variables that impact inequities in enrollment they found that the
data on enrollment could not be meaningful in isolation of other institutional
activities such as recruitment and retention. The team now wanted to look at which
students groups were retained. They found that Hispanic students consistently had
the highest fall-to-fall first-year retention rate and that White and Asian American
students had relatively equal first-year retention rates. African-American and
Native American students had the lowest first-year retention rates. For Hispanic

96
and African American first-year students, the fall-to-fall retention rate had declined
each year over a four year period. For African American first-year students the rate
has declined 10 percentage points and for Hispanic students the rate has declined
7.2 percentage points during that time, while the White and Asian American first
year students ’retention rate had remained relatively stable.
Table 4. Riverton Lake College – Freshmen Retention by Ethnicity
50.0 %
60.0%
70.0 %
80.0%
90.0%
1999-00 88.1% 71.9% 77.6% 63.6% 78.0%
2000-01 83.5% 72.6% 83.6% 66.7% 79.0%
2001-02 80 .9 % 61.5% 78.8% 77.8% 76.7%
Latino African American Asian American Native American White
Fall-to-Fall Freshmen Retention by Ethnicity, 1999 - 2002
Figure 11.

What stood out to the team was the low percentage of African-American students
(proportional to their representation on campus) on nearly all of the outcome
measures that were collected. This was also true for Hispanic students on several
measures. However, when the team learned that a large percentage of African
American students who were retained at Riverton were athletes, they wanted to
investigate this finding. The revelation of this particular inequity generated content
reflection for each team. Sean, Karl and Laura each had experiences in working
with African American student athletes, but from different vantage points. Unlike
other underrepresented minority students, the African American student athletes
were from families who had attended college. This fact was interesting to Sean and
prompted him to delve into this further, “Sean explained that African American

97
transfer students are primarily student athletes from student athlete families, hence
they are not first generation [indicating that these students were intentionally
recruited] Sean said that orientation is going on with the athletes…that he had just
recently met with the football team. This sparked an idea for him. He now wants to
know how the Diversity Scorecard relates to athletics. He said, “50% of the team is
new freshmen or transfer students. This is anecdotal. I don’t know those figures for
sure but how did this relate to retention Laura.” Karl’s experience with the African
American athletes was primarily with football players in Academic Review,
“We’ve worked with a lot of athletes, especially football players, in academic
review. Football is part of an on-going conversation at Riverton. We all noticed
that 42% of the football team was on AR.”
The conversation that followed focused on the fundamental question, “if
African American athletes were specifically recruited, why were they not retained
and what were the factors impacting this problem?” The team members realized
that there were many questions left unanswered, for example, Laura relayed that
although the students are surveyed at withdrawal, the survey does not capture
whether the withdrawal is voluntary or involuntary, financial, or familial (i.e. death
in the family). This lead the CUE facilitator to question this trend “asking if this
[athletics]was for some students playing on the football team, the only chance at
getting a college degree. Laura: You’re saying it’s like a double-edged sword.
Students are getting in because they’re players, but if we don’t help them see that
there’s more than playing they will never get a college degree. Among students,

98
they see the football players going through a revolving door. Almost half of the
football team is new every year so this gets reinforced.” This interaction between
the CUE facilitators and the team displays how DSP lens served to raise the team’s
awareness that failure to retain African American student athletes was about
institutional accountability, rather than blaming the students. Their awareness was
especially heightened by the fact that these students were intentionally recruited or
transferred to Riverton. Advisement was a key issue for Sean in his role in
Academic Affairs at Riverton, “[Sean] said when he started his job as Associate
Dean he instituted sending students a “meet with me” letter if they received 3 or
more mid-term warnings. Many people did not go see him. Many students were on
the football team. He remarked five out of the 16 football players that “I’ve been in
constant contact with.”
This particular content related to African American student retention
intensified the team’s efforts to probe further into other aspects of institutional
practices such as tracking advisement, student withdrawals but most importantly,
the courses commonly taken by African American students, especially athletes. In
addressing retention, several questions needed to be answered that related to course
access and successful course completion. “Success” was defined as completing the
course with a grade of “C” or higher. The CUE facilitators were very instrumental
in helping the team make sense of the data and posed questions to the team. This
was a powerful stimulus for content reflection and a deeper understanding of this
problem. During one of the first site visits, the CUE facilitators asked the team to

99
consider examining “gateway” and “gatekeeper” courses. Gatekeeper courses were
defined by the team as those courses with the highest percentage of students who
receive a grade of “D” or lower (note: a “D” grade or lower places a student at-risk
for academic probation) whereas Gateway courses defined as those courses that:
(a) are commonly taken by all students (e.g., first-year course sequence) and are
required in the course sequence for a major (e.g., introductory courses such as Intro
to Sociology). At this site visit meeting, the CUE facilitators asked, “Psych 101
was brought up as a gatekeeper course. What other courses are gatekeepers? How
do you define gateway courses? Another way to look at this is to ask, what are the
“hot” majors that will eventually lead to high-paying or technical jobs or careers
(i.e. medicine)? Which pre-requisite courses associated with these majors are
gatekeepers? Karl was very receptive as was Laura, “We read the Toolbox book.”
Karl said, “We also asked ourselves, what classes do first year students take? Once
we knew what courses they take, we looked at Calculus & Chemistry. Laura stated
firmly, for us, we define gateway courses as portals into a major”. The team
discovered that math and interdisciplinary courses, such as philosophy were serving
as “gatekeepers”. Karl shared, “As a math faculty, we know there is a problem but
we don’t know what to do about it….At Riverton they teach math for what you
might need if you don’t plan on going into technical field. The team found that not
only were the math department and interdisciplinary courses serving as
gatekeepers, but they were gatekeepers for the largest percentage of students (e.g.,
where the percentage of students who do not pass first-year sequence courses in

100
these areas exceeds the percentage of students enrolled, by ethnicity). Additionally,
for Hispanic students, the percentage of students who do not pass exceeded the
percentage of enrolled Hispanic students in biology, earth science, history,
interdisciplinary courses, and political science.
Inequities in Institutional Receptivity and Excellence Perspectives
Although much of the team’s focus concentrated on the DSP perspectives of
access and retention, concerns surfaced in the areas of institutional receptivity and
excellence. As defined by the DSP framework, institutional receptivity was the
ability to create and support an accommodating and responsive environment for
underrepresented minority students (Bensimon, 2004). According to the team’s
recorded findings, when examining the ethnicity of students compared to the
ethnicity of faculty, Hispanic students exceeded the percentage of Hispanic faculty
by 19 percentage points, whereas White faculty exceeded the percentage of White
students by 24 percentage points, in Fall 2001.
Table 5- Riverton Lake College –Student and Faculty Ethnicity, Fall 2001

0%
10%
20%
30%
40%
50%
60%
70%
80%
Students 2% 5% 8% 26% 54%
Faculty 1% 2% 11% 7% 78%
N . ative Am African Am. Asian Am. Latino White
Student and Faculty Ethnicity, Fall 2001
Figure 14.

101
Thus, for Hispanic students, the likelihood of encountering faculty of the
same ethnicity at Riverton was less likely than for White students. The data is
represented in Table 5. When addressing excellence, the team to considered
enrollment and achievement. In terms of access to the courses and enrollment, the
team had already identified “gatekeeper” and “gateway” courses from their
previous work in defining the problems through the retention perspective. The team
also touched upon the “achievement” when disaggregating the data related to
retention, academic review and specifically, African American athletes. For African
American and Hispanic students, the percentage of students who received
departmental distinction was lower than the percentage of enrolled students for the
2000-2001 and 2001-2002 academic years. The same condition existed for the low
percentage of African American students recognized for excellence in those years.
Under representation of minority students in student leadership arenas and honor
societies was also revealed when the team members reflected on the problems and
content in retention and student athletes, “Sean said that he wanted to track athletes
and academic honors. Laura said, Karl could set up the data by athletes and grades.
Laura said, “We have never tried to understand these students beyond the
numbers.”
In summary, the evidence team, with the support of the CUE facilitators,
identified that Riverton had significant inequities in four primary areas: 1) a decline
in underrepresented minority enrollment with the largest decline in Hispanic
students – from 368 to 328 (3 percentage points) in four-year period, 2) the fall-to-

102
fall retention rate for African American first-year students had declined by 10
percentage points and for Hispanics the rate had declined 7.2 percentage points, 3)
that the math department and interdisciplinary courses may be serving as
gatekeepers for the largest percentage of students and 4) the ethnic mix of students
compared to the ethnic mix of faculty, the percentage of Hispanic students
exceeded the percentage of Hispanic faculty by 19 percentage points, whereas
White faculty exceeded the percentage of White students by 24 percentage points.
On all measures of excellence, Hispanic students were underrepresented when
comparing the percentage of Hispanic students who were recognized for excellence
in relation to the percentage of Hispanic students enrolled. The same condition
existed for African American students in regards to those measures. Table 6
provides a representation of students with departmental distinction by major and
gender.
Table 6- Students with Departmental Distinction in the
Major, by Gender 00-01 and 010-02

0%
20%
40%
60%
80%
100%
00-01 % Award 80.3% 19.7%
00-01 % Graduates 62.9% 37.1%
01-02 % Award 78.6% 21.4%
01-02 % Graduates 57.9% 42.1%
Female Male
Students with Departmental Distinction in the Major, by
Gender 00-01 and 01-02 Academic Years

103

The team’s ability to collectively and individually engage in content reflection
enabled them to identify these problems and inequities among the Hispanic and
African American students, and at this juncture the team’s discourse progressed
towards formulating questions that could lead to strategies and measures to monitor
the indicators. The team was now poised to engage in process reflection, the phase
in which there is deeper inquiry into the content or facts of the problems. Within
the context of the DSP, process reflection, an approach used to refine or further
define problems, would be akin to defining fine-grained measures. To illustrate
how this team engaged in process reflection, I will discuss the team’s activities in
identifying the fine-grained measures and how process reflection supported their
efforts to address inequities in enrollment and retention.
Using Process Reflection to Identify Fine-grained measures
While content reflection focuses upon gaining a deeper insight and
understanding into the facts or substance that comprise a problem, process
reflection enables the individual to enter the next phase of defining the problem by
asking “what else might be causing the problem or what else needs to be know to
address the issues.” In other words, once there is recognition of a problem, the team
needed to think about how to advance towards uncovering all plausible elements
that create or perpetuate the problems. When the team members used the “vital
signs” and content reflection to uncover the gaps in their student outcomes, the
“uncovered gaps” did not show why the gaps existed. In this instance, process

104
reflection entails “unpacking or deconstructing the gap” through continued
exploration of the data and other sources of information. Even though the team
spent time grappling the content of the problems, they eventually achieved a deeper
understanding and became cognizant of other variables impacting the problem
areas. These factors needed to be taken into account when considering how to
address the content. Therefore, rather than turning to the most obvious or
commonly employed solutions, process reflection compels the individual to
address conditions and causative factors that have been garnered through content
reflection towards a remedy. In order to formulate ways to address broad areas of
inequitable outcomes such as enrollment and retention, the team members needed
to engage in “second-order level of inquiry”, which required them to take a deeper
look at the existing indicators. They mined and collected additional data to help
detail and “track down” when and where problems surfaced within and throughout
the systems. Second-order inquiry was apparent when Laura began to contemplate
the constellation of variables that could influence retention. Laura and Sean sensed
that students who worked were more vulnerable and at risk for failing or
withdrawing from courses but did not have data to validate this assumption. “Laura
brought up the next thing she had for the group. She handed out a survey she had
distributed to students on work hours. The survey I sent out went to about 200
students who are on the Dean’s list and about 200 students who are on academic
probation or suspension.” As she passed this out, Sean stated, “I have some
anecdotal information I want to share. This week I’ve been meeting with students

105
who are in academic difficulty. I’ve been asking them how many hours per week
that they work. They have reported working 30-35 hours per week.” This scenario
demonstrates how the team members ventured to produce a distinctive, microscopic
or fine-grained measures to help make sense of the factors impacting retention.
When the team begins to contemplate how to use this information they
engage in process reflection. To show how the team used process reflection to
formulate measures to evaluate outcomes, I will focus on two problems that
emerged from the DSP perspectives of access, specifically enrollment and retention
Finer-grained measures- Enrollment.
As defined in the previous section, the process of developing “fine-grained
measures” allowed the team members to distinguish particular trends and student
outcomes that were not readily apparent from the initial disaggregated data. To
produce greater detail, the team members thought of other data sources that could
be examined to reveal new information. A few of the examined areas that lead to
the development of fine- grained measures included analyzing data related to
applications, admittance and enrollment rates by ethnicity.
To illustrate how the team members used process reflection to create fine-
grained measures, let us revisit the Native American Indian students in relation to
financial aid and enrollment. If we were to simply look at financial need as the
primary reason for the withdrawal of Native American students from Riverton, the
simple solution would be to obtain sources of funding such as scholarships and
loans. However, we must bear in mind that among Native American families,

106
seeking loans is highly discouraged. It is then unlikely, that providing counsel and
sources of funding would be the appropriate process to resolve the issues of
financial need and retention for Native American Indian students. For other ethnic
students, financial aid packages could influence the ability to gain access to an
institution and be equally as crucial to a student’s retention at the institution.
Recalling that the institution was primarily tuition driven, this meant that many
students may have needed a variety of funding sources including public or private
scholarships, grants or loans. If funding became a hardship for students, they may
have been forced to withdraw from the institution. It was important for the
evidence team to identify and fully understand the spectrum of factors leading to a
declining retention rate in underrepresented minority students before speculating
how to solve the problem. Sean’s statement acknowledges the pervasiveness of
financial aid as an influential variable, “these issues of Financial Aid that keep
cropping up over and over again. We need to keep looking at this”.
Sean’s question was characteristic of the type of statements that
encouraged process reflection. Through content reflection, the team discovered
that many of the African American students were athletes. As relayed by the CUE
facilitators, Sean’s inquiry was now aimed towards increasing overall enrollment of
African American students, “Sean: What would it take to get a critical mass of
African American students at Riverton Lake College that mirrors the typical entry
and exit path of all other ethnic groups (e.g., those who start at their first year at
Riverton and those whose primary or secondary consideration for attending is not

107
dmittance and Yie ld Rates by Ethnicity, 2002
related to athletic participation).” Although the data revealed a decline in
enrollment both Hispanic and African American students, the team focused upon
increasing access in the African American group. The field notes heavily
referenced the team’s conversations regarding the African American student
population more so than the discussions revolving the Hispanic students. There
may have been a few reasons for this, 1) one could surmise that the team felt there
would need to be a concerted, targeted effort to recruit first generation, non-athlete,
African American students to increase the enrollment numbers and 2) the data
revealed that many Hispanic students were retained even though they may have low
performance. Subsequently, the team looked at the yield rate of applicants by
ethnicity in relation to the number of overall enrollment. Developing measures
that would ultimately be helpful in the recruitment of students became the focus of
the team’s efforts. Although the team found a decline in the number of applicants
from both groups, they found that Hispanic students, if admitted, were more likely
to enroll than any other ethnic group. An example of one year of data is illustrated
in Table 7.
Table 7- Admittance and Yield Rates by Ethnicity
A
A p p lic at io ns A d mit t e d A d mit R at e Enro lle d Y ie ld R at e
American Native 19 16 8 4 .2 % 6 3 7.5%
African American 6 7 4 7 70 .1% 11 2 3 .4 %
Latino 263 212 80.6% 80 37.7%
As ian American 14 1 117 8 3 .0 % 3 0 2 5.6 %
White 696 566 81.3% 137 24.2%
International 148 39 26.4% 10 25.6%
Unknown 187 152 81.3% 34 22.4%
To tal 15 2 1 114 9 75.5% 30 8 26.8%
Figure 8.

108
Identifying the finer –grained measures was the initial step in developing ways to
address this problem in access and subsequent enrollment. In terms of access, the
team felt the overall goal should be: “to increase the number and percentage of
enrolled African American students African American student application rate
(number and percentage), African American student admittance rate (number and
percentage, African American student yield rate (number and percentage). This
team was ambitious in its efforts and wanted to move forward with “solutions”. For
instance, Laura thought that it would be helpful to look at the financial aid
packages in relation to family or student income to see if the tuition was
prohibitive, “Laura wanted to look further at Federal Grants …..which are based on
income. This might tell them something about the income level of students at
Riverton....Laura pointed out that 50% of Hispanic students have low to low-
medium incomes levels but if compared to the rest of the students it has no real
world meaning.” Though these findings laid the foundation for possible solutions,
it was not within the scope of this team’s mission to formulate specific recruitment
strategies to increase access. Being aware of this, the CUE facilitators, were able to
redirect Laura’s efforts from defining a specific strategy by discussing how the
evidence team could incorporate this into their recommendations for the President’s
report. The CUE facilitators encouraged process reflection in the team by giving
suggestions that were more closely aligned with the DSP’s framework thereby
refocusing the team’s discussions, “[CUE facilitator] You might want to suggest
that a team of evidence monitors be developed, or a task force that looks at some of

109
the things you’ve pointed out. These recommendations are more about process, not
how to solve the problems in your scorecard. We want you to build in processes
and keep the momentum going once we’ve finished this project. After looking at
this data, we want to focus your attention on these 4-5 areas. We will describe each
of these areas and put in some charts. Next comes the recommendations. These
aren’t recommendations in the sense of how to solve the problems, but they are
recommendations about how to keep the scorecard going.”
While discussing issues in access, the team members became aware that
access could not be looked at in isolation from retention. In examining the data, the
evidence team members became acutely aware of the interrelatedness of the DSP
perspectives and how this coincided with how their own reflective processes, that
is, in order to find plausible recommendations for the scorecard, they often needed
to reexamine the data which would yield new content and a deeper understanding
of a particular problem. Sean restates how inquiry is a dynamic, ongoing process,
energized by the teams’ inquisitiveness, “Sean interjected to remind the team that
without this exploratory process, they would never have ventured into this facet of
problem, questions about these areas of retention have come up because of the data
we have looked at”.
Process Reflection- Inequities in Retention.
As previously seen, this team had an acumen for generating new knowledge
from data, (content reflection) which, in turn, stimulated them to think or reflect
upon the need for greater detail and a deeper understanding when approaching the

110
problem (process reflection). Therefore, by breaking down the disaggregated data
by additional categories, such as gender and major, the team learned additional
facts and developed finer-grained measures that would enable them to construct
goals and measures that could be internally monitored and sustained by the
institutional actors.
Upon re-examining the disaggregated retention data by ethnicity, the team
found that while the Hispanic students consistently had the highest fall-to-fall first-
year retention rate at 80.9% (for the specific time frame), African-American and
Native American students had the lowest first-year retention rates at 61.5% and
77.8%, respectively. Another “micrscopic look” at the data on African American
student retention revealed 1) African American student athletes may have retained
at a higher percentage than the total number of African American students retained
and 2) a significant number of African American athletes were in Academic
Review (AR). Karl shared this fact at one of the meetings which is conveyed by
CUE facilitators in lengthy text from the field notes, “’we all noticed that 42% of
the football team was on AR. This is something we’ve known for a while –
experientially. We now have the data to prove it.’ Karl said that this is a project he
has wanted to work on ever since he started gathering the data on the student
athletes who are on probation. He said, ‘We’ve worked with a lot of athletes,
especially football players, in academic review. I think we could tag the incoming
students and check this out further.’” African American athletes were not
specifically referred to in number nor did the team know if the evidence would

111
demonstrate a relationship between athletes in AR and their ethnicity. This was
important and as Sean had recognized “most of our information is anecdotal, now
we’ll have a way to show this.” This excerpt demonstrates how the team members
used content reflection to harness new information about the problem and process
reflection to gain a better appreciation of the facts when developing one of the
measures to monitor retention. Although the CUE facilitators did not elaborate on
Karl’s comment that the African American student athletes could be “tagged”, I
interpreted Karl’s statement to be positive within the context of the discussion.
The information that emerged from the data validated the team’s assumptions and
increased their awareness of the academic vulnerabilities of African American
student athletes. “Tagging” could alert advisors and serve to heighten their
awareness and vigilance in tracking the academic performance of African
American student athletes. When the data was disaggregated further, the team
learned in fact, that, “most of the African Americans at Riverton are athletes. We
need to monitor African American athletes. They have an increased need in
attention and need to be shepherded closely.” Having the knowledge that these
students were at risk for being placed on probation and AR, the evidence team felt
that a way to tackle the problem would be through the advisors who were
positioned to be proactive. They could provide anticipatory guidance with
curriculum choices, time management and study techniques during the sports
season when time for studying would need to be balanced with academic demands.
A fundamental premise of the DSP was to support a sense of empowerment among

112
institutional actors to develop contextually meaningful solutions when addressing
inequitable outcomes in relation to race and ethnicity; and this evidence team
understood the importance of translating their findings to other institutional actors.
Additional support for student athletes could take shape in a variety ways but the
specific strategies needed to be developed by those faculty advisors and other
institutional actors, such as coaches, who would be in contact with the students.
Finer-grained measures-Retention
Reflecting about the inequities in retention led the team to search for finer –
grained measures to monitor the problem. The field notes indicate that Karl
suggested several avenues to monitor retention that had not been drawn upon,
“[We] continue to be concerned about African American student retention- African
American athletes and transfers. How can we shape these concerns into goals? Do
we need more data?” Laura responds to Karl by adding, “Could you set up the data
by athletes and grades. We have never tried to understand these students beyond
the numbers”. In addition to these categories, the CUE facilitators knew that Laura
had experience with students and exit interviews and encouraged her to think about
this in relation to retention, specifically, other factors to consider when looking at
how the institutional data related to retention, “Laura thought, [we] should identify
the factors that influenced or determined a student’s voluntary or involuntary
withdrawal. If a student has a problem like illness or death in the family so they get
another chance if they appeal. Most students state their reason as leaving as
financial, but they are never probed.” Demographic factors that influenced

113
retention could also be captured from existing data, such as, whether a student was
“out of state”, campus resident or commuter, or the type of financial aid package.
In terms of process reflection, this is another example of how the team’s deeper
level inquiry and collaboration with the CUE facilitators, stimulated them to
respond to the content of the problem and think of alternative tactics they might
otherwise not have pursued.
At an interim site visits during the first year of the project, the DSP
research staff and DSP Project Director were invited to attend and were highly
instrumental in encouraging the evidence teams’ process reflection. During this
interim meeting, several key topics were discussed, but the pivotal question posed
to the team members by the CUE researchers was to make a critical distinction in
identifying the “Gatekeeper” and “Gateway” courses. The Project Director’s
question prompted a conversation that segued to deeper level of inquiry by the team
members. To distinguish the courses that may have been acting as gatekeepers, the
Project Director queried, “Do you monitor attrition in the Math/Science/Etc.
majors? In terms of Hispanics, how are they doing? After quantitative reasoning
you don’t take calculus unless you’ll use it in your major? In response to these
questions, ”Sean offered , “We should pick two things to focus on. Math and
gateway courses sound good and noted “Math GPAs are the lowest across the
board” (referring to all ethnicities).
As discussed previously in this chapter, the two areas with the highest
percentage of students receiving below a “C” grade were math and

114
interdisciplinary, primarily the first-year writing seminars. For Asian American
and Hispanic students, biology seemed to be a gatekeeper, while White and African
American students’ philosophy appeared to be serving as a gatekeeper.
Additionally, for Hispanic students, the percentage who did not pass courses in the
earth science, history, and political science departments exceeded the percentage of
Hispanic students overall.
The team also found that across the board for African American, Asian
American and White students, math was an area where the percentage of students
who did not pass exceeded the percentage of students enrolled within the particular
ethnicity as illustrated in Table 8.
Table 8- Number of Student Who Do Not Pass by Ethnicity

Number of Students Who Do Not Pas s by
Ethnicity BIOL ECON ES HIST INTD M ATH PHIL PLSC Grand Total
African A merican 1 1 1 1 4 4 2 1 15
As ian A merican 5 1 1 6 8 1 22
Latino 8 4 3 12 18 16 2 7 70
International (Non Citizen) 1 1 1 4 4 11
Other or Unknown 2 1 1 3 4 4 1 2 18
W hite 10 9 4 15 32 39 16 9 134
Grand Total 27 17 9 33 68 75 21 20 270
Area (e.g., Department and/or Courses)
Figure 13.

There were several areas that the team wanted to delve into and the CUE
facilitators suggested looking at a few measures that could help Riverton address
retention. When the CUE facilitators pointed out that “many campuses had
problems with the “gateway” courses”, they also suggested that the team look at the
number of students who withdrew from the courses. In doing so, the team
discovered that the freshman writing courses may be acting as gatekeepers among
the African American and Hispanic students. Upon seeing this evidence, Karl
shares, “I had never thought about gateway courses — ever.” [Sean] “I have never

115
seen things like this {data}. This project has given us opportunities to do new kinds
of thinking.”

Table 9 Freshman “Gatekeeper” Courses by Ethnicity
0.0%
10 .0 %
20.0%
30.0%
40.0%
50 .0 %
60.0%
70 .0 %
80.0%
% P opulat ion 5.5% 8.7% 25.8% 49.0% 3.8%
Averag e % Do No t Pas s 5.6% 8.1% 25.9% 49.6% 6.7%
BIOL 3.7% 18.5% 29.6% 37.0% 7.4%
ECON 3.7% 5.9% 23.5% 52.9% 5.9%
ES 3.7% 0.0% 33.3% 44.4% 11.1%
HIS T 3 .7% 3 .0 % 3 6 .4 % 4 5.5% 9 .1%
IN TD 14 .8 % 8 .8 % 2 6 .5% 4 7.1% 5.9 %
MATH 14.8% 10.7% 21.3% 52.0% 5.3%
PHIL 7.4% 0.0% 9.5% 76.2% 4.8%
PLSC 3.7% 5.0% 35.0% 45.0% 10.0%
% No t Pas s % No t Pas s % No t Pas s % No t Pas s % No t Pas s
African American As ian Am. Latino W hite Other
Freshmen "Gatekeeper" Courses, by Ethnicity
(Pe rce n tage of Fre shm e n Re ce iving D,F, NC in De pts Liste d)
Figure 12.

Through process reflection, the team members discussed which indicators
should be monitored and designed the following measures in response to the
inequities in retention: Fall-to-Fall freshman retention rate by ethnicity (e.g.
freshman to sophomore year retention), semester-to-semester retention rate by
ethnicity, incoming applicants retention rate by ethnicity amount and type of
financial aid by retention rate by ethnicity, “Gatekeeper” courses in freshman year
by ethnicity, and completion rate and grade by ethnicity. One of the most intensive
and critical meetings for the team was the “Presidential Report Writing Retreat”.
This is where the team and CUE facilitators strategically sifted through the data
analysis of the past several months and chose the indicators and measures that

116
would construct the scorecard. The CUE facilitator focused the team with an
opening comment, “The Riverton team needed to review and pick their measures.
About three or four for each perspective. They also needed to tie these measures to
the strategic plan. Consequently, the team identified several goals and measures for
the Riverton Lake College’s Diversity Scorecard. The goals and measures and
their corresponding Scorecard perspectives are outlined in Appendix B.
Although this section has focused upon process reflection as a conduit to
defining finer-grained measures, I feel that it is important to emphasize that while
the team members used process reflection to determine how to face the inequities,
they continued to ask questions, bring data and uncover more information about the
substance or content- exhibiting how the steps in critical reflection are not achieved
as separate, discrete actions that are finite, but are dynamic processes that are
intertwined throughout the collaborative discourse.
In seeking to achieve a greater understanding of how to deal with emerging
phenomena, one is compelled to ask more questions and pursue deeper knowledge.
During the planning meeting to discuss how to write the Report to the President,
Laura summarizes this learning process when she states, “Usually we would see
data and sit there in silence. But then we began asking questions of the data and a
pattern emerged. When we didn’t get the answers we were looking for, we asked
more/new questions of the data… She didn’t want to cause more work for the team,
but is it possible to revisit some of the questions/issues in detail that came up when

117
they were re-examining data and writing their report ?.....We make space for
reflective learning and deeper level conversations.”
Premise Reflection
Recalling from Chapter Two, premise reflection is a higher order of
reflective thinking and the vehicle for transforming meaning and perspectives. It is
in the act of challenging our fundamental beliefs, perceptions, thoughts and actions
(meaning schemes) that we are able to change our existing assumptions and taken-
for- granted beliefs into a new structure of assumptions and beliefs from our
experiences (meaning perspectives)(Mezirow, 1990). More importantly, premise
reflection obliges us to question the very existence of the problem. The primary
charge for the evidence team was to define problems by initiating a process of
inquiry, using existing institutional evidence to generate knowledge or evidence of
differences and conditions in educational outcomes for ethnically diverse students,
particularly for African Americans and Hispanic. During this project, the team met
and analyzed existing data that was disaggregated by race and ethnicity. Once
inequities in educational outcomes were identified (content reflection), the team
members continued to explore the data, asking Karl to disaggregate the data by
more specific measures such as gender or major called finer-grained measures
(process reflection). At this juncture the team members considered how to
approach the information with recommendations that would be presented to the
President and key administrative members at Riverton Lake College.

118
In analyzing the text for statements that convey the processes of critical
reflection, I interpreted most of the extracted dialogue to be that of content and
process reflection, that is, defining the substance of the problem and development
of methods and recommendations (finer-grained measures) based upon one’s
response to the content. Since our reflective thoughts and feelings are largely tacit
and experiential, they are not easily captured for literal interpretation. For that
reason, premise reflection calls for an in depth re-evaluation of the underlying
basis of a problem where the individual is urged to as ask "why" a specific meaning
perspective or problem exists in underlying practices. For instance, the
disaggregated data revealed that Hispanic students with low GPA’s were retained.
This fact was incongruent with the team’s conventional beliefs and assumptions
about retention and triggered a lengthy discussion. This fact was discussed several
times throughout the field notes at various intervals. In the following statement,
Sean ventures to say that although the retention for Hispanic students is high, they
really do not know how or why this positive outcome is driven, especially in light
of the fact that some of the students have low GPA’s and was counterintuitive,
“Sean contextualized that they retain Hispanics at very high rates, and they want to
understand what they are doing right to retain and graduate them. Laura: We need
to find out why are Hispanics so successful. We need to answer this so we can try
to apply some of our findings to help other students.” This enduring analysis and
inquiry is an example of this team’s commitment to probe and ask the question
why, “The group mentioned that they wanted to know why they are good at

119
retaining Hispanic students even when they have a low GPA. Sean said, “We
graduate Hispanics better than any group of students. We want to know why”....”or
Karl’s comment, “We do retain students who are not highly prepared. But the
question to ask, can we refine our understanding of what makes them
retain/succeed? How can we do better–looking at institutional practices.” If we
keep in mind that Karl was originally “very skeptical” about the DSP, his statement
could be interpreted as premise reflection since it conveys that Karl has an insight
and desire to find out “why” this outcome has occurred within the Hispanic student
population. There were no statements from the team members or notes from the
CUE facilitators that suggested that anyone knew how or why Hispanics with low
GPA’s would be retained at higher rates than students of other ethnic groups.
Furthermore, when the team members discussed this outcome, they approached it
as an opportunity to delve into the data for a more profound understanding of how
the institution’s practices generated this outcome.
Premise reflection may have also been triggered by the fact that institutional
practices or systems were ineffective in capturing students at risk. Throughout the
meetings, the team discussed the various student groups that were placed on
academic probation and invested much time in addressing the African American
athletes in Academic Review. However, Laura pointed out that although there was
a system in place to “warn” students of poor academic performance, there was no
way of knowing how students responded to the warning, “Laura not only look at
students who were on probation, but whether the warning system had an affect or

120
not….the system they have in place to help students was not working.” When
discussing retention, her statement suggests that she has moved beyond the obvious
content of the problem, “…where in the cycle are we not succeeding? They’re not
new problems, we can’t keep harping that these problems exist.” Laura was not
blaming the students for failing, but that the system they have in place may be
failing the student. This is the perspective one who understands that student
outcomes is a reflection of institutional accountability and conveys an attitude and
value that is more powerful than rectifying a “systems problem”.
On several occasions, Sean would revalidate the fact that Riverton was an
institution that made strides in dealing with issues of race and equity. Sean
explained, most of the institutions of higher education are working on enrolling
students of color, “Riverton can work on other issues of diversity…… At Stage I, a
campus needs to focus on enrolling students. Campuses, like Riverton, are prepared
for Stage II Diversity, which is measuring outcomes related to student
achievement.” With this in mind, Laura’s experiences with underrepresented
minority students gave her an insight that led her to ask fundamental philosophical
questions, that is, why have these problems evolved in an institution that takes such
pride in its achievements in diversity. She goes on to elaborate that “One of the
issues I have been working on is balancing providing services for students of color
but not wanting people to associate people of color with needing help. Yes,
students do go into this office for assistance, but that doesn’t mean that they need
help much more than other students.” In considering the African American student

121
athletes and the percentage in Academic Review, the data did in fact reveal that
their performance was low. However, because of the African American athletes
notoriety on campus, it would appear that students of color “needed more help”,
fueling the myth. Laura knew that this type of situation was perpetuated in the
institution and therefore, prior to the DSP, would not have been questioned.

Transformative Learning.
If the ultimate goal of critical reflection is to have perspective
transformation, then elements of transformative learning can be gleaned from the
text. It is important to note that the CUE facilitators recognized that the evidence
team attributed a vast amount of importance to their participation in the DSP and
this is demonstrated by their vigilance. Sean perceived himself, Laura and Karl as
co-learners and conduits for Riverton’s organizational learning. Sean began the
meeting by reintroducing the Riverton team to everyone. He explained the role of
each individual and stressed that each team member was a co-learner. He
explained that Karl helped him learn the statistics he never learned in college.
Laura helped make meaning of the data and tell the story.
During the initial sessions, Karl focused on presenting data and assisting the
team in “deciphering” the facts and figures. It was not until the data was
disaggregated and revealed the inequitable outcomes in minority groups that he
conveyed what he had learned or how his attitudes had changed, “Doing this
project I’ve found many ways of thinking about data. I’ve even learned new

122
techniques as an IR person. I’m not sure if this is where you would put it, but I
want it understood that at first “I was very skeptical about this project. However I
have found the approaches to be really useful. To see the differences on paper was
especially helpful.” “ Karl added, “I had never thought about gateway
courses — ever. “ I have found the approaches to data very useful. This makes it
easy when tying it into other things I’m doing or committees I’m on….There are a
lot of mythologies. Doing this project I’ve found many ways of thinking about
data.”
Karl talked about the fact that he learned more about getting data out of the
system. For example “he mentioned disaggregating data by ethnicity and how he
would do this and bring that information to their meetings”. Although I am unable
to determine that Karl’s learning was sustained over time, his statement reflects that
disaggregating data by ethnicity is a strategy that he has integrated into his way of
thinking. Karl said that he wants to get even more outcome data on athletes
disaggregated by ethnicity, transfer status, team, and gender. He said that this is a
project he has wanted to work on ever since he started gathering the data on the
student athletes who are on probation. He said, “I don’t know what I’ll find, but I
can get the data. The CUE facilitator commented that she appreciated how “he is
always interested in finding data on outcomes and this is not a common practice in
higher education.”
In contrast, Laura’s awareness of equity, race and academic performance
existed prior to her participation as a team member and her learning moments were

123
different than Karl’s. Laura’s learning manifested in statements that related to her
gaining a new meaning perspective as a result of obtaining new knowledge from
the disaggregated data. The CUE facilitators’ notes imply that Laura’s knowledge
was deepened through critical reflection as in the case of the Native American
student, loans and financial aid. For Laura, the “lesson learned” came with the
revelations garnered from the project and teams’ reflective discourse. Her intuition
and hunches were often validated by the data. She often questioned why a
particular attitude existed when she was advising minority students and how her
own assumptions might impact her ability to advise minority students. When the
data invalidated common assumptions about student athletes, Laura’s commented,
“Don’t you think the mythology fuels our ethos? We are willing to do so much for
so little…. Laura pointed out that the target enrollment numbers vary. Debunking
the argument that there’s no room for them (African American students. We’re still
in shock you guys, can you tell? There have been lots of myths and it involves a lot
of work In this next phase we want to get more ideas about how we can get this
out there. Ideally this will become a core part of the institutional culture.” In
keeping with Riverton’s founding philosophy of providing academic excellence for
all students, the team often discussed how the DSP could act as a supporting
structure, essentially a scaffold, to enhance institutional practices with informed
decision making derived from evidential inquiry.
Similar to Laura, Sean supported the tenets of the DSP from the inception
of the project. The field notes exhibit Sean’s enthusiasm numerous times

124
throughout the project. Several of his comments project his positive attitude and
commitment, “ Sean brings an understanding and appreciation for organizational
learning. He also has an interest in how students of color navigate their way
through the university. Sean is interested in institutional change.” At Riverton’s
Assessment Committee Meeting, one of the last meetings recorded in the collection
of field minutes, Sean is quoted, “this (the presentation) makes my soul sing gladly
because it reminds the institution of its responsibility.” The fact that transformative
learning may have occurred with Sean is represented by his own account of his
feelings while serving on the DSP team,
Sean added, “This project is training me to think critically. I now look at
some of the mythologies and ask about supportive data. What I want to underscore
is the process I’ve had with learning. I want to share all the wonderful things I’ve
learned.” Sean said, “I have never seen things like this [data].” He has never
looked this closely at student data and had never looked at financial aid data before.
“This project has given us opportunities to do new kinds of thinking], That’s the
exciting thing about this project—I’m learning. I’m learning lots about how I can
look at data…. Also, I’ve learned different ways of perceiving….which has been a
transformative experience.”
Sean often expressed what he felt he was learning and how this learning
could lead to institutional change as exemplified in the field notes “this project for
the Riverton team has very much been aimed at institutional learning, particularly
for underrepresented students. In fact, it has created a paradigm change in our

125
thinking…. It was not about a blame chain, just about very intense self-reflection
and institutional learning….On this campus when we talk about issues and
problems we often talk about mythologies. Evidence-based practices provide a
positivism stating where we are and where we need to improve. For so long we’ve
depended on the mythologies. Making sense of evidence is part of the critical
thinking process…. We make space for reflective learning and deeper level
conversations.”
Within the document of the Report to the President, in the context of the
first recommendation, “Continued Inquiry by the Diversity Scorecard Team” the
team address their perception of how the Diversity Scorecard can impact
institutional change through engaging in learning experiences that are
transformative:
“The Riverton Lake College Diversity Scorecard team has found the
Scorecard approach and process--of evidentiary inquiry into the state of equity in
student outcomes and potentially enabling or inhibiting practices that contribute to
these outcomes--a “high” learning experience, transformational, and likely to bring
about or has already resulted in changes in practices.”
Conclusion
From March of 2001 through March of 2004, the DSP Evidence Team at
Riverton Lake College, Karl, Laura and Sean engaged in collaborative discourse
with each other and CUE facilitators that lead to critical reflection, specifically
content, process and premise reflection. The existing data was disaggregated by

126
race and ethnicity and acted as a stimulus or trigger for “disorienting dilemmas” or
challenges to the beliefs, perceptions, attitudes and practices of these institutional
actors. By identifying inequities and by critically reflecting upon the content of the
problems, the members were able to respond to the new knowledge (inequities),
reflect upon and identify how to create indicators (fine grained measures) to
monitor how institutional practices promoted or tolerated inequities or inhibited
equitable outcomes. Elements of transformative learning were present among the
team members, but to varying degrees. As with all stage theories, the progression
through the “steps” are fluid and dynamic and do not occur in a rigid sequential
order. However, statements from the individual team members convey that
learning occurred and that reflecting upon the data was a critical part of the process.

127

CHAPTER 5: DISCUSSION
The primary goal of the Diversity Scorecard Project was to assist
institutions of higher education in identifying inequitable educational outcomes
among historically underrepresented minority groups, specifically Hispanic and
African American students.
The purpose of this study was twofold. One reason was to evaluate the
presence of learning among the members in one evidence team that participated in
the Diversity Scorecard Project. Specifically, I wanted to understand what
(knowledge, content) and how (processes, dynamics and strategies) the individuals
learned about inequities. The other intention of this study was to gain insight into
the elements that encouraged or inhibited the team members’ learning about
inequitable educational outcomes. When reviewing and coding the evidence team’s
meeting minutes and field notes I applied components of Mezirow’s transformative
learning theory, specifically using the three stages of critical reflection (content,
process and premise reflection) as the conceptual framework (Appendix C).
In this chapter, which is divided into four sections, I will provide an
overview of my findings. Four important findings will open the discussion
including the limitations of the study, followed by the implications and
recommendations for practice. Finally, I will close with recommendations for
future studies.

128

Important Findings
In conducting this study, the most important finding that emerged was that
learning occurred among the individual group members and the group as a whole.
For this team, their participation in the DSP was a catalyst for the critical reflection
that lead to knowledge. The members not only learned new knowledge about the
educational outcomes of minority students but also gained insight into existing
inequities that were reflected in their educational outcomes. The DSP provided the
team with a “do-able” strategy and process that could be replicated by others
throughout the institution as a means of awareness of inequities among other
institutional actors and leaders. The team’s learning was recognized in the
following ways:
1. The Diversity Scorecard Project was a catalyst for learning among the
team members, especially Karl, the Institutional Researcher.
2. The team members learned how to use institutional data to gain
information and knowledge about educational inequities and they were able
to contextualize this new knowledge.
3. Critical reflection was supported and made possible by the team leader,
the teams’ dynamics and the CUE facilitators.
The DSP as a Catalyst for Learning.
A core principle in Mezirow’s transformative learning theory is that a
“disorienting dilemma” must occur to trigger critical reflection. In this study, the

129
disaggregated data served as the primary catalyst for the team’s critical reflection.
When the members agreed to participate in the DSP, they were aware that one of
the project’s primary goals was to disaggregate data on educational outcomes by
race and ethnicity to determine whether the institution was achieving academic
success for Hispanic and African American students. The directors of the project
emphasized equitable educational outcomes because they felt that student diversity,
a characteristic that was prominent in the case study institution as well as on all the
other participating colleges, masked the growing inequality being experienced by
those students who represented the diversity these institutions pointed to with a
sense of pride and accomplishment (Bensimon, 2005).
There was content that was new knowledge for all of the members, such as
analyzing disaggregated data. However, perspective transformation was most
evident in one member, Karl, whose statements convey that he moved from a
skeptic to an advocate of the DSP. Although Sean and Laura may not have
experienced learning to the extent of Karl’s change in perspective, the collaborative
interactions, discourse and reflection resulted in a team that was able to convey the
premises and mission of the Diversity Scorecard Project and its value as a tool for
assessing institutional performance.
Through this process, the team members learned from the data that many of
the educational outcomes among minority students were not consistent with their
longstanding assumptions and perceptions. For example, when the data revealed
that many of the African American athletes did not fall into the category of “first

130
generation” students, this came as a surprise to the members. This assumption was
so deeply entrenched in their thinking that without the impetus of the DSP, this
issue may never have been questioned.
Sean, Laura and Karl were uniquely skilled to participate on this project,
but as the institutional researcher, Karl was essential for the data collection and
analysis. It was critical for him to understand that the DSP had a specific approach
to the data and this strategy was vital to the achievement of the project’s objectives.
Of the three team members, it was Karl who initially displayed some
reticence about the value of the project. However, as he disaggregated the data by
race and ethnicity, he saw how the existing data served to inform the team with
new facts about minority student outcomes. For Karl, the DSP proved to be an
effective, useful tool to help him interpret data. He felt that the DSP enabled him to
think about data (content reflection) and refine his understanding and approaches
(process reflection) in new ways and that he had learned techniques that he could
use in other settings. I elaborate on Karl’s learning because, although he never
obstructed the process, he was admittedly skeptical. In spite of his skepticism, he
still learned a new way of analyzing and interpreting data. More importantly, Karl
ultimately “bought into” the relevance of the DSP tenets as a way to bring
institutional practices in line with equity. Stemming from his desire to “get the
word out”, he recognized how he was advantageously positioned to bring the
DSP’s techniques and principles forward to other groups of institutional actors.
Although this study did not look at sustainable learning among the team members

131
or the dissemination to other institutional actors, Karl’s statements indicated that he
wanted to make a concerted effort to incorporate and apply the DSP principles into
other institutional ventures.
How to use institutional data to learn and create new knowledge
When Merriam (2005) discusses Mezirow’s concept of critical reflection in
practice, she emphasizes that meta-cognition is an inherent requisite for critical
reflection to occur. Although Mezirow believes that the ability to critically reflect is
part of the adult developmental process, Merriam posits that the adult learner must
already possess developmentally mature cognitive and emotional functioning.
Therefore, the team members needed to be cognitively able and emotionally willing
to explore ethnic issues. When the team members learned a specific, structured
approach to data analysis, that is the dissagregation of data, they learned a skill.
Polkinghorne (2004) would refer to this method of learning as the technical-rational
approach.(p.30). He further explains that although an answer is generated from a
calculation that all would agree upon, the debate ensues when the decision to take
action centers upon the values and assumptions of those interpreting the answers
(p.30). The act of gathering and disaggregating the data by race and ethnicity was a
new institutional strategy of presenting quantitative evidence of educational
outcomes. To gain an understanding into how these numbers or percentages
signaled inequities in educational outcomes required a meta-cognitive process or
critical reflection.

132
According to Bensimon (2004) when individuals recognize that inequitable
outcomes exist, they are more likely to be motivated to take action, “In order to
bring about change in an institution, individuals must see, on their own, and as
clearly as possible, the magnitude of inequities (awareness). They then must
analyze and integrate the meaning of these inequities (interpretation), so that they
are moved to act upon them (action)” (p.45). This awareness must then create a
shift in one’s traditional paradigm of teaching and learning, generating actions that
can penetrate the deep-seated barriers that exist within the institutional culture.
Therefore, the members of the evidence team not only learned “how to” analyze the
existing data in a new way, but also gained insight into how they felt about their
findings through critical reflection. For this type of learning to occur, Argyris
contends that institutional actors must be open and willing to create new knowledge
which may uncover, expose or bring to light evidence that is “uncomfortable” to
discuss, especially for those who are vested in maintaining a status quo. Therefore,
institutional actors must grapple with the discomfort that arises from an increased
insight or awareness which subsequently spans the barriers that inhibit learning or
change (Argyris, 1977).
While reviewing the field notes, statements such as Karl’s convey an
openness to acknowledge that this new information has broadened his perspectives
about equity, “I’ve never looked at the “gatekeeper” courses that way- ever…the
data has always been here, we just never looked at it this way”.

133
Although Karl became committed to the project, at the beginning he was
not convinced of how the “numbers” could provide evidence about race and
ethnicity. This was not the case with Sean and Laura who had an intuitive “hunch”
that the numbers would validate the existence of inequities, some which they
suspected but were hard to get to because they were concealed by the typical
methods of data analysis used in higher education, i.e., the use of aggregate data.
This shift in thinking was clearly expressed by Sean, "it has created a paradigm
change in our thinking". All three team members had the willingness, motivation
and administrative experience to participate on this project and successfully
complete a scorecard that represented the inequities at Riverton.
The team leader and team dynamics relation to Critical Reflection and Learning
The team at Riverton displayed positive attributes (openness, willingness,
motivation) that were conducive to individual and collective learning. Some of the
questions I would now pose as a result of this case study are: Could learning and
critical reflection occur with other personality types and leadership styles? Could
learning and critical reflection occur with more members, or members from other
departments who may not have had an interest in statistical data, evidence based -
practice, or diversity issues? Would critical reflection take place if members did not
have a previous working relationship with each other?
The evidence team at Riverton was a small group, consisting of three
individuals. They described themselves as collegial and this characteristic was also
expressed in the facilitator’s debriefing notes. The members also had pre-existing

134
professional relationships with each other which was another factor that may have
enhanced their ability to engage in reflective processes. The CUE facilitators noted
that this team had “No stifling, they have a perfect dynamic.” Culture, race and
gender are inherently sensitive issues that cause discomfort and discussions, for
example, that involve race and racism tend to be avoided. Although this team did
not directly discuss racism the focus on specific racial and ethnic groups did not get
in the way of the team’s inquiry. There was not overt resistance to the requirement
of disaggregating the data by race and ethnicity, not even from Karl who at the
beginning was skeptical. Had this group of individuals not shared a sense of
mission and commitment in terms of equity and excellence, their learning outcomes
may have been greatly altered or non- existent. In other words, if skepticism or
cynicism had been the driving force uniting the team, then learning would have
been far less likely to happen during the project.
This team, one might say, had good chemistry. They also had a shared
commitment to the college, students and to their work. These characteristics of the
group made it possible to broach sensitive topics that other institutional actors
within the college might have considered too “taboo” within the context of the
college culture. As mentioned in the last chapter, this college was proud of its
diversity and conversations about inequities could have been seen as undermining
achievements touted by the institution.
While Laura and Sean shared a similar communication style when
expressing their feelings, thoughts or insights, Karl’s pragmatic approach is no less

135
noteworthy in helping us interpret how the dynamics of this team lead to learning,
“Karl said, ‘There are several things that make this work. One of them is the
combination of people and having an IR person who’s a mathematician.” As we
have seen previously, Karl’s focus and learning was geared toward the more
structural aspects of DSP such as data gathering and preparing reports, but Laura
and Sean’s interpretive insights complemented Karl’s data work and ultimately
influenced how he felt about the “work” and outcomes of the DSP. I can surmise
that the fundamental principles and strategies of the DSP could be learned by any
team, that is data could be disaggregated and analyzed. The more complex issue is
whether a larger group, with varying degrees of interest in diversity and equity
engage in the critical reflection that is required for learning to happen. Based on
this study, I would venture to say that the learning that the DSP was designed to
foster is contingent on having an institutional culture and leadership that supports
the project and is not intimidated by the problems that might be revealed by fine-
grained measures of educational outcomes by race and ethnicity. At this college,
which had the official designation of being Hispanic Serving, there was
considerable risk in disaggregating data and what it might reveal about the
educational achievement of Hispanics.
Distributed Leadership. In the field notes it is apparent that this team was
not going to be discouraged by the time and labor demands of the project. In many
ways, they were ready to take the challenge. One of the ways the team members
worked to make this happen was to be realistic about how they would organize

136
their time to accomplish the “work” of the DSP within the project’s timeline. The
CUE facilitators were quick to notice the delegation of leadership, “Sean asserted
himself as the leader of the meeting - Sean writes the agenda each week, leads the
meetings, and gives assignments to Karl and Laura…. commented, “They have
very specific assignments for each other. It is very impressive to me that Sean is
very interested in this project. He is very motivated. He takes his time and energy
to dealing with the project.”
Another significant factor influencing the team dynamics was the leadership
approach. Sean, the “leader” of this evidence team, was knowledgeable, respected
and committed. Additionally, he was an ardent proponent for equity in diverse
student populations and was already philosophically aligned with the DSP’s
principles and premises. The CUE facilitators capture his enthusiasm for the
project and his willingness to facilitate the meetings throughout the field notes. He
knew the strengths of the team members and often commented on their talents.
Because of his interest in organizational learning, he often took the opportunity to
show how the scorecard interconnected with other institutional initiatives.
Additionally, Laura insight and contextual understanding of the data gave
her the ability to compliment and augment Sean’s leadership. As mentioned in
Chapter 4 Laura was able to help Karl make sense of new knowledge about the
educational outcomes by organizing the data into a narrative. This combined
leadership was a powerful ingredient in promoting group and individual reflection
and learning and this was often acknowledged by the CUE facilitators, “Riverton

137
is a unique example of a DSP team. Much of it has to do with the team’s
leadership.”
The CUE Facilitators as Purveyors of Learning. A strategic difference in
the framework of the DSP from other assessment models in higher education was
the use of practitioners as the “researchers”. In accordance with one of the most
salient principles of the DSP, the evidence team would drive the process of
developing their “scorecard” based upon the evidence that was garnered from
existing routine institutional data. Although each team member had some
knowledge of diversity issues and data analysis, they had never worked on a project
of this particular nature. As Bensimon explains, the staff and researchers from the
Center of Urban Education functioned in a unique role to facilitate and support the
evidence team members in their work (Bensimon, 2004, p.48). The CUE
researchers and facilitators were also sensitive to the fact that this process was new
to the evidence team and that it was important to remain cognizant of adult learning
principles when engaging with them.
During the analysis and coding of the field notes, I realized that this team’s
critical reflection and learning was facilitated by the CUE researchers in four ways:
they served to inform, clarify, reflect, and coach the teams’ course of action and
progression. Two key concepts from Vygotsky’s cognitive learning theory,
scaffolding and zone of proximal development came into play when the CUE
facilitators worked with the group. To briefly review these concepts, scaffolding is
used to help individuals learn new information by providing opportunities to foster

138
the development of cognitive skills. The readiness of an individual or group to
move to the next level of competency has been described by Vygotsky as the “zone
of proximal development”, that is, the distance between the actual development and
level of potential development. As the teacher uses scaffolding to assist the
performance of the students, the CUE facilitators helped the team members in a like
manner. As an example of how assisted performance occurred, consider the
development of “fine grained measures” from the team’s analysis of the data
related to the gatekeeper courses. The interpretation of existing data, disaggregated
by race and ethnicity was a new concept and skill for the team members, and at
each meeting they would discuss the results of the data analysis prepared by Karl.
Once the members had proficiency in this process, the CUE facilitators would pose
questions that would assist the members in identifying more discrete measures in
order to engage in process reflection.
The CUE facilitators not only informed the team about facts and figures, but
shared a philosophical viewpoint. It was a way of helping the group move away
from the sheer mechanics of analyzing the data, fostering a deeper questioning to
bare the more elusive elements of what the data may be indicating. Of all of the
team members, Karl was probably affected the most in terms of a shifting paradigm
and using the content to gain a new perspective. The CUE facilitators became
adept at recognizing when and how to pose challenging questions and cultivated
the members’ ability to delve into the issues, in effect, the CUE facilitators used
scaffolding as a strategy to move the members from a superficial analysis of the

139
inequities to a more profound understanding, therefore encouraging critical
reflection.
Several statements in field and debriefing notes depict the CUE facilitators’
sensitivity towards the team’s readiness to learn new concepts or to have their
existing “habits of mind” challenged. At intervals, when examining the data, the
team members would brainstorm but needed further direction or clarification in
how to approach the information. Because the CUE facilitators had experience in
interpreting this type of data, they were able to clarify many of the facts that were
obscured or simply not seen through the DSP lens. Clarification was not only used
in assisting the team with statistical content. A broader expanse of inquiry and
understanding was also prompted by experiential knowledge and insight from
working on the project. The facilitators also used reflective statements. Often, the
reflective statement was posed as a question rather than an answer which acted as
an impetus for the team members to engage in critical reflection.
During the meetings, the team members often revealed their own
perceptions and assumptions of other institutional actors, students and stakeholders,
and were concerned about how the scorecard would be received. Since the CUE
facilitators were not embedded in Riverton’s “culture”, they could often take on the
role of “the fly on the wall” or “outsider looking in”. It was apparent that many of
the issues that surfaced during the project were long standing problems, such as the
poor retention of out of state students, students that Riverton had actively recruited.

140
Having fresh perspectives from the CUE facilitators revitalized the team’s interest
in reestablishing and updating their goals based on evidence.
Finally, to foster the evidence team’s enthusiasm, the CUE facilitators
“coached” Karl, Sean and Laura. They observed and gave feedback that was timely
and meaningful within the context of the DSP, reinforcing or demonstrating
alternative actions to meet a goal.
In discussing these interactions between the team and facilitators, it appears
that the process of critical reflection was enhanced and supported by the DSP staff.
The question lies in whether critical reflection could have taken place without the
presence of outside facilitators. It is also plausible to consider the type of
preparation and level of proficiency an individual may need to facilitate groups
through complex problem solving. Familiarity with the DSP’s premises,
background and data analysis were requisite, not to mention the ability to
effectively communicate. In this case study, the CUE facilitators were well
prepared and a “good fit” with the team which greatly influenced how this team
progressed through the project.
Limitations
One of the limitations to this study is that the investigation was conducted
on only one institution of higher education out of 14 that participated in the
Diversity Scorecard project, therefore, I cannot compare this team’s learning to the
other teams. Bauman’s (2002) comparative analysis of learning at the team level
among the 14 teams that participated in the DSP revealed that groups that had

141
specific characteristics in fact learned at different levels and that a constellation of
factors influenced how and what these groups learned. Although the unit of
analysis for the Riverton case study was each individual team member rather than
the team,, there were learning characteristics among the individuals from this study
that resonate with Bauman’s findings related to group learning. The individuals at
Riverton possessed similar characteristics as Bauman’s “High Learning” groups,
that is, high learning groups based their judgments on institutional data and were
willing to surrender their experiential knowledge or assumptions when the evidence
invalidated it (Bauman, 2002, p.105).
The findings in this case study supported that the participation prompted
reflection and enhanced the team’s learning, but there is a significant constraint in
stating that transformative learning occurred. Additionally, this study did not look
at how this learning was translated into organizational learning.
Another limitation was the inability to corroborate the field notes with
another data-gathering method, such as interviews. The meeting minutes and
debriefing notes were taken and recorded over an 18-month period and
subsequently archived. The clarity, accuracy and interpretation of the dialogue was
highly dependent upon the facilitators’ ability to capture the words and essence of
the conversation in the written text. Therefore, my interpretations may be skewed
by biases held by the facilitators. As mentioned earlier, a facilitator’s experience
and proficiency in assisting a groups’ dynamics is also critical and may have a
significant impact upon group learning. The fact that the CUE facilitators and this

142
team had a very positive rapport raises the question of whether assisted
performance can be successful in the absence of an individual or groups’
receptivity to new information and learning. Good interpersonal relationships are
obviously desirable but it would not be rare to find them lacking in academic units
and among individuals who work in related areas.
Implications and Recommendations for Practice
Implications for Institutional Learning.
Using Data to Change “habits of mind.” In discussing how individuals
learned through the lens of an adult transformative learning model, specifically
using critical reflection, there must be a motivating stimulus and opportunities to
engage in critical reflection. The stage must be set with guidance and support to
sustain the process. The Diversity Scorecard Project was the catalyst for learning in
this case study and in the comparative study of the 14 teams conducted by Bauman
(2002).
The project provoked thoughtful, conscious inquiry into the educational
outcomes of minority students. Without conscious, intentional, action, assumptions
and practices that impact equity, remain unquestioned and unchallenged; and as
Renner (2003) pointed out, the persistence of educational inequities has been
sustained over the past five decades, demonstrating little progress in closing the
achievement gap in Hispanic and African American students. As Bensimon (2004)
asserts, the willingness of individuals to critically examine themselves and their
institutions is an essential perquisite to achieve an increased awareness of

143
inequities. She further explains that a common and difficult barrier to overcome is
the inclination to “look outside of ourselves for the source of problems or to avoid
examining them at all”(p.45). The importance of critical self-reflection cannot be
overemphasized if there is an authentic commitment by institutional actors to
acknowledge shortfalls in promoting equitable educational outcomes.
In order to engage in critical reflection and for learning to occur, the team
members needed to be open and willing to create new knowledge by using the
disaggregated data to drive their learning. They learned how to produce and use
evidence, and in the process became “user-driven researchers”. McEwen and
McEwen (2006) defined “user-driven” research as a method employed by those
“inside” of the institution to improve the effectiveness of their work, whether the
work is an academic program or administrative policy. For many educators, the
innovative use of data driven evidence to inform practice may be perceived as an
arduous and time-consuming process. Becoming a “user-driven researcher”
requires deliberation and persistence, which may be seen as a daunting task in an
already overcommitted, overburdened group of administrators and educators. One
could surmise that using evidenced based strategies, to evaluate institutional
performance is not embraced by academic communities because it is time
consuming and resource-intensive.
Although the process of data gathering and interpretation may seem
deceptively straight forward at first, it is a complex process that requires
instructional support, time and training.

144
Organizational learning in higher education is often motivated by concerns
about quality, accountability, and efficiency and often results in mandates that
colleges and universities meet with perfunctory reports to demonstrate compliance.
Historically, organizational assessment and change has been an “outside-in”
process, that is, consultants or pressures outside of the institution were the triggers
for change. Conversely, one of the most important and unique facets of the DSP
was its emphasis in supporting an “inside-out” examination of institutional
practices by those closest to the problems. Bensimon (2004) purports, that rather
than delivering a “packaged intervention”, the DSP called upon individual
institutional actors to explore their own assumptions, attitudes and actions. There
was also a foundational belief, that institutional actors were more likely to find the
causes and solutions to inequities if they were empowered in defining the problems.
An opportunity, such as the DSP, provides a structure and skill that
empowers an institution and its actors to internally drive the process. By learning
how to use evidence, contextually meaningful interventions can be developed by
those closest to the problems. Organizational learning occurs through the
individuals and groups that comprise the organization- without these conduits, new
knowledge cannot be transferred to the greater whole (Yorks and Marsick, 2000).
Although it was not the intention of this study to consider the impact of
facilitators upon learning, they played a positive role in assisted performance with
the team. The level of proficiency in understanding data, facilitating group

145
dynamics, and general, broad knowledge of the project’s conceptual model are
noteworthy variables to consider.
Conclusion
The evidence team at Riverton Lake College completed their initial charge
and developed a Scorecard that reflected the indicators to assess institutional
performance in producing equitable educational outcomes for minority students in
the four scorecard perspectives: access, retention, institutional receptivity, and
excellence. During the course of this project, inequities were identified and
investigated to produce contextually meaningful monitors to guide performance
and improvement. However, another important outcome from participating on the
team was that these individuals learned. There was evidence that feelings and
attitudes were positive throughout the process and that the team used elements of
critical reflection to identify and define problems related to equity.

146
REFERENCES
Argyris, C. (1991). Teaching Smart People How to Learn. Reflections, 4(2), 4-15.

Argyris, C. (1977). Double Loop Learning in Organizations. Harvard Business
Review:(Sept/Oct. 1977) 115-125.

Argyris, C.& Schon, D.(1996). Organizational learning II: Theory, method and
practice. New York, Addison-Wesley.

Bauman, G.L. (2002). Developing a culture of evidence: Using institutional data to
Identify educational outcomes. Unpublished dissertation, University of
SouthernCalifornia, Los Angeles, California.

Bauman, G.L., Bustillos, L.T., Bensimon, E.M., Brown, M.C., Bartee, R.D. (2005).
Achieving Equitable Educational Outcomes with All Students: The
Institution’s Roles and Responsibilities. Monograph commissioned by the
Association for American Colleges and Universities

Belenky, M., & Stanton, A.V.(2000). Inequality, Development and Connected
Knowing.In Jack Mezirow & Associates (Eds.) Learning as
transformation. San Francisco: Jossey-Bass. (71-101).

Bennet, C.I.(1995)Preparing teachers for cultural diversity and National Standards
of Academic Excellence. Journal of Teacher Education, 46, 259-266.

Bensimon, E.M. (2004). The Diversity Scorecard. Change (January/February), 45-
52.

Bensimon, E.M. (2000). Designing and Implementing a Diversity Scorecard
A Proposal Submitted to The James Irvine Foundation : The Center
for Urban Education September 9, 2000.

Bensimon, E.M. (2005) Closing the achievement gap in higher education: An
organizational learning perspective. New Directions for Higher Education:
131(Fall, 2005).

Bensimon, E. M. (2005). Equality as a fact, equality as a result: A matter of
institutional accountability. Monograph commissioned by the American
Council on Education.

147
Bensimon, E.M., Polkinghorne, D.E., Bauman, G.L., & Vallejo, E. (2004).
Doing research that makes a difference. The Journal of Higher Education,
75, (1) 105-126.

Bensimon, E. M. & Polkinghorne, D. (2003). The accountability side of
equity,Diversity Digest, 7(1,2).

Boyce, M. E., (2003). Organizational learning is essential to achieving and
sustaining change in higher education. Innovative Higher Education, 28,
119-136.

Boyd, R.D. and Myers, J.G. Transformtive education. International Journal of life
Long Education 7, no 4. 1988 261-284

Bromley, D.M. (1990). Academic contributions to psychological counseling: A
philosophy of science for the study of individual cases. Counselling
Psychology Quarterly, 3(3), 299-307.

Byrnes, J.P. (1996). Theories of cognitive development and learning. In J.P. Byrnes
Cognitive development and learning in instructional contexts. Boston:
Allyn and Bacon.

Cranton, P. (1996). Professional Development as Transformative Learning. San
Francisco: Jossey-Bass, 75-117

Daft,D.L.& Huber,G.P.(1987). How organizations learn: A communication
framework.Research in the Sociology of Organizations,(5), 1-36.

Daigre, E.(2001) Toward a critical service-learning pedagogy: A Freirean approach
to civic literacy. In "Service Learning," In Hagedorn, L Serra, ed., Sound
Instruction: Ready to Use Classroom Practice (Chattanooga, TN: Rapid
Intellect Group, 2001)
Dirkx, J. M. (1998). Transformative learning theory in the practice of adult
education: An overview. PAACE Journal of Lifelong Learning, 7, 1-14.

Dirkx, J. M., Meziorw, J. Cranton, P.(2006). Musing and reflections on the
meaning, context, and process of transformative learning: A dialogue
between John M. Dirkx and Jack Mezirow. Journal of Transformative
Education. 4, 124.

148
Easterby-Smith, M.(1997). Disciplines of organizational learning: Contributions
and critiques. Human Relations, 50, 1085-1113.

Friere P. (1970). Pedagogy of the Oppressed. New York: Herder and Herder
(1972)

Fullan, M. (1993). Change forces: Probing the depths of educational reform.
London: Falmer Press.

Hallett, R.(2003) Assessing reflection: classroom based research and professional
development processes and practices. ATN Evaluations and Assessment
Conference, November , 2003, University of Australia, Adelaide, South
Australia.
Retrieved: 2/7/06 http://www.unisa.edu.au/evaluations/index.htm
http://www.unisa.edu.au/evaluations/Full-papers/HallettFull.doc

Howard, G.(1999). We can't teach what we don't know: white teachers, multiracial
schools. New York: Teachers College Press.

Johnson, L.B. (1965). “To fulfill these rights”. Commencement address at Howard
University, June 14, 1965.

Kegan, R. (2000). What “forms”transforms? A constructive-developmental
approach to transformative learning. In Jack Mezirow & Associates (Eds.)
Learning as transformation. San Francisco: Jossey-Bass. (35-70).

Kasl,L., Marsick, V.& Denchant, K.(1997). Teams as learners. Journal of Applied
Behavioral Science, 33, 227-246.

Kember, D.(1999).Determining the level of reflective thinking from students'
written journals using a coding scheme based on the work of Mezirow.
International Journal of Lifelong Education, 18, 18-30.

Kezar, A., & Eckel. P. (2002). Examining the institutional transformation process:
The Importance of sensemaking, inter-related strategies and balance.
Research in Higher Education. 43(3), 295-328

Kezar, A., & Eckel, P. (2002). The effect of institutional culture on change
strategies in higher education: Universal principles or culturally responsive
concepts? The Journal of Higher Education, 73(4), 435-460

McGlynn-Provitera, A. The Status of opportunity: The Hispanic Outlook in
Higher Education: Paramus: May 23, 2005. Vol. 15, Iss. 17, pg. 18

149

Merriam S.B.(2002). Qualitative research in practice. San Francisco: Jossey-Bass
(6-12).

Merriam, S.B.(1998). Qualitative research and case study applications in education.
San Francisco: Jossey-Bass.

Mezirow, J.(1998). On critical reflection. Adult Education Quarterly.48,(3) 185-
198

Mezirow, J.(1991) Transformative dimensions of adult learning. San Francisco:
Jossey- Bass).

Mezirow, J.(1994). Understanding transformation theory. Adult Education
Quarterly, 44,(222-32).

Mezirow, J.(2000). Learning to think like an adult. In Jack Mezirow & Associates
(Eds.) Learning as transformation. San Francisco: Jossey-Bass. (3-33).

Mezirow, J.(2003). Transformative learning as discourse. Journal of
Transformative Education, 1, 58-63.

McEwen, E.K., McEwen, P.J (2003) Making sense out of research
Thousand Oaks: Sage Publications, Corwin Press.

National Center for Education Statistics.U.S. Department of Education. Available:
www.nces.ed.gov.

Nieto, S. (2000). Placing equity front and center: some thoughts on
transforming teacher education for a new century. Journal of Teacher
Education, 51, 180-187.

Piaget, J. (1952). The origins of intelligence in children. New York. International
Universities Press.

Piaget, J.& Inhelder, B.(1969).The psychology of the child. New York: Basic
Books.

Polkinghorne, D.E.(2004). Practice and the human sciences: the case for a
judgment- based practice of care. New York: SUNY.

150
Polkinghorne, D.E ,(1997) Reporting Qualitative Research as Practice," In
: Representation and the Text: Reframing the Narrative Voice , Ed. W.G.
Tierney and Y.S. Lincoln 3-21 Albany: State University Press.

Plack and Greenberg (2005) The Reflective Practitioner: Reaching for
Excellence in Practice Pediatrics, 116 (6),1546-1552.

Renner, K.E. (2003). Racial equity and higher education. Academe, 89 (1), 38-43.

Sarason, S. (1982) The culture of the school and the problem of change. Boston:
Allyn &Bacon.

Senge, P., Kleiner,A., Roberts, C.,Ross, R., Roth,G., & Smith, B. (1999). The
dance of change: The challenges in sustaining momentum in learning
organizations. New York, NY: Doubleday.

Shrivastava, P. (1983). A typology of organizational learning systems. Journal of
Management Studies (20), 7-28.

Solomon, P. (1995). Beyond prescriptive pedagogy: teacher inservice education for
cultural diversity. Journal of Teacher Education, 46, 251-258.

Stake, R.(1995).The art of case research. Thousand Oaks, CA: Sage Publications.

Taeber,C. (Ed).(1996). The statistical handbook of women in America (2
nd
Ed.).
Phoenix: Oryx Press.

Taylor, E.W.(1997). Building upon the theoretical debate: A critical review of the
empirical studies of Mezirow’s transformative learning theory. Adult
Education Quarterly, 48(1) 34-59

Taylor, E.W.(2001). Transformative learning theory: a neurobiological perspective
of the role of emotions and unconscious ways of knowing. International
Journal of Lifelong Learning, 20: 218-236.

Taylor, E.W.(2000). Analyzing research on transformative learning theory, In
Mezirow, J., (2000). Learning to think like an adult. In Jack Mezirow &
Associates (Eds.) Learning as transformation. San Francisco: Jossey-
Bass. 29-310.

151
Tharp, R.G. & Gallimore, R. (1988). Rousing minds to life. Cambridge
University Press, Melbourne, Australia.

United States Census Bureau, Available:
http://www.census.gov/econ/census02/

Weick, K.E., & Westley,F.(1996). Organizational learning: Affirming an
oxymoron. In S.R. Clegg, C. hardy, and W.R. Nord (Eds.), Managing
organizations. Thousand Oaks Sage Publications.

Williams, B.(2000). Developing critical reflection for professional practice through
problem- based learning. Journal of Advanced Nursing. 34: 27-34.

Yorks, L,& Marsick, V. Organizational learning and transformation. In Jack
Mezirow & Associates (Eds.) Learning as transformation. San Francisco:
Jossey-Bass. (253-280).

Yin, R.K. (1994).Case study research: Design and methods (2
nd
ed.). Beverly Hills,
CA: Sage Publishing.

Yin, R.K. (2003).Case study research: Design and methods (3rd ed.). Beverly
Hills, CA: Sage Publishing.

152
Appendix A
Sample Field Notes

K CUE 2

CUE = CUE Facilitator 1
L = Laura
K = Karl
CUE = CUE Facilitator 2
S = Sean
Guest = Guest

S
Guest

CUE 1 L

This was not a typical site visit. This meeting served as a way to help develop a
positive connection with the Academic Vice President. The Riverton team met with
the CUE facilitators last week to go over the agenda. Sean wanted us (CUE
facilitators) to explain the DS project so we prepared a presentation.

The guest sat next to me (CUE facilitator 2)for the meeting. He was expected to sit
next to Sean, but he chose the chair next to me. For this reason, I did not take
copious notes.
I introduced myself. I guided him through the presentation (The full presentation
can be found on the G-drive so this will not be repeated). He asked a few questions
and made some comments. I with briefly go into each of these.

Schedule of meetings December 2000-March 2004

Year 1- (12/00-12/01) 14 meetings
Year 2-(1/02-10/02) 12 meetings
Year3-(6/03-9/03) 3 meetings
Year4-(3/04) 2meetings

153

Appendix B

Riverton Lake College- Diversity Scorecard Goals and Measures by Perspectives

ACCESS
Goal 1: To increase the number and percentage of enrolled African American students
Measure 1: African American student application rate (number and percentage)
Measure 2: African American student admittance rate (the number and percentage of students
who apply that are admitted)
Measure 3: African American student yield rate (the number and percentage of students who
are admitted that enroll)
RETENTION
Goal 1: To increase the freshmen-to-sophomore retention rates by ethnicity
Measure 1: Fall-to-fall freshman retention rate by ethnicity (e.g. freshman to sophomore year
retention)
Measure 2: Semester-to-semester retention rate by ethnicity
Measure 3: Incoming quintile retention rate by ethnicity
Measure 4: Amount and type of financial aid by retention rate by ethnicity
Measure 5: “Gatekeeper” courses in freshman year by ethnicity
Goal 2: To increase the number and percentage of target group students’ “success” in “gateway”
courses.
Measure 1: Completion rate and grade by ethnicity

INSTITUTIONAL RECEPTIVITY
Goal 1: To develop/enhance a climate of respect and appreciation of different cultures, ethnicities,
gender, sexual orientation, and social class on campus
Measure 1: Campus climate survey (NSSE
2
) scores by ethnicity
Measure 2: Hiring rates for faculty/staff of color
Measure 3: Rates of bias/discrimination/harassment incidents
Measure 4: Senior survey diversity- and satisfaction-related question results by ethnicity (e.g.,
question 11, and supplemental questions 30-35 and 38)

EXCELLENCE
Goal 1: To increase the number of African American, Asian American, and Hispanic students who
are identified as student leaders
Measure 1: Number and percentage of students in POET leaders by ethnicity
Measure 2: Number and percentage of RAs by ethnicity
Measure 3: Number and percentage of OWLs by ethnicity
Measure 4: Number and percentage of club leaders by ethnicity
Measure 5: Number and percentage of academic department award recipients by ethnicity
Goal 2: To increase the number and percentage of African American, Asian American, and Hispanic
students on the Dean’s Honor List (e.g., honor roll)
Measure 1: Number and percentage of students on Dean’s Honor Roll by ethnicity

154

Appendix C

Transformative Learning –Conceptual Framework

Assumptions-myths,
experiences, Frames
of Reference

Catalyst for Reflection
Disaggregate Data New Awareness

Content
Reflection
What ?
DSP
Design

Process
Reflection
How ?

Premise
Reflection
Why?
Why does this
problem exist-
What are the facts
What is the evidence
What else do I need
How –Strategies to
approach problem
Transformative
Learning-
New Meaning –
Perspectives
Change in
Attitudes-
New Frames of
Reference
Dispelled Myths

Abstract (if available)

Abstract This study is an evaluation of learning in individual institutional actors during their participation in a project that involved them 14 campus teams whose responsibility was to examine data on students' educational outcomes disaggregated by race and ethnicity. The purpose of the project was to assist campus teams in the construction of a diversity scorecard to continuously monitor the progress of minority students toward the attainment of equitable educational outcomes. The premise of this project, known as the 'Diversity Scorecard', was that, by becoming aware that inequities exist among historically underrepresented minority students, institutional actors would then be compelled to take responsibility for addressing them.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

An evaluation of individual learning among members of a diversity scorecard project evidence team

PDF

Creating an equity state of mind: a learning process

PDF

Achieving equity in educational outcomes through organizational learning: enhancing the institutional effectiveness of community colleges

PDF

Equitable choices in higher education: investigation of an equity-based faculty search committee development program

PDF

The influence of organizational learning on inquiry group participants in promoting equity at a community college

PDF

Making equity & student success work: practice change in higher education

PDF

Participation in higher education diversity, equity, and inclusion work: a relational intersectionality of organizations analysis

PDF

Language and identity in critical sensegiving: journeys of higher education equity agents

PDF

The college selection process of high-achieving Latino students

PDF

Remediating artifacts: facilitating a culture of inquiry among community college faculty to address issues of student equity and access

PDF

Examining the relationship between students’ pre-college experiences and outcomes in diversity courses

PDF

Changing the landscape of institutional assessment on transfer: the impact of action research methods on community college faculty and counselors

PDF

Elementary principal leadership and learning outcomes for low socioeconomic status Hispanic English learners

PDF

The “lost boys” of higher education: African American males from basic skills through university transfer

PDF

Racing? to transform colleges and universities: an institutional case study of race-related organizational change in higher education

PDF

AB 705: the equity policy – race and power in the implementation of a developmental education reform

PDF

A LatCrit analysis of the educational access, attainment, and life outcomes of Mexican origin people

PDF

Success in the sticky: exploring the professional learning and instructional practices that are sticky for distinguished secondary STEM educators of students historically…

Asset Metadata

Creator Bruning, Madeleine D. (author)

Core Title Beyond access: an evaluation of attitudes and learning towards achieving equitable educational outcomes in higher education

School Rossier School of Education

Degree Doctor of Education

Degree Program Education

Publication Date 11/15/2006

Defense Date 10/24/2006

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag equity outomes,Higher education,OAI-PMH Harvest

Language English

Advisor Bensimon, Estela Mara (committee chair)

Creator Email bruning@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-m153

Unique identifier UC1385472

Identifier etd-Bruning-20061115 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-30559 (legacy record id),usctheses-m153 (legacy record id)

Legacy Identifier etd-Bruning-20061115.pdf

Dmrecord 30559

Document Type Dissertation

Rights Bruning, Madeleine D.

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Repository Name Libraries, University of Southern California

Repository Location Los Angeles, California

Repository Email cisadmin@lib.usc.edu