University of Southern California Dissertations and Theses

A cluster analysis of the differences between expert and novice counselors based on real time training

(USC Thesis Other)

A cluster analysis of the differences between expert and novice counselors based on real time training

PDF

Download

Open document

Flip pages

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content INFORMATION TO U SER S
This manuscript has been reproduced from the microfilm master. U M I films
the text directly from the original or copy submitted. Thus, some thesis and
dissertation copies are in typewriter face, while others may be from any type of
computer printer.
The quality of this reproduction is dependent upon the quality of the
copy subm itted. Broken or indistinct print, colored or poor quality illustrations
and photographs, print bleedthrough, substandard margins, and improper
alignment can adversely affect reproduction.
in the unlikely event that the author did not send UM I a complete manuscript
and there are missing pages, these will be noted. Also, if unauthorized
copyright material had to be removed, a note will indicate the deletion.
Oversize materials (e.g., maps, drawings, charts) are reproduced by
sectioning the original, beginning at the upper left-hand comer and continuing
from left to right in equal sections with small overlaps.
ProQuest Information and Learning
300 North Zeeb Road, Ann Arbor, M l 48106-1346 USA
800-521-0600
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
A CLUSTER ANALYSIS OF THE DIFFERENCES
BETWEEN EXPERT AND NOVICE COUNSELORS
BASED ON REAL TIME TRAINING
by
Chen Z. Oren (Opochinsky)
A Dissertation Presented to the
FACULTY OF THE GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment o f the
Requirement o f the Degree
DOCTOR OF PHILOSOPHY
EDUCATION (COUNSELING PSYCHOLOGY)
December 2001
Copyright 2001 Chen Z. Oren (Opochinsky)
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
UM I Number 3065829
Copyright 2001 by
Oren (Opochinsky), Chen Zvi
All rights reserved.
___ __®
UMI
UMI Microform3065829
Copyright 2002 by ProQuest Information and Learning Company.
All rights reserved. This microform edition is protected against
unauthorized copying under Title 17, United States Code.
ProQuest Information and Learning Company
300 North Zeeb Road
P.O. Box 1346
Ann Arbor, Ml 48106-1346
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
UNIVERSITY OF SOUTHERN CALIFORNIA
THE GRADUATE SCHOOL
UNIVERSITY PARK
LOS ANGELES, CALIFORNIA 90007
This dissertation, written by
......................
under the direction of h. Dissertation
Committee, and approved by all its members,
has been presented to and accepted by The
Graduate School, in partial fulfillment of re
quirements for the degree of
DOCTOR OF PHILOSOPHY
Dean of Graduate Studies
DISSERTATION COMMITTEE
Chairperson
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
DEDICATION
This dissertation is dedicated to my parents who did not get to share the
completion of this experience with me. In there own way, they were both
instrumental in instilling in me the desire, motivation, and dedication to devote many
years of my life to achieve the highest degree in psychology.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
ACKNOWLEDGEMENTS
I would like to express my deep appreciation and thanks for my advisor,
mentor, and friend, Dr. Joan Rosenberg. Dr. Rosenberg was instrumental in the
completion of this study in a timely and professional matter. Her wisdom, patience,
support, mentoring, motivation, perfectionism, and sense of humor enabled me to
combine learning and enjoyment in this challenging journey.
I would like to thank Yifat Oren for her patience, commitment, and support
through 10 years of higher education and three academic degrees. I will never forget
her contribution and support.
Dr. John Brekke and Dr. Richard Clark, my committee members provided
valuable feedback to this study. Dr. Clark’s expertise, feedback and belief in the
importance of the topic provided me much encouragement and motivation.
My sister and brother and their families have been always a safety net
providing great support in time of need. I would like to thank them for their
unconditional love, care and help.
Finally, I would like to thank my friends and colleagues Ferdinand Arcinue,
Lindsey Buckman, and John Carter, who helped me with data entry and analysis and
in many other ways. You all played a major a role in my successes and I wish you
all well.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
TABLE OF CONTENTS
Page
DEDICATION.......................................................................................................... ii
ACKNOWLEDGEMENTS......................................................................................H i
LIST OF TABLES................................................................................................... vi
LIST OF FIGURES................................................................................................. vii
ABSTRACT........................................................................................................... viii
CHAPTER
1 INTRODUCTION.............................................................................. 1
Critique of the Research Literature........................................4
2 LITERATURE REVIEW.................................................................. 1 1
Development of Expertise.....................................................11
Identifying Expertise: Difficulties Inherent in Defining an... 17
Expert
Novice - Expert: The Differences......................................... 18
Expertise and Medicine.........................................................20
Expert-Novice: Application to Counseling...........................26
Development of Expertise in Counseling: Definition,..........26
Descriptions, and Assumptions about Expertise
Empirical Findings................................................................30
Training Model based on Expertise Principles.....................37
Summary of the Literature...................................................43
Purpose of Study..................................................................46
Research Hypotheses...........................................................46
3 METHODOLOGY...........................................................................48
Design..................................................................................48
Participants/Raters...............................................................48
Procedure.............................................................................52
Data Analysis.......................................................................56
4 RESULTS.........................................................................................59
Preliminary Analyses...........................................................59
Hypothesis 1 .........................................................................82
Hypothesis 2 .........................................................................83
Hypotheses 3 & 4 .................................................................84
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
V
5 DISCUSSION...................................................................................88
Hypothesis 1...........................................................................88
Hypothesis 2 .........................................................................96
Hypothesis 3 .........................................................................97
Hypothesis 4........................................................................100
Other Findings....................................................................101
Implications........................................................................105
Limitations of the Study..................................................... 107
Recommendations for Future Research............................. 109
REFERENCES.......................................................................................................112
APPENDICES
A INFORMED CONSENT................................................................. 119
B SUBJECT GENERAL INFORMATION SHEET............................122
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
LIST OF TABLES
Table
Page
1 Novice Demographics........................................................................ 49
2 Expert Demographics......................................................................... 53
3 Agglomeration Table for Novice Counselors/Hierarchical............... 60
Cluster Analysis
4 Novice Cluster Membership and Example Items............................... 68
5 Agglomeration Table for Expert Counselors/Hierarchical Cluster.. .72
Analysis
6 Expert Cluster Membership and Example Items.................................80
7 Univariate ANOVA for the Three Variables Effect on Expert’ 86
Cluster Membership
8 Univariate ANOVA for the Three Variables Effect on Expert’s 86
Cluster Members
9 Group Means for Novices’ 5-Cluster Solution....................................86
10 Group Means for Novices’ 5-Cluster Solution...................................87
11 Breakdown of The Feedback Comments into Expert and Novice ... 89
Clusters
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
vii
LIST OF FIGURES
Figure Page
1 Dendogram for Novice Counselors................................................... 63
2 Dendogram for Expert Counselors.................................................... 76
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
ABSTRACT
The expert - novice (cognitive psychology) literature provided the theoretical
framework for this study, which was designed to explore and identify the differences
between novice and expert clinicians’ thought processes, schemas and
conceptualizations as they related to clinical feedback provided during actual
counseling sessions
This dissertation used a computer, audio and video technology as a research
methodology (Real-Time Training; RTT; Rosenberg, 2000) for accessing the thought
processes of novice and expert counselors in real time. This innovative research
methodology allows the differences between novice and expert counselors’
conceptualizations in two different and separate levels to be explored. This included
identifying: a) the different type of feedback comments novice and expert counselors
provide while observing a counseling session and b) the differences in how these
feedback statements were categorized by novice and expert counselor raters.
Ten novice and 10 expert counselor raters sorted 150 typed feedback
statements into categories based on themes and similarities. Seventy five statements
were made by novice counselors and 75 by an expert counselor during actual
counseling sessions. The feedback statements were made during counseling practica
across six different semesters. Cluster analysis was used to examine the differences
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
in novice and expert counselor raters’ categorization of the feedback statements and
a Discriminant Factor Analysis (DFA) was used to analyze the type of feedback
comments novice and expert counselors provided during the live supervision (RTT)
process.
The cluster analysis revealed that expert counselor raters produced fewer (4)
clusters than the novice counselor raters (5) by integrating three novice clusters into
one category. Expert and novice raters computed the categorization tasks in almost
equal time. The DFA suggested that, overall, novice and expert counselors made
different type of feedback comments, however, the time in the semester the feedback
comments were made did not influence the type of comments made. Results are
consistent with prior findings in the general expertise literature and the counseling
expertise literature suggesting that experts’ superior performance may be due to their
more efficient use of cognitive schemas.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1
CHAPTER 1: INTRODUCTION
Counseling is a complex and ambiguous process that requires the ability to
process a large amount of information and the ability to integrate many different
skills (Johnson & Happner, 1989). Earlier, Loganbill, Hardy, and Delworth (1981)
identified three sets of learning tasks that counseling psychologists in training need
to acquire: “factual knowledge and theoretical concepts, skills, and personal self-
knowledge” (p. 6). Hill and O’Grady (1985) later stated that therapists need to
negotiate a large amount of data such as: presenting problems, diagnoses, goal
setting, behavioral observations, clinical hypotheses, and personal reactions, in an
incredibly quick and sophisticated manner in order to establish an overall treatment
plan. Further, students pursuing a career in psychology providing psychotherapy
need to learn a significant amount of counseling related information before working
with clients (e.g., psychological theory and diagnostic skills) including learning how
to apply and integrate this knowledge with actual clients, in real situations.
This transition from theory and classroom learning to clinical situations with
clients is a challenging evolution which generally takes a significant amount of time
to master. Bernard and Goodyear (1998) assert: “clinical supervision provides the
crucible in which supervisees can blend formal theory learned in the classroom and
practitioner driven knowledge gained through practice and incorporate them as their
own working knowledge (p. 4).” For the purpose of this dissertation, the terms,
trainee, supervisee and novice will be used interchangingly. Clinical supervision
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2
also serves the purpose of protecting the welfare of clients. Ideally, an expert
(supervisor) transfers his or her own clinical expertise while simultaneously
monitoring a novice's performance, as the student gains experience and develops
knowledge. A supervisor faces the task of attending to client issues and concerns (to
protect the welfare of the client), monitoring a trainee’s performance, and monitoring
his or her own reactions to the client and trainee. Clearly, attending to these multiple
levels for the purpose of facilitating the trainee’s growth is a complex and
challenging task.
Various training methods and models have been developed (e.g. self-report,
audiotape, process notes, videotape, and live forms of supervision) to assist
supervisors’ training tasks as they train novice counselors (Johnson & Heppner,
1989). In the last 20 years, supervision training has been evolving and improving
with the development of new technology and research, which has focused on the
efficacy of clinical training and on the integration of new models and conceptual
fiumeworks. Essentially, many theories about supervision were developed to help
supervisors address the learning process of trainees across the trainees’
developmental stages (Bernard & Goodyear, 1998) and a significant number of these
theories focus on how supervisees change as they gain training and supervised
experience (Blocher, 1983; Loganbill, Hardy & Delworth, 1982; Stoltenberg, 1981).
Overall, these theories tend to emphasize counselors’ (supervisees) development of
communication skills and the optimal supervisory environment to promote
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
supervisees’ satisfaction and growth in each stage (Bernard & Goodyear, 1998;
Etringer, Hillerbrand, & Claibom, 1995) rather than how complex knowledge can
best be transferred from an expert to a novice through training. Additionally, the
models that were based on the assumption that several distinct stages of experience
occur throughout counselor training were criticized because differences in trainee
characteristics across levels of experience appeared to exist only between early
beginners and intem-level trainees (Holloway, 1987).
As research was conducted in the area of supervision, psychologists and
educators also witnessed an increase in information regarding human learning and
the development of expertise drawn from the field of cognitive psychology (e.g.,
Anderson, 1981; Chi, Glaser, & Farr, 1988; Ericsson & Chamess, 1994; Ericsson &
Lehman, 1996). Whereas some academic disciplines (e.g. education, medicine,
computer science, music, and sport psychology) have integrated expertise principles
to enhance the learning process in their respective disciplines, clinical training lags
far behind in applying expertise principles to the clinical training of new
psychotherapists or counselors. With a growing body of research regarding the
performance of experts combined with the knowledge about the limitations of the
developmental supervision models, researchers finally recognized the need to
improve clinical training and supervision by integrating principles from the field of
cognitive psychology (Binder, 1993; Fuqua, Johnson, Anderson & Newman, 1984;
Johnson & Happner, 1989; Rosenberg, 1997). These researchers believe that a
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4
counselor’s cognitive processing is critical to his or her ability to effectively develop
and formulate behavioral responses and to generalize acquired skills to new
situations in the therapeutic situation (Moran, Kurpius, Brack & Brack, 1995).
However, only a limited number of studies seeking to understand the cognitive
principles of expertise have been conducted in the area of counseling and
psychotherapy training. Further, most of the available studies are descriptive in
nature and/or are fraught with numerous methodological limitations.
Critique of the Research Literature
First, there are methodological problems. While theoretical literature (e.g.
Binder, 1993; 1999; Goodyear, 1997; Hillerbrand, 1987; Locke & Covell, 1997;
Lichtenberg, 1997; Skovholdt, Ronnestad & Jennings, 1997) and qualitative studies
(Jennings, & Skovholt, 1999) are important, there is a strong need for well-designed
empirical studies. Almost all of the empirical studies regarding counseling expertise
are analogue in nature and use retrospective methods to explore expert and novice
counselors’ cognitive process (e.g. transcripts, videotapes of sessions. Interpersonal
Process Recall; IPR; and questions or free association after the conclusion of
counseling session(s) were concluded).
The examination of novice and expert counselors’ cognitive processes and
schemas in such a manner poses some limitations. Analogue studies do not closely
resemble the type of cognitive processes and/or problem solving skills (e.g. use of
memory, speed and type of decision making, case conceptualization and accurate
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5
diagnosis) counselors are called on to make in clinical settings. Retrospective
studies are more remote and as such the subject’s memory can play a role by
reporting only the salient cognitive patterns. Further, thoughts are dynamic and
constantly change which may reduce the reliability of the study; the thought
processes a subject may report in retrospective studies may be different than the
actual thoughts processes the same subject may experience in real-time with a client.
For example, reading transcripts or reviewing a videotape of a counseling session
provides counselors more time to think and process important clinical information.
Further, transcripts and audiotapes cannot capture non-verbal information from a
session. In clinical settings counselors are required to respond immediately to many
levels of information (verbal and non-verbal information).
Also, it is common for empirical studies to attempt to explore differences
between novice and expert counselors by focusing on one counseling session in the
form of vignette, transcript or videotape (e.g. Cummings, Hallberg, Martin, Slemon,
and Heibert, 1990; Martin, Slemon, Hiebert, Hallberg, and Cummings, 1989;
Kivlighan & Quigley, 1991; Mayfield, Kardash, and Kivlighan, 1999; O’Byran, &
Goodyear, 1997). This singular focus does not allow a researcher to track changes in
novice counselors as they develop clinical expertise. In fact, there are no available
studies reviewing the specific changes that occur during this evolution.
Consequently, the available information regarding possible changes in novice
counselors’ schemas and cognitive processes are being interpolated primarily from
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
6
research in other domains. There is a real need for research designs which span a
greater number of sessions to provide empirical information regarding the
development of expertise in novice counselors.
Second, there are theoretical and practical definitional problems. Whereas,
the development of expert performances in a specific domain has been demonstrated
and studied in different fields, Dawes (1994) argued that psychotherapy expertise is
nothing but a myth. He based his argument on many studies (e.g. Frank & Frank,
1993; Seligman, 1995; Wampold, Mondin, Moody, Stich, Benson & Ahn, 1997)
which concluded that credentials, experience, and theoretical orientation of
psychotherapists are unrelated to patient outcomes. A major limitation in
determining whether counseling expertise exists is the difficulty in measuring the
effectiveness of psychotherapy outcome.
Assuming that counseling expertise exists, there appears to be an absence of a
clear definition of what constitutes an expert counselor (Goodyear, 1997). Years of
counseling experience have been most widely used as a proxy for expertise; however
years of expertise alone have not been shown to be an appropriate measure of
expertise (Goodyear, 1997; Jennings & Skovholdt, 1999). Further, Skovholdt,
Ronnestad, and Jennings (1997) noted that expertise actually occurs after thousands
hours of practice and an average of at least 15 years of professional practice. In
addition, they noted that in order to eliminate a narrow range of experience
differences between novice and expert counselors, research studies should use a
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7
range difference of at least 10 years (novice to expert) of professional counseling
experience. Consequently, Jennings and Skovholdt (1999) suggested a new standard
for defining counseling expertise in research that includes: a) years of experience (at
least 15 years), b) qualification (holding at least one license), and c) peer-selection.
However, none of the previous empirical studies adhere to all aspects of the above
definition of expertise. For example, the mean range of years of experience in the
O’Byme and Goodyear (1997) study was less then 10 years. The Kivlighan and
Quigley (1991) investigation used the criterion of at least 1000 hours of group
counseling experience to define counseling expertise, although, most of the other
studies defined expertise by 10 or less years of experience (Cummings et al., 1990;
Martin, et al., 1989; Mayfield, Kardash, & Kivlighan, 1999).
Third, there are sampling problems. Samples are frequently small, so that the
results cannot be generalized (e.g. Brammer, 1997; Cummings, Hallberg, Martin,
Slemon, & Heibert, 1990; Martin, Slemon, Hiebert, Hallberg, & Cummings, 1989).
None of the studies controlled for subject’s ethnicity as a possible confound.
Further, some studies use academicians as expert counselors (e.g. Brammer, 1997).
However, academicians who do not practice counseling and/or psychotherapy on a
regular basis do not adequately represent or resemble expert practitioner-counselors.
Additionally, novice and expert counselor subjects are usually selected from the
same training program. For example, Mayfield, Kardash, and Kivlighan (1999) used
novice and expert counselors selected from the same counseling psychology training
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
8
program; the counselors in the Cummings et al. (1990) study were all working in the
same university counseling center and shared a similar theoretical orientation
(cognitive-behavioral). Mayfield et al note: “sharing a common training program
and theoretical orientation may create homogeneity in the participants’
conceptualization and limit the generalizability of the findings” (p.513).
Fourth, although researchers and practitioners call for the integration of
principles of expertise into the training of novice counselors and psychotherapists
(Binder, 1993; 1999, Hillerbrand, 1989; O’Byme, Clark, Malakuti, 1997) only three
studies attempted to apply cognitive principles of expertise into the development of
new supervision training models (Rigazio-Diglio, Daniels, & Ivey, 1997; Rosenberg,
1997; Stolenberg, McNeil & Delworth, 1997). Thus, there is a need to develop and
empirically test more training models grounded in the cognitive psychology
literature. These models and research studies could provide tools to enhance the
clinical training of novice counselors and psychotherapists.
Finally, Binder (1999) called for the use of integrative computer programs to
help novice counselors breach the gap between theoretical classroom teaching and
hands-on experiential clinical skills required to conduct psychotherapy with real
clients. He suggested that computer technology could enhance novice counselors’
problem solving and clinical skills and serve as an important research tool.
However, only the Brammer (1997) study used computer technology to study
counseling expertise. Brammer used a computerized client as a stimulus to elicit
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
9
subjects’ thoughts and conceptualization. The analogue nature of the study limits the
generalzability of this study’s findings.
In summary, the existing research literature fails to adequately address the
various aspects of counseling expertise. Most of the studies are analogue and
retrospective in nature and have been hampered by sampling and methodological
problems. Further, there is a real need for research studies to explore the
development of expertise in novice counselors across several counseling sessions.
Overall, the existing studies do not consistently apply the knowledge gained through
the expertise literature to develop training models that may improve novice
counselors’ clinical skills. Thus, research is needed which avoids some of the
limitations that have plagued other studies.
This study addressed some of the major gaps noted above. This investigation
particularly addressed the following limitations noted in previous studies: a) the
analogue and retrospective nature, b) the lack of studies of greater duration that
allows for exploration of changes in novice counselors’ schemas and thought
processes in their transition toward expertise, c) a consistent adherence to an
appropriate definition of novice and expert counselors, and d) the lack of integration
of principles of expertise and computer technology to the training of new counselors
and psychotherapists.
This empirical study contributed to the professional counseling expertise
literature by using an innovative and groundbreaking research tool utilizing
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
10
computer technology and audio/visual aids (Real-Time Training; RTT; Rosenberg,
1998). RTT allowed the researcher of this study to access and identify for the first
time the thought processes of novice and expert counselors in real-time (during an
actual counseling session). The computer technology also allowed for the tracking
of changes in novice counselors’ thought processes over a period of 12 to 15
sessions. In addition, RTT is theoretically grounded in the principles described in
the expertise literature and was developed to enhance the clinical training of novice
counselors and psychotherapists. Using the computer in this manner allowed the
researcher to tap into the thought processes of novice and expert counselors. Finally,
this study closely followed Jennings and Skovholdt’s (1999) definition of expertise.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
11
CHAPTER 2: REVIEW OF THE LITERATURE
Although there is a limited amount of research that explores the development
of counseling expertise, related areas of research can help provide important
background information on this growing area of study. This chapter will review the
general literature of expertise, mostly theoretical in nature, which suggests that
expertise can be developed and that there are cognitive differences between novice
and expert performers. Next, related research literature from the medical field which
requires similar cognitive functioning and decision making as the counseling field,
will be reviewed to highlight unique theoretical concepts and research methodology
that could contribute to the study of counseling expertise. Finally, the existing
literature of counseling expertise will be reviewed.
General Findings
Development of Expertise
In recent years, cognitive psychologists have focused their efforts on
understanding the development of expertise in different domains (Ericsson &
Chames, 1994; Ericsson & Lehmann, 1996; Ericsson & Smith, 1991). Ericsson and
Smith (1991) explained that: “the study of expertise seeks to understand and account
for what distinguishes outstanding individuals in a domain from less outstanding
individuals in that domain as well as from other people in general” (p. 2). Ericsson
and Smith (1991) also described expert performance as “consistently superior
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
12
performance on a specified set of representative tasks for a domain” (cited in
Erricson & Lehman, 1996, p.277).
Before delving into the literature to explore what constitutes expert
performance and what are the components of expertise it is important to examine
whether or not superior performance in a specific domain is innate or acquired
(Howe, Davidson, & Sloboda, 1998). Lehmann (1998) asserts that previous expert
levels of performance can routinely be achieved today by many non-expert but
motivated people. Recent evidence supports motivation and deliberate practice,
“highly structured activity, the explicit goal of which is to improve
performance”(Ericsson & Chamess, 1994; p. 386) as essential determinants of expert
performance (Erricson, 1998; Ericsson, Rrampe, & Tesch-Romer, 1993; Schneider,
1998). Ericsson, Krampe, and Tesch-Romer (1993) argue that " ‘ the differences
between expert performers and normal adults reflect a life-long period of deliberate
effort to improve performance in a specific domain” (p. 400). They claim that
deliberate practice requires effort, and it is not inherently enjoyable nor does it
provide any immediate external or monetary rewards, but rather generates costs
related to teachers, coaches, and training settings. However, individuals engage in it
for the sole purpose to improve performance. Later, Ericsson and Chamess (1994)
noted that “deliberate practice differs from other activities in its ability to provide
optimal opportunities for learning and skill acquisition” (p. 739). Ideally, deliberate
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
13
practice will be structured as a limited (time-wise) daily activity during long-periods
of time in an effort to avoid exhaustion.
Ericsson, Krampe, and Tesch-Romer (1993) extended Bloom's (1985b) three
phase development toward adult expertise model (using deliberate practice) to a
four-phase model. The first phase begins with a novice’s introduction to a particular
domain and ends with the initial instruction and deliberate practice. Phase two is
characterized by extended periods of practice and ends with the individual’s
commitment to pursue the domain’s activities full-time. The third phase consists of
the individual’s full-time commitment to engage in deliberate practice for the
purpose of improving performance, and ends by making a living as a professional in
the specific domain or ending the full time commitment to the domain’s activity. In
the fourth and final phase, the expert is able to make an innovative contribution to
the domain and to surpass the knowledge of his or her teachers. The model argues
that with increased experience and deliberate practice the combination of innate
talent and practice can not be differentiated. Ericsson and Lehmann (1996) conclude
that: “with the single exception of height, current evidence for domain-specific
characteristics (talent) among expert performers can be better accounted for by
extended intense practice, which causes physiological, anatomical, and even
neurological adaptations in the body (p.278)”.
Janelle, Hillman, Apparies, Murray, Meili, Fallon and Hatfield (2000)
provided empirical evidence to support the notion that expertise is related to
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
14
physiological change in the body. Based on the study of rifle shooters they found
that expert rifle shooters demonstrated significant differences in cortical activation
and gaze behavior than novice rifle shooters during rifle shooting competition. Thus,
it appears that the influence of innate ability on expertise is small and it is not a
limiting factor in the development of expertise.
Whereas deliberate effort is described as the major contributor and
distinguished characteristic of expertise, deliberate practice integrates within its
structure two other important components of expertise development: self-monitoring
and feedback. Ericsson and Chamess (1994) noted that repetition and successive
refinement through feedback and self-monitoring are typically used as training
activities (deliberate practice) to improve performance. Consequently, Ericsson and
Lehman (1996) believe the central components of helping someone develop
expertise regardless of domain include the constant interaction of: a) direct
experience of deliberate practice, b) self-reflection, (monitoring and reflecting on
one's own performance), and c) receiving ongoing constructive feedback from
others. In fact, the role of deliberate practice, self-reflection and feedback in
enhancing performance has been studied separately in different domains such as
sport psychology, business settings, and medicine, yet they are rarely presented or
studied together. It also has been noted that in order to become an expert in a certain
domain one must dedicate him or herself to gain long-term experience and
engagement in the three central components of expertise. In reviewing several
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
15
studies in many different domains, Ericsson and Smith (1991) concluded that
typically at least 10 years of full-time preparation is required to become an expert.
This “10-year rule” of expertise may be due to the fact that the development
of expertise also requires changes in cognitive processes. Specifically, the
development of expertise is dependent on the understanding and use of declarative
and procedural knowledge. Declarative knowledge is based on facts and is stored in
the form of concepts, principles and propositions focusing on “what”, and “why”
information. An example of declarative knowledge from the clinical domain is the
knowledge that the symptoms of major depression are changes in sleep patterns,
interest, energy and concentration level, appetite, pervasive guilt, psychomotor
retardation, and suicide ideation. Procedural knowledge is functional knowledge of
cognitive actions which are based on classification and change procedures in mental
schemas and which focus on “how” and ‘"when” type of knowledge (Bruning,
Schraw, & Ronning, 1999) and can be represented as “if/then” statements. A
clinician’s unconscious and automated understanding that if a client reports suicide
ideation then the clinician should ask the client whether he or she has thoughts,
intent, plans or means to commit suicide is a related example for procedural
knowledge.
Declarative knowledge is conscious, slowly activated, quickly learned and
easily modified while procedural knowledge is unconscious, quickly activated yet
slowly learned, difficult to modify but durable over time (Clark & Blake, 1997).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
16
Thus, the rapid retrieval of procedural knowledge enables more efficient problem
solving behavior (Anderson, 1981, 1983) and expert performance. Researchers have
asserted that expertise is gained through the automatization of declarative knowledge
into procedural knowledge. The transformation process of declarative knowledge
into procedural provides the performer with an unconscious understanding of how to
accomplish tasks.
For example, Anderson (1983) asserted that the development of expertise in a
specific domain involves three cognitive stages. In the first stage, the cognitive
stage, people are faced with novel problems and use declarative knowledge in the
form of trial and error in an effort to problem-solve. With constant practice and
monitoring, skill corrections begin. In the second stage, the associative stage, the
novice gains perceptual understanding and keeps on detecting and correcting errors
and inefficient problem-solving skills and goal-directed behaviors. Practice,
feedback and self-monitoring help the novice make connections between goals and
outcomes; these efforts move the novice into the autonomous stage where the
knowledge about the specific domain becomes proceduralized, automated, and
unconscious. In this last stage, practice is still required to increase speed and
efficacy of the goal-directed skills. Thus, knowledge in this final stage becomes
routine and helps explain experts’ ability to consistently perform at a superior level
on a specified set of representative tasks. Kirsch and Lynn (1999) believe that
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
17
between 75%-90% of expert performance involves experts drawing on their pre
existing and long-time practiced automated-routine knowledge in their domain.
Identifying Expertise: Difficulties Inherent in Defining an Expert
Ericsson and Chamess (1994) point out that the dynamics and nature of some
domains lend themselves to measuring performance better than others. Some
activities such as sport performance are standardized which enables clearer
identification of expert performance (i.e., time required to complete a task, amount
of goals or baskets scored, etc.). However, other domains measure expertise in less
precise ways. For example, the arts and sciences award prizes to outstanding
performances on the basis of published articles or books, discovery of new
inventions, or special artistic abilities. Social sciences typically use statistical
measures of outliers (extreme scores on a measure) to identify exceptional
performance (typically, at least two standard deviations above the mean). Ericsson
and Chamess (1994) suggested that in some domains there is no evidence of
sustained superior performance across all tasks between novice and experts yet
experts may have more domain-specific knowledge and better problem-solving skills
in some important tasks that may distinguish them from non-expert performers (i.e.,
financial investors and possibly counselors). Thus, it is recommended that in such
domains a focus should be geared toward the distinction between perceived and
actual expert performance.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
18
Novice - Expert: The Differences
After reviewing the perceived cognitive mechanism by which expertise may
be learned and developed, it is important to note what the differences are between
expert and novice performance. Glasser (1992) reported that an expert’s proficiency
is domain specific, where competence in one domain is not necessarily transferable
to other areas. Accordingly, Ertinger, Hillerbrand and Claibom (1995) state that:
“experts and novices regardless of discipline differ in the way they encode
information, organize information in memory, and in the way they use this
information in problem solving” (p. 5). Olson and Biolsi (1991) asserted earlier that
experts hold a large body of specific knowledge (which is not available to novices)
in a specific domain on which they may capitalize to generate their outstanding
performance. Sternberg and Horvath (1995) believe that an expert’s ability to draw
upon this larger knowledge base and to do so more effectively than novices (when
problem solving) is the most fundamental difference between experts and novices.
Glasser (1992) noted that experts appear to store abstract knowledge in large and
meaningful problem-relevant schematic categories and show superior skills in
pattern recognition, whereas novices tend to organize their knowledge into categories
based on linear, superficial and irrelevant cues (inert knowledge) that may not enable
problem solving.
In addition, experts and novices make different use of their short-term
memory. Experts use selective searches of their memory (due to the automatization
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
19
of the knowledge) and thus demonstrate a superior use of memory. As a result,
experts are able to “free” more memory for the purpose of problem solving (Glasser,
1992). Consequently, Ericsson and Chamess (1994) noted that in a number of cases
novices have been able to recall an equal amount of information as experts have (in a
specific domain) but novices demonstrated a much less sophisticated representation
of the information. They concluded that “ the critical aspects of experts’ working
memory is not the amount per se but rather how the information is stored and
indexed in long-term memory” (p. 736). Experts’ sophisticated and efficient storage
of information enables them to be mostly unaffected by interruptions of unrelated
activity while storing new information in short-term memory. Hillerbrand (1989)
reports that with the acquisition of declarative knowledge experts also develop
metacognitive knowledge - knowledge about one’s cognitive processing, thinking
resources and capabilities. Metacognitive knowledge enhances an expert’s ability to
monitor his or her own problem solving skills, progress toward goals, use
information-gathering strategies, and apply different interventions.
Chi, Glaser, and Farr (1988) summarized several distinct differences between
experts and novices which include: a) experts perceive and understand problems in
their domain at a deeper level than do novices, who tend to rely on surface and
sometimes irrelevant details, b) experts tend to possess superior short and long-term
memories, c) experts are faster than novices in executing tasks related to their
domain, d) experts perform the tasks with fewer errors than do novices, e) experts
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
20
are able to perceive large and meaningful patterns of information, and f) experts self
monitor their own behavior more frequently and more effectively than novices.
Research findings regarding expert performance have been studied in many
different domains such as sports psychology, music, literature, mathematics, chess,
and medicine over the past 40 years (Ericsson & Chamess, 1994; Ericsson & Smith,
1991). Examination of the findings from all of these fields is beyond the scope of
this paper. However, medical training seems to offer the closest parallel to the
training of psychotherapists. Physicians, nurses, psychologists and other mental
health providers are all concerned with the training of novices and desire to: a)
integrate classroom theory into actual practice, b) ensure that novices gain expertise
in diagnosis, and develop skills in making immediate decisions regarding treatment.
It is apparent that both fields involve complex mental skills while working with
human physical and mental dysfunctions. Locke and Covell (1997) suggested that
the counseling psychology field could benefit by drawing from research in the
medical field. Therefore, before examining the limited research findings regarding
expertise in the field of psychology, a number of relevant studies conducted in the
medical field will be highlighted.
Expertise and Medicine
A majority of medical studies focus on clinical diagnosis, decision-making,
and problem based learning. Patel & Groen (1991) concluded that medical experts
hold a large body of specialized knowledge which they use to solve problems.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
21
Furthermore, medical experts tend to use forward reasoning (goal directed thinking
such as conceptualization and diagnostic conclusion) in making correct diagnoses
while less experienced medical practitioners tend to generate plausible diagnoses in
efforts to confirm and disconfirm information received from clients (backward
reasoning). These medical findings support prior general findings where experts have
been found to engage in forward reasoning toward a goal (conceptualization and
diagnostic conclusion), whereas novices tend to think backward since they are less
certain about the relationship, the presenting problems and the end goal (diagnosis
and treatment plan; Gick, 1986). Consistent with the general findings of expertise it
seems that medical trainees also use trial and error reasoning for making diagnoses
while experts have already fine-tuned their mental schemas and can immediately use
forward reasoning. In addition, Patel and Groen report that although medical
novices and experts demonstrated no real differences in their ability to recall relevant
and irrelevant cues, experts using forward reasoning made significantly more
accurate diagnoses. Patel and Groen also noted that medical experts use backward
reasoning only when it is directly linked to the solution of a problem. Thus, when
only relevant information is present and there is a need for backward reasoning
experts will automatically refer to it, since it is the most appropriate and direct
method of problem solving.
In a related study, Patel, Arocha and Kaufmann (1994) suggested that
medical experts utilized working memory representation based on facts that
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
22
summarize clinical findings and that allowed flexible reasoning about diagnostic
alternatives. This schematic representation allowed experts to reevaluate incorrect
hypotheses and seek diagnostic alternatives. These findings suggest that, although at
times both novices and experts may demonstrate incorrect diagnoses, experts will
still demonstrate a higher level of performance and accuracy due to their larger and
more flexible knowledge base.
Offredy (1998) examined the differences between novice and experienced
nurses’ decision making and found that expert nurses are better than newly qualified
nurses at: a) collecting clinical data efficiently and understanding its importance, b)
recognizing the probabilistic relationship between data and patient conditions, and c)
developing more detailed and appropriate interventions and using heuristics in their
decision making.
Benner, Tanner, and Chesla (1996) conducted a comprehensive interpretive
study of nursing in critical care practice between 1988 and 1994. Results of their
study suggested that the clinical judgment of experienced nurses depends as well as
on abstract diagnostic labeling on a comprehensive understanding of the client’s
illness and life story; the integration of both sets of information is advanced clinical
knowledge gained through experience with many clients in several similar situations.
Thus, Benner et. al, suggested that expert nurses do not limit themselves to
theoretical diagnostic criteria.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
23
Benner et. al, also proposed a five-stage skill-acquisition model of novice
nurses in their development toward becoming experts. The model asserts that good
decisions are those made intuitively by professionals with expertise, and that such
expertise represents the end-point of the five-stage sequential model. In the first
stage, the Novice stage, novices are perceived as having no experience in situations
in which they are expected to perform, and find themselves behaving on the basis of
context-free rules and trials and errors. In the second stage, the Advanced
Beginners’ stage, the novices start to demonstrate more acceptable types of
performance and have gained enough experience to identify some recurring meaning
in the situations to which they are exposed. In the third stage, the Competent state,
nurses start to perceive their actions as part of a longer-term plan, which enables the
development of efficiency and organization in their work. In the Proficient stage, the
practitioner perceives things in a more comprehensive and holistic way. A nurse in
this fourth stage is able to rapidly alter behavior toward long-term plans when
unexpected patterns of care occur. Finally, in the last stage, the Expert stage, the
practicing nurse does not need to rely anymore on guiding rules, but rather uses
intuition in familiar situations. However, when a new or challenging situation occurs
the expert nurse is able to draw back on deductive reasoning to move toward a
solution of the problem.
The two unique contributions Benner et. al, (1996) added to the existing
literature included: a) the use of intuition as a major distinguishing factor between
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
24
novice and expert nurses, and b) a five-stage model of skills acquisition in the
transformation of novice to expert practitioner. These formulations have not been
studied nor applied to the field of counseling training and supervision.
Lamond and Famell (1998) compared the knowledge information and
accuracy in decision making of seven novice and seven expert nurses regarding the
treatment of pressure sores. Lamond and Famell found that expert nurses were able
to make more accurate decision-making based on their ability to focus on the salient
aspects of a situation. The importance of this study lies in two of its methodological
aspects. First, the study identified expert and novice nurses based on peer review
and length of experience. Ward managers were asked to identify individuals who
were considered to be 'expert' in wound care. As will be later noted, incorporating
both years of experience and peer-selection is an important aspect of the definition of
expert counselors. Second, the study asked an 'expert' panel to determine what
constitutes a 'correct' decision in order to have means for measuring and comparing
the novice and expert nurses' accuracy of diagnosis and decision making. This
research method may also be useful for studying diagnosis and decision-making in
the counseling field.
Offerdy (1998) conducted a descriptive study using observations and
interviews of 20 registered nurses with different experience levels (in terms of years)
to examine nurses’ decision making. This study identified important factors that
relate to experts’ decision making processes, which include: “a) the ability to
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
25
recognize patterns in clinical situations to fit with patterns previously seen, b) an
appreciation of the consequences of inappropriate action and, c) the ability to
concentrate simultaneously on complex and sometimes masked patient cues as well
as multiple treatment options” (p. 988). Consistent with the observations noted by
Benner, Tanner and Chesla (1996), Offerdy noted that expert nurses use more
intuition and pattern recognition while novices use more hypothesis generation for
clinical decision making.
Taylor (1997) conducted an interpretive qualitative study using observations
and interviews based on novice and expert nurses’ ability to attend to cues. The
study defined cues as “sensory stimuli which can trigger the problem solving process
and can be picked up by any of the nurse's senses. The cues which are obvious in the
nurse-patient interaction are often referred to as signs (objective data) and symptoms
(the patient's subjective data)” (p. 336). The study used 15 students from an
undergraduate Bachelor of Nursing degree (novice) and 15 registered nurses who
had completed a tertiary course as entry to practice into nursing (experts) in medical-
surgical hospitals and rehabilitation wards. This study found that novice nurses
demonstrated very few problem-solving skills, which was directly related to their
inability to recognize cues in the clinical setting. On the other hand, expert nurses
identified many of the patient's signs and symptoms. They used a wide range of cues
to assess the patient (i.e. observing the patient's facial expressions and weight
bearing capacity). Identifying and attending to clients cues was directly related to
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
26
expert nurses’ problem-solving skills which enabled them to incorporate all
information, signs, symptoms, test results and other related information into a more
complete picture of the patient risk assessment.
Expert-Novice: Application to Counseling
Johnson and Happner (1989) suggested that in order to develop more
specialized training models we first need to understand how expert and novice
counselors cognitively process information. Thus, the knowledge of how new
information has been processed and integrated with existing knowledge by experts in
counseling settings will provide answers regarding how to better teach the
acquisition of clinical skills to novice psychotherapists. Thus, this section will
examine the existing findings regarding expertise in psychotherapy and highlight the
need for broadening the body of knowledge related to the characteristic differences
between novice and expert counselors.
Development of Expertise in Counseling:
Definition. Descriptions, and Assumptions About Expertise
Martin, Slemon, Hiebert, Hallberg, and Cummings (1989) acknowledged that
most of the domains in which expert performance were studied such as physics,
mathematics, and chess are well structured whereas counseling is much more
flexible, complex, and of a multifaceted nature. Furthermore, as noted earlier,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
27
Ericsson and Chamess (1994) indicated that in some domains (counseling is one of
those domains) it is more difficult to define and measure expertise. Accordingly,
Lichtenberg (1997) described clinical practice as multifaceted involving different
skills such as interviewing, assessment, diagnosis, the application of a variety of
intervention strategies, etc. He added that clinical problems are “fuzzy” and may be
effectively solved by different orientations and approaches. Thus, Lichtenberg
suggested that clinical judgment should be viewed as an ill structured problem where
expert counselors should be better than novice counselors in transforming ill
structured clinical problems into a set of better-structured and solvable sub-problems.
In addition, expert counselors should be better able to identify and use procedures,
frameworks, and interventions and assessment strategies when dealing with clinical
problems.
Based on the complexity of counseling and psychotherapy, Skovholdt,
Ronnestad, and Jennings (1997) suggested that the more commonly used concept of
expertise be modified; this means: a) including the construct of the client-counselor
working alliance, b) understanding that it takes approximately 15 years of daily
practice to become an expert counselor, c) maintaining a greater difference in years
of experience between expert and novice counselors when researching expert-novice
differences, d) including self-reflection as an important expertise developmental
factor, and e) stressing the importance that professors who teach
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
28
psychotherapy/counseling skills also practice counseling in order to become expert,
themselves.
Goodyear (1997) described several possible criteria such as experience, level
of training, demonstrated effectiveness, aesthetics, perceptions of others, legal status,
and cognitive processes that could be used to define expertise related to counseling
psychology. However, these criteria also pose some problems. For example, some
of these criteria are inter-related to and/or associated with experience and training
level (e.g., the legal system defines psychologists’ expertise by training levels and
experience; aesthetics seems to be gained with years of experience). Additionally,
demonstrated effectiveness is a complex criterion itself since: a) psychotherapy
outcome is difficult to measure, and b) outcome effectiveness in psychology tends to
be independent of orientation, degree or experience. Further, although years of
experience are the most widely used criteria to distinguish between novice and
expert counselors, years of experience alone have not been found to be a clear
differentiating factor of levels of expertise in the counseling field (Skovholdt,
Ronnestand, & Jennings, 1997). Thus, it may be more appropriate as illustrated by
O’Byme and Goodyear (1997) and Jennings and Skovholdt (1999) to use both years
of experience (ideally, 15 years and a wide range of difference between novice and
expert counselors’ experience) and peers’ perception of who is an expert counselor
when examining the development of expertise in counselors.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
29
Researchers have generalized findings to the field of counseling psychology
by drawing from medical and other fields. For example, Johnson and Heppner
(1999) noted that because novice counselors may be cognitively overwhelmed by
theory and information they have just learned, when asked to describe an intake with
a client they relate many facts but are unable to describe the central elements of the
client’s dynamic and problems. At the same time, as the result of experience and
practice, expert counselors typically are able to identify the relevant facts and
integrate the information into a diagnosis and treatment plan.
Glasser (1992) concluded that experts’ knowledge is highly procedural and
goal oriented. Earlier, Hillerband (1989) noted that becoming an expert counselor
requires the transfer of declarative knowledge to procedural knowledge (which
makes experts’ proficiency more efficient and accurate) by practice, experience and
the use of feedback. Further, Hillerband also asserted that since experts hold
metacognitive knowledge, expert counselors may be able to better “monitor their
progress toward goals, information they attend to, their comprehension of clients,
case conceptualizations, and information-gathering strategies, and their application
of alternative reasoning and problem solving techniques with ambiguous or novel
problems” (p. 294).
Like Lichtenberg (1997), Etringer, Hillerbrand, and Claibom (1995) earlier
noted that the type of problems raised in counseling are ill-structured problems, and
they recommended that researchers examine whether expert counselors exhibit
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
30
similar skills to experts in other fields. Specifically, experts tend to engage in
forward reasoning toward a goal (conceptualization and diagnostic conclusion),
while novices tend to utilize backward reasoning since they are less certain about the
relationship, the presenting problems and the end goal (diagnostic and treatment
plan).
Empirical Findings. In an attempt to empirically explore some of these assumptions,
a number of researchers have used different methodological designs to examine the
differences between novice and expert counselors. Research in this area is still quite
limited; the study of expertise in counseling and psychotherapy training is
developing and thus is only in its exploratory phase. The absence of a clear
definition of counseling-related expertise (due to the complex, flexible, and
multifaceted nature of counseling) has made conducting empirical research a
challenging task. In addition, the difficulty in conducting long-term treatment
outcome studies to examine the differences between novice and experienced
counselors has further limited the type of studies conducted and also limited
researchers' abilities to generalize findings to everyday clinical settings.
Martin, Slemon, Hiebert, Hallberg, and Cummings (1989) conducted one of
the first quantitative studies (using Conceptual Mapping Task; CMT) examining the
novice and expert conceptualization process. This study provided only limited
support for the differences between novice and expert counselors. The study found
that novice counselors formulated more concepts than the experienced counselors.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
31
Martin et al. interpreted the results as evidence that “experienced counselors hold
extensive abstract, general knowledge of counseling which enables them to
conceptualize specific counseling situations efficiently and parsimoniously, whereas
novice counselors engage in more extensive and unique conceptual work with each
client” (p. 395). However, the limitations of this study may explain why the study
revealed only limited significant differences. The expert counselors were defined by
at least four years of experience. Four years of expertise is not consistent with the
10-15 years of experience that Jennings and Skovholt (1999) suggested is necessary
for counseling expertise. Further, years of experience alone do not provide an
adequate representation of counseling expertise. Additionally, the counselors in the
study related information about two different clients; using information about two
clients makes it difficult to ascertain whether competing sets of information had any
influence on the different number of concepts generated by the counselors. Also,
Martin et al, (1989) suggested that the results of the study “are limited to the measure
and the number of concepts considered by the participating counselors” (p. 398) and
called upon researchers to use other research approaches to examine the
conceptualization differences between expert and novice counselors.
Cummings, Hallberg, Martin, Slemon, and Heibert (1990) conducted a
follow-up study by qualitatively examining the content of the Concept Mapping Task
of two novice and two experienced counselors from the Martin et al (1989) study.
The two expert counselors had more than four years of counseling experience
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
32
whereas the two novice counselors were in the second year of a master’s program in
counseling psychology. All four counselors worked in the same university-
counseling center. As in the Martin et al. study, expertise was defined by over four
years of counseling experience which, again, is not consistent with Jennings and
Skovholt (1999) 10-15 years definition of development of expertise in counseling.
Additionally, the difference in years of experience between the novice and expert
counselors seems to be narrow. Other limitations of the study are the small sample
size which limits the ability to generalize the findings of the study and the fact the all
of the participants worked in the same university-counseling center. Sharing the
same training model and orientation (all four participants used a cognitive behavioral
approach to counseling) may create homogeneity in the participants’ counseling
conceptualization. Nevertheless, the Cummings et al study found that experience
and training provided experienced counselors with “well developed schemata for
explaining how change occurs in short-term counseling and for conceptualizing
individual client problems” (p. 127). Further, experienced counselors schemas
tended to be dominated with domain-specific, thematic concepts. On the other hand,
novice counselors tended to attend to unrelated and/or irrelevant information. These
findings are consistent with expertise findings from other fields where experience
and training contribute to experts’ deep-level schemata that can be used
automatically to accurately conceptualize situations (in counseling and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
psychotherapy to conceptualize clients problems) by differentiating relevant from
irrelevant information.
Kivlighan and Quigley (1991) tried to address Hillerbrand’s (1989) call for
expertise research related to group counseling settings. In this study, Kivlighan and
Quigley used multidimensional scaling to examine novice and experienced
counselors’ conceptualization of group process. Novice and experienced counselors
were asked to view videotape of a group counseling session and rate the similarity of
each pair of group members. The expert counselors were defined by hours of group-
therapist leaders’ experience (at least 1000 hours) and not by years of experience or
peer-nomination. The experienced group counselors appeared to demonstrate more
psychological dimensions and a deeper and more complex representation of the
group dynamics. In addition, experienced and novice counselors described the
dimensions of the group process somewhat differently. Experienced counselors
conceptualized group members on three dimensions: dominant/submissive,
friendly/unfriendly, and supporting therapeutic work/hindering therapeutic work,
while novice counselors conceptualized group members on two dimensions:
dominant/submissive and high/low rate of participation.
In an attempt to replicate the previous studies with different statistical
methods (cluster analysis) Mayfield, Kardash, and Kivlighan, (1999) examined
novice and expert counselors’ differences in knowledge structure about clients. The
median experience level of the experienced counselors was 12 years. The counselors
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
34
were also recruited by one of the researchers on the basis of their prominence in the
therapeutic community. Although the selection of subjects was an advance from the
previous studies it seems insufficient to identify expert counselors using one peer
researcher. Four experienced and five novice counselors read transcripts from a
counseling session, categorized clients’ statements and mapped the relationship
between the different categories. Mayfield, Kardash, and Kivlighan found that
novice counselors had more concept categories than expert counselors. Thus, expert
counselors used their abstract and well developed knowledge base and mental
schemas to problem solve; this appears to have helped them structure clients’
information more efficiently and parsimoniously than novice counselors.
Furthermore, novice counselors formed their categories on the basis of temporal time
order (when statements where made during a session). Finally, experienced
counselors completed some of the tasks significantly faster than the novice
counselors. Mayfield, Kardash, and Kivlighan concluded that experts demonstrate a
superior use of memory and as a result are able to “free” more memory for the
purpose of problem solving while novice counselors seem to attend to surface detail
and require more time to process information, possibly due to their less developed
domain specific knowledge base. Their findings are consistent with research results
in other domains. This study, however, had several limitations; like the Cummings
et al. study, the novice and expert counselors were associated with the same program
which may have contributed to homogeneity bias, the study used analogue data
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
35
which may not accurately represent real counseling settings, and, finally, the study
did not control for treatment outcome.
O’Byme and Goodyear (1997) examined the amount and type of information
expert and novice psychologists sought while clinically assessing a client. Although
this study employed a combination of peer-nominated expert counselors and years of
experience, their range of clinical experience was 6.5 to 26 years with a mean of 8.4
years of experience while the novice counselor’s range of experience was 0-9 years.
The subjects were given a short description of a “made-up” client and were asked to
list what else they would like to know about that client. The study revealed that
expert counselors requested more information than did the novices and focused less
on crisis aspects of the client. It appeared that the experts were gaining more
information to relate to their existing knowledge base; this approach would likely
enable them to gain a more complete picture of the client before diagnosing or
creating a treatment plan. By focusing more on the client’s underlying
characteristics rather than client crisis, experts were better able to distinguish
between relevant and irrelevant information. However, expert and novice counselors
did not differ in their level of confidence regarding their assessment.
Brammer (1997) examined the relationship between psychologists’
experience and training levels and types of clinical inquiry. The study used an
artificial intelligence program simulating a date rape client. Psychologists and
trainees ranging in their experience and training levels were asked to interact with a
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
36
computerized client by asking questions and responding to the client’s answers.
Brammer (1997) found that experience was significantly related to the type of
questions asked. More experienced counselors paid more attention to the client and
specifically sought diagnostic information. In addition, trying to compare the
information provided by the client with their pre-existing knowledge base and
theory, experienced counselors asked more forward goal directed questions and
fewer client history questions. Clinical experience and training levels were unrelated
to counselors’ confidence and perceived expertise. One major limitation of this
study was that approximately 50% of the participants were academicians who do not
practice psychotherapy in everyday clinical settings. The study is also limited based
on the relatively small number of participants in each experience group, thus creating
a narrow range of years of experience. In addition, the analogue nature of the
experiment (using a computerized client) does not represent a real clinical setting.
Ganzach (1997) examined the clinical judgments of 29 clinicians, 13 staff
psychologists, and 16 trainees. This study did not specify how the experts were
defined other than their being staff psychologists. Each clinician judged 861 MMPI
profiles on a scale ranging from least psychotic (1) to most psychotic (11). The
study revealed that in judging the degree of pathology of mental patients, experts
assign heavier weight to the more pathological information than novice
psychologists. This seems to be congruent with the other findings that expert
counselors focus on more important and relevant cues than novice counselors.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
37
Finally, Jennings and Skovholdt (1999) conducted a qualitative study to
examine the cognitive, emotional, and relational characteristics of master therapists.
This study did not compare expert and novice counselors yet it provides some insight
into the cognitive characteristics of master therapists. This study seems to have
addressed some of the limitations of previous studies, especially the Cummings et al
(1990) qualitative study, by examining a larger number of subjects (10) and by
defining expertise by years of experience (all or full time practitioners ranging 21-41
years of practice), qualification (all held at least one license) and peer-selection.
Jennings and Skovholdt seem to have integrated the most exclusive definition of
expert/master counselor. The study found that master therapists: a) are vicarious
learners, strongly valuing and seeking continuing education, b) rely on their broad
and rich accumulated life and work experience in their therapeutic work (they
perceive their experience as a major contribution to the depth and knowledge about
human behavior), and c) value cognitive complexity and the ambiguity of human
nature and, as a result, use multiple and complex criteria when diagnosing clients and
when judging therapy outcomes.
Training Model based on Expertise Principles.
Different researchers and professionals have attempted to draw from the expertise
literature and apply these findings to clinical and counseling training and
supervision. Ertinger et al. (1995) noted that researchers and practitioners who are
looking to enhance clinical training and develop expertise in novice counselors
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
38
should focus on the following questions: a) How do training and supervision
activities help novice counselors develop therapeutic schemas? b) How do
counselors-in-training develop knowledge structures? c) How do advanced or expert
clinicians structure their knowledge? and d) How can knowledge schemas be used in
clinical work and diagnostic or conceptualization activities?
Hillerbrand (1989) noted that traditionally the focus of group supervision is
to enhance trainees' internal affective dimensions and communication skills. He
suggested that in order to improve novices' learning, supervisors should also focus
on enhancing trainees specific cognitive skills: a) novices should be informed about
the specific skills acquired in supervision, b) novices should be encouraged to talk
aloud the reasoning process in order to make the cognitive process overt, c) novices
should be encouraged to identify when and why particular skills were effectively
used, d) novices should be encouraged to leam when and how to use analogies in
their cognitive process, and e) novices should be encouraged to articulate the
metacognitive skills they used to double-check and monitor their attention, memory,
comprehension, conceptualization, problem solving, hypothesis-testing, and progress
toward goals (p. 66-67). Hillerbrand focused primarily on the development of
novices’ self-reflection and metacognitive skills. As previously mentioned, these
skills have been documented to be important aspects of expert’s performance and
thus appear to be critical for novices’ development of expertise.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
39
O’Byme, Clark, and Malakuti (1997) asserted that “the literature of expertise
tends to primarily focus on what the advanced expert can do rather than how they do
it or how they leam to do what they do” (p. 321). They recommended applying the
following conditions into clinical training in order to enable the development of
expertise in counselors: a) teach the declarative knowledge that will help novices
handle novel problems as they develop, b) teach problem-solving procedures, c)
support the development of both knowledge and motivation, d) arrange internship
experiences for the systematic practice of productions, e) define activities that
develop advanced expertise, f) select an adequate context for training, and g)
sequence training in the order in which the learner will apply the knowledge.
Binder (1993) believed that in order maximize psychotherapists’ training an
“anchored instruction” training model should be utilized. An anchored instructional
model is a structured teaching format that promotes depth of comprehension and
accessibility of acquired knowledge and is designed around the following three
conditions: a) the knowledge to be learned must be explicitly linked or made relevant
to specific problem situations (Chi, Glaser & Farr, 1988), b) the knowledge must be
acquired through active involvement of the learner in a manner that encourages both
memorization and comprehension of the material (Bransford, Sherwood, Kinzer, &
Hasselbrand 1982), and, c) the knowledge must be acquired in a context as similar as
possible to the conditions of actual practice in order to leam to recognize features of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
40
the problems as they appear in reality and to practice selecting relevant from
irrelevant material (cited in Rock & Bransford, 1990, p. 313).
Additionally, the use of computer environments may serve to enhance and
develop learning (Jong & Joolingen, 1998). Binder (1999) recommended the use of
interactive computer technology in clinical training to further enhance trainees’
learning, and believes that interactive, multimedia computer technology can "fill the
gap between the classroom teaching of declarative knowledge about what therapy is
and the generally unsystematic teaching through supervision of therapeutic strategies
and tactics associated with procedural knowledge” (p. 715). Binder noted that
interactive computer programs that enable trainees to recognize communication
patterns and receive immediate narrative feedback about their performance could
enhance novice counselors’ clinical skills “by structuring sequences of therapeutic
problem-detection and problem-solving exercise under conditions that stimulate
actual clinical experiences” (pp. 715-716). Despite the fact that some domains of
psychology have utilized computer technology and cognitive psychology principles
to enhance learning (e.g. remote learning, interactive and simulation computer
programs), of clinical training lags far behind other disciplines in applying these
principles to improve the training of psychotherapists.
Although researchers and professionals called for the integration of cognitive
principles to develop expertise in novice counselors it appears that there is a lack of
training in the supervision models that are grounded in cognitive and expertise
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
41
theory. Rosenberg (1999) identified Stolenberg, McNeil and Delworth (1997) and
Rigazio-Diglio, Daniels, and Ivey (1997) as the only researchers who have attempted
to incorporate cognitive principles into comprehensive supervision models.
Recently, Rosenberg (1999) developed a model for clinical training, which utilized
several principles described in the expertise (cognitive psychology) literature along
with interactive computer technology. The result was a methodology for clinical
training (Real-Time Training; RTT; 1998), which involves the combined use of a
computer, and audio-visual technologies with a live supervision approach. RTT is
an applied training model for novice counselors based on self-reflection, deliberate
practice, and constructive feedback. It also meets all the criteria for an anchored
instructional model, integrates metacognitive and other cognitive skills, and attempts
to address the drawback of previously used supervision models.
RTT involves the traditional videotaping of a trainee (novice counselor)
while he or she is counseling a client behind a one-way mirror. Trainees (novices)
and a supervisor (expert) observe the counseling session through an observation
window or via a T.V. screen. However, unique to RTT, the computer, videocamera,
TV and VCR are linked together in such a way that the supervisor (expert) and peer-
trainees (novices) can type real-time feedback (using a computer keyboard) as the
session progresses. The feedback comments are superimposed in real-time with the
videotape of the session. The result is a videotape of a counseling session that
includes text-based feedback of the expert and novices commentary (or thought
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
42
process). Thus, when the trainee who conducted the session reviews the tape, the
session and the expert’s supervisory feedback and peer’s comments are viewed
simultaneously, as in the case of a foreign film with subtitles with the text scrolling
as the movie continues. Thus, the trainee receives the same feedback that was
provided in real-time to his or her peers observing the session; there is no
interruption to the flow of the session or to the trainee’s own thought process.
The use of RTT provides researchers a new research tool and methodology to
examine the differences between expert and novices. It also addresses some of the
limitations of previous research methods used in the counseling expertise field. The
computer maintains an ongoing record of the typed feedback comments and can also
label the source (expert versus novice) of the comments. For the first time,
researchers can examine expert and novice counselor’s thought processes, schemas
and conceptualizations of an actual counseling session dealing with ill structured or
fuzzy problems in an actual clinical setting - as opposed to the more typical
retrospective look at clinician’s thought processes (e.g. Interpersonal Process Recall;
IPR; Kagan, 1980). Examining the immediate mental schemas and
conceptualizations of novice and expert counselors while a therapy session takes
place may better resemble the type of decision making novice and expert counselors
are called on to make in clinical settings. In addition, RTT can track feedback
comments of many counseling sessions. This represents an important advancement
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
43
over most of the studies previously mentioned, which gathered data only from a
limited number (one or two) counseling sessions.
Summary of the Literature
Some common conclusions can be drawn from the studies reviewed here. In
general, the development of expertise is a long-term process (at least 10 years),
which is typically achieved through deliberate practice, self-reflection and feedback.
Expertise has been studied in many different domains. Some common differences
between novice and expert performance have be found across domains. The medical
literature highlighted here may contribute to the study of counseling expertise by
focusing on different aspects of diagnosis and the utilization of different research
methods for examining expertise. It seems that most the studies in the medical field
define expertise on the basis of both years of experience and either peer-selection or
some type of board certification (registered nurses, specialty in a medical area, etc).
The field involving applied psychology has been lagging behind other fields
in studying counseling expertise. However, initial exploratory studies suggest that
some of the general findings of expertise are also applicable to counseling and
psychotherapy. First, it appears that expert counselors’ cognitive processes are
geared toward reaching conclusions toward diagnosis in a timely manner. Second,
expert counselors appear to problem solve by developing complex pattern
recognition and selecting important from irrelevant information. Third, expert and
novice counselors appear to have different mental representations of clients and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
44
consequently to ask different types of questions. Fourth, it appears that many
researchers suggest that psychotherapy and counseling training could be enhanced by
the integration of principles of expertise and interactive computer programs for
supervision and clinical training.
However, some common limitations can also be highlighted following a close
examination of the empirical studies that have been conducted in counseling and
psychotherapy regarding the development of expertise. All the studies gathered data
only from a limited number (one or two) of counseling sessions. It is also important
to note that the experimental studies used retrospective methods (e.g. transcripts,
videotapes of sessions, and questions or free association after the conclusion of
counseling session(s)). Thus, the novice and expert counselors’ conceptualizations
were determined after the fact. It also appears that most of the studies did not
balance for ethnicity of the expert and novice counselors or for their different
orientation or program affiliation. In addition, the studies did not consistently in
adhere to the Skovholdt, Ronnestand, and Jennings (1997) and Goodyear (1997)
recommendations of a) avoiding a narrow range of years of experiential difference
between novice and expert counselors b) using a 15 (or more) year of experience
criteria for experts counselors and c) combining years of experience with peers’
identification of expert counselors as better definition of expert counselors.
There is a real need to conduct more studies examining the development of
expertise in counseling. Future studies should address some of the limitations of the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
45
studies noted above and focus on experimental methods using counseling data
gathered during therapy sessions in counseling setting. Such studies will enable the
examination of novice and expert counselors’ immediate mental schemas and
conceptualizations. Studies of such a nature may better resemble the type of
decision-making novice and expert counselors are called on to make in clinical
settings.
Etringer et al. (1995) addressed the methodological issue noted above and
stated that it is difficult to extrapolate research from cognitive psychology to
psychotherapy or clinical training because of the methodological differences between
the areas. Cognitive psychology is typically conducted in controlled laboratory areas
whereas the training of counselors is conducted in uncontrolled clinical settings.
Fuqua, Johnson, Anderson, and Newman (1984) called for researchers “to
experiment with different assessment procedures in the counseling training setting
such as thought listing and think aloud approaches” (p. 93). The advancement of
technology allows for new research methods (e.g. Real-Time Training; RTT;
Rosenberg, 1999) to be utilized. Such methods enable researchers to tap into novice
and expert counselor’s thought processes (and record these thought processes) even
in the “ill structured” and “fuzzy” clinical setting. As a result, it is now possible to
examine cognitive differences between expert and novice counselors in real time.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
46
Purpose of the Study
The purpose of this study is to utilize computer technology to access novice
and expert counselors thought process. The identification of novice and expert
counselors’ thought processes in real time will help shed light into the differences
between novice and expert counselors’ thought processes. The differences between
expert and novice counselors will be examined in two ways. First, using RTT, the
study will look at the different types of real-time/immediate feedback comments
(thought processes) of novice counselors (trainees) and an expert counselor
(supervisor). These feedback comments were provided to novice counselors
conducting therapy sessions in an actual clinical setting. Second, novice and expert
counselor raters will categorize the different feedback provided by the trainees and
supervisor. This will allow a further examination of the cognitive differences
(schemas) between novice and expert counselors. The study will also look at the
changes in the feedback (thought process and case conceptualization) of novice
counselors with time (number of session during a semester). Finally, the study will
examine the differences in task speed between novice and expert counselors.
Research Hypotheses
1. Expert counselor raters will produce fewer categories/clusters than the novice
counselor raters.
2. Expert counselor raters will spend less time than novice counselor raters sorting
the feedback comments into categories.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
47
3. Expert counselors will differ in the feedback comments they provide while
observing live counseling sessions than the feedback comments provided by
novice counselors.
4. As time progresses in the semester novices’ feedback will become more similar
to that of the expert counselor.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
48
CHAPTER 3: METHODOLOGY
Design
Counselor trainees (novice counselors) enrolled in masters and doctoral level
practicum saw clients in a community mental health center across an academic
semester. Trainees behind a one-way mirror were observed and videotaped (using
RTT) by a Ph.D. level practicum instructor (over 20 years of clinical experience) and
counselor trainees peers. The trainees and supervisor used computer keyboards to
type feedback about the observed counseling session. Each comment was preceded
by the label of the feedback provider (student or supervisor) and was superimposed
on a videotape of the counseling session. Data for this dissertation included 75
randomly selected comments out of the hundreds comments made by novice peers
and 75 randomly selected comments out of the hundreds comments made by the
supervisor.
Participants/Raters
Ten novice raters and 10 expert raters served as participants in this study.
Novice Counselors. For the purpose of this study novices were defined as counselor
trainees who had completed an introductory counseling skills (or practicum) course
and who had also completed or was currently enrolled in a practicum course at the
time of the study. Overall, novices saw a mean of 7.8 clients (range 1 -14) for a mean
of 74.2 sessions (range 3-140) sessions. Although all novices came from academic
settings, they varied in the type of training received, the university, theoretical
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
49
orientation and the amount of training received. The novice counselors included two
men and eight women. Their mean age was 28.2 years (range = 23-41 years). The
sample was consisted of 9 Caucasians and one Latina. The sample included 8
novices seeking a master degree in Marriage and Family Therapy (MFT) and 2
novices seeking a doctoral degree in clinical psychology from 6 different psychology
programs in the southern California area. All but three novices had some
paraprofessional counseling experience ranging from 6 months to three years. The
novices varied in the amount of training they had received with an average of 1.5
semesters of practica (range 1-3). The novices’ theoretical orientations spanned
across the spectrum of psychotherapy orientations (e.g. humanistic, psychodynamic,
cognitive, spiritual, and eclectic). Efforts were made to include more men and a
broader ethnic mix of subjects yet conflict of time was a real obstacle. See Table 1
for novices’ demographics.
Table 1
Novices’ Demographics
Code Age Gender Ethnicity Degree Program Orientation
N1 30 Female White M.A. MFT Humanistic
N2 28 Female Latina M.A. MFT Cognitive
Behavioral
N3 31 Female White M.A. MFT Eclectic
N4 27 Female White M.A. MFT Psychodynamic
N5 25 Female White PSYD Clinical Cognitive
Behavioral
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 1 (continued)
N6 23 Female White M.A. MFT Integrative
N7 41 Male White PSYD Clinical Psychodynamic
N8 31 Male White M.A. MFT Spiritual
N9 23 Female White M.A. MFT Cognitive
Behavioral
N10 23 Female White M.A. MFT Eclectic
Table 1 (continued)
Code Number
of
Clients
Seen
Number
of
Clients’
Hours
Paraprofessional
Experience
N1 1 1-2 Years
N2 9 69 6-12 Months
N3 10 98 0
N4 8 104 0
N5 12 100 2-3 Years
N6 14 140 0-6 Months
N7 8 120 0
N8 6 75 2-3 Years
N9 5 19 2-3 Years
N10 5 14 6-12 Months
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
51
Expert Counselors. All the expert clinicians were licensed psychotherapists
providing psychotherapy services for at least 15 consecutive years with a minimum
number of client hours at 10 hours a week during that 15 year period. Further, the
clinicians were identified as experts in their field by holding at least one license and
achieving expertise by contributing to the field as presidents or board members of
psychological organizations, authors of books and publications, clinical directors, or
identified as expert in a specialty area such as ethics and cross-cultural areas. This
definition of counseling expertise is consistent with two out of the three criteria
suggested by Jennings and Skovholdt (1999); years of experience (over 15 years and
a range difference of at least 10 years between novice and expert counselors) and
qualification (holding at least one license). The third criterion, peer selection, is
based on the assumption that counselors can identify peer counselors as experts in
the field. However, there is no evidence to support the above claim. Further, it is
arguable that a client’s perception treatment and effectiveness of treatment outcome
can serve as better criteria for identification of expert counselors. However, clients
who attend psychotherapy hold confidentiality and, as noted earlier, psychotherapy
outcome is difficult to measure. Consequently, peer selection was not used as a
criterion of expertise for this study.
The expert sample included five males and five females with a mean age of
57.4 (45-70). Overall, experts held a license for a mean of 22.9 years (range 15-34)
and saw a mean of 21.4 clients per week (range 10-35). The sample included two
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
52
licensed social workers, four licensed counseling psychologists, and four licensed
clinical psychologists who were either affiliated with private practice, university
counseling centers, or community mental health centers. The experts also varied in
their type of degree, license, affiliation and theoretical orientation. The experts also
described various theoretical orientations (e.g., psychodynamic, existential,
cognitive, and eclectic).
Differences in the ratio of men and women between the novice and the expert
groups appears representative of the gender shift in the mental health field in the last
20 years where more women have been seeking psychotherapy careers. The mean
range of difference of years of practice experience between novice and expert
counselors was approximately 20 years, which is consistent with Jennings and
Skovholdt (1999) suggestion of adhering to a range of at least a 10 year practice
difference between novice and expert counselors. The researcher also attempted to
include African American and Latino expert subjects. However time conflict
prevented the approached subjects to participate in the study. See Table 2 for
experts' demographics.
Procedure
Data was collected from six semesters of counseling practica. Each
counseling trainee saw at least one client for approximately 12 weeks. Each trainee
was required to go behind a one-way mirror and be provided feedback via RTT at
least two times during the semester. Base on client load, an attempt was made to
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
53
observe two to three trainees per week and provide them with text-based feedback
from peer observers (novices) and the practicum instructor (expert) using the RTT
methodology.
The RTT methodology involves providing text-based feedback to trainees
observing a counseling session as the session is occurring. Both the expert
(practicum instructor) and the novices (trainees) observing the session were able to
provide this text-based feedback while a counseling session is being conducted.
These feedback statements served as data for this investigation. Each new entry
made by a novice or the expert was considered an individual statement and could be
Table 2
Experts’ Demographics
Code Age Gender Ethnicity Degree Degree
Year
El 58 Female White MSW 1973
E2 61 Male White ED.D. 1971
E3 56 Male White Ph.D. 1972
E4 59 Male White Ph.D. 1984
E5 51 Male Asian
American
Ph.D. 1980
E6 70 Male White Ph.D. 1958
E7 63 Female White Ph.D. 1976
E8 45 Female Chinese
American
Ph.D. 1985
E9 54 Female Asian
American
MSW 1976
E10 57 Female White Ph.D. 1972
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
54
Table 2 (Continued)
Code Orientation Number
of
Licensed
Years
Type of
License
Number
of
Clients
per
Week
El Psychodynamic 25 LCSW 12
E2 Psychodynamic 30 Psychologist 10
E3 Existential 25 Psychologist 25
E4 Existential 16 Psychologist 25
E5 Cognitive
Behavioral
18 Psychologist 25
E6 Existential &
Psychodynamic
34 Psychologist 12
E7 Cognitive
Behavioral
23 Psychologist 25
E8 Psychodynamic
& Cognitive
15 Psychologist 20
E9 Psychodynamic 22 LCSW 25
E10 Eclectic 21 Psychologist 35
as simple as “good intervention” or “nice question” to a statement which included
five or six sentences to explain a concept such as, “ by her body posture, lack of eye
contact, and the fact that her affect is rather flat, she seems so disinterested. I
wonder why she seems so reluctant to really have a conversation, to engage with
you? She is not too present, or at least not willingly”. Each time a novice counselor
provided feedback, his or her statement would be preceded by the word
“STUDENT’; likewise, each time the expert provided feedback, her statement was
preceded by the word “SUPERVISOR”. Thus, an initial level of coding
differentiating novice from expert therapists was completed at the time of recording
using RTT.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
55
This process generated 1898 comments by the trainees and supervisor alike.
Data that was generated spanned across several practica classes from different
practica classes, clients, clinical situations, and times in the semester (representing
potentially different developmental stages in trainees training). The statements were
later coded by: a) who made the comment (expert vs. novice), b) the time of the
session during that week (first, second or third session during the day), c) year the
counseling session took place (1998 or 1999), d) semester of the academic year the
session took place (Fall or Spring), e) the week during the 15 week semester the
session took place (1-12), and f) the time during the session the statement was made
(1 for the first statement, 10 for the 10th statement etc...). For example, the code
EA9911-1 represents the first statement made by an expert (E), during the first out of
3 sessions during that week (A), in the 1999 year (99), spring semester (1), the
statement typed in the session (1).
After the coding of the data, 75 feedback statements made by the expert and
75 feedback statements made by novices were randomly selected using a random
numbers table. The selected statements were then pasted on an index card (the
feedback statement was pasted on the front side of the index card and the code on the
back side of the index card). Each feedback statement was pasted on a separate card.
The statements were randomly ordered (again using a random numbers table before
each presentation to a new participant, in order to circumvent any effect bias in
rating due to the prior ordering or pairing of statements.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
56
Participants were asked to complete a brief demographic questionnaire which
included questions such as, “Please describe your licensure (e.g., licensed
psychologist, licensed clinical social worker, licensed M.F.T., etc.)?, or in what year
were you first licensed? (See Appendix B).
The novice and expert clinician raters were trained individually on how to
sort the feedback cards, prior to starting the task. The instructions the raters were
given were as follows: “Please sort the cards into categories based upon any themes
or similarities you perceive among the statements. There is no right or wrong
answer.” No limitations were placed on the number of categories, nor amount of
time to start or complete the task. After identifying categories, the raters were also
asked to label each category based upon the common theme they used to place the
statement into that particular category. Finally, each rater was asked to define each
category: “ Please define each category by listing the qualities or attributes that made
you sort the cards into the specific category.”
Data Analysis
This study was exploratory in nature and sought to identify the differences
between novice and expert counselors’ thought processes using feedback comments
made using RTT methodology. Cluster analysis was used to initially group the data
gathered in this study. Borgen and Barnett (1987) stated that:
“cluster analysis is a classification technique for forming homogeneous group
within complex data set. Typically, we do not know a priori the natural
groupings or subtypes, and we wish to identify groups within a data set. We
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
57
wish to form classifications, taxonomies, or typologies that represent different
patterns in the data. Clustering algorithms are used to search a data set to
partition the objects (or people) into relatively distinct groups. Cluster analysis is
most often used as an exploratory technique. If the research area is relatively
new, clustering may be a productive early step to identify and structure the
subgroups that are of potential value in understanding the research problem” (p.
460).
Previously, Mayfield et al. (1999) had used structure analysis to analyze
Concept Mapping Tasks (CMT) in order to examine differences in how experienced
and novice therapists structured their knowledge of the client. Borgen and Barnett
(1987) noted that the most widely used clustering method is the hierarchical method.
Hierarchical cluster analysis is a step-by-step statistical procedure that builds clusters
by locating the most similar objects/statements/concepts before moving to the next
cluster. The end result is a tree dendogram. This study used a hierarchical cluster
analysis to analyze the data.
Next, a Chi square analysis was conducted to examine whether the level of
expertise, the semester/ practicum group, or the session number within each week,
were related to the type of feedback comments made.
Next, a Discriminant Factor Analysis (DFA) with the type of feedback
(clusters) as the dependent variable and the time in the semester (week number) and
expertise level (expert or novice) as the independent variables were conducted in
order to examine the relationship between the feedback type, level of expertise of
feedback provider, and the session number the feedback was made. When the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
58
dependent variable is categorical, Betz (1987) suggested using DFA to analyze such
data. In this study the categories/clusters the expert and novice counselors sorted
together (type of feedback) were categorical variables; these were used as the
dependent variable.
Finally, to test the differences between novice and expert counselors’ time to
complete the sorting task a Mann- Whitney U test was used to compare the amount
of time raters spent on sorting the index cards into categories. In a similar task,
Mayfield et al. (1999) used the Maim-Whitney U test, a nonparametric statistic
because the “very small sample sizes involved (4 novice and 5 expert counselors)
and the likelihood that assumption of normality does not hold” (P. 511). It seems
that this study also consist of a relative small sample sizes (10 novice and 10 expert
counselors) and consequently will benefit form using the Mann-Whitney U test.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
59
CHAPTER 4: RESULTS
Preliminary Analyses
Two similarity matrices were formed (one for the novice raters and one for
the expert raters) from the 150 selected statements by totaling the number of raters in
each group (experts and novices) who placed each pair of items into the same group.
The attained numbers were then divided by the number of raters (10 in each group)
to obtain a Russel-Rao measure of similarity. A hierarchical cluster analysis was
performed on the similarity matrices using an agglomeration algorithm that joins the
two closest items at each step. The analysis is based on the principle that like items
(close items) have higher frequencies than dissimilar items. As a result, the pair of
items that were grouped together more frequently across the 10 raters are assumed to
be more similar than the items that were grouped together less frequently. Thus, the
similarity score for each pair of items can range form 0 to 1 where 0 represents total
distant and dissimilarity and 1 represents complete closeness and similarity. The
proximity of the clusters for each matrix was measured by the average between
groups linkage method. The number of clusters in each matrix was determined by
calculating the largest drop in the proximity value in each step that two clusters are
joined. This drop indicates the greatest discontinuity and the presence of data
clusters.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Novice
The agglomeration table (see table 3) and the dendogram (see figure 1) for the
novice matrix revealed two large drops in the proximity values indicating two
possible solutions: a five-cluster solution or a three- cluster solution.
Table 3
Agglomeration Table for Novice Counselors/Hierarchical Cluster Analysis
Cluster Coefficients Stage Next
Combined Cluster Stage
First
Appears
Stage Cluster 1 Cluster
2
Cluster 1 Cluster 2
1 119 125 1.100 0 0 114
2 110 121 1.100 0 0 127
3 89 126 1.000 0 0 23
4 48 86 1.000 0 0 140
5 32 78 1.000 0 0 14
6 85 138 .900 0 0 11
7 123 127 .900 0 0 23
8 3 120 .900 0 0 27
9 113 118 .900 0 0 14
10 29 93 .900 0 0 15
11 33 85 .900 0 6 17
12 44 83 .900 0 0 16
13 14 54 .900 0 0 17
14 32 113 .850 5 9 18
15 29 62 .850 10 0 19
16 40 44 .850 0 12 52
17 14 33 .833 13 11 36
18 32 90 .825 14 0 19
19 29 32 .800 15 18 33
20 59 149 .800 0 0 90
21 30 133 .800 0 0 35
22 97 129 .800 0 0 44
23 89 123 .800 3 7 34
24 82 108 .800 0 0 84
25 94 106 .800 0 0 56
26 55 105 .800 0 0 51
27 3 100 .800 8 0 52
28 66 75 .800 0 0 38
29 38 69 .800 0 0 48
30 42 57 .800 0 0 64
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 3 (continued)
31 13 23 .800 0 0 76
32 10 22 .800 0 0 66
33 29 37 .763 19 0 36
34 89 115 .750 23 0 37
35 30 35 .750 21 0 51
36 14 29 .731 17 33 53
37 89 111 .700 34 0 56
38 66 72 .700 28 0 55
39 99 145 .700 0 0 81
40 143 144 .700 0 0 104
41 61 142 .700 0 0 88
42 124 135 .700 0 0 92
43 45 101 .700 0 0 54
44 96 97 .700 0 22 89
45 46 88 .700 0 0 99
46 77 80 .700 0 0 96
47 53 68 .700 0 0 83
48 38 63 .700 29 0 101
49 52 58 .700 0 0 85
50 8 50 .700 0 0 89
51 30 55 .683 35 26 57
52 3 40 .667 27 16 78
53 14 148 .657 36 0 58
54 45 47 .650 43 0 96
55 66 79 .633 38 0 79
56 89 94 .625 37 25 102
57 30 92 .620 51 0 76
58 14 15 .620 53 0 77
59 146 147 .600 0 0 118
60 6 141 .600 0 0 104
61 102 140 .600 0 0 83
62 4 137 .600 0 0 115
63 130 136 .600 0 0 90
64 42 132 .600 30 0 88
65 39 128 .600 0 0 82
66 10 107 .600 32 0 78
67 19 104 .600 0 0 109
68 18 103 .600 0 0 103
69 1 98 .600 0 0 103
70 11 95 .600 0 0 114
71 65 74 .600 0 0 86
72 43 73 .600 0 0 126
73 24 67 .600 0 0 102
74 16 27 .600 0 0 93
75 7 12 .600 0 0 131
76 13 30 .600 31 57 80
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 3 (Continued)
77 14 21 .600 58 0 87
78 3 10 .583 52 66 87
79 66 139 .575 55 0 106
80 13 112 .563 76 0 91
81 99 122 .550 39 0 107
82 39 117 .550 65 0 101
83 53 102 .550 47 61 122
84 36 82 .550 0 24 94
85 52 76 .550 49 0 105
86 5 65 .550 0 71 124
87 3 14 .545 78 77 112
88 42 61 .533 64 41 110
89 8 96 .533 50 44 111
90 59 130 .525 20 63 107
91 13 41 .511 80 0 117
92 34 124 .500 0 42 106
93 16 114 .500 74 0 132
94 36 91 .500 84 0 115
95 60 84 .500 0 0 108
96 45 77 .500 54 46 105
97 26 70 .500 0 0 113
98 20 64 .500 0 0 125
99 46 56 .500 45 0 123
100 17 25 .500 0 0 139
101 38 39 .489 48 82 110
102 24 89 .481 73 56 116
103 1 18 .475 69 68 125
104 6 143 .475 60 40 111
105 45 52 .473 96 85 129
106 34 66 .473 92 79 118
107 59 99 .467 90 81 117
108 60 81 .450 95 0 122
109 2 19 .450 0 67 144
110 38 42 .443 101 88 116
111 6 8 .440 104 89 126
112 3 116 .435 87 0 113
113 3 26 .426 112 97 127
114 11 119 .425 70 1 142
115 4 36 .425 62 94 130
116 24 38 .423 102 110 133
117 13 59 .420 91 107 138
118 34 146 .413 106 59 132
119 87 150 .400 0 0 128
120 109 134 .400 0 0 137
121 49 71 .400 0 0 135
122 53 60 .400 83 108 129
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
63
Table 3 (Continued)
123 9 46 .400 0 99 138
124 5 31 .400 86 0 131
125 1 20 .387 103 98 135
126 6 43 .378 111 72 133
127 3 110 .369 113 2 128
128 3 87 .356 127 119 149
129 45 53 .354 105 122 134
130 4 51 .350 115 0 136
131 5 7 .350 124 75 143
132 16 34 .343 93 118 134
133 6 24 .342 126 116 137
134 16 45 .336 132 129 141
135 1 49 .333 125 121 140
136 4 131 .314 130 0 141
137 6 109 .314 133 120 139
138 9 13 .309 123 117 144
139 6 17 .288 137 100 142
140 1 48 .281 135 4 146
141 4 16 .275 136 134 145
142 6 11 .269 139 114 148
143 5 28 .267 131 0 145
144 2 9 .257 109 138 146
145 4 5 .246 141 143 147
146 1 2 .234 140 144 147
147 1 4 .172 146 145 148
148 1 6 .134 147 142 149
149 1 3 .115 148 128 0
Figure 1
Dendogram for Novice Counselors
Dendogram using Average Linkage (Between Groups)
Rescaled Distance Cluster Combine
C A S E 0 5 10 15 20
Label Num
+----------+-----
NB99110-33 89 -+----------+
EC9915-3 126 -+ +- — +
NB0016-10 123
-------- +----- +
+-+
EB9914-9 127
--------+ I +---+
NA9837-2 115
NB99111-9 111 ----+ I X
EA9835-10 94
-------------------+- + ■
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
64
Figure 1 (Continued)
NA98311-4 106 + I I '
NB9916-8 24 -------------------------+----- + I
EB99111-5 67---------------------------+ I
NA00113-4 61 +----------+ +---+
NB98312-4 142 + +---+ I I
NB98312-3 42 +-----------+ I II I
NB9837-5 57 + +----+ I I I
NA9834-3 132 -------------------------+ +-+ I
NB99110-27 38 +-----+ I I
EA99112-31 69 + +------------+ I I
NB00113-13 63 + +-+ +-+
NB9916-15 39 -------------------------+-+ I II
EA9835-11 128 -------------------------+ +-— + I I
NB99110-15 117 + I I
NB98311 97 +-----+ I I
EA00111-2 129 + +----------+ I I
EB9834-2 96 + +---+ I +-+
NB98310-4 8 +----------+ I I I I
NB9838-7 50 + +---- + I I
EB99112-21 143 +------------+ I II
EC99110-8 144 + +-+ I I
NB9838-8 6 -------------------------+-----+ I I
NB9839-2 141 -------------------------+ I I
EA9915-6 109 +-----+ I
NA9838-5 134 + I
NC98311-9 11 +------------------ + I
EB9834-17 119 + + + +----- +
EC98311-12 95 -------------------------+ +----------+ I
NA9837-10 43 -------------------------+--------+ I I
NB98312-2 73 -------------------------+ I I
EB9912-22 17 +----------+ I I
EB9912-20 25 + +-+ I
NB99112-8 56 +---------------+ I I
EA99112-17 125 + + + I
NA9834-18 9 + I
EA9914-6 7 -------------------------+--------------+ j
EC99110-11 12 --------------------------i- I I
EB9833-5 65 -------------------------+-+ +— + I
EA98311-5 74 -------------------------+ + + I I I
EA9915-1 5 + +— + +-+ I
EB9911-15 31 + II I
EA9915-4 28 + I I
EA9918-3 86 -------------------------+ I I
NA9834-30 147 -------------------------+-------------------+ I I
EA9832-14 146 -------------------------+ I +---1
EA9835-9 82 +--------------+ II I
NA9834-35 108 + + + II I
EB9834-5 4 + +-----+ II I
EB98324-21 36 +-+ +-+ I I I
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
65
Figure 1 (Continued)
EA9913-8 91
EA99112-14 51
NB9836-3 52
EA9833-11 58
EA9833-8 76
NB98310-5 77
EA9837-9 80
EA9837-18 45
EA00114-7 101
EB9838-6 47
EB98311-1 53
NB0015-13 68
EA9911-9 102
EB9835-14 140
EA9835-13 60
NA00114-8 110
NA9914-1 81
EB0022-8 34
NB98311-7 16
NB98311-13 27
NAOO114-20 114
NA9839-9 66
NA98312-2 75
NA9834-31 72
NB00114-27 79
EB98310-3 139
NA9839-14 124
EB9833-6 135
NA0022-13 34
NB9839-4 131
EA9836-17 48
EA9836-14 137
NA00111-2 49
NA99111-13 87
EC98312-6 18
NA0012-4 103
NB00114-10 1
EA9918-6 98
NA0026-3 71
NC9837-5 19
NA9837-14 104
NA98311-15 2
NB00111-19 46
NB99110-6 88
NA99111-11 13
NA9913-7 23
NB0022-3 55
NB99111-6 105
----------------------------------------------------------+ j j +_+ z
+ j j j
+------------- + z z z
+ +----- + z z z
+ + + z z z
----------------------------------------+--------------------------+ I I I I
----------------------------------------+ z I I I I
+_+ z z + _ + zz
----------------------------------------+-----+-------------------+ I I I
--------------------------------------+ z j j
------------------------------+ + j j j
--------------------------------- + +------------------+ j j j
+_+ I I I I
-----------------------------------------+ +_+ j j
---------------------------------------------------------------------+------+ i i i i
------------------------------- + +-+ +-+ I
---------------------------------------------------------------------+— + I I
— ---- ---- -------------—-----— — — -f J_ +—!-
-----------------------------------------+-------- + i i
--------------------------------------------- + +--------------- + j
----------------------------------------------------------+ j j
-------------------+ +— + i i
----------------------------------------X +_+ X z
--------------------------------------------------+ x-----------+ ! J
+--------+ z z
---------------------------------+--------------------- + j z z
----------------------------------------+ x +-+ i
--------------------------------------------------- + j j
----------------------------------------------------------------------+ j
--------------------------------------------- +----------+ j
-------------------------------------------------------+ +~f I
+ + + r
+ J x
------------------------------------------j -------- f . _ j ---------[ _ J
-------------------------------------------------------+ +-------------------+ J z J
+ ------------+ + - +
-------------------------------------------------------+ J
+ z J
+------------- + +_z
-------------------------------------------------------- 1------------------+-----------------------1- x I
+ J x x
----------------------------------------+--------------------------+ z x x
----------------------------------------+ J I I I
-------------------------x---------------------------+ X I I I
--------------------------+ I I +— + I
-------------------------+------------+ +_+ +-----------+ x I
--------------------------+ +— + I I I I I I
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
M M
Figure 1 (Continued)
NC00114-9 30
NB99112-6 133
NB99110-17 35
EB9836-7 92
EA9833-19 112
NB9838-5 41
NA00114-13 20
EB9911-7 64
NB99111-3 99
NB98310-12 145
EC9915-6 122
NB0022-4 59
NB0026-7 149
EA99110-2 130
NB98310-7 136
NB98310-2 26
EB9916-I 70
EA99111-7 10
EC98311-3 22
EA9918-1 107
EA9833-13 44
EA9835-7 83
EB0022-2 40
EA99111-3 3
EB9838-11 120
EA9913-6 100
EA9912-13 93
NB99110-10 121
NAOOil-6 15
EB9831-4 85
EB9835-6 138
NB0024-2 33
NC9837-11 14
EB9831-16 54
EA9839-5 29
EA9833-6 62
EA9911-3 32
EB98310-10 78
EB98310-1 113
EB99112-15 118
EA9835-6 90
EB98312-1 37
EA98311-12 148
NC98312-5 21
NB9838-9 116
NA0017-3 150
-+— + i i i i i i
-+ + - + + - + + - + i i
+ i i i i i
-------------------- + i +_+ +.
------------------------------ + j j
------------------------------+ j
----------------------------------------- + -------+ j
--------------------------------- + j j
+ + +-+
+ +----------+ j
+ + - +
.+--------------------------- + j
-+ + -------+
+ +
+ ------ +
----------- + I
+ I
+--+ I
+ I I
I I
-+ I I
-----+ +---+ I I
-+ + I I I I
-+-----+------- + I I I
+ I I +-+
-+------------- + I I I I
-+ +---+ I +---+ I I
+ I I I I I I
-+ I +---+ I I I
-+------+ I I I I I I
-+ +-----------+ I I I I I I
-+------+ I I I I I I I
-+ I I I I I I I
.+----------+ +— + I i III
-+ I I I I I +-+ +-
-------- j. - | — j . j +------ j . I I I
I I I I I I I I
-+--+-+ +-+ I I I I
-+ I I I I I I
+ I I I I I
---------+ I I I I
--------------------- + I I I
--------------------------------- + j X
+ T
+
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
67
Upon inspection of the proximity values it was evident that the these values
dropped substantially between the steps for a four-cluster and a five-cluster solution
(a drop of 0.43 from 0.246 to 0.203) and between the steps for a two-cluster and a
three-cluster solution (a drop of 0.04 from 0.172 to 0.132). These drops of
approximately 0.04 are larger than that for surrounding steps (between 0.015 to 0.03)
indicating a relatively greater discontinuity. A closer examination of the 2 possible
solutions revealed that clusters 2,3, and 4 in the five-cluster solution were integrated
into one cluster in the three-cluster solution. Three criteria were used to determine
that the five-cluster solution represented the best solution described by the data.
First, the primary researcher and a licensed psychologist closely examined the
statements of the clusters created by the two possible solutions and concluded that
clusters 2, 3, and 4 were distinct from each other and as a result represent three
independent and different clusters. Second, the drop in the proximity value for the
five-cluster solution (0.043) was larger then the drop in the proximity value for the 3-
cluster solution (0.04). Third, a similar study examining novice counselors’ ratings
of counseling conducted by Kivlighan and Coleman (2000) also proposed a five-
cluster solution with similar attributes.
Naming the clusters. Table 4 contains example items that best represent each of
the five clusters.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
68
Table 4
Novice Cluster Membership and Example Items
Cluster Name Example Item Example Item
Deepening and
Clarifying
#111 “So what does she do when
things become too much to
handle”.
#073 “Have you tried
any other method of
escape aside from using
drugs”.
Didactic
Information
#004 “What did they mean by
incompetent? What is your
understanding of being
incompetent - here is another
example of someone asking the
counselor a question that is
potentially difficult to answer.
That’s why learning a way to
respond to these types of questions
is so important”.
#147
“Education.. .cycles of
violence related to
domestic violence
families...client should
be praised for leaving
him”.
Procedural
Information
#036 “If you think about the
question regarding your own
postures in here, notice what
feelings/experience she was
evoking in you. That is important
information”.
#077 “Now seems that
it would be a good time
to point out her
disinterest, esp. since
this is the 2n d time he
has brought up the idea
that he feels that she is
not listening and she
has reacted by looking
at her nails, her clothes,
etc.”.
Conceptualization #104 “In Asian-American
literature, talks about collective
identity and Asia-American
women identifying in relation to
others -daughter, wife, mother.
She has lost that”.
#035 “Sounds like she
either approaches her
relationships from a
place of weakness
(please like and accept
me - 1 don’t like
myself) or a place of
anger (why don’t you
treat me as I would like
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
69
Table 4 (Continued)
Positive #150 “It was nice that C. made his #044 “Getting more
use of ‘uncomfortable’ visible so it upset about what? Nice
Reinforcement aware in the room”. question - she should
keep clarifying”.
In the first cluster which contained 45 statements (30% of the statements), novice
counselor raters appeared to group together comments related to seeking information
and/or clarification, and questions related to challenge or confront the client in an
effort to deepen the session (e.g., “She can’t even say his name? What does that
mean?” and, “should, by whose standards?”). Therefore, this cluster was named
Deepening and Clarifying. The second cluster contained 9 items (6% of the
statements). These statements were related to explanations, examples, and education
provided to the trainee regarding the client (e.g., “this is an example of moving back
and forth between the actual content of the session and how the process becomes
relevant in overall life problem solving”). Consequently, this cluster was labeled
Didactic Information. Cluster 3 contained 41 items (27.3% of the statements).
These statements were related to pointing out to the trainee process information
related to time (e.g., here-and-now), linking and structure of the session (e.g., “also,
timing re: the depression she mentioned may be relevant”, “talking with her husband
yesterday is the likely culprit about why she feels bad today - link temporal events
together”). Statements in this cluster often included procedures about procedures
(metacognitive process; how to do therapy). Thus, this cluster was called time and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
70
Procedural Information. Cluster 4 contained 24 items (16% of the statements) and
included statements that were related to therapist’s cognitive ability such as,
hypothesis testing, conceptualization, assessment, and suggestions for treatment
(e.g., “her sense of self is very wrapped up in whether she is appearing to be perfect,
petty, ugly, etc. - externals”). Therefore, this cluster was named Conceptualization.
Cluster 5 contained 31 items (20.6% of the statements) that included statements
about the trainee’s effective performance (e.g., “good trying to establish how sad is
different from depressed for her”, “great rapport”). Consequently this cluster was
labeled Positive Reinforcement.
After naming the clusters, the researcher compared the clusters’ names with
the names the novice raters gave to the categories (unanalyzed categories) and also
with the only three existing studies that have examined comments made during live
observation of counseling sessions. The three studies looking at live supervision
include Heppner, Kivlighan, Burnett, Berry, Goedinghaus, Doxsee, et. al., (1994)
who described the underlying dimensions for supervisory interventions during
supervision; Holmes, Kivlighan, and Caple (2000), who used clusters to analyze the
dimensions described by Heppner et al., and Kivlighan and Coleman (2000) who
also used cluster analysis to examine novice counselors’ rating of live supervision
comments. For example, cluster 1 in the current study related to information
seeking, clarifications, and challenging questions to deepen the session and was
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
71
named Deepening and Clarifying. Three out of the 10 (30%) of the novice counselor
raters named a grouped category as deepening or expanding/exploring. Additionally,
Heppner et al., named one the dimensions in their study Deepening the Existing
Directions, and Kivlighan and Coleman (2000) labeled one of the clusters as
Challenging the Client.
Cluster 2 in this study related to statements that provided explanations,
examples, and/or education to the trainees was named Didactic Information. Three
out of the 10 (30%) of the novice counselor raters named a grouped category as
direct questions or getting more information about educational issues. Heppner, et
al., (1994) labeled a dimension as being didactic, Holmes et al., (2000) named an
intervention as Didactic/Confrontive, and Kivlighan and Coleman (2000) called a
cluster Request for Information or Details.
In this study, cluster 3 included process information and linking information
and was labeled Procedural Information. Five of the novice raters (50%) labeled a
category as process or linking together, whereas Kivlighan and Coleman (2000)
found a cluster to relate to themes and patterns in client’s behavior.
Cluster 4 in this study related to conceptualization as therapist’s cognitive
ability such as, hypothesis testing, assessment, and possible treatment plans and was
named Conceptualization. Four of the novice raters (40%) named categories as
assessment/conceptualization whereas Heppner et al, (1994) labeled a dimension as
clarifying cognitive tasks.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
72
Finally, cluster 5 in this study was associated with comments praising the
trainees’ performance and was named positive reinforcement. Eight of the novice
raters (80%) grouped categories based on encouragement, praising and
reinforcement, whereas all the studies listed above also named categories based on
support to or joining the intern, and positive reinforcement.
Based on the comparison of type of clusters across these four studies as well
as a comparison to the novice rater’s categories, it appears that external validity
exists regarding the naming of clusters identified in this investigation.
Expert
The agglomeration table (see table 5) and the dendogram (see figure 2) for
the expert matrix indicated a four-cluster solution. Upon inspection of the proximity
Table 5
Agglomeration Table for Expert Counselors/Hierarchical Cluster Analysis
Cluster Coefficients Stage Next
Combined Cluster First Stage
Appears
Stage Cluster 1 Cluster 2 Cluster 1 Cluster
2
1 113 120 1.000 0 0 11
2 37 93 1.000 0 0 4
3 62 78 1.000 0 0 8
4 33 37 1.000 0 2 15
5 17 18 1.000 0 0 141
6 3 10 1.000 0 0 14
7 138 148 .900 0 0 8
8 62 138 .900 3 7 19
9 127 129 .900 0 0 34
10 54 121 .900 0 0 18
11 83 113 .900 0 1 18
12 85 90 .900 0 0 15
13 35 59 .900 0 0 17
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 5 (Continued)
14 3 15 .900 6 0 19
15 33 85 .900 4 12 16
16 32 33 .860 0 15 33
17 35 41 .850 13 0 37
18 54 83 .817 10 11 32
19 3 62 .808 14 8 39
20 58 144 .800 0 0 36
21 132 143 .800 0 0 66
22 75 135 .800 0 0 35
23 97 119 .800 0 0 87
24 109 111 .800 0 0 41
25 103 110 .800 0 0 128
26 73 106 .800 0 0 87
27 94 96 .800 0 0 76
28 38 42 .800 0 0 71
29 23 30 .800 0 0 68
30 21 29 .800 0 0 94
31 22 28 .800 0 0 80
32 44 54 .780 0 18 33
33 32 44 .769 16 32 39
34 123 127 .750 0 9 79
35 12 75 .750 0 22 38
36 58 68 .750 20 0 64
37 35 145 .733 17 0 62
38 12 34 .733 35 0 52
39 3 32 .730 19 33 40
40 3 107 .716 39 0 63
41 61 109 .700 0 24 65
42 124 147 .700 0 0 85
43 63 141 .700 0 0 71
44 60 140 .700 0 0 107
45 84 139 .700 0 0 83
46 133 136 .700 0 0 84
47 108 134 .700 0 0 67
48 2 131 .700 0 0 74
49 126 128 .700 0 0 113
50 25 125 .700 0 0 65
51 69 115 .700 0 0 70
52 12 114 .700 38 0 72
53 55 112 .700 0 0 100
54 88 105 .700 0 0 128
55 77 101 .700 0 0 69
56 81 99 .700 0 0 109
57 49 87 .700 0 0 106
58 47 80 .700 0 0 77
59 4 76 .700 0 0 95
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 5 (Continued)
60 65 72 .700 0 0 116
61 20 57 .700 0 0 131
62 31 35 .675 0 37 104
63 3 70 .670 40 0 73
64 58 74 .667 36 0 77
65 25 61 .650 50 41 76
66 6 132 .650 0 21 79
67 64 108 .650 0 47 74
68 23 104 .650 29 0 75
69 27 77 .650 0 55 115
70 50 69 .650 0 51 108
71 38 63 .650 28 43 98
72 12 36 .640 52 0 97
73 3 100 .633 63 0 78
74 2 64 .633 48 67 100
75 23 46 .633 68 0 104
76 25 94 .630 65 27 80
77 47 58 .625 58 64 97
78 3 118 .618 73 0 101
79 6 123 .611 66 34 98
80 22 25 .607 31 76 103
81 149 150 .600 0 0 105
82 5 142 .600 0 0 120
83 84 137 .600 45 0 115
84 117 133 .600 0 46 113
85 66 124 .600 0 42 102
86 82 98 .600 0 0 121
87 73 97 .600 26 23 122
88 91 92 .600 0 0 132
89 86 89 .600 0 0 90
90 9 86 .600 0 89 124
91 1 67 .600 0 0 103
92 48 51 .600 0 0 136
93 11 43 .600 0 0 122
94 14 21 .600 0 30 101
95 4 19 .600 59 0 129
96 13 16 .600 0 0 139
97 12 47 .592 72 77 99
98 6 38 .583 79 71 118
99 7 12 .575 0 97 107
100 2 55 .570 74 53 110
101 3 14 .570 78 94 111
102 45 66 .567 0 85 119
103 1 22 .561 91 80 120
104 23 31 .560 75 62 109
105 56 149 .550 0 81 123
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 5 (Continued)
106 49 71 .550 57 0 131
107 7 60 .550 99 44 119
108 39 50 .533 0 70 117
109 23 81 .522 104 56 126
110 2 122 .514 100 0 126
111 3 40 .512 101 0 148
112 95 146 .500 0 0 135
113 117 126 .500 84 49 134
114 53 102 .500 0 0 129
115 27 84 .500 69 83 121
116 65 79 .500 60 0 125
117 8 39 .500 0 108 133
118 6 24 .500 98 0 127
119 7 45 .493 107 102 130
120 1 5 .477 103 82 127
121 27 82 .475 115 86 125
122 11 73 .475 93 87 142
123 26 56 .467 0 105 138
124 9 52 .467 90 0 132
125 27 65 .458 121 116 130
126 2 23 .453 110 109 135
127 1 6 .452 120 118 133
128 88 103 .450 54 25 134
129 4 53 .433 95 114 137
130 7 27 .422 119 125 137
131 20 49 .417 61 106 147
132 9 91 .412 124 88 143
133 1 8 .412 127 117 142
134 88 117 .400 128 113 140
135 2 95 .395 126 112 136
136 2 48 .381 135 92 140
137 4 7 .379 129 130 141
138 26 116 .375 123 0 139
139 13 26 .360 96 138 143
140 2 88 .358 136 134 145
141 4 17 .336 137 5 144
142 1 11 .328 133 122 149
143 9 13 .321 132 139 144
144 4 9 .307 141 143 146
145 2 130 .291 140 0 146
146 2 4 .288 145 144 147
147 2 20 .237 146 131 148
148 2 3 .206 147 111 149
149 1 2 .187 142 148 0
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
76
Figure 2
Dendogram for Expert Counselors
Dendogram using Average Linkage (Between Groups)
Rescaled Distance Cluster Combine
CASE 0 5 10 15 20
Label Num +---------- +----------+----------+----------+-------- +
EB98310-1 113 -+----+
EB9838-11 120 -+ +----+
EA9835-7 83 --------+ +-+
EB9831-16 54 --------+— + +-+
NB99110-10 121 ------- + I I
EA9833-13 44 + I
EB98312-1 37 -+ +-+
EA9912-13 93 -+-----+ I I
NB0024-2 33 -+ +-+ I I
EB9831-4 85 --------+ +------+ I
EA9835-6 90 ------- + I I
EA9911-3 32 + I
EA9833-6 62 -+---- + I
EB98310-10 78 -+ + + I
EB9835-6 138 ------- + I +---+
EA98311-12 148 ------- + +------+ I
EA99111-3 3 -+----+ I I I
EA99111-7 10 -+ + + I +-+
NA0011-6 15 ------- + I II
EA9918-1 107 + I I
EB9916-1 70 + +---+
EA9913-6 100 + I
EB99112-15 118 + +— +
NC98312-5 21-- +------------ + I I
EA9839-5 29 + +-+ +----------------+
NC9837-11 14 + I I
EB0022-2 40 ----------------------------------+ I
NA00114-13 20 +-----------------+ I
NB9837-5 57 + +------------1
NA00111-2 49 +--------+ I I
NA99111-13 87 + +------- + I
NA0026-3 71 + i
EB9912-22 17 -+------------------------------------------ + I
EC98312-6 18 -+ I I
EB9834-5 4 +----- + I I
EA9833-8 7 6 + + + I I
NC9837-5 19 + +---+ I I
EB98311-1 53 ----------------------------------+— + I I I
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
77
Figure 2 (Continued)
EA9911-9 102
NA9839-14 124
NA9834-30 147
NA9839-9 66
EA9837-18 45
EA9835-13 60
EB9835-14 140
NA98312-2 75
EB9833-6 135
EC99110-11 12
NA0022-13 34
NA00114-20 114
EB98324-21 36
EB9838-6 47
EA9837-9 80
EA9833-11 58
EC99110-8 144
NB0015-13 68
EA98311-5 74
EA9914-6 7
EB9833-5 65
NA9834-31 72
NB00114-27 79
EA9835-9 82
EA9918-6 98
NB98310-5 77
EA001I4-7 101
NB98311-13 27
EB0022-8 84
EB98310-3 139
EA9836-14 137
EA9913-8 91
EB9836-7 92
EA9918-3 86
NB99110-33 89
NA9834-18 9
NB9836-3 52
NA99111-11 13
NB98311-7 16
NB0026-7 149
NA0017-3 150
NB99112-8 56
NB98310-2 26
NB9838-9 116
EC9915-3 126
EA9835-11 128
NB99112-6 133
NB98310-7 136
NB99110-15 117
+ j +-+ i
-------------------------------------+----------+ i i i i
-------------------------------------+ +_+ i i i i
-------------------------------------------------+ +— + i i i i
------------------------------------------------ + j i i i i
---------------------------------------------+ -------------------+ j i i i i
-------------------------------------+ j +— + +-+ i i
---------------------------- +_+ i i i i i i
-------------------------+ +_+ i i i i i i
-----------------------------+ +_+ i i i i i i
------------------------------------------+ + — + + — + i i i i
-------------------------------------+ +_+ x i i i i
-------------------------------------+ x j I I I I
-------------------------------------+— + +_+ i i i i
-------------------------------------+ i i i +___ + i i
----------------------- j --- j . ^ --- ) . I I I H h
-------------------------+ +----------+ I X x I I
-----------------------------+ +_+ x I I I
-----------------------------------------+ x I I I
---------------+ I I I
.+----------------------+ I I X
-+ +-+ I I I
--------------------------+ XI I I
-----------+-------------+_+ x I
+ I I I
-+-+ I I I
.+ +--------------+ x I I
— + +-+ I I
-+---------+ I I I
-+ +--------+ I I
+ I I
-----------+ + x I
+ I I I
+ +----+ x I
-+------- + I II I
.+ +— + II I
---------+ +-+ I
.+----------------------+ I I x
-+ I I I I
-+-+ +-+ I I
.+ +----------+ I +— I
---------------+ +---------+ x x
------------------------------------+ I I I
-------------------------------------+ x x
.+ + I I
-+ +-----+ I I
-+ + I I I I
-+------+-----+ +-+ I I
-------+ II I I
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Figure 2 (Continued)
NA0012-4 103
NA00114-8 110
NB99110-6 88
NB99111-6 105
EA9836-17 48
EA99112-14 51
EC98311-12 95
EA9832-14 146
NA9914-1 81
NB99111-3 99
NB99110-17 35
NB0022-4 59
NB9838-5 41
NB98310-12 145
EB9911-15 31
NA9913-7 23
NC00114-9 30
NA9837-14 104
NB00111-19 46
NB0022-3 55
EA9833-19 112
NA98311-15 2
NB9839-4 131
NA9834-35 108
NA9838-5 134
EB9911-7 64
EC9915-6 122
EA99110-2 130
NB98311 97
EB9834-17 119
NB98312-2 73
NA98311-4 106
NC98311-9 11
NA9837-10 43
EA99112-31 69
NA9837-2 115
NB9838-7 50
NB9916-15 39
NB98310-4 8
NB99110-27 38
NB98312-3 42
NB00113-13 63
NB9839-2 141
EB9914-9 127
EA00111-2 129
NB0016-10 123
NA9834-3 132
EB99112-21 143
NB9838-8 6
.+---------------------+ j !
-+ +--+ I
- + + --------
I
+ ------------------------------- +
--------------------------------- + j
----------------------------------------------------+----------+ j
----------------------------------------------------+ j i
------------------ +------------- + j I
---------------------------+ j i i
-+-+ I II
-X + ----------------+ + -------+ + _ +
-—+ +-+ I I I
----------------------------------------- + + ---------------- + I I I
--------------------------------------------- + J 2 J 2
+--------- + +_+ 2 2
------------------------------- + + _ + J I I
+ j
+ ----------------+ x
--------------------------------------+ +_+ j
------------------------------------------+----- + j j j
-------------------- + +---+ I I
-------------------- +-+ I------ +--+
------------------------------------------+ +_+ j
---------------------- + T
+ ---------------+
- + + -
I I
------ + + j
+ I
-+-+ I
-+ + + I
+ + - + +
+ X------------+ I
---------------------------------------------+ J J
-+-------+ I I
-+ +— + I I
-------+-+ I I I
------------- + +----- + x I
-+------- + I I +— +
-+ + + I I I
--------- + +-+ +-+ I
--------------- + ------------------ + J I I I
-------+ +-+ I I I
+ J J J
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
79
Figure 2 (Continued)
NB9916-8 24 + +---+
EA9915-1 5 --------------------------- +------- +
NB98312-4 142 --------------------------- + I
EC98311-3 22 +------------+ I
EA9915-4 28 + I I
EA9835-10 94 +----------+ +-+ I
EB9834-2 96 + I I I I
EA9915-6 109 +-----+ +-+ I I
NB99111-9 111 + +-+ I +----- +
NA00113-4 61 -------------------- + +-+ I
EB9912-20 25 -------------------- +-+ I
EA99112-17 125 -------------------- + I
NB00114-10 1 --------------------------- +-+
EB99111-5 67 --------------------------- +
values it was evident that the proximity values dropped substantially between the
steps for a three cluster and a four-cluster solution (a drop of 0.05 from 0.288 to
0.236). This drop is larger than that for surrounding steps (between 0.03 to 0.01)
indicating a relatively greater discontinuity. This abrupt decrease in the matrix value
indicates that a four-cluster solution best describes the data. Additionally, a visual
inspection of the dendogram also suggested a four-cluster solution as the appropriate
solution for the expert data. Naming the clusters. Table 6 contains example items
that best represent each of the four clusters. Cluster 1 contained 36 statements (24%
of the statements) and as in the case of the novice raters, expert raters appeared to
group together comments in this cluster that related to information seeking for the
purpose of deepening and clarifying (e.g., •‘ what does it mean to explode”).
Consequently, this cluster was named Deepening and Clarifying. The second cluster
is the largest cluster grouped by the expert raters and contained 81 items (54% of the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
80
statements). This cluster included comments related to conceptualization in term of
therapist’s cognitive ability such as, hypothesis testing, assessment, and treatment
interventions (e.g., “I would have kept the focus on his gambling and discuss with
him how he uses it to distract him from other thoughts and feelings”) and therefore
was named Conceptualization. Cluster 3 is the smallest cluster and contained 7
statements (4.6%). This cluster seems to focus on comments describing non-verbal
body and voice qualities (e.g., “how can she process her body language with her, or
should she”). Consequently, this cluster was named Observation of Non-Verbal
Process.
Table 6
Expert Cluster Membership and Example Items
Cluster Name Example Item Example Item
Deepening and
Clarifying
#141 “So how are all of
these things connected”.
#025 “What happened in
May that things changed
her appetite?”.
Conceptualization #128 “Feelings of
disappointment preventing
her from reaching out to
others”.
#066 “Due to the brief
nature of this
relationship, it may be
helpful for the client to
address the emotional
issues relating to his
pressures”.
Observation of Non-
Verbal Process
#87 “D. has effectively
been using her voice to
calm, bring down the level
o f‘defensiveness’ that
was hindering their
communication a bit”.
#49 “This isn’t a big
thing, but uncrossing
your arms might be more
open looking”.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
81
Table 6 (Continued)
Positive Reinforcement #37 “Good follow up re:
asking about suicidal and
homicidal thoughts”.
#22 “Nice. Good
confrontations and
attempts to engage.
How does hiding from
you help? What is she
crying about?”.
Cluster 4 contained 26 statements (17.3%) and consistent with the novice raters, this
cluster includes statements associated with praising the trainees’ performance (e.g.,
“nice...good question”). Therefore, this cluster was also named Positive
Reinforcement.
The experts’ clusters names were also compared to the names given to the
categories described in the three studies examining live supervision and with the
expert raters’ names for clusters (unanalyzed categories). Clusters 1,2, and 4 have
similar names as the novice clusters. Cluster 3 is distinct from any of the novice
clusters and is the only cluster that was compared to the existing studies examining
comments made during live supervision.
Cluster 1 was named deepening and clarifying. Three of the expert raters
(30%) labeled categories as Deepening or Clarification. Cluster 2 relates to
conceptualization in term of therapist’s cognitive activities such as, hypothesis
testing, assessment, and treatment interventions and was named Conceptualization.
Four of the expert raters (40%) labeled categories in terms of assessment,
conceptualization or treatment interventions. Cluster 4 included statements
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
82
associated with praise of the trainees activities and was called Positive
Reinforcement. Four expert raters (40%) named categories in terms of praising,
complements and reinforcement.
Cluster 3 was named observation of non-verbal process. Two expert raters
(20%) named categories in-terms of non-verbal process and none of the other studies
distinctly emphasized non-verbal cues as a name for category or dimension from live
supervision. Except for cluster 3, which seems to represent a new dimension not
addressed by existing literature, the names given to the expert clusters in this study
appear to be consistent with the names provided to the categories by the expert raters
and is also similar to the content and names provided in the live supervision studies.
Hypothesis 1
The first hypothesis, that expert counselor raters would produce fewer
categories/clusters than the novice counselor raters was supported by the cluster
analysis. The number of clusters produced by the experts was compared with the
number of clusters produced by the novice raters. As shown in Table 3 and Figure 2,
expert raters sorted the feedback comments into four categories (Deepening and
Clarifications, Therapist’s Cognitive Activity, Observations of Non-Verbal Process,
and Positive Reinforcement). As shown in Table 1 and Figure 1, novice raters sorted
the feedback comments into five categories (Deepening and Clarification, Didactic
Information, Procedural Information, Therapist’s Cognitive Activity, Positive
Reinforcement).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
83
Additionally, a series of chi-square analyses were used to examine whether
cluster membership of the statements was influenced by contextual variables such as,
the semester/cohort, or the session sequence within each week (client) for both the
novice and the expert clusters. In other words, the analysis examined whether the
clusters were consistent across contexts, or whether certain types of feedback
statements only arise in some semesters/cohorts and or session sequence (client). The
analyses revealed that for the expert clusters, neither practicum cohort or the session
sequence had a significant relationship with the cluster membership. For the
semester effect, Chi square = 10.84, df = 6, p= 0.09. For the session sequence effect,
Chi square = 3.12, df = 6, p = 0.79. Therefore, for the expert clusters, the type of
statements made was not dependent on the semester or the session sequence. For the
novices’ clusters, the type of statements were dependent on the semester; Chi square
= 16.31, df = 8, p = 0.038. Additionally, the session number (client) was close to
being significantly related to the type of statements made; Chi square = 15.30, df = 8,
p= 0.053. This may suggest a mild relationship between the type of statements and
the session sequence (client). Consequently, it appears that novice’s categorization
may be influenced by context related factors. .
Hypothesis 2
The second hypothesis predicted that Expert counselor raters would spend less
time than novice counselor raters sorting the feedback comments into categories; this
hypothesis was not supported. To test the hypothesis a one tailed Maim-Whitney U
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
84
test was calculated. The mean rank for the novice counselors was 10.75 while the
mean rank for the expert counselors was 10.25 with a Mann-Whitney U of 47.5.
There was no significant difference between expert counselors and novice counselors
in the time spent sorting the 150 cards into categories, p = .853 > .05.
Hypotheses 3 and 4
Hypotheses 3 and 4 were tested together via DFA and are therefore reported
together in this section. The third hypothesis predicted that expert counselors would
differ in the type of feedback comments they would provide while observing a live
counseling session than the feedback comments provided by novice counselors.
Hypothesis 4 predicted that the time of the semester in which the novices’ feedback
comments were made would impact the type of statements made. More specifically,
it was expected that as the weeks in the semester progressed, novices’ feedback
would become more similar to that of the expert counselor. Both hypotheses were
tested by a DFA with three predictors. The first predictor was the effect of expert vs.
novice on the feedback comment (hypothesis 3). The second predictor was the effect
of time in the semester on the feedback comments (hypothesis 4). A third predictor
was added to test the interaction between level of expertise and time of the semester.
Interaction term was based on an effect coding of expert status (weeks in the
beginning of the semester were scored low for expert and high for novices, whereas
weeks later in the semester were scored high for experts and low for novices). This
measures any systematic differences in timing between expert and novices.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
85
Outcome was tapped by cluster membership according to the expert and novice
cluster solutions (4-cluster solution for expert raters and 5-cluster solution for novice
raters). Therefore, the data tests the effect expertise level, time during the semester
and systematic time on the type of statements made. Univariate ANOVA for the
three variables effects on cluster membership are displayed in Table 7(expert) and
Table 8 (novice). Stepwise DFA with a criterion to entry, p < 0.05 was used. Only
the level of expertise entered the equation. Hypothesis 3 was supported by the
analysis (level of expertise had a significant effect on the type of feedback
statement made). However, Hypothesis 4 was not supported by the analysis (time in
the semester did not effect the type of statements expert or novice counselors made).
For the expert raters, Wilks Lambda = 0.911, p = 0.0036, approximately 9%
of the variance in the expert cluster membership is accounted for expert status; for
the novice raters’, Wilks Lambda = 0.871, p= 0.0005, approximately 13% of the
variance was accounted for novice status. Given that level of expertise made a
difference in cluster membership the group mean of expertise status by cluster
membership was examined to determine which type of cluster was most frequently
typed by novice counselors and which type of cluster was most frequently typed by
expert counselors (see Table 9 for novice raters and Table 10 for expert raters). The
analysis suggests that, overall, expert and novice counselors make different type of
comments.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
86
Table 7
Univariate ANOVA for the Three Variables Effect on Expert’s Cluster Membership
Variable F-ratio Significance
Timing 0.9466 0.4199
Week 1.5031 .2162
Level of Expertise 4.7030 0.0036
Table 8
Univariate ANOVA for the Three Variables Effect on Expert's Cluster Membership
Variable F-ratio Significance
Timing 0.3085 0.8719
Week 2.7935 0.0284
Level of expertise 5.3579 0.0005
Table 9
Group Means for Novices’ 5-Cluster Solution
Novice Cluster Level of Expertise*
1 - Deepening and Clarification - .15556
2 - Didactic Information .55556
3 - Procedural Information .02439
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
87
Table 9 (Continued)
4 - Therapist’s Cognitive Activities -.58333
5- Positive Reinforcement .48387
Table 10
Group Means for Novices’ 5-Cluster Solution
Expert Cluster Level of Expertise*
1 - Deepening and Clarifications - .22222
2 - Therapist’s Cognitive Activity -.06173
3 - Observation of Non-Verbal Process - .42857
4 - Positive Reinforcement .61538
* The variable level of expertise was coded 1 for experts and -1 for novices so that
values above 0 indicate predominant expert’s statements and values below 0
predominant novice statements.
For the novice five-cluster solution, clusters 2 (Didactic Information) and 5
(Positive Reinforcement) included predominantly expert statements, while cluster 4
(Conceptualization) included predominantly novice statements. For the expert four-
cluster solution, cluster 4 (Positive Reinforcement) included predominantly expert
statements and cluster 3 (Observation of Non-Verbal Process) included mostly
novice statements.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
88
CHAPTER 5: DISCUSSION
In this chapter, the findings of the study will be summarized, and the meaning
and significance of the study will be discussed, beginning with the hypotheses.
Next, other important findings will be addressed followed by identification of
implications for training, limitations of the study and recommendations for future
research.
Hypothesis 1
The results supported hypothesis 1. A comparison of the cluster analyses for
novice and expert counselor raters indicated that expert counselor raters’ produced
fewer clusters (4 clusters) than the novice counselor raters (5 clusters). It appears
that, overall, novice and expert counselor raters conceptualized the Deepening and
Clarifying cluster (cluster 1 for novice and expert counselors) and the Positive
Reinforcement cluster (cluster 5 for novice and cluster 4 for expert counselors)
similarly. There were a number of statements that were not identically sorted into
the categories such as item number 007,017, and 144 for the Deepening and
Clarifying cluster and items number 022, 026, and 150 for the Positive
Reinforcement cluster, yet overall the clusters were almost identical. However, it
appeared that expert counselor raters integrated novice counselor raters’ Didactic
Information (cluster 2), Procedural Information (cluster 3), and Conceptualization
(cluster 4) clusters into one cluster, Conceptualization (cluster 2; see table 11).
Additionally, the expert raters identified a cluster that was not identified by the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
89
Table 1 1
Breakdown of The Feedback Comments into Expert and Novice Clusters
Item # Code Novice Cluster Membership Expert Cluster
Membership
008 NB98310-04 1 1
Oil NC98311-09 1 1
006 NB98308-08 1 1
024 NB99106-08 1 1
017 EB99102-22 1
073 NB98312-02 1 1
094 EA98305-10 1 1
096 EB98304-02 1 1
106 NA98311-04 1 1
125 EA99112-17 1 1
127 EB99104-09 1 1
129 EA00111-02 1 1
132 NA98304-03 1 1
143 EB99112-21 1 1
144 EC99110-08 1
038 NB99110-27 1 1
115 NA98307-02 1 1
063 NB00113-13 1 1
007 EA99104-06 1
067 EB99111-05 1 1
134 NA98308-05 1
097 NB98311-05 1 1
123 NB00106-10 1 1
117 NB99110-15 1 2
119 EB98304-17 1 1
126 EC99105-03 1 2
128 EA98305-11 1 2
089 NB99110-33 1 2
056 NB99112-08 1 2
069 EA99112-31 1 1
043 NA98307-10 1 1
025 EB99102-20 1 1
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
90
Table 11 (Continued)
141 NB98309-02 1 1
061 NA00113-04 1 1
009 NA98304-18 1
050 NB98308-07 1 l
109 EA99105-06 1 1
039 NB99106-15 1 1
042 NB98312-03 1 1
142 NB98312-04 1 1
057 NB98307-05 1
002 EC99110-11 1 2
095 EC98311-12 1 2
065 EB98303-05 1 2
111 NB99111-09 1 1
031 EB99101-15
2 2
146 EA98302-14 2 2
082 EA98305-09
2 2
005 EA99105-01 2
1
086 EA99108-03 2 2
147 NA98304-30
2 2
108 NA98304-35 2
1
028 EA99105-04
2
1
004 EB98304-05 2 2
066 NA98309-09 2
076 EA98303-08 2
110 NA00114-08
2
114 NA00114-20 2
048 EA98306-17 2
075 NA98312-02 2
072 NA98304-31 2
049 NA00111-02
036 EB98304-21 2
071 NAOO106-03 0
087 NA99111-13
081 NA99104-01
2
051 EA99112-14 2
079 NB00114-27
2
077 NB98310-05
2
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
91
Table 11 (Continued)
103 NA00102-04
« * >
3 2
027 NB98311-13 3 2
098 EA99108-06 3 2
101 EA00114-07 3 2
091 EA99103-08 3 2
060 EA98305-13
■ * >
3
2
016 NB98311-07 3
2
139 EB98310-03 3 2
080 EA98307-09 3
2
045 EA98307-18
- >
3
2
084 EBOO102-08 3
2
001 NB00114-10 3 1
140 EB98305-14 3 2
018 EC98312-06 3
2
131 NB98309-04 3
2
124 NA98309-14 3 2
068 NB00105-13 3
2
058 EA98303-11
* - *
3
2
052 NB98306-03
- ■ »
3 2
102 EA99101-09 3
2
074 EA98311-05 3
2
053 EB98311-01 3
2
034 NA00102-13 3
2
135 EB98303-06 3
2
137 EA98306-14
■ * >
3
2
047 EB98308-06 3
2
149 NBOO106-07 4 2
099 NB99111-03 4
2
122 EC99105-06 4
2
104 NA98307-14 4
2
064 EB99101-07 4
2
019 NC98307-05 4 2
055 NBOO 102-03 4 2
002 NA98311-15 4 2
035 NB00110-17 4 2
105 NB99111-06 4 2
059 NBOO 102-04 4 2
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
92
Table 11 (Continued)
136 NB98310-07 4 2
145 NB98310-12 4 2
046 NB00111-19 4 2
041 NB98308-05 4 2
088 NB99110-06 4 2
030 NC00114-09 4 2
092 EB98306-07 4 2
023 NA99103-07 4 2
112 EA98303-19 4 2
013 NA99111-11 4 2
020 NA00114-13 4
* ■ >
0
130 EA99110-02 4 2
i ^ n
Ioj NB99112-06 4 2
150 NAOO107-03 5 2
029 EA98309-05 5 4
032 EA99101-03 5 4
021 NC98312-05 5 4
033 NBOO 104-02 5 4
037 EB98312-01 5 4
044 EA98303-13 5 4
070 EB99106-01 5 4
093 EA99102-13 5 4
107 EA99108-01 5 4
118 EB99112-15 5 4
022 EC98311-03 5 1
010 EA99111-07 5 4
003 EA99111-03 5 4
083 EB98305-07 5 4
090 EA98305-06 5 4
138 EB98305-06 5 4
113 EB98310-01 5 4
120 EB98308-11 5 4
100 EA99103-06 5 4
026 NB98310-02 5 2
078 EB98310-10 5 4
062 EA98303-06 5 4
116 NB98308-09 5 2
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
93
Table 11 (Continued)
040 EB00102-02 5
015 NA00101-06 5 4
085 EB98301-04 5 4
054 EB98301-16 5 4
014 NC98307-11 5 4
121 NB99110-10 5 4
148 EA98311-12 5 4
novices, Observation of Non-Verbal Process (cluster 3). This cluster has not been
previously identified in the research literature on live supervision.
It appears that experts had a more developed cognitive schema and were able
to more efficiently perceive the large bits of information (the 150 feedback
statements), which enabled them to: a) categorize the feedback comments into four
clusters that integrated three novice clusters into one, and b) identify a distinct and
subtle cluster (Observation of Non-Verbal Process). These findings are consistent
with the expertise literature that has noted that experts: a) perceive and understand
problems in their domain at a deeper level than do novices, b) appear to store
abstract knowledge in large and meaningful problem-relevant schematic categories,
c) show superior skills in pattern recognition, and d) are able to more efficiently
perceive large and meaningful patterns of information (Chi, Glaser and Farr, 1988;
Glasser, 1992).
Findings in this study also support findings previously identified in the
literature on counseling expertise. The fact that expert raters produced fewer clusters
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
94
than novices is consistent with the Martin et al. (1989) and the Mayfield et al. (1999)
findings’ where novice counselors formulated more concepts than experienced
counselors on a Cognitive Mapping Task (CMT). However, the innovative nature of
the current investigation is an important contribution to the counseling expertise
literature: this is the first study that allows researchers to tap directly into novice and
expert counselors’ thought process in real time as opposed to using retrospective
methods. The consistency of the findings in this study with the expertise literature in
general helps confirm previously held views regarding expert and novice
functioning. These findings also lend support to the growing literature on expertise
in psychotherapy which suggest that expert counselor’s well-developed schemas
allow them to structure clinical information efficiently and distinguish between
relevant and surface details.
Expert counselor raters’ integration of Didactic Information (cluster 2),
Procedural Information (cluster 3), and Conceptualization (cluster 4) clusters into
one cluster, Conceptualization (cluster 2) might be explained by novice counselor
raters’ tendencies to look at idiosyncratic case features while conceptualizing clinical
cases, whereas expert counselors tended to focus on similar thematic features
(Mayfield et al., 1999. By viewing themes and patterns the expert raters viewed
didactic, procedural, and conceptualization information as one integrated cluster that
was defined by attributes related to conceptualization. Novice counselors, on the
other hand, have limited experiences conducting psychotherapy; consequently they
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
95
have less well-developed and instead appeared to use case specific details to sort the
Conceptualization cluster into three separate clusters.
Expert and novice counselor raters categorized the Deepening and Clarifying
and Positive Reinforcement clusters in a similar manner. One possible explanation is
that the statements these two clusters represented more surface detail information
which is much more content based, basic, global and obvious. Past research on
novices suggests that they are more likely to attend to superficial or surface detail in
early stage of their development; this finding lends further support to that
observation.
However, expert and novice counselor raters may have used different
categorizing and problem solving skills to conceptualize less structured and more
complex information such as conceptualization (and process-oriented) information.
This explanation is consistent with Lichtenberg’s (1997) notion that counseling
psychology expertise should be viewed and studied in the context of “ill-structured
problems (the parameter of the problems are generally not clear and or at best
loosely defined” p. 223). He suggested that experts and novices might use different
problem solving skills; expert counselors may have a greater breadth and depth of
clinical experience which enables them to organize fuzzy and complex information
into more meaningful patterns (e.g. categorization and conceptualization of clinical
information). Categorizing different feedback statements which were randomly
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
96
selected from different semesters, different clients and different counselors seems to
represent a task that fits the description of an “ill structured problem”.
Hypothesis 2
The second hypothesis predicted that expert counselor raters would spend
less time than novice counselor raters sorting the feedback comments into categories.
The results of a one tailed Mann-Whitney U analysis showed that the novice and
expert groups had similar rank means, thus offering no support to the analysis. This
finding is inconsistent with previous findings in the literature. Chi, Glaser and Farr
(1988) suggested that experts are faster than novices in executing tasks related to
their domain, and Mayfield et al., (1999) found that experienced counselors were
faster at some CMT tasks such as reading session transcripts and sorting statements
into categories than novice counselors. Chi, Glaser, and Farr’s (1998) finding relates
to general expertise in a specific domain and not necessarily to psychotherapy
expertise. It may be that the “ill structured problems” and the complex nature of
counseling expertise contributed to the insignificant finding in the current
investigation.
The current study also demanded that the novice and expert counselor raters
perform a novel and unfamiliar task; the novelty of the task may also explain why
the novice and expert group did not differ in the amount of time required to complete
the task. Additionally, the Mayfield (1999) study used a small sample size of novice
(5) and expert (4) counselors, which could have contributed to the significant
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
97
findings in their study. The current study used a larger sample size (10 and 10 expert
counselor raters); a larger sample size may have evened out the time differences
between the novice and expert group.
Finally, it is apparent from the means of the expert (51:25) and the novice
(53:26) groups that, on the average, counselors in both groups completed the task
within one-hour. Many of the novice and expert counselors completed the task
within their work settings and thus it is possible that counselors from both groups
attempted to complete the task within a counseling session time-frame to allow
themselves time to return to work within a reasonable time.
Hypothesis 3
Overall, the results supported the third hypothesis, which predicted that
novice and expert counselors would differ in the type of feedback comments they
would provide while observing a live counseling session. The stepwise DFA with a
criterion entry of 0.05 was significant for level of expertise and suggested that,
overall, expert and novice counselors made different type of feedback statements.
For the novice five-cluster solution, Didactic Information (cluster 2) and Positive
Reinforcement (cluster 5) included predominantly expert statements, while
Conceptualization (cluster 4) included predominantly novice statements. For the
expert four-cluster solution, Positive Reinforcement (cluster 4) included
predominantly expert statements and Observation of Non-Verbal Process (cluster 3)
included mostly novice statements.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
98
Didactic Information included statements related to explanations, examples,
and/or education to the trainees. Positive Reinforcement included statements related
to praising the trainees’ performance and encouraging them to continue working in a
certain direction. Consequently, it is not surprising that the expert counselor made
most of the statements in these two clusters; using RTT, it was possible to
simultaneously provide examples, offer encouragement and answer novice
observers’ typed questions as the counseling session was being conducted behaind a
one-way mirror.
Novice counselors made most of the conceptualization comments within the
novice counseling rater’s 5-cluster solution. This observation is consistent with the
study conducted by Brammer (1997) who suggested that experienced therapists
appear to ask fewer client-history questions; instead experts providing psychotherapy
appear to use prototype themes and depend on their previous experience, in their
conceptualization of the client. It is not surprising that novice counselor raters
focused on content information related to cognitive activities involved with
assessment and treatment planning, while the expert counselor attended more to
provide positive reinforcement, explanations, and examples. The role of a supervisor
in a practicum setting is to teach and train novices while attending to process
information as well as the welfare of the client.
It is surprising that the expert counselor raters identified the Observation of
Non-Verbal Process cluster that included mostly novice statements. Experienced
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
99
counselors would have commonly address non-verbal process information associated
with body language. There may be three possible explanations for this finding.
First, the practicum instructor introduced the notion of non-verbal process
information as important conceptualization and intervention skills in the first lecture
and throughout the semester. It seems that the experienced counselor successfully
transferred her knowledge to the trainees; this transfer of knowledge is evidenced by
the feedback comments made by novice trainees in the practica classes, as they
applied their knowledge to peers conducting counseling sessions behind a one-way
mirror. Second, it is possible that the random selection of items contributed to the
unequal distribution between novice and expert counselors’ statements in this cluster.
In other words, a different random selection of items may have produced a more
equal distribution of statements made by novice and expert counselors in this cluster.
An erratic and unequal distribution between clusters also occurs in terms of
semesters (e.g., the clusters included 77 statements made in fall 1998 and only 28
statements made in spring 2000). Finally, it appears novice counselors may have
attended to and commented on obvious and surface details (the most observable
observations that had been discussed in class) while novice counseling raters
involved in the categorization task lacked experience and automated procedural
information to integrate their knowledge in terms of conceptualization, (the mean
counseling experience for novice counselors was 74.2 hours of therapy experience
which seems to be less than the required amount of experience hours to automate
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
100
counseling knowledge) The fact that novice counselor raters sorted the items that
expert counselor raters included in the Observation of Non-Verbal Process cluster
across four different clusters provides further support to the above explanation.
Hypothesis 4
Hypothesis 4, that the time of the semester in which the novices’ feedback
comments were made would impact the type of statements made was not supported
by the findings. Based on the concept of transfer of knowledge from the expert to
the novice counselors, it was expected that as the weeks in the semester progressed,
novices’ feedback would become more similar to that of the expert counselor.
However, the DFA indicated that the time in the semester that the statements were
made was not significantly related to the type of statements made by novice or expert
counselors. This is inconsistent with Kivlighan and Coleman (2000) who suggested
that novice counselors may have been looking to the experienced counselor for cues
on what to type as feedback comments to their peers, since the experienced
counselor may have made each type of feedback comment in earlier sessions than the
novice counselors. Again, the random selection of the items in this study created an
unequal distribution of items as far as time in the session and time in the semester.
This may have influenced the relationship between levels of expertise, the type of
statements and the time in the semester the statements were made. Another
explanation may be that the expert counselor’s lecture before sessions and
supervision provided after counseling sessions wee conducted influenced the type of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
101
feedback comments that were provided by the novices. It is possible that the novice
counselors modeled the expert counselor’s feedback and interventions and applied
his/her education/suggestions immediately in the same session or week in such a way
that time differences associated with type of feedback could not be measurable or
detected.
Other Findings
There are other findings in this study worth noting. One interesting finding
was suggested by a series of chi-square analyses examining whether the cluster
memberships were consistent across contexts, or whether certain types of feedback
statements only arise in some semesters/cohorts and or session sequence (client).
The results showed that, for the expert cluster membership, the type of statements
were consistent across factors where neither practicum cohort nor the session
sequence had a significant relationship with the cluster membership. However, for
the novice cluster membership the type of feedback statements made were dependent
on the semester and may have suggest a mild relationship with the session sequence
(the Chi square analysis was close to be significant).
These findings appear to indicate that experts’ categories/clusters were
consistent and universal regardless of contextual variables such as semester or
session sequence. It appears that novices’ categorization may be influenced by
context related factors; again they may have used superficial, content bound factors
to sort the feedback comments into categories. This appears to be consistent with
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
102
previous literature findings. Martin et al. (1989) found that, for specific client’s
problems, novice counselors generated more concepts by attending to each client as
an individual case; they interpreted the results as evidence that “experienced
counselors hold extensive abstract, general knowledge of counseling which enables
them to conceptualize specific counseling situations efficiently and parsimoniously,
whereas novice counselors engage in more extensive and unique conceptual work
with each client” (p. 395).
Cummings et al., (1990) showed that expert counselors consistently and
repeatedly used domain-specific, thematic concepts to understand counseling and
client’s problems overtime. Expert counselors’ well-developed schema to explain
how change occurs and conceptualizing individual client problems enables expert
counselors to be consistent over time regardless of surface and contextual variables.
On the other hand, novice counselors do not possess a well-developed schema and
seem to attend to surface level details dependent on the descriptive characteristic of
the client’s problems.
Finally, Mayfield et al. (1999) explained that novice counselors’ categories
tended to focus on case specific details and varied from novice to novice whereas all
experienced counselors used similar non-case-specific categories. Thus, the finding
of the current study supports the findings of previous research which has suggested
that novice counselors use contextual factors that may be irrelevant while
conceptualizing clinical information while expert counselors use their experience and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
103
automated knowledge to consistently conceptualize clinical information regardless of
these same contextual factors.
Another important finding from this investigation is the comparison of the
type of clusters identified by novice and expert counselors, in this study, with other
categories, clusters, interventions, and intentions identified by previous studies. This
comparison may begin to clearly highlight some of the basic tasks or core issues
addressed in psychotherapy and clinical training. The current study revealed a
number of clusters representing the different type of comments counselors may make
in counseling: Deepening and Clarifying, Didactic Information, Procedural
Information, Conceptualization, and Positive Reinforcement for novice counselors
and Deepening and Clarifying, Conceptualization, Observation of Non-Verbal
Process, and Positive Reinforcement for expert counselors. These clusters appear to
be consistent with other studies which have examined the type of activities occurring
in psychotherapy and supervision. Kivilghan and Coleman (2000) suggested that
novice counselors' raters sorted feedback cards into 5 clusters: Challenging Client
and Request for more Information/Details (related to Deepening and Clarifying and
Didactic Information), Timing/Planning of Interventions and Themes/Patterns
(similar to Procedural Information in the novice cluster and Conceptualization in the
expert cluster), and Reinforcement (Positive Reinforcement).
Additionally, Heppner et al., (1994) described six underlying dimensions for
supervisory interventions: a) Directing and Instructions vs. Deepening the Existing
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
104
Directions, b) Clarifying Cognitive Tasks and Goals vs. Encouraging Emotional
Reactions, c) Confronting the Client vs. Encouraging the Client, d) Being Didactic
and Distant vs. Emotionally Involved, e) Joining the Trainee vs. Challenging the
Trainee, and f) Providing Directions vs. Resignation. Holmes et al., (2000) cluster
analyzed the above dimensions into four foci that supervisors use during live
supervision: a) Didactic/Confrontive, b) Supportive/Enabling, c) Emotionally
Focused, and d) Alliance Focused. Kivilghan and Coleman (2000) earlier discussed
the similarity between these studies’ identified activities with their own findings and
there appears to be a close relationship between the clusters identified by Kivilghan
and Coleman and this current investigation. Thus, there is a strong consistency
across all of the above studies; this consistency may, once again, help us understand
the core activities or tools involved in conducting psychotherapy.
Another line of research examining common activities in counseling involves
the study of intentions. Hill and O’Grady (1985) defined intentions as “ a therapist’s
rationale for selecting a specific behavior, response, mode technique or intervention
to use with a client at any given moment within a session” (p. 3). This study of
intentions suggest that there were a set of 19 intentions: set limits, get information,
give information, support, focus, clarify, hope, cathart, cognitions, behaviors, self-
control, feelings, insight, change, reinforce change, resistance, challenge,
relationship, and therapist needs. Hill, Helms, Tichenor, Spiegel, O’Grady, and
Periy (1988) found some overlap in the original intentions and integrated the 19
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
105
intentions into a unique set of only 7 intentions: set limits, assess, provide support,
educate, explore, restructure, and promote change. In a separate study, Stiles,
Startup, Hardy, Berkham, Rees, Shapiro, and Reynolds (1996) identified seven
clusters out of the 19 intentions: treatment context, session structure, affect,
obstacles, encouraging change, behavior and cognition-insight.
Comparing the study of intentions with the study of other activities used in
counseling, along with the results from the current investigation continues to lend
support to the idea that there are core activities counselors use during counseling
sessions with clients. The current study used innovative technology which, for the
first time, allowed researchers to access the thought process of novice and expert
counselors in real-time (while observing a counseling session) rather than through
retrospective methods. Results from this study supports and confirms existing
research and seems to further help identify the core issues and activities therapists
engage in while conducting counseling or psychotherapy.
Implication
Counseling is characterized by an ever changing relationship between a client
and a therapist. This relationship incorporates thoughts, feelings and behaviors
which are difficult to measure. The literature discussing expertise was drawn from
studies that could directly measure observable activities representing expertise (e.g.
chess competition or solving math problems). The research literature regarding
counseling expertise has used retrospective methods and loose definitions of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
106
expertise. Discussions about counseling expertise have relied on general findings of
expertise and there have been a limited number of studies which have examined
differences between novice and expert counselors.
The major implication and contribution of the current investigation may lay
in its innovative and rigorous research method. For the first time, this investigation
shows directly that the same type of differences between novices and experts that
have been discussed in the expertise literature are also found in the field of
psychotherapy. The use of RTT made the current study a first of a kind. RTT
enabled the researchers to access novice and expert counselors’ thought processes in
real-time while observing a counseling session. Thus, this study examined the
differences between novice and expert counselors “in action” as opposed to previous
studies which have used retrospective research methods.
This study looked at the differences between novice and expert counselors
from two different levels. Differences between the type of feedback comments
novice counselors and an expert counselor make during live supervision and the
differences between novice and expert counselor raters’ categorization of these
feedback comments into clusters were studied.
This investigation addressed the more loosely defined differences between
expert and novice participants that has occurred in previous psychotherapy research
by applying stringent criteria in selecting novice and expert participants for this
study. This study used criteria discussed by Jennings and Skovholdt (1999) for
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
107
defining expertise: a) at least 15 years of experience, b) qualifications (holding at
least one license), and c) peer-selection. This study also followed Ronnestad and
Jennings (1997) suggestion to eliminate the narrow range of experience differences
between novice and expert counselors often seen in psychotherapy research. They
suggested that research studies use a range difference of at least 10 years of
professional counseling experience between novice and expert participants. As
opposed to all of the studies reviewed in chapters 1 and 2, this investigation applied
all the criteria noted immediately above.
Overall, experts held at least one license for a mean of 22.9 years. Peer
selection was applied as well; all the experts had demonstrated expertise by
contributing to the field as elected presidents or board members of psychological
organizations, authors of books and publications, serving as directors of counseling
centers, or being identified as an expert in a specialty area such as ethics or
multicultural issues. Additionally, the mean difference between novice and expert
counselor raters was approximately 20 years, which is more than the 10 and 15-year
range difference suggested by Jennings and Skovholt (1999) and by Ronnestad and
Jennings (1997).
Limitations of the Study
The major limitation of this study was that the expert’s feedback comment
was drawn from only one supervisor working with several practicum trainees. This
may limit the ability to generalize the expert feedback comments. The expert’s
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
108
supervision style, process oriented comments, and theoretical orientation may have
influence the type of feedback comments made.
Another limitation involves the random selection of the sampled items. The
150 feedback comments that were sorted into categories were randomly selected
from a pool of approximately 2000 statements. The items selected appeared to
represent an unequal distribution of statements in terms of semester/cohort (e.g., fall
98 had 77 statements, spring 2000 had only 28 statements). The distribution between
clusters was erratic and may have affected the data and the relationship between the
type of statements, the semester, and the session sequence. For example, the
Kivlighan and Coleman (2000) study suggested that novice feedback type was
consistent across contextual factors such as semester and session sequence though
this investigation found that novice statement were dependent on the semester and
had only a mild relationship to session sequence. This unequal distribution of the
statements may have contributed to the difference in findings.
In selecting the novice and expert counselor raters the researcher of the current
investigation attempted to adhere not only to the stringent definition of expertise but
also to gender and ethnicity selection bias. The expert counselor rater participants
included five males and five females and Asian American and Caucasian subjects.
However, due to time conflict the researcher was not successful in including African-
American or Latino expert counselors. The novice counselor rater participant pool
included only two males. Furthermore, all subjects with the exception of one Latina
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
109
were white. Some of the differences found in this investigation may have been
influenced by ethnic differences.
Finally, the differences between expert and novice counselors were examined
based on categorization and conceptualization skills. In reality, the goal of
counseling is to help clients achieve their goals and promote change. This study, as
all other counseling expertise studies, did not use a longitudinal research method to
explore differences between novice and expert counselors in terms of promoting
change and helping clients achieve their goals. In other words, the study did not use
counseling outcome measures as a means to assess the differences between novice
and expert counselors.
Recommendations for future Research
Future research should use more than one expert counselor’s feedback
comments to assess differences between novice and expert counselors. Using
several expert counselors who have differing theoretical orientations and supervision
styles would increase the generalizability of the findings. This would also allow
researchers to better assess RTT as a research and training method.
A qualitative study examining how the trainees perceived the type of comments
using RTT may be important in order to assess the trainees experience, their
perceived learning experience modeling expert’s behavior, and their rationale for the
type of feedback comments they have provided. Such a study may help us better
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
110
understand the novice counselor’s learning experience and transition toward
becoming experts themselves.
A longitudinal research design exploring the differences between novice and
expert counselors using counseling outcome measures and client change should be
used. This will more closely resemble the actual counseling expereince. If a similar
study is conducted an attempt should be made to include more ethnic novice and
expert counselor raters, in order to explore the role of ethnic diversity within the field
of counseling expertise.
Future studies may include non-counselors, beginning counselors, advanced
trainees and expert counselors as counselor raters sorting clinical statements into
categories in order to examine whether differences exist in different levels of
expertise and compare these differences with raters that are not associated with the
counseling field at all. This research design may shed light on the transition of
novices toward gaining expertise and compare with non-counseling raters to explore
whether expertise in counseling exists.
Finally, future research should compare the items each novice and expert
counselor rater sorted into each category and explore their rationale and qualities by
which they decided to sort the cards into their respective categories. This type of
study could assess the differences between novice and expert counselors regarding
their knowledge (declarative vs. procedural knowledge) and cognitive errors while
sorting the cards into categories. Experts often have difficulty explaining their
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Ill
procedural/automated knowledge and a study of this sort may shed light on how
experienced and novice clinicians organize their knowledge.
Using an innovative research method to examine a developing field within
the counseling filed was inspiring. I hope that other researchers would continue to
conduct research using RTT to further explore the field of counseling expertise and
other related fields.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
112
REFERENCES
Anderson, J. R. (1981). Cognitive skills and their acquisition. Hillsdale. NJ:
Erlbaum.
Anderson, J. R. (1983). The architecture of cognition. Cambridge. MA:
Harvard University Press.
Benner, P. A., Tanner, C. A., & Chesla, C. A. (1996). Expertise in Nursing
Practice: Caring. Clinical Judgment, and Ethics. New York: Springer Publishing
Company.
Bernard, J. M., & Goodyear, R. K. (1998). Fundamentals of clinical
supervision. Needham Heights, MA: Allyn & Bacon.
Betz, N.E. (1987). Use of discriminate analysis in counseling psychology
research. Journal of Counseling Psychology. 34(4). 393-403.
Binder, J. L. (1993). Is it time to improve psychotherapy training? Clinical
Psychology Review. 13. 301-318.
Binder, J. L. (1999). Issues in teaching and learning time-limited
psychodynamic psychotherapy. Clinical Psychology Review. 19.705-719.
Blocher, D. H. (1983). Toward a cognitive developmental approach to
counseling supervision. The Counseling Psychologist. 11(1). 27-34.
Bloom, B. S. (1985b). Generalization about talent development. In B. S.
Bloom (ed.), Developing Talent in Young People (pp. 507-549). New York:
Ballantine Books.
Borgen, F. H., & Barnett, D. C. (1987). Applying cluster analysis in
counseling psychology research. Journal of Counseling Psychology. 34(4). 456-468.
Brammer, R. (1997). Case conceptualization strategies: The relationship
between psychologist’s experience level, academic training, and mode of clinical
inquiry. Educational Psychology Review. 9(3). 333-351.
Bransford, J. D., Sherwood, R. D., Kinzer, C. K., & Hasselbring, T. S.
(1987). Macro-context for learning: Initial findings and Issues. Applied Cognitive
Psychology. 1.93-108.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
113
Bruning, R.H., Schraw, G. J., & Ronning, R. R. (1999). Cognitive
Psychology and Instruction (Third Edition). NJ: Prentice-Hall.
Chi, M., Glaser, R., & Farr, M. (1988). The Nature of Expertise. Mahwah.
NJ: Erlbaum.
Clark R. E. & Blake. S. (1997) Analyzing cognitive structures and processes
to derive instructional methods for the transfer of problem solving expertise. In S.
Dijkstra and N. M. Seel (Eds.) Instructional Design Perspectives. Volume II. Solving
Instructional Design Problems. Oxford, Pergamon.
Cummings, A. L., Hallberg, E. T., Martin, J., Slemon, A., & Heibert, B.
(1990). Implications of counselor conceptualizations for counselor education.
Counselor Education and Supervision. 3 0 .120-134.
Cunning, N. J., & Stewart, N. R. (1983). Effects of discrimination training
on counselor training response choice. Counselor Education and Supervision. 23.
46-61.
Dawes, R. M. (1994). House of Cards: Psychology and Psychotherapy Built
on Mvth. New York: New York.
Ericsson, K. (1998). Basic capacities ca be modified or circumvented by
deliberate practice: A rejection of talent accounts of expert performance. In Howe,
M. J. A., Davidson, J. W., & Sloboda, J.A. (1998). Innate talents: Reality or Myth?
Behavioral and Brain Sciences. 21.399-442.
Ericsson, K., & Chames, N. (1994). Expert Performance: Its structure and
acquisition. American Psychologist. 49(8). 725-747.
Ericsson, K, Krampe, R. & Tesch-Romer, C. (1993). The role of deliberate
practice in the Acquisition of Expert Performance. Psychological Review. 100(3).
363-406.
Ericsson, K., & Lehman, A. C. (1996). Expert and exceptional performance:
evidence of maximal adaptation to task constraints. Annual Review of Psychology.
47. 273-305.
Ericsson, K., & Smith, J. (1991). Prospects and limits of the empirical study
of expertise: An introduction in K. A. Ericsson & J. Smith (Eds.). Toward a general
theory of expertise (p p . 1-38). Cambridge, MA: Cambridge University Press.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
114
Etringer, B. D., Hillerbrand, E., & Claibom, C. D. (1995). The transition
from novice to expert counselor. Counselor Education & Supervision. 35(11.4-17.
Fuqua, D. R., Johnson, A. W., Anderson, M. W., & Newman, J. L. (1984).
Cognitive methods in counselor training. Counselor Education & Supervision.
24(1), 85-95.
Frank, J. D., & Frank, J. B. (1993). Persuasion and Healing: A Comparative
study of Psychotherapy. Baltimore: MD, John Hopkins University Press.
Ganzach, Y. (1997). Theory and configurality of clinical judgments of
novice and expert psychologists. Journal of Applied Psychology. 82C 6 '). 954-960.
Gick, M. L. (1986). Problem solving strategies. Educational Psychologist.
21,99-120.
Glasser, R. (1992). Expert knowledge and processes of thinking. In
Goodyear, R. K. (1997). Psychological expertise and the role of individual
differences: An exploration of issues. Educational Psychology Review. 9(31.251-
265.
Halpem, D. F. (1992). Enhancing thinking skills in the sciences and
mathematics. Hillside, NJ: Lawrence Erlbaum Associates, Publishers.
Heppner, P. P., Kivlighan, D. M., Burnett, J. w. Berry, T. R., Goedinghaus,
M., Doxsee, D. J., Hendricks, F. M., Krull, L.A., Wright, G. E., Bellatin, A. M.,
Durham, R. J., Tharp, A., Kim, H., Brossart, D. F., Wang, L., Witty, T. E., Kinder,
M. H.., Hertel, J. B., & Wallace, D. L. (1994). Dimensions that characterize
supervisory interventions delivered in the context of live supervision of practicum
counselors. Journal of Counseling Psychology, 41,227-235.
Hillerbrand, E. (1989). Cognitive differences between experts and novices:
Implications for group supervision. Journal of Counseling & Development. 67(5).
293-296.
Hill, C. E., & O’Grady, K.E. (1985). List of therapist intentions illustrated
in a case study and with therapists of varying theoretical orientations. Journal of
Counseling Psychology. 32C 1L 3-22.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
115
Hill, C. E., Helmes, J. E., Tichenor, V., Spiegel, S. B., O’Grady, K.E., &
Perry, E. S. (1988). Effects of therapist response modes in brief psychotherapy.
Journal of Counseling Psychology. 35(3). 222-233.
Holmes, P., Kivlighan, D. M., & Caple, R. (2000). The immediate effects of
live supervisory interventions on counselors treatment adherence on session impact.
Unpublished manuscript. University of Missouri, Columbia: MO.
Howe, M. J. A., Davidson, J. W., & Sloboda, J.A. (1998). Innate talents:
Reality or Myth? Behavioral and Brain Sciences. 21.399-442.
Holloway, E. J. (1987). Developmental models of supervision: Is it
development? Professional Psychology: Research and Practice. 18(31.209-216.
Janelle, C. M., Hillman, C. H., Apparies, R. J, Murray, N. P., Meili, L.,
Fallon, E. A, & Hatfield, B. D. (2000). Expertise differences in cortical activation
and gaze behavior during rifle shooting. Journal of Sport and Exercise Psychology.
22.167-182.
Jennings, L., & Skovholt, T. M. (1999). The cognitive, emotional and
relational characteristics of master therapists. Journal of Counseling Psychology.
46(1), 3-11.
Johnson, W. C., & Heppner, P. P. (1989). On reasoning and cognitive
demands in counseling: Implications for counselor training. Journal of Counseling
and Development. 67.428-429.
Jong, T. & Joolinger, W. R. (1998). Scientific discovery learning with
computer simulations of conceptual domains. Review of Educational Research.
68(2), 179-201.
Kagan, N. (1980). Influencing human interaction-eighteen years with PR.
In A. K. Hess (Ed.), Psychotherapy Supervision: Theory. Research and Practice.
New York: Wiley, 262-286.
Kivlighan, D. M., & Coleman, M. N. (2000, August). Stimulating reflection-
in-action: A taxonomy of reflective comments made by experienced trainers and
novice peers. . In Joan I. Rosenberg (Chair), New clinical training paradigms:
Cognitive psychology, technology, and transfer of knowledge. Symposium
presented at the 108th annual convention of the American Psychological Association.
Washington, DC.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
116
Kivlighan, D. M., & Quigley, S. T. (1991). Dimensions used by experienced
and novice group therapists to conceptualize group process. Journal of Counseling
Psychology. 38.415-423.
Kirsch, I., & Lynn, S. J. (1999). Automaticity in clinical psychology.
American Psychologist. 54(7L 504-515.
Kurpius, D. J., & Morran, D. K. (1988). Cognitive-behavioral techniques and
interventions for application in counselor supervision. Counselor Education &
Supervision. 27(41368-376.
Lamond D. & Famell S. (1998) The treatment of pressure sores: a
comparison of novice and expert nurses' knowledge, information use and decision
accuracy. Journal of Advanced Nursing 27,280-286.
Lehmann, A. C. (1998). Historical increase in expert performance suggest
large possibilities for improvement of performance without implicating innate
capacities. In Howe, M. J. A., Davidson, J. W., & Sloboda, J.A. (1998). Innate
talents: Reality or Myth? Behavioral and Brain Sciences. 21. 399-442.
Lichtenberg, J. W. (1997). Expertise in counseling psychology: A concept
in search of support. Educational Psychology Review. 9 (3 \ 221-237.
Locke, T. E., & Covell, A. J. (1997). Characterizing expert psychologist
behavior: Implications from selected expertise literature. Educational Psychology
Review. 9(3). 239- 249.
Loganbill, C., Hardy, E., & Delworth, U. (1981). Supervision: A conceptual
model. Counseling Psychologist. lOfll. 3-42.
Lorr, M. (1983). Cluster Analysis for Social Scientists: Techniques for
Analyzing and Simplifying Complex Blocks of Data. CA: Jossey-Bass Inc.
Martin, J., Slemon, A. G., Hiebert, B., Hallberg, E. T., & Cummings, A. L.
(1989). Conceptualizations of novice and experienced counselors. Journal of
Counseling Psychology. 36(41. 395-400.
Mayfield, W. A., Kardash, C. M., & Kivlighan, D. M. (1999). Differences
in experienced and novice counselors’ knowledge structures about clients:
Implications for case conceptualization. Journal of Counseling Psychology. 46(4).
504-514.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
117
Moran, D. K., Kupius, D. J., Brack, C. J., & Brack, G. (1995). A cognitive-
skills model for counselor training and supervision. Journal of Counseling and
Development 73f41. 384-393.
O’Byran, K. R. (1993). The relationship between psychological expertise
and clinical assessment: A comparison of novice and expert performance.
Unpublished Doctoral Dissertation. University of Southern California.
O’Byran, K. R., Clark, R. E., & Malakuti, R (1997). Expert and novice
performance: Implications for clinical training. Educational Psychology Review.
9(3), 321-332.
O’Byran, K. R., & Goodyear, R. K. (1997). Client assessment by novice and
expert psychologists: A comparison of strategies. Educational Psychology Review.
9(3), 267-278.
Offfedy, M. (1998). The application of decision making concepts by nurse
practitioners in general practice: Issues And Innovations In Nursing Practice.
Journal of Advanced Nursing. 28(51.988-1000.
Olson, J. R. & Biolsi, K. J. (1991). Techniques for representing expert
knowledge. Chapter 10 in K. A. Ericsson & J. Smith (Eds.). Toward a general
theory of expertise (pp. 240-285). Cambridge, MA: Cambridge University Press.
Patel V. L., Arocha J. F., & Kaufmann D. R. (1994). Diagnostic reasoning
and medical expertise. In The Psychology of Learning and Motivation. (Ed). D
Medin, 30 (187-251). New York: Academic.
Patel, V. L, & Groen, G. J. (1991). The general and specific nature of
medical expertise: A critical look. In K. A. Ericsson & J. Smith (Eds.). Toward a
General Theory of Expertise. New York: Cambridge University Press.
Rigazio-Diglio, S. A., Daniels, T. G., & Ivey, A. E. (1997). Systemic
cognitive developmental-integrative approach to psychotherapy supervision. In C.
E. Watkins, Jr. (Ed.). Handbook of Psychotherapy and Supervision. New York, Jon
Wiley & Sons: 223-245.
Rock, D. L., & Bransford, J. D. (1990). An empirical evaluation of three
components of the tetrahedron model of clinical judgement. Journal of Nervous and
Mental Disease.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
118
Rosenberg, J. I. (1999). From novice to expert using anchored live
supervision: Enhancing clinical training with computer-assisted, real-time feedback.
Unpublished Manuscript.
Schneider, W. (1998). Innate talent or deliberate practice as determinants of
exceptional performance: Are we asking the right question. In Howe, M. J. A.,
Davidson, J. W., & Sloboda, J.A. (1998). Innate talents: Reality or Myth?
Behavioral and Brain Sciences. 21.399-442.
Seligman, M. E. (1995). The effectiveness of psychotherapy: The consumer
report study. American Psychologist 50(12 ). 965- 974.
Skovholdt, T. M., Ronnestand, M. H., & Jennings, L. (1997). Searching for
expertise in counseling, psychotherapy, and professional psychology. Educational
Psychology Review. 9(3). 361-369.
Sternberg, R. J., & Horvath J. A. (1995). A prototype view of expert
teaching. Educational Researcher. 9-17.
Stiles, W. B., startup, M., Hardy, G. E. H., Berkham, M., Rees, A., Shapiro,
D.A., & Reynolds, S. (1996). Therapist session intentions in cognitive-behavioral
and psychodynamic-interpersonal psychotherapy. Journal of Counseling
psychology. 43 (4), 402-414.
Stoltenberg, C. D. (1981). Approaching supervision from a developmental
perspective: The counselor complexity model. Journal of Counseling Psychology.
28,59-65.
Stoltenberg, C. D., & McNeill, B. W. (1997). IDM Supervision: The
Integrated Developmental Model of Supervision. San Francisco, Josse-Bass.
Taylor, C. (1997). Problem solving in clinical nursing practice. Journal of
Advanced Nursing. 26(2). 329-336.
Wampold, B.E., Mondin, G.W., Moody, M., Stich, F., Benson, K., and Ahn,
H. (1997). A meta-analysis of outcome studies comparing Bona Fide
psychotherapies empirically, “ all must have prizes”. Psychological Bulletin. 122HT
203-215.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
119
Appendix A
Informed Consent
University of Southern California
Division of Counseling Psychology
INFO RM ED CONSENT FO R NO N-M EDICAL RESEARCH
CONSENT TO PARTICIPATE IN RESEARCH
A Cluster Analysis of the Differences between Novice and Expert Clinicians based on
Real-Time Training.
You are asked to participate in a research study conducted by Chen Oren, M.A., and Joan I.
Rosenberg, Ph.D. from the Department of Counseling Psychology at the University of
Southern California. The results will contribute to the completion of Chen Oren’s Ph.D.
dissertation. You are selected as a possible participant in this study because you are either a
novice clinician or an expert clinician. Your participation is voluntary.
PURPOSE OF THE STUDY
The purposes of this study are to gather information about the differences in expert
and novice clinician's thought processes, schemas and conceptualizations as they
relate to clinical problems for the purpose of enhancing the training of novice
clinicians. Your participation in this study helps us to provide the field with a
description of how novice and expert clinicians perceive counseling session content,
consequently it will provide us with information on how to better train novice
clinicians to become competent clinicians.
PROCEDURES
If you volunteer to participate in this study, you are asked to complete the following
demographic questionnaire that will take about 5 minutes and a card-sorting task. The time
required for the card-sorting task will be tailored to the individual needs you may have to
complete the task. Please sort the cards based upon any themes or similarities you perceive
among the statements.
POTENTIAL RISKS AND DISCOMFORTS
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
120
There are no potential negative effects, risks, and discomforts from participating in this study.
POTENTIAL BENEFITS AND PAYMENTS FOR PARTICIPATION
You are expected to gain no benefits or financial payments from participating in this study.
INFORMATION ABOUT THE SAMPLE
On the checklist at the end of this consent form you are asked to let us know if you would
like to receive information about the results of this study. You may also choose not to
receive any information. Research is a long and complicated process. Obtaining general
information from a project may take years. Even if there is general information from a
project, there may not be personal information for every participant.
CONFIDENTIALITY
Information obtained in the card sort will only be reported in an aggregated form without
any potential identifiable descriptions connected to individuals. Any information that is
obtained in connection with this study and that can be identified with you will remain
confidential and will be disclosed only with your permission or as required by law.
When the results of the research are published or discussed in conference, no information
will be included that would reveal your identity. The completed card sort information will
be stored in a locked filed cabinet at Dr. Joan Rosenberg’s office. Chen Oren and Dr. Joan
Rosenberg will be the only people who will have access to the data. The data will be
destroyed five years after the completion of the dissertation.
PARTICIPATION AND WITHDRAWAL
You may choose whether to be in this study or not. If you volunteer to be in this study, you
may withdraw at any time without consequences of any kind. You may also refuse to
answer any questions you don’t want to answer and still remain in the study. The
investigator may withdraw you from this research if circumstances arise which warrant
doing so.
IDENTIFICATION OF INVESTIGATORS
If you have any questions or concerns about the research, please feel free to contact Chen
Oren or Joan Rosenberg at (310) 477- 3242 at the Division of Counseling Psychology, USC,
WPH 500, Los Angeles, CA 90089-0031.
RIGHTS OF RESEARCH SUBJECTS
You may withdraw your consent at any time and discontinue participation without
penalty. You are not waiving any legal claims, rights or remedies because of your
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
121
participation in this research study. If you have any questions regarding your rights
as a research subject, contact the office of the Vice Provost for Research, Bovard
Administration Building, Room 300, Los Angeles, CA 90089-4019, (213) 740-6709
or upirb@usc.edu.
SIGNATURE OF PARTICIPANT
I understand the procedures described above. My questions have been answered to
my satisfaction, and I agree to participate in this study. I have been given a copy of
this form.
Name of Participant Phone Number
Signature of Participant Date
Please indicate by checking and initialing the category below what type of
information you want to receive. It is your responsibility to let the investigator
know if your address and/or telephone number changes. The contact
information is in this informed consent form under “Identification of
Investigators”.
□ General Information about what the study found
□____Specific Information about what the study found about me
□ I do not want any information about my sample
SIGNATURE OF INVESTIGATOR
I have explained the research to the subject and answered all of his/her questions. I believe
that he/she understands the information described in this document and freely consents to
participate. My signature as witness certified that the subject signed this consent form in my
presence as his/her voluntary act and deed.
Name of Investigator
Signature of Investigator Date (must be the same as
subject’s)
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
122
Appendix B
Subject General Information Sheet
Novice Information Sheet
1) Code#.
2) Age__
3) Gender M or F (circle one)
4) Race/Ethnicity:________________
5) Prior degree(s) attained (e.g., Ph.D., Psy.D., M.S., M.A., B.S., B.A., etc.):
6) Name of university you are currently enrolled
7) Current degree you are pursuing:__________
From what type of program/department you are earning your highest degree (circle
one)?
a. M.F.T.
b. Clinical Psychology
c. Counseling Psychology
d. Social Work
e. Other (please describe)___________________________________
8) Current year in graduate school
9) What is your theoretical orientation?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
123
Appendix B (continued)
10) Have you completed an introductory course on counseling/therapy skills?
11) Have you already completed your first counseling practicum?
Indicate the type of clients you have worked with thus far and the number of
sessions with each (e.g. 1 adult-8 sessions; 1 child-4 sessions; 1 couple- 5
sessions):
12) If you have any prior or current counseling/paraprofessional experience
(outside of practicum), circle the collective duration of these experiences:
a. 0-6 months d. 2-3 years
b. 6-12 months e. 4 or more years
c. 1-2 years
Describe the nature of any experiences:
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
124
Appendix B (continued)
Subject General Information Sheet
Expert Information Sheet
1) C ode#____________
2) Age_______________
3) Gender M or F (circle one)
4) Race/Ethnicity:________________
5) Degree(s) attained (e.g., Ph.D., Psy.D., M.S., M.A., B.S., B.A., etc.):
6) From what type of program/department you have earned your highest degree
(circle one)?
a. M.F.T.
b. Clinical Psychology
c. Counseling Psychology
d. Social Work
e. Other (please describe)___________________________________
7) University/Hospital/Agency with which you are affiliated:
8) What is your theoretical orientation?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
125
Appendix B (continued)
9) Please describe your licensure (e.g., licensed psychologist, licensed clinical
social worker, licensed M.F.T., etc.)_______________________
10) In what year were you first licensed?__________________
11) Have you been licensed continually since that time?_____________
12) What is your average number of client contact hours per week (include
individuals, couples, family, children, and group therapy, but not assessment)
13) What type of clients/client problems do you work with?
14) Have you always maintained this number of hours/week since licensure? (if
not, please explain).
15) Have there been periods since licensure in which you have not worked with
clients at all? (e.g., teaching only, research only, etc.).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

An adolescent -parent conflict resolution skills training program for ethnically diverse families: A program evaluation study

PDF

Identity in midlife lesbians: A kaleidoscopic view

PDF

Childhood videotaped neuromotor and social precursors of schizophrenia: A prospective investigation

PDF

A phenomenological inquiry into the essential meanings of the most intense experience of religiosity and spirituality

PDF

Atypicality: Benefit and bane. Provocation, trigger, typicality, and the expression of aggression and generalization

PDF

Can individual creativity be enhanced by training? A meta analysis

PDF

Development and validation of the Cooper Quality of Imagery Scale: A measure of vividness of sporting mental imagery

PDF

An integrative analysis of racism and quality of life: A comparison of multidimensional moderators in an ethnically diverse sample

PDF

Emotion regulation as a mediator between family-of-origin aggression and marital aggression

PDF

Gender differences in motivation for sexual intercourse: Implications for risky sexual behavior and substance use in a university and community sample

PDF

Change beliefs and academic outcomes: A construct validation

PDF

Impact of language and culture on a neuropsychological screening battery for Hispanics

PDF

Anorexia nervosa: A developmental perspective

PDF

Concordant and discordant drug use in intimate relationships: A longitudinal study

PDF

Gender differences in symptom presentation of depression in primary care settings

PDF

Considering the impact of culture on the therapeutic relationship: A look at Latino school-age children in a special education setting

PDF

Heterosexuals' attitudes toward lesbians and gay men and the attitude functions they serve: Correlates, stability and gender differences

PDF

A daily diary approach to compare the accuracy of depressed and nondepressed participants' estimation of positive and negative mood: A test of the depressive realism hypothesis

PDF

A longitudinal examination of gender and ethnicity across two models of community treatment for individuals with schizophrenia and related disorders

PDF

Associations and mechanisms among attention deficit hyperactivity disorder symptoms, cognitive functioning, and drinking habits

Asset Metadata

Creator Oren (Opochinsky), Chen Zvi (author)

Core Title A cluster analysis of the differences between expert and novice counselors based on real time training

School Graduate School

Degree Doctor of Philosophy

Degree Program Counseling Psychology

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag OAI-PMH Harvest,psychology, clinical

Language English

Contributor Digitized by ProQuest (provenance)

Advisor Rosenberg, Joan (committee chair), Brekke, John (committee member), Clark, Richard (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c16-200116

Unique identifier UC11339081

Identifier 3065829.pdf (filename),usctheses-c16-200116 (legacy record id)

Legacy Identifier 3065829.pdf

Dmrecord 200116

Document Type Dissertation

Rights Oren (Opochinsky), Chen Zvi

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA