University of Southern California Dissertations and Theses

Collaborative stimulation in team design thinking

(USC Thesis Other)

Collaborative stimulation in team design thinking

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

Collaborative Stimulation in Team Design Thinking

A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
DEPARMENT OF AEROSPACE AND MECHANICAL ENGINEERING
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(MECHANICAL ENGINEERING)
December 2013

Copyright 2013 Jonathan Sauder
ii

Epigraph

“Two people are better off than one, for they can help each other succeed.”

Old Jewish Proverb

iii

Acknowledgements
Isaac Newton said in a letter "If I have seen a little further it is by standing on the
shoulders of giants", and the same couldn't be more true for the work presented here.
The first and primary giant supporting this work is my research Advisor, Prof. Yan Jin.
Many of the concepts presented here are ones which he suggested during our many long, but
invigorating discussions. Without his guidance, patience, wisdom, and encouragement this work
would have never been completed. I would also like to thank the dissertation committee, Prof.
Geoff Shiflett and Prof. Stephen Madigan, for vetting this work and contributing various
insights.
Another giant who specifically influenced this work is Dr. Oren Benami, through his
creative cognition approach to conceptual design. His many years of effort provided the
foundation for this research. I would also like to thank my lab mates, Dr. Chang Chen, Dr.
Winston Chiang, Newsha Khani, James Humann, Cherry Liu and Rajendra Duwarahan for all
their insights and inspiring discussions.
In addition to those who have influenced my research from an academic side, I would
like to thank my parents, Frank and Kathy Sauder. My father proof-read a number of my papers,
and encouraged me to keep going a number of times when I was ready to quit. My mother
provided the educational foundation for me to exceed to the point of getting into the Ph.D.
program, as she was my instructor from kindergarten through high school.
I would like to thank my wife, Kali Sauder, who has been a constant supporter of my
work, encouraging me each step of the way. She also undertook the thankless task of
iv

proofreading my entire dissertation word for word, greatly improving the readability. Finally, all
glory goes to Him who reveals mysteries in this work not because of any wisdom that I have, but
in order that understanding may be made known to you.
This work is supported in part by the National Science Foundation under Grant No.
CMMI-1131422. Any opinions, findings, and conclusions or recommendations expressed in this
paper are those of the authors and do not necessarily reflect the views of the National Science
Foundation.

v

Table of Contents
Epigraph ........................................................................................................................................ ii
Acknowledgements ...................................................................................................................... iii
List of Tables ................................................................................................................................ ix
List of Figures ................................................................................................................................ x
Abstract ........................................................................................................................................ xii
1 Introduction ........................................................................................................................... 1
1.1 Motivation and Background ........................................................................................... 1
1.2 Research Approach and Issues........................................................................................ 3
1.3 Thesis Overview ............................................................................................................. 4
1.4 Thesis Organization ........................................................................................................ 4
2 Related Work ........................................................................................................................ 6
2.1 Introduction ..................................................................................................................... 6
2.2 Engineering Design Theory and Methodology ............................................................... 7
2.2.1 Systematic Design View ............................................................................................. 7
2.2.2 Axiomatic Design ....................................................................................................... 8
2.2.3 Idea Generation Approaches ..................................................................................... 10
2.3 Creative Cognition ........................................................................................................ 13
2.3.1 General Theory ......................................................................................................... 13
2.3.2 Extended to Design ................................................................................................... 14
2.3.3 Protocol Analysis ...................................................................................................... 15
2.4 Group Creativity ........................................................................................................... 16
2.4.1 Aggregate Models ..................................................................................................... 16
2.4.2 Process Models ......................................................................................................... 17
2.4.3 Creativity Metrics ..................................................................................................... 19
2.5 Summary ....................................................................................................................... 21
3 Merging Group Creativity and Creative Cognition ........................................................ 23
3.1 Introduction ................................................................................................................... 23
3.2 Design Contributions of Creative Cognition ................................................................ 23
3.3 Design Contributions of Group Creativity .................................................................... 24
3.4 Group Creativity’s and Creative Cognition’s Gap........................................................ 25
3.4.1 Existing Research...................................................................................................... 26
3.5 The Proposed Interactive Cognition Approach ............................................................. 27
3.5.1 Proposal of Collaborative Stimulation ...................................................................... 27
vi

3.6 Research Questions ....................................................................................................... 28
4 Collaborative Stimulation: Approach, Framework and Types ...................................... 30
4.1 Introduction ................................................................................................................... 30
4.2 Creative Cognition Foundation ..................................................................................... 30
4.2.1 Design Entities .......................................................................................................... 31
4.2.2 Thought Processes .................................................................................................... 31
4.2.3 Design Operations ..................................................................................................... 32
4.3 Relevant Concepts from Group Creativity ................................................................... 32
4.4 Collaborative Stimulation Framework .......................................................................... 33
4.5 Types of Collaborative Stimulation .............................................................................. 35
4.5.1 Prompting .................................................................................................................. 36
4.5.2 Seeding ...................................................................................................................... 36
4.5.3 Correcting ................................................................................................................. 37
4.5.4 Clarifying .................................................................................................................. 37
4.5.5 Influence of Each Collaborative Stimulation Type .................................................. 38
4.6 Limitations .................................................................................................................... 39
4.6.1 Generative and Exploratory Processes ...................................................................... 40
4.6.2 Group Size ................................................................................................................ 40
4.6.3 Ignoring Social Inhibitions and Procedural Issues.................................................... 41
4.7 Summary ....................................................................................................................... 41
5 Developing an Experimental Approach- Two Pilot Experiments .................................. 43
5.1 Introduction ................................................................................................................... 43
5.2 Past Approaches to Analyzing Collaboration ............................................................... 43
5.3 Core Empirical Challenges ........................................................................................... 44
5.4 Pilot Experiment 1 – Concurrent Collaborative Protocol Analysis .............................. 45
5.4.1 Participants ................................................................................................................ 46
5.4.2 Task and Materials .................................................................................................... 46
5.4.3 Experiment Design.................................................................................................... 47
5.4.4 Procedure .................................................................................................................. 47
5.4.5 Results ....................................................................................................................... 48
5.5 Pilot Experiment 2 – Retrospective Collaborative Protocol Analysis .......................... 49
5.5.1 Participants ................................................................................................................ 49
5.5.2 Task and Materials .................................................................................................... 50
5.5.3 Experiment Design.................................................................................................... 51
5.5.4 Procedure .................................................................................................................. 51
5.5.5 Results and Analysis ................................................................................................. 53
5.6 Summary ....................................................................................................................... 55
6 Evaluation of Collaborative Stimulation: Experiment, Analysis, and Results ............. 56
6.1 Introduction ................................................................................................................... 56
6.2 Research Questions to be Answered ............................................................................. 56
6.3 Experimental Methods .................................................................................................. 57
vii

6.3.1 Subjects ..................................................................................................................... 57
6.3.2 Task and Materials .................................................................................................... 57
6.3.3 Experiment Design.................................................................................................... 58
6.3.4 Procedure .................................................................................................................. 59
6.4 Applying Protocol Analysis .......................................................................................... 60
6.4.1 Identifying Episodes and Segmenting ...................................................................... 61
6.4.2 Coding Scheme ......................................................................................................... 62
6.4.3 Example Coding........................................................................................................ 65
6.5 Analyzing the Novelty of Ideas .................................................................................... 69
6.5.1 Quantifying Idea Novelty ......................................................................................... 69
6.5.2 Example Novelty Analysis ....................................................................................... 70
6.6 Results ........................................................................................................................... 71
7 Observed Collaborative Stimulation Patterns ................................................................. 74
7.1 Introduction ................................................................................................................... 74
7.2 Collaborative Stimulation and Thought Processes ....................................................... 74
7.3 Collaborative Stimulation and Novelty ........................................................................ 79
7.4 Collaborative Stimulation and Analogies ..................................................................... 84
7.5 BICB Scores and Stimulation Frequency ..................................................................... 86
7.6 Individual and Collaborative Stimulation of Memory Retrieval .................................. 90
8 Implications for Collaboration and Recommendations .................................................. 93
8.1 Introduction ................................................................................................................... 93
8.2 Development of Questioning Techniques ..................................................................... 93
8.3 Encouraging Early Collaborative Stimulation .............................................................. 94
8.4 Creative Diversity Training .......................................................................................... 97
8.5 Enhancing Memory Retrieval Stimulation ................................................................... 99
9 Contributions and Future Work ..................................................................................... 100
9.1 Contributions............................................................................................................... 100
9.1.1 CTS Model and Collaborative Stimulation............................................................. 100
9.1.2 Retrospective Protocol Analysis ............................................................................. 101
9.1.3 Recommendations for Creative Collaborative Design ........................................... 101
9.2 Future Work ................................................................................................................ 102
9.2.1 Collaborative Stimulation and GSP Model............................................................. 102
9.2.2 Experimental Methods and Analysis ...................................................................... 103
9.2.3 Interventions and Industry Applications ................................................................. 104
10 Bibliography ...................................................................................................................... 106
11 Appendix A- Experimental Material .............................................................................. 118
11.1 Problem 1 .................................................................................................................... 118
11.2 Problem 2 .................................................................................................................... 118
viii

12 Appendix B- Protocol Analysis Examples ...................................................................... 119
12.1 Raw Transcripts Divided by Episodes ........................................................................ 119
12.1.1 Collaborative Dialog ........................................................................................... 119
12.1.2 Collaborator 1 Retrospective .............................................................................. 120
12.1.3 Collaborator 2 Retrospective .............................................................................. 121
12.2 Segmented and Coded Episodes for Episodes 3-4 ...................................................... 123
12.2.1 Collaborative Dialog ........................................................................................... 123
12.2.2 Collaborator 1 Retrospective .............................................................................. 124
12.2.3 Collaborator 2 Retrospective .............................................................................. 125
ix

List of Tables
Table 5-1 Pilot experiment 2 results ............................................................................................. 54
Table 5-2 Pilot experiment comparison ........................................................................................ 55
Table 6-1 Coding Scheme ............................................................................................................. 64
Table 6-2 Segmented and coded transcripts ................................................................................. 67
Table 6-3 Collaborative stimulation results .................................................................................. 71
Table 6-4 Novelty results .............................................................................................................. 72
Table 6-5 Team results.................................................................................................................. 73
Table 7-1 Collaborative stimulation and thought processes relationship strength matrix ............ 78

x

List of Figures
Figure 1-1 Areas of collaboration research ..................................................................................... 2
Figure 2-1 Research focus .............................................................................................................. 6
Figure 3-1 An interactive cognition approach .............................................................................. 27
Figure 4-1 Generate, Stimulate Produce (GSP) Model (Jin & Benami, 2010) ............................. 31
Figure 4-2 Collaborative thinking stimulation (CTS) model ........................................................ 34
Figure 4-3 A breakdown of stimulation in team design ............................................................... 36
Figure 5-1 Pilot experiment 1 interface ........................................................................................ 46
Figure 5-2 Pilot experiment 1 process .......................................................................................... 48
Figure 5-3 Pilot experiment 2 design ............................................................................................ 51
Figure 5-4 Pilot experiment 2 procedure ...................................................................................... 53
Figure 6-1 Experiment design ....................................................................................................... 58
Figure 6-2 Experiment Process ..................................................................................................... 59
Figure 6-3 Transcript, in episodes and segments .......................................................................... 61
Figure 6-4 Skateboard locking arm and lock location .................................................................. 68
Figure 7-1 Probability of collaborative stimulation stimulating specific thought processes ........ 75
Figure 7-2 Percent of thought processes stimulated by each collaborative stimulation type ....... 76
Figure 7-3 Stimulation and average novelty ................................................................................. 79
Figure 7-4 Distribution of stimulated ideas .................................................................................. 80
Figure 7-5 Working principles average novelty ........................................................................... 81
Figure 7-6 Collaborative stimulation and hierarchy levels ........................................................... 82
xi

Figure 7-7 Average concept novelty per hierarchy level .............................................................. 83
Figure 7-8 Structural and surface analogies .................................................................................. 85
Figure 7-9 Inventive and explanatory analogies ........................................................................... 85
Figure 7-10 Analogies and collaborative stimulation type ........................................................... 86
Figure 7-11 Team avg. BICB vs. stimulation frequency .............................................................. 87
Figure 7-12 Individual BICB vs. stimulation frequency .............................................................. 87
Figure 7-13 Percent Design Entity Inspired.................................................................................. 88
Figure 7-14 Experimental stimulation breakdown ....................................................................... 90
Figure 7-15 Collaborative stimulation: control vs. experimental ................................................. 91
Figure 8-1 When design and collaboration start ........................................................................... 95

xii

Abstract
In both design education and industry it is often assumed that collaboration encourages
creativity, although research in brainstorming has shown that this is not always the case. There is
an opportunity to develop more effective collaborative methods, based on research. This work
lays the groundwork for new methods, by extending creative cognition to group creativity by
proposing an interactive cognition approach. Specifically, designers' interactions, through
questions and shared design entities, stimulate creativity relevant thought processes. Various
types of collaborative stimulation are identified, namely, prompting, seeding, correcting, and
clarifying. An experiment using collaborative retrospective protocol analysis was developed to
determine if the hypothesized types of collaborative stimulation exist and how influential each
type was by exploring stimulation patterns. Patterns were found between collaborative
stimulation and thought processes, novelty of ideas, and analogies. Specifically, prompting had a
strong relationship with memory retrieval, seeding and correcting had strong relationships with
transformation, and clarifying had moderate to strong relationships with memory retrieval,
association, and transformation. Also, seeding and correcting were found to create the most
novel ideas. Additional patterns were found between the past creative experience of individuals
and stimulation frequency, and designers were found to be stimulated more often in collaborative
than individual settings. The implications of these patterns for developing new methods to
promote group creativity are discussed.
1

1 Introduction
1.1 Motivation and Background
Two key issues face modern engineering designers: complexity and globalization. The increased
complexity of current design projects means increased need for collaboration among multiple
engineers, teams, and companies. At the same time, globalization has made it necessary for
companies to continuously innovate in order to remain competitive. Therefore engineers must
produce designs which are creative. These two issues provide the motivation to explore how
collaboration (which is already required) can encourage creativity. This raises the motivating
question of this research, “How can the collaborative design process be made more effective for
creativity?”
Studies of group creativity can be categorized into types of control and size of
collaboration groups, as shown in Figure 1-1. A homogenous group is composed of individuals
who are equally positioned and have neither specific authority nor power from a governing body.
In an authority based case, there is a “boss” who is in charge of most decisions. An internal
power situation occurs when there are elements of either political or technical power at play in
the group, and not all group members are equal. Outside power involves individuals outside the
group who have direct authority over the inner workings of the group. Along the dimension of
size and structure, the first is a small group, which contains several people without a coordinator.
The second is a slightly larger team with a coordinator. An organization forms when there is a
formal structure and a set of shared rules. Most studies in group creativity are devoted to the
2

areas of professional teams and business innovation. More recently, the design community has
focused on design groups, which are smaller and are simpler in regards to control mechanisms.

Figure 1-1 Areas of collaboration research
The research areas of business innovation, professional teams, and design groups have
proposed many solutions for making the collaborative design process more effective. These
solutions fit into the two categories of organizational approaches, which suggest organizational
structures and policies to promote creativity (Amabile, Conti, Coon, Lazenby, & Herron, 1996;
Wilde, 2010; Woodman, Sawyer, & Griffin, 1993) and technique approaches, which suggest
collaborative methods for creativity (Gallupe et al., 1992; Osborn, 1957; Warr & O’Neill, 2005).
While many of these approaches do improve idea generation, some, like brainstorming, have
questionable effects (Diehl & Stroebe, 1987). There is an opportunity to discover new effective
ways of collaborating by taking a different approach. Research on creative cognition (Finke,
Ward, & Smith, 1996) in the area of engineering design has generated new insights regarding
design methodologies by taking a deeper approach which considers the individual’s thinking
processes (Jin & Benami, 2010; Shah, Smith, Vargas-Hernandez, Gerkens, & Wulan, 2003).
However, creative cognition has not yet explored the collaborative setting. The goal of this
3

research is to extend creative cognition to collaboration, so more effective collaborative methods
can be developed from its insights.
1.2 Research Approach and Issues
In the past, several authors have made steps towards combining cognition and
collaboration (Nijstad & Stroebe, 2006; Stempfle & Badke-Schaub, 2002). While these works
have made advancements in their own way, they have not focused on how collaboration
influences multiple specific thought processes. This research proposes an interactive cognition
approach (Figure 1-1) which examines how a designer’s thinking processes are influenced by
other designers in a group setting. This approach led to the proposal of collaborative stimulation:
the idea that thought processes are collaboratively stimulated through shared design entities.
The proposed idea of collaborative stimulation has the goal of providing a firm base for
developing methods which will lead to the collaborative stimulation of thought processes for
improved group creativity. It accomplishes this by combining knowledge from current
approaches in group creativity and creative cognition; using group creativity to identify types of
collaborative interactions and creative cognition to identify thought processes. The challenge is
to capture relationships between collaborative interactions and thought processes in order to
identify how collaboration stimulates creativity relevant thought processes, which will provide a
base for developing new methodologies.

4

1.3 Thesis Overview
The overall objective of the proposed research is to understand how collaboration influences
the thought processes which have been identified as related to creativity. The specific objectives
are:
1. To explore how collaboration stimulates thought processes, by extending creative
cognition to collaboration by proposing an interactive cognition approach.
2. To develop an experimental methodology to test and evaluate the concept of
collaborative stimulation.
3. To identify patterns between collaborative stimulation and specific thought processes and
creativity metrics.
Accomplishing these research objectives will provide a foundation for developing new methods
to increase stimulation of thought processes allowing for improved group creativity.
1.4 Thesis Organization
A brief summary of each chapter in the dissertation is provided below.
Related Work: Reviews relevant research in the areas of engineering design theory and
methodology, creative cognition and group creativity.
Merging Group Creativity and Creative Cognition: Identifies the design contributions from
the two foundations for this work, and identifies a research gap between them. Describes how an
interactive cognition approach can bridge the gap and resulting research questions.
5

Collaborative Stimulation: Approach, Framework, and Types: Develops the concept of
collaborative stimulation, shows CTS (Collaborative Thinking Stimulation) model framework
and specifies the types of collaborative stimulation. Limitations are also mentioned.
Developing and Experimental Approach- Two Pilot Experiments: Describes developing an
experimental method which identifies thought process and collaborative stimulation.
Evaluation of Collaborative Stimulation: Experiment, Analysis, and Results:- Explains the
experiment to verify collaborative stimulation, discusses protocol analysis, and states results.
Observed Collaborative Stimulation Patterns: Identifies patterns found between collaborative
stimulation and thought processes, novelty, analogies, and BICB scores.
Implications for Collaboration and Recommendations: Defines practical implications of
collaborative stimulation findings and suggests best practices for industry.
Contributions and Future Directions: Lists the contributions this work has made to relevant
fields and discusses future work.

6

2 Related Work
2.1 Introduction
Literature that is relevant to this research comes from three key areas, engineering design theory
and methodology, creative cognition, and group creativity. From the area of engineering design
theory and methodology, axiomatic and systematic views of design and various idea generation
methods are explored. In the area of creative cognition the focus is the general underlying theory
of creative cognition, its extension to design, and the related experimental method, protocol
analysis. Aggregate models, process models and creativity metrics are the focus in the area of
group creativity. This is displayed in the Venn diagram in Figure 2-1.

Figure 2-1 Research focus
7

2.2 Engineering Design Theory and Methodology
Design theory and methodology explores how the engineering design process occurs, and
develops methods to improve it. The aim of this research is to contribute to the field design
theory and methodology by providing information to develop more effective idea generation
techniques. In the literature review, two of the most popular views on design methodology,
systematic design and axiomatic design, are examined. Then various idea generation techniques
developed to encourage creativity in the design process are explored.
2.2.1 Systematic Design View
The systematic view of design is a methodology that is based on observations of the real design
process from both the industry and academic settings (Pahl & Beitz, 1996). Four main phases of
the product development processes were identified: planning and task clarification, conceptual
design, embodiment design, and detail design.
Planning and task clarification occurs at the beginning of the project. First, the task needs
to be clarified by collecting details on existing constraints and by clearly outlining the
requirements which must be fulfilled. These activities should generate a requirements list which
will guide the rest of the design process.
Conceptual design occurs when the requirements list is developed into a principle
solution. To develop the principle solution, first the essential problems are defined and function
structures solving these problems are created. Working principles which will suitably fulfill these
function structures are then identified and combined into a single working structure. This
structure is then evaluated against both technical and economic criteria to determine if it is an
8

adequate principle solution. This dissertation focuses on collaboration occurring in the
conceptual design phase.
Embodiment design takes the principle solution from the concept phase and develops it
into a layout which lines up with both technical and economic criteria. Usually, multiple
preliminary layouts are created and then combined or eliminated to form the final definitive
layout. The final layout produced by this phase must be detailed enough to check the strength,
function, spatial constraints and cost of the design.
Detail design takes place when the layout is developed to create a final design. After this
stage is completed, all the forms, dimensions, materials, production methods, and costs are
determined. The result of this phase is product documentation, including engineering drawings
which define the build specifications of the design.
2.2.2 Axiomatic Design
Axiomatic Design (Suh, 1990, 2001) is a framework which consists of two key concepts, a
design space composed of layers and domains and two design axioms for guiding design
decision making. Layers are defined by the level of abstraction, and total number of layers
defines the complexity of the design (more complex designs have more layers). Domains are
primary categories in the product development process. They consist of the customer domain, the
functional domain, the physical domain, and the process domain.
In the customer domain, customer needs (CN’s) are identified forming a list of what the
end user wants. These CN’s inspire functional requirements (FR’s), existing in the functional
domain. FR’s determine what functions the design must perform in order to fulfill the CN’s. The
FR’s are used to develop design parameters (DP’s) calling out what the design will do and look
9

like to fulfill the FR’s. DP’s exist in the physical domain. Finally DP’s are fulfilled by the
process variables (PV’s) which exist in the process domain. PV’s are specific information that
completes the design, such as dimensions or materials.
The development of a design in the axiomatic process occurs in a “zig-zag” motion
between domains and depth of detail. The “zig-zag” process is guided by two design axioms
which determine the best designs and are the second key concept of axiomatic design. The first is
the independence axiom and the second is the information axiom. The independence axiom
states each design element should be independent of all others, such that elements such as
functional requirements or design parameters do not overlap. This allows for the optimization of
each element independently. The information axiom states that if deciding between two design
concepts which meet the first axiom, the concept which has the least uncertainty should be
chosen.
The axiomatic design process lays out a methodology for the development of ideas and
for the interactions between CN’s, FR’s, DP’s and PV’s. As this research specifically explores
early design, most of the focus will exist on CN’s, FR’s, and DP’s. CN’s will define the problem,
where as FR’s and DP’s create design entities, where collaborative interactions are observed. It
should be noted, while the development of engineering design methodologies is important, in
reality, research has found that collaborative work tends toward not following a prescribed
methodology (Stempfle & Badke-Schaub, 2002). Instead, collaborating designers tend to jump to
a solution much more quickly, skipping over steps which would have them considering more
options. Therefore, while design methodologies provide guidance, it is important to understand
the thinking behavior of designers in collaborative settings.
10

2.2.3 Idea Generation Approaches
Many tools, methods, and organizational structures have been suggested to assist in the
generation of creative ideas. They can be organized into three categories, individual technique
approaches, collaborative technique approaches, and organizational approaches.
2.2.3.1 Individual Technique Approaches
A number of individual idea generation techniques were proposed along with Pahl and Beitz’s
(1996) systematic approach to design. They include basic approaches used frequently by
individuals and complex methods used less often. Examples of basic approaches are analysis
(studying elements in the design and their interrelationships) and synthesis (fitting multiple parts
together to create something new). Examples of complex approaches are the method of forward
steps (where as many different solutions to a problem as possible are generated), the method of
backwards steps (where the final goal is considered, and then the possible ways that goal could
be accomplished are backwardly generated) and design catalogs (which contain many types of
solutions from machine elements to materials).
Like design catalogs, TRIZ (acronym for "theory of inventive thinking" in Russian) is a
history-based idea generation technique (Altshuller, 1984) developed after the investigation of
thousands of patents. However, unlike design catalogs, it prescribes a methodology by having
engineers examine contradictions which occur in a designs (for example, strong but lightweight).
It then provides the designer with ideas to solve these contradictions from past solutions which
come from patent examination.
Other methods, instead of providing a prescribed way to generate ideas, provide a
framework to encourage idea generation. The morphological matrix provides a method to
11

identify multiple potential solutions to different functions a design must fulfill, and then combine
these solutions into new ideas (Ritchey, 1998). Fishbone diagrams, where branches off a key
idea are identified, is a method which provides an organized way of exploring all the aspects of a
problem or solution (Michalko, 2001). Random stimuli of unrelated ideas has been suggested as
a method to encourage new ways of thinking (De Bono, 1970).
2.2.3.2 Collaborative Technique Approaches
In the area of collaborative design, many idea generation techniques have been established
to increase creativity. Perhaps the most well known is brainstorming. Brainstorming can either
occur in face-to-face groups (Osborn, 1957) or electronically (Gallupe et al., 1992), and consists
of individuals stating all potential solutions to a problem as they think of them. The 6-3-5
method, similar to brainstorming, is another popular idea generation technique (Rohrbach, 1969).
Six participants generate three ideas, which they pass five times. During each pass, the
participants take the ideas they have been given, and develop the concepts further.
C-sketching has been developed specifically for the field of design (Shah, Vargas-
Hernandez, Summers, & Kulkarni, 2001). It is similar to the 6-3-5 method, except there are five
designers who work on one sketch at a time and pass the sketches four times. Each individual
modifies the sketch passed to them during each round. For design, the C-sketch method has been
found to be more effective than the 6-3-5 method.
Other collaborative idea generation techniques have suggested each design team member
have a notebook which they keep for a period of time and then swap with another member in the
design team (Michalko, 2001). With the start of the digital age, many technological tools have
been developed to assist group creativity. Various studies have explored different types of
12

technological collaboration tools. In particular, there has been a focus on using tabletop screens
and projections (Sundholm, Artman, & Ramberg, 2004; Terrenghi, Fritsche, & Butz, 2006; Warr
& O’Neill, 2005). Personal electronic devices like tablets and PDA’s (Warr & O’Neill, 2005)
and Smart Boards (Sundholm et al., 2004) are also a popular tools for collaboration.
2.2.3.3 Organizational Approaches
Organizational approaches consist of recommending organizational structures and policies which
promote creativity. Amabile et al. (1996) proposes there are several key elements in
organizations which influence creativity: encouragement of creativity, freedom or autonomy,
adequate resources, pressures, and organizational impediments. Each was empirically found to
be positively related to creativity, except for organizational impediments (e.g. policies which
discourage creativity and experimentation) and negative pressures (there are also positive
pressures for creativity). West (2002) also mentions the importance of the organizational climate
and the necessity of intra-group safety within the project team itself. Creativity will not occur if
team members cannot trust each other. Woodman et. al. (1993) interestingly demonstrates that
rewards will actually decrease creativity if the rewards are rigorously based on the outcomes.
This is because the risk of not receiving a reward becomes too high to encourage creativity.
McGrath and Krackhardt (2003) examined how networks can be used to influence
organizational change. Managers must get support in order to spur an organization’s adoption of
creative ideas. Developing ties across departments, introducing innovation at the periphery of the
organization, and utilizing friends of randomly selected organizational members as change
agents are methods which can ease innovation adoption across an organization.
13

In the area of team design, Sosa and Marle (2010) found the best way to predict if
individuals would stimulate a creative idea in collaborative design was not the creative results of
their past interactions, but rather how their peers rated the creativity of their interactions. Wilde
(2010) suggests setting up teams with diverse problem solving approaches to enhance creativity.
Woodman et. al. (1993) also predicts team diversity leads to creativity. However, Polzer, Milton
and Swann (2002) caution that team diversity is only positive if interpersonal congruence (how
well group members can understand each other’s perspectives) is high. Otherwise, diversity will
negatively impact creativity.
2.3 Creative Cognition
The creative cognition approach is one of the two foundations this dissertation is built on. The
review starts by examining general work on creative cognition, and then proceeds to explore
work which has extended creative cognition to design. The section concludes with an overview
of work on the subject of protocol analysis, a necessary method to conduct experiments related to
creative cognition.
2.3.1 General Theory
The creative cognition approach was introduced by Finke, Ward and Smith (1996), and
emphasized creativity results from the same thought operations applied in everyday life. Their
work proposes the Genplore model which divides creative thought processes into generative
processes (diverging to more ideas) and exploratory processes (converging to fewer ideas). A
divergent/convergent cycle exists between these two types of processes (generative processes
lead to exploratory processes, which then lead to more generative processes). The ways in which
14

ideas are generated, interpreted and explored are governed by constraints. As the cycle continues
entities move from being initial ideas to fully formed concepts until a final solution is produced.
Nijstad and Stroebe (2006) take a cognitive approach to creative idea generation,
although they limit their exploration to the process of memory retrieval, or how memories
relevant to the problem are retrieved from long term memory to working memory. When an
image is identified as relevant it is brought into the working memory and is used to generate a
potential solution. In exploring how collaboration influences an individual’s thought processes,
they found taking turns interferes with both stages of the process. Furthermore, ideas mentioned
by collaborators help to activate problem relevant information, and cognitive failures
(incomplete thought processes) are critical in determining a brainstorming session’s participant's
persistence, enjoyment, and satisfaction.
2.3.2 Extended to Design
Benami and Jin (2002; also see Jin & Benami, 2010) extended creative cognition to the
design process. They created the cyclical generate-stimulate-produce (GSP) model of the
creative design process. The cycle consists of design operations which generate design entities,
which stimulate cognitive processes, which produce more design operations. The GSP model
divides cognitive processes into generative (divergent) and exploratory (convergent) processes.
The cycles occur in a divergent and convergent manner until a final design is reached. Their
work forms the core foundation for collaborative stimulation.
There have been additional extensions of creative cognition to design. Shah et al. (2003)
developed an approach to align experiments occurring in creative cognition research with those
occurring in design. Specifically, they focused on incubation (testing the effect of taking a break
15

from a design problem in both the laboratory and design setting) with the hope of expanding to a
more complete design ideation model in the future. Chusilp and Jin (2006) take a cognitive
approach which focuses on the iterations of specific processes in design. The three iteration
loops of problem redefinition, idea stimulation, and concept reuse, are identified. These loops
iterate between the design stages of analyze, generate, compose and evaluate. Different from the
afore-mentioned GSP model, generative thought processes are divided into the stages of generate
and compose. Generate consists of the thought process of memory retrieval, where as the
compose stage consists of the thought processes of transformation and association.
2.3.3 Protocol Analysis
A common approach to the investigation of thought processes during an activity is to use
protocol analysis (Cross, Christiaans, & Dorst, 1997). To conduct protocol analysis, subjects
verbalize their thoughts while they are performing the activity being studied. Their verbalized
thoughts are then transcribed and a coding scheme is applied to this transcript. The coding
scheme provides a way to uniformly identify thought processes occurring in the transcript. To
ensure transcripts are encoded correctly, intercoder reliability is used. Multiple coders apply the
coding scheme to the same transcript and then the difference between the encoded transcripts is
compared. An agreement of 70% or higher is generally considered acceptable (van Someren,
Barnard, & Sandberg, 1994). There are a number of design studies which use protocol analysis in
individual (e.g. Gero & Mc Neill, 1998; Jin & Benami, 2010; Suwa, Purcell, & Gero, 1998) and
collaborative settings (Stempfle & Badke-Schaub, 2002) to identify thought patterns. Also, the
analysis of verbal protocols has been used in design to identify aspects like uncertainty (Schunn,
2010) or social interactions (Cross & Clayburn Cross, 1995).
16

Latent semantic analysis (LSA) is another approach involving the detailed examination of
a text. It is a purely computational approach focusing on conversation context, rather than word
order or syntax (Dong, 2005). LSA can be applied to the dialog of a conversation (Dong,
Kleinsmann, & Deken, 2013) or to individual progress statements taken in the middle of the
design process (Fu, Cagan, & Kotovsky, 2010). However, the use of LSA in design is focused on
comparing mental models (Dong et al., 2013; Fu et al., 2010), not thought processes.
2.4 Group Creativity
Group creativity is the second area of research foundational to this work. While there are a
number of collaborative idea generation techniques which have already been covered, this
section focuses on models of group creativity. The models can be divided into two categories:
aggregate models and process models. Creativity metrics, or measures of creativity, are also
discussed in this section.
2.4.1 Aggregate Models
Aggregate models view group creativity as the sum of each team member’s individual creative
contribution, sometimes with a multiplier effect. Pirola-Merlo and Mann (2004) propose a model
which starts with each individual’s creativity. Influencing each individual’s creativity is the
organization’s climate, created by the perspectives of the work environment held by each team
member. The individual creativity is summed into team creativity which is then summed over
time into the team’s total creative output. Because most projects have a long duration, each
individual’s output and therefore the team’s creative output will change over time.
17

Taggar (2002) proposes an aggregation model which is built on Amabile’s work.
Amabile et. al. (1996) studied the work environment extensively to see what would positively
and negatively influence individual creativity. Taggar (2002) explored how the individual factors
of agreeableness, extraversion, conscientiousness, general cognitive ability, and openness to
creativity related to individual creativity. He then went on to study the team-relevant creativity
processes of team motivation, domain relevant skills, and creativity-relevant processes. His
research was able to draw correlation coefficients between these different factors and individual
creativity (for example, team motivation and creativity-relevant processes had the strongest
influence in individual creativity). Finally, individual creativity is aggregated into team
creativity, with this aggregation being influenced by team-relevant creativity processes.
Shalley and Perry-Smith (2008) propose an aggregation model, which explores team
creative cognition. They define creative cognition as consisting of problem
identification/formulation and conceptual breadth. The key element which influences creativity,
they propose, is diverse relationships beyond the team stimulating each member’s individual
creative cognition. The individual creative cognition then infused into the group’s creative
cognition. The most relevant part of their work is postulating collaborative interactions allow
total team creativity to be more than just the aggregate creativity of each team member. This
occurs because interactions make each member more effective through the cross fertilization of
ideas, a concept providing the inspiration for proposing collaborative stimulation.
2.4.2 Process Models
Process models view group creativity as a set of related processes. Sonnenburg (2004) proposes
communication is the driving force behind group creativity, supported by a process model. He
18

creates a new term and model, the creaplex, defined as “a specific type of communication system
from which collaborative creativity emerges”. There are four main dimensions to the creaplex
model: the type of communication, the course of performance, the working style, and relation
between the implication of solution and the nature of problems. He proposes group creativity
occurs in the stages of problem finding, problem acceptance, preparation, incubation,
illumination, verification, modification, and solution. While the model has these stages occurring
linearly, the work states in the real world there are concurrent occurrences, feedback loops, and
interrelations between all the stages, making it rather complex.
West (2002) proposes a model which seeks to explain the complex interplay of ideas in
collaboration on a team. The key processes he proposes are group task characteristics (relating to
the assignment the group is expected to complete), group knowledge and diversity of skills (the
combination of experiences each member brings to the team), integrating group processes (how
the group comes together and works together), and external demands (influences from the
organization and outside world). These processes effect two outputs, creativity and innovation
implementation. The two differ as creativity is the generation of a novel idea, where as
innovation is the application of the idea in a work setting.
Stempfle and Badke-Schaub (2002) have analyzed the collaborative process specifically
for design by identifying thinking operations the team members experience. They identified key
thinking operations as being exploration, generation, comparison and selection. The thinking
operations influence both content-based task work and process-based team work. Task work
focuses on idea generation, where as team work focuses on communication and the development
of the team. One of their interesting findings was that teams will naturally choose the first good
19

idea they invent, instead of searching for multiple ideas. This works well in settings where a
standard solution and efficiency is needed, but not in settings requiring true innovation.
2.4.3 Creativity Metrics
There is also substantial work in the area of measuring creativity by either quantifying the
creative ability of a specific individual or the creative outcome of a project. Knowing how
creative someone is or the creativity of a specific outcome is useful to explore the influence of
individual creativity on team creativity, learn how to set up teams, and discover methods that
effectively enhance creativity.
Measuring an individual’s creativity can occur through two types of tools: inventories of
creative behaviors and skill measuring tools. Inventories of creative behaviors give an individual
a score based on the creative activities in which they have participated. To take an inventory, the
individual will answer a series of question, similar to a survey. Some of the most common
inventories include the Biographical Inventory of Creative Behaviours (BICB) (Silvia, Wigert,
Reiter-Palmon, & Kaufman, 2012), the Creative Achievement Questionnaire (CAQ) (Carson,
Peterson, & Higgins, 2005), and the Creative Behavior Inventory (CBI) (Hocevar, 1979). The
CAQ and CBI also measure the frequency with which an individual has participated in creative
activities, while the BICB only provides a list of activities in which the individual has
participated. However, the BICB can be completed more quickly, and was found to have similar
accuracy to the CAQ and CBI.
Skill measuring tools are questionnaires and tests which attempt to measure an
individual’s aptitude for creativity regardless of the creative activities in which they have
participated. The Torrance Test of Creative Thinking (TTCT) examines an individual on both
20

verbal and non-verbal creativity skills, to measure their effectiveness at producing creative ideas
(Torrance, 1974). Another test which explores creativity-related skills is the Creativity
Reasoning Test (CRT) which asks the individuals a series of riddles. There are two versions of
the test, one for children and another for adults (Cropley, 2000). The Creativity Personality Scale
(CPS) asks individuals to identify which adjectives on a list of 30 describe them. They receive a
score from -12 to 18 for creativity, depending on which adjectives they choose (Gough, 1979).
Another self-rated measurement of creativity is the Creatrix Inventory (C&RT) which asks the
user to rate how much a certain behavior relates to them on a 9 point scale. It then plots their
score in a two dimensional matrix with the axes of risk taking and creativity (Cropley, 2000).
There are also multiple ways to measure the creativity of an idea, most of which require
judges to make the determination. Before getting into specific methods, it must be observed all
measurement methods break the creativity of a product into several categories. This is because it
is challenging to judge creativity in and of itself as it is amorphous and hard to define. Therefore,
each method applies different terms to define a product’s creativity.
The Consensual Assessment Technique (CAT) assumes if a product is creative, expert
judges will agree on the product's creativity. A creative product is measured by multiple sub-
categories under the areas of creative, technical, and aesthetic. The judges ratings in each area
are then compared to each other and if inter-judge reliability is high, the measurements are
deemed as valid (Amabile, 1982). The Student Product Assessment Form (SPAF) was designed
to measure the creativity of projects created in gifted student classes. The form determined
creativity of a product by examining the process of creating the product (8 aspects) and assessing
the product itself (7 aspects). Each aspect was rated by an instructor on a 1-5 scale, to determine
the total creativity (Reis & Renzulli, 1991). The Creative Product Semantic Scale (CPSS)
21

consists of 55 adjective pairs with a 7 point scale. Judges rate and describe the product using a
scale between the opposing adjectives. The results are then used to determine the product’s
novelty (how new a product is), resolution (how well a product performs its intended task), and
elaboration and synthesis (how well the product relates to the user)(O’Quin & Besemer, 2006).
One of the most widely used creativity measurement ratings in the field of design is
Shah’s metrics. They differ from the previous methods as the metrics can be used to measure the
creativity of a group of ideas produced, rather than just one product. The metrics consist of
novelty, variety, quality and quantity (Shah, Smith, & Vargas-Hernandez, 2003). Novelty
measures how unique a set of ideas are compared to the other ideas generated. Variety examines
how broad a specific group of ideas is compared to the other groups of generated ideas. Quantity
measures purely the number of ideas generated within a group. Quality is determined by a set of
judges who subjectively rate the performance of each idea, and is checked by assessing
reliability between the judge's scores (Shah et al., 2003). Another approach to measuring the
creativity within a group of ideas is through adjusted linkography. The linkograph is a set of
connections which show how ideas are related to each other, and from this network, a value for
creativity can be obtained (van der Lugt, 2000).
2.5 Summary
In summary, three key areas related to this research have been explored: design theory and
methodology, creative cognition, and group creativity. Multiple models and perspectives from
each field have been explored by reviewing research from the fields of design, psychology, and
business.
22

Design theory and methodology provides the landscape for where this work seeks to make
its contributions by providing insights for the development of new idea generation
methodologies for collaborative design. The areas of creative cognition and group creativity
provide the foundation for this work. Many of the ideas reviewed in these two areas provide
inspiration for various aspects of collaborative stimulation, which will be proposed. The next
chapter will review the contributions creative cognition and group creativity have made to the
field of design, and a gap will be identified.
23

3 Merging Group Creativity and Creative Cognition
3.1 Introduction
Group creativity and creative cognition are the foundations for this work. In this chapter, it is
explained how these two areas of research contribute to design and how a gap exists between
these two fields. The details of this gap are then discussed as well as the research questions
created.
3.2 Design Contributions of Creative Cognition
Creative cognition’s unique approach of considering specific processes has led to a number of
insights for the development of methodologies. It has been found that more ambiguous and less
mature concepts provide the best stimulation (Jin & Benami, 2010). For example, if a designer is
given the challenge to design a human powered boat, providing working concepts, like chain
drives and pedals, is better than giving a bicycle as inspiration. A better understanding has been
developed of key components of ideation methods, like how incubation or proactive stimuli
influence quality, quantity, novelty and variety (Vargas-Hernandez, Shah, & Smith, 2010).
Additionally, creative cognition has generated insights on preventing fixation, when a designer
focuses on a single solution or way of solving a problem. Fixation can be reduced by providing
the designer with abstract or non-standard problem statements (Finke et al., 1996) as many times
the statement can falsely over constrain a problem. Another way of reducing fixation is to
introduce inspiration examples (Perttula & Sipilä, 2007). However, for the examples to work
24

effectively, and not just increase fixation, it was found they were required to have a low amount
of commonality.
3.3 Design Contributions of Group Creativity
The aggregate and process models of group creativity have produced a number of suggestions
applicable for increasing the creativity of designers. For example Sonnenburg (2004) proposes
constant communication in collaboration hurts the illumination (or “a-ha”) stage of creativity.
Shalley and Perry-Smith (2008) suggest teams should be formed from individuals who have a
high diversity of contacts as it is even more important than the diversity of the team. Taggar
(2002) lists many creativity-supporting behaviors adopted by team members: team citizenship,
effective communication, involving others, and providing feedback.
However, more frequently used and well known ways of encouraging group creativity are
developed in the professional realm, instead of by researchers. This is especially prevalent in
idea generation methodologies, like brainstorming (Osborn, 1957), the 6-3-5 method (Rohrbach,
1969), design random stimuli (De Bono, 1970), visualizing the problem solving process (Catford
& Ray, 2004), fishbone diagrams, and shared design notebooks (Michalko, 2001). Unfortunately,
many of these methods are not developed considering the results of research, but instead are
created based on what seems to be a good idea.
While many of these methods do encourage idea generation, some like brainstorming
have negative effects. While brainstorming was designed to encourage ideas from a group
(Osborn, 1957), it was soon found collaborative brainstorming produced a lower quantity and
quality of creative ideas than individuals working alone (Taylor, Berry, & Block, 1958). Reviews
of multiple brainstorming studies have come to the same result, identifying brainstorming’s lack
25

of producing a high quantity and quality of ideas, due to social inhibitions and procedural issues
(Diehl & Stroebe, 1987; Lamm & Trommsdorff, 1973; Mullen, Johnson, & Salas, 1991). Social
inhibitions are social reasons why individuals do not share ideas, such as being mentally lazy or
worrying about evaluation (Diehl & Stroebe, 1987). Procedural issues are efficiency issues
which occur because of multiple collaborators. For example, only one person can talk at a time
(Mullen et al., 1991) and when sharing ideas, others have to process the concepts which were
shared (Diehl & Stroebe, 1987), reducing ideation effectiveness. Interestingly, more recent
research on brainstorming shows the method has positive stimulating effects, but they are not
strong enough to overcome the problems described here (Brown, Tumeo, Larey, & Paulus, 1998;
Dugosh, Paulus, Roland, & Yang, 2000).
Methods of encouraging group creativity which are based on research have been found to
be effective, as can be observed with the C-sketch design method (Shah et al., 2001). It was
developed by considering group creativity research and has outperformed other invented
methods. Additionally, all the research in the area of brainstorming has provided a foundation for
how to improve the method (although still nominal groups generally outperform collaborative
groups). For example, it has been found when ideas are shared briefly, to avoid procedural
issues, individuals will produce more ideas than those who have not had exposure to shared ideas
(Perttula, Krause, & Sipilä, 2006).
3.4 Group Creativity’s and Creative Cognition’s Gap
By comparing and contrasting literature concerning creative cognition and group creativity, it
can be observed both provide high value, particularly in the area of developing methodologies.
However, a clear gap exists between the two areas. Creative cognition explores the thinking
26

processes of each designer (Chusilp & Jin, 2006; Finke et al., 1996; Jin & Benami, 2010), but
does not explore the influence of collaborative interactions. Group creativity examines team
interactions, but treats individuals as “black boxes”, not investigating individual thinking
processes (Pirola-Merlo & Mann, 2004; Sarmiento & Stahl, 2008; West, 2002). Models
extending creative cognition to design have provided the foundation for insights on
methodologies (Chusilp & Jin, 2006; Jin & Benami, 2010; Vargas-Hernandez et al., 2010), and
methods derived from group creativity research like C-sketch (Shah et al., 2001) have been
observed to be effective. Therefore, extending creative cognition to group creativity in design to
will provide the basis for creating more effective collaborative methodologies.
3.4.1 Existing Research
There is other research which comes close to bridging the gap between creative cognition and
group creativity, but unfortunately falls short of the depth needed to analyze fundamental design
thought processes. Stempfle and Badke-Schaub (2002) take a cognitive approach to the
engineering design process (but not creativity). However they do not take the exploration of
thought processes deep enough, only breaking them down into the categories of exploration and
generation. Similarly Tang, Lee, and Gero (2011) explore how collaboration through traditional
and digital sketching influences high and low level thought process groups, but do not examine
its influence on specific thought processes. In contrast, Jin and Benami’s (2010) research goes
several layers deeper than this into specific individual thought processes.
Perhaps the deepest and most complete work examining collaborative idea generation from
a cognitive perspective was by Nijstad and Stroebe (2006). They model how the memory
retrieval process, which brings ideas from long term memory to working memory, is influenced
27

by collaboration. But this work does not explore other generative processes, beyond memory
retrieval, in detail. Also, it does not specifically look at group interactions, as the group is treated
as external to the individual with anything from the group merely being an input or an output. It
should be noted, while the title to Shalley and Perry-Smith's (2008) research sounds like it
bridges the gap between creative cognition and collaboration, they instead use an individual
cognition perspective to model team interactions.
3.5 The Proposed Interactive Cognition Approach
The gap between creative cognition and group creativity can be bridged by proposing an
interactive cognition approach, which considers both the individual's thought processes and how
collaboration influences them. Interactive cognition (Figure 3-1) proposes the total team
creativity is greater than the sum of each person’s individual creativity, as it is also influenced by
the individuals’ interactions with the team.

Figure 3-1 An interactive cognition approach
3.5.1 Proposal of Collaborative Stimulation
This work aims to bridge the gap by proposing collaborative stimulation, a specific instance of
the interactive cognition approach. The fundamental concept behind collaborative stimulation is
that specific collaborative interactions stimulate specific thought processes. The collaborative
stimulation effect has already been eluded to by research in brainstorming (Brown et al., 1998),
28

group creativity models (West, 2002), and memory retrieval (Nijstad & Stroebe, 2006).
However, as of yet there has not been the identification of specific types of collaborative
stimulation. Also, relationships have yet to be drawn between collaborative stimulation and
thought processes. By combining the depth of creative cognition research with collaborative
aspects identified in group creativity research, powerful insights into the collaborative design
process may be generated.
One of the key challenges of investigating collaborative stimulation in depth is observing
specific thought processes occurring in the collaborative setting. This is one of the key reasons
past work has not fully extended creative cognition to collaboration. To overcome the challenge,
this work will need to modify current experimental methodologies.
3.6 Research Questions
There are both theoretical and methodological research questions concerning collaborative
stimulation. The theoretical research questions are:
1. How are creativity relevant thought processes stimulated by interactions between
designers?
2. What are the types of collaborative stimulation created by designers’ interactions?
3. How influential is each type of collaborative stimulation?
The answers to the theoretical research questions can be used to infer how collaboration impacts
group creativity, which will form a base for suggesting new collaborative design methodologies.
In order to verify the answers to the theoretical research questions, an experiment must be
conducted which identifies thought processes. This leads to the experimental methodology
research question:
29

1. How can a subject’s thought processes be observed in a collaborative setting where the
designer thinks about his/her own design, observes others’ designs and talks to other
designers simultaneously?

30

4 Collaborative Stimulation: Approach, Framework and
Types
4.1 Introduction
This chapter describes the development of collaborative stimulation, starting with its strong roots
in creative cognition, extending a creative cognition model to collaboration. First, the specific
backgrounds from creative cognition and group creativity are discussed. Next, the model which
orients collaborative stimulation is considered, and then the types of collaborative stimulation are
identified. It concludes with limitations on the current view of collaborative stimulation.
4.2 Creative Cognition Foundation
The collaborative stimulation framework is based on Jin and Benami’s (2010) generate-
stimulate-produce (GSP) model of creativity in conceptual design, which was expanded from the
Geneplore model (Finke et al., 1996). The basic model consisted of design entities, which
stimulate thought processes, which produce design operations which generate new design
entities (Figure 4-1).

31

Figure 4-1 Generate, Stimulate Produce (GSP) Model (Jin & Benami, 2010)
This cycle continues as pre-inventive design entities (immature designs) become knowledge
entities (or mature designs), until a final design is reached or may be terminated if the designer is
unable to obtain a satisfactory design. Jin and Benami (2010) identified specific elements in each
of these stages.
4.2.1 Design Entities
Design entities are immature design ideas which exist in the categories of form, function and
behavior. Form is the appearance a design takes, or the physical space within which it exists.
Function is the purpose a design serves, or the need it fulfills. Behavior is how a design acts, or
the interaction properties of a design. The proposed collaborative stimulation approach states that
collaborative interactions occur through design entities. Design entities, which designers realize
are meaningful and relevant to the problem, stimulate thought processes.
4.2.2 Thought Processes
Thought processes are mental operations which create new concepts. The processes can be
divided into two categories, generative and exploratory. Jin and Benami (2010) identified
generative (divergent) thought processes relevant to creative design as memory retrieval,
32

association and transformation. Memory retrieval occurs when a designer remembers a concept.
Association occurs when a designer creates connections between two concepts. Transformation
occurs when a designer alters a concept. Exploratory (convergent) thought processes relevant to
creative design were identified as solution analysis and problem analysis. Solution analysis is
scrutinizing a design entity which has been created. Problem analysis is exploring the problem
statement. Thought processes produce design operations to realize concepts.
4.2.3 Design Operations
Design operations bring the concepts from thought processes to realization. There are two
categories of design operations: external and internal. External design operations can be
observed when a designer is working through a problem, where as internal design operations
occur within the designer's mind. External design operations consist of sketch, talk, write, point
and simulate and create design entities that are shared. Internal design operations consist of
question, supposition, declaration, suggestion, explanation and computation (Jin & Benami,
2010).
4.3 Relevant Concepts from Group Creativity
From a group creativity standpoint, collaborative stimulation takes an approach which combines
both aggregate (Pirola-Merlo & Mann, 2004; Shalley & Perry-Smith, 2008; Taggar, 2002) and
process (Sonnenburg, 2004; Stempfle & Badke-Schaub, 2002; West, 2002) based models. The
results of the individual’s thought processes are aggregated through external design entities.
However, this aggregation does not just build independently, but rather continues to influence
the next thought processes and design operations yet to occur.
33

Group creativity research also identifies specific ways collaboration can encourage new
ideas. First, it has been proposed recalling memories will be more prevalent in collaboration as
designers stimulate each others’ memories (Brown et al., 1998; Sarmiento & Stahl, 2008; West,
2002). Sarmiento and Stahl (2008) also identify two other interactions collaboration creates
beyond remembering: referencing and bridging. Referencing is the naming of specific concepts,
so they can be shared. Bridging is purposeful actions used to overcome boundaries or mental
blocks. Another result of collaboration is finding a way to incorporate each others’ concepts,
when two designers disagree. Accommodating different ideas can be reached by abstracting the
concepts to a level of agreement, and then finding a new solution (Jin, Geslin, & Lu, 2007).
Finally, research has identified how analogies are useful in the explanation of concepts to
another designer (Glynn & Takahashi, 1998; Goldschmidt, 1997).
4.4 Collaborative Stimulation Framework
The collaborative thinking stimulation (CTS) model establishes a framework for collaborative
stimulation by extending the GSP model to collaboration by proposing each designer’s thinking
interacts through shared external design entities (Figure 4-2).
34

Figure 4-2 Collaborative thinking stimulation (CTS) model
It can be observed in the CTS model that each designer engages in the same individual
processes as the GSP model (shown in grey), but also externally interacts with the other designer
(shown in white). At times, the designer stays within their own individual cycle while other
times they share ideas with or are stimulated by their collaborator. The focus of this work is on
the lower half of the CTS framework answering the research question, “How are creativity
relevant thought processes stimulated by interactions between designers?” by hypothesizing:
External design entities stimulate thought processes through collaborative stimulation (bottom of
Figure 4-2). While collaborative stimulation influences both exploratory and generative thought
processes, the current focus is on the stimulation of generative thought processes, consisting of
memory retrieval (remembering an idea from the past), association (drawing relationships
between two design entities), and transformation (altering a design entity) (Jin & Benami, 2010).
35

4.5 Types of Collaborative Stimulation
In the team setting, both collaborative and individual stimulation exist (see Figure 4-2 and Figure
4-3). Individual stimulation occurs when a designer is stimulated by their own design entities
(which can be internal or external), where as collaborative stimulation occurs when a designers is
stimulated by their collaborators external interactions. The focus in this section is identifying the
different types of collaborative stimulation.
There are two mechanisms through which collaborative stimulation influences generative
thinking processes: design entity initiated and question initiated. Design entity initiated
stimulation occurs when a design entity the collaborator created stimulates a generative thinking
process and is attributed to the diversity in group knowledge and skills (West, 2002). Stimulation
through this mechanism tends to be spontaneous and will only occur if the designer is paying
attention to the ideas their collaborator is sharing. Question initiated stimulation occurs when a
question (or assumed question) from the collaborator stimulates a generative thinking process
and is due to the different views of the task and external demands (West 2002). Stimulation
through this mechanism is somewhat forced.
To answer the second research question, “What are the types of collaborative stimulation
created by designers’ interactions?”, specific types of collaborative stimulation are proposed.
They occur through the two mechanisms listed above and are the design entity initiated
prompting and seeding and the question initiated correcting and clarifying (Figure 4-3). These
types have been identified from the literature review, personal industry experience, and
qualitative pilot experiment observations.
36

Figure 4-3 A breakdown of stimulation in team design
4.5.1 Prompting
Research regarding collaboration’s influence on memory shows ideas from team members will
often stimulate an individual’s memory (Nijstad & Stroebe, 2006). Brainstorming research
agrees, finding collaboration has stimulating effects on memory retrieval (Brown et al., 1998;
Dugosh et al., 2000). In fact, the reasoning behind brainstorming is to have another’s ideas
stimulate oneself. Prompting, based on these research concepts, occurs when a design entity
developed by a collaborator reminds the designer of a memory. It is expected prompting will
stimulate the thought process of memory retrieval most often, although occasionally an
association, which would connect a memory to the case at hand, is also expected.
4.5.2 Seeding
Building on another’s idea is one of the key steps in collaborative design (Holsapple & Joshi,
2002) and is often the basis of aggregation models of group creativity (Pirola-Merlo & Mann,
2004; Shalley & Perry-Smith, 2008; Taggar, 2002). There are multiple creative design
techniques which are based on this effect, including the 6-3-5 technique (Rohrbach, 1969) and C-
37

sketch (Shah et al., 2001). While ideal models indicate individuals would continuously
contribute to a collaborative idea, more frequently individuals build on only parts of others’ ideas
and then combine the ideas (Kvan, 2000). Rather than being a continuous phenomenon, building
on each other’s ideas occurs sporadically throughout the design process. From these
observations, the collaborative stimulation of seeding is proposed to occur when a design entity
developed by a collaborator is furthered by an individual. It is expected that seeding will
stimulate the thinking process of transformation.
4.5.3 Correcting
Critiquing and then revising the design based on those critiques is a key component of the
ontological design process (Holsapple & Joshi, 2002). A model by Stempfle and Badke-Schaub
(2002) proposes the evaluation of ideas (a challenge or question) reveals a concept’s
inadequacies and encourages the generation of new ideas. Correcting comes from these
observations and occurs when the designer is asked a question or challenged by a collaborator
and alters the design entity to resolve the raised issue. Correcting is expected to most
significantly influence transformation, but also moderately influence association and have a
minor influence on memory retrieval.
4.5.4 Clarifying
Elaboration is an important practice which supports creativity as it opens opportunities for
additional insights (Vyas & Nijholt, 2009). Clarifying is a form of elaboration which occurs
when the designer is challenged or asked a question by a collaborator (or senses they do not
understand a concept) and attempts to clarify their idea by explaining it in a different way,
leading to the occurrence of generative thinking processes. Clarifying is expected to have a
38

heavy influence on association and a moderate influence on both transformation and memory
retrieval.
Analogies are one way clarifying can specifically be used to explain concepts which
require clarification (Goldschmidt, 2011). This can often be seen in education as educators
explain challenging concepts using analogies (Glynn & Takahashi, 1998). These explanatory
analogies may become generative analogies, creating new concepts. Generative analogies consist
of the three generative thought processes of memory retrieval, association, and transformation
(Jin & Benami, 2010).
4.5.5 Influence of Each Collaborative Stimulation Type
The third research question, regarding the influence of each collaborative stimulation type, will
be answered by finding patterns between each type of collaborative stimulation and thought
processes, novelty and stimulation frequency from empirical results.
The first pattern to be explored will be the relationship between each type of
collaborative stimulation and thought processes. It will be examined how likely each type of
collaborative stimulation results in memory retrieval, association, and transformation.
Additionally, all the collaboratively stimulated thought processes will be examined to perceive
what type of collaborative stimulation can be attributed to them. Each type of collaborative
stimulation is expected to be uniquely influential toward various types of thought processes.
The second pattern to be investigated is each type of collaborative stimulation’s influence
on novelty. The influence of collaborative stimulation on specific ideas is determined by the
creativity of an idea a collaborative stimulated thought process produces. As there were many
ideas generated, it was desired to use a method that measures the creativity of one idea in
39

comparison to multiple groups of ideas. Shah’s metrics were decided upon, as the process is
more straight forward than linkography. When using Shah’s metrics, the creativity of an idea is
measured through novelty, variety, quantity and quality (Oman, Tumer, Wood, & Seepersad,
2013; Shah et al., 2003). However, it was decided focusing on novelty would be most effective
in order to measure the creativity impact of collaborative stimulation. Variety and quantity were
not selected as they focus only on sets of ideas and do not rate individual ideas (Shah et al.,
2003). While these two metrics are useful to measure creativity in an overall process, it would be
difficult to observe what impact collaborative stimulation had on specific ideas. As quality is
qualitative, it was determined to measure the impact of collaborative stimulation through novelty
as it is quantitative (Shah et al., 2003).
The final pattern to be explored is the frequency of collaborative stimulation. There are
some types of collaborative stimulation which should occur more frequently than others. If those
which occur less frequently are especially influential with regards to a desired thought process or
have a strong relationship to novelty, it could be useful to design methods to encourage that type
of collaborative stimulation.
4.6 Limitations
Going deep into thinking processes is inherently challenging. Therefore, there are many aspects
of collaboration that the current proposal of collaborative stimulation approach, and even the
CTS model, does not yet incorporate which are discussed in this section. It should be noted, that
simply because these are considered limitations does not mean they are not important or do not
need to be addressed. However, it is important to first understand how collaboration influences
thought processes through stimulation before trying to understand all the limitations. Focusing
40

on the stimulation of thought processes without applying any limitations would make the
research unclear. These limitations create areas for future work, which are discussed at the end of
this dissertation.
4.6.1 Generative and Exploratory Processes
Thought processes can be divided into two groups, as previously explained: generative and
exploratory. Generative processes are divergent, generating more ideas whereas exploratory
processes are convergent, reducing ideas to the concepts which will work (Finke et al., 1996).
Generative processes relevant to engineering design creativity consist of memory retrieval,
association, and transformation. Exploratory processes relevant to engineering design consist of
solution analysis and problem analysis (Jin & Benami, 2010). In order to keep a manageable
scope, collaborative stimulation currently only focuses on the stimulation of generative cognitive
processes. This keeps the focus on processes which create new ideas.
4.6.2 Group Size
Collaborative stimulation and the CTS model are currently examining a special case of groups:
dyads, made up of only two people. The argument could be put forward that dyads are not really
a group, and do not really collaborate. These claims are backed up by research which has shown
dyads to interact differently than any group of three or more (Moreland, 2010). However, there is
not currently knowledge regarding an individual’s thought process being altered by two
individuals collaborating. Therefore, it is important to first understand the interactions between
two individuals, before moving on to three or more. Even Moreland (Moreland, 2010), who
claims dyads are different from groups, mentions that dyads can reveal important information
about groups. Also, adding a third person greatly increases the complexity of analysis.
41

4.6.3 Ignoring Social Inhibitions and Procedural Issues
As mentioned in brainstorming research, collaboration has been found to negatively influence
creativity through social inhibitions and procedural issues. However, it can be noticed the CTS
model does not deal with either of these. Procedural issues and social inhibitions are inhibitors
which occur at the organizational and social interactions levels, but are believed to have
minimized impact on this research. This is justified as collaborative stimulation focuses on the
stimulation of thought processes, which occur much deeper than the social structure.
Additionally, issues like social loafing will be more prevalent on larger teams, rather than dyads
(Latané, Williams, & Harkins, 1979) who are the focus of this work. It is also expected that
social inhibitions are more applicable to teams working on projects subjective in nature (e.g.
dealing with emotional or political issues) instead of engineering. Procedural issues are also
more relevant to larger teams, as elements like production blocking are less impactful if the team
is composed of fewer individuals (Diehl & Stroebe, 1987). Because of the minimized impact and
the lack of clarity incorporating these factors would cause, it has been decided to ignore these
issues for the present time.
4.7 Summary
This chapter has discussed how related work directly contributes to the collaborative stimulation
framework and collaborative stimulation. Collaborative stimulation has been defined as
occurring when collaboratively generated design entities stimulate a thought process. Specific
types of collaborative stimulation were defined as the design entity initiated prompting and
seeding and the question initiated correcting and clarifying. Limitations on this work’s approach
42

were then discussed. The next step is to experimentally verify the existence of collaborative
stimulation and to empirically identify patterns.
43

5 Developing an Experimental Approach- Two Pilot
Experiments
5.1 Introduction
This chapter discusses the challenges in developing an experimental approach and the two pilot
experiments used to develop the approach. It starts by examining past approaches to analyzing
collaboration and then discusses the two specific new methods which were developed and tested
through pilot experiments.
5.2 Past Approaches to Analyzing Collaboration
To evaluate the proposal of collaborative stimulation, an experimental methodology must be
established which allows for the observation of individual thinking processes in the collaborative
setting. A typical approach would be to use protocol analysis (Cross et al., 1997), where subjects
think aloud while they are working through a design process. In the collaborative setting, the
typical approach is to analyze the conversation. Sometimes the conversation is simply analyzed
for social interactions (Brereton, Cannon, Mabogunje, & Leifer, 1997; Cross & Clayburn Cross,
1995) while other times actual protocol analysis is conducted, applying a coding scheme to a
dialog transcript (Stempfle & Badke-Schaub, 2002; Wiltschnig, Christensen, & Ball, 2013).
From a social perspective, Brereton et al. (1997) investigated how collaborative
interactions influence the design process by either focusing it on a specific concept or
transitioning it to a new idea. Cross and Clayburn Cross (1995) explore the aspects of roles and
relationships, planning and acting, information gathering and sharing, problem analyzing and
44

understanding, concept generating and adopting, conflict avoiding and resolving in collaborative
design.
Protocol analysis has been applied to group dialogs in order to identify thinking
interactions. Wiltschnig et al. (2013) examine how collaboration influences problem/solution co-
evolution between designers and links to creative activities, like analogising. Stempfle and
Badke-Schaub (2002) specifically apply protocol analysis to a team’s dialog transcript to identify
underlying thought operations. They state it is valid to apply protocol analysis to conversation in
order to observe thought operations because of the work (Goldschmidt, 1997) which compares
individual verbalizations to group dialogs. Goldschmidt (1997) states the intimate nature of
sharing occurring in design team conversations is close to the internal speech individual
verbalizations produce. Similarly Tang et al. (2011) uses protocol analysis to explore the
influence of digital versus traditional sketching, and collocation versus non-collocation, on high
and low level thinking activities. However, none of the past collaborative approaches obtain
individual protocols over the length of the design process.
5.3 Core Empirical Challenges
In order to identify specific thought processes and how they are stimulated, as occurring in the
CTS model, it is necessary to obtain protocols from each designer, in addition to the group
dialog. The past approaches of only analyzing the conversation transcript does not work, as
collaborative stimulation involves both external (shared) and internal (private) thoughts.
Therefore a modified protocol analysis approach is required, which will answer the research
question “How can a subject’s thought processes be observed in a collaborative setting where
45

the designer thinks about his/her own design, observes others’ designs and talks to the other
designers simultaneously?”
From the research question two specific challenges are derived:
C1: Prevent a subject’s verbalized thoughts (when they are not talking to their partner) from
influencing their collaborator.
C2: Observe thought processes when individuals talk with each other, and thus cannot
continuously verbalize their thoughts.
Two different methods, concurrent and retrospective collaborative protocol analysis,
were developed from current protocol analysis techniques to solve C1 and C2 so individual
protocols in the collaborative setting could be obtained.
5.4 Pilot Experiment 1 – Concurrent Collaborative Protocol
Analysis
Collaborative concurrent protocol analysis used a physical barrier between designers that
allowed conversation but prevented verbalized thoughts from being communicated. This was
accomplished by having two designers work remotely using Skype, employing screen share and
a push-to-talk feature. The solution allowed the designers to verbalize their thoughts
continuously while working through the design problem, but prevented the collaborator from
hearing their verbalizations, thus resolving C1. Both the verbalized thoughts and the
conversation were recorded through the computer’s microphone. It was theorized C2 would not
be an issue as, when a designer was talking, what they were saying was also what they would be
thinking about.
46

5.4.1 Participants
Four mechanical engineering graduate students from the University of Southern California, 2
females and 2 males, were recruited for the study on a volunteer basis. All students were
members of the IMPACT lab and thus had exposure to design practices. However, the students
were all novice designers with less than a year of work experience. For this experiment, the
students were placed in teams of two. The study was reviewed and approved by the IRB.
5.4.2 Task and Materials
In order to collaborate while being physically separated, each student was provided with a laptop
computer. To communicate with each other, the students used Skype with a push-to-talk button
and screen sharing allowing them to share drawings in Microsoft Paint. Skype took up half of
each students screen and Microsoft Paint took up the other half (Figure 5-1 shows the screen
setup). In addition, the computer also made an audio recording of the entire conversation and
individual verbalizations.

Figure 5-1 Pilot experiment 1 interface
47

In addition to the laptops, students were presented with a design problem statement asking them
to design a spill proof cup for use on airlines. To sketch concepts, students were encouraged to
use the sketch area on Microsoft Paint. As each student was in a different location, two
facilitators were required to stay with the students at all times.
5.4.3 Experiment Design
As this experiment was designed to test a new experimental methodology, a traditional
experiment design approach with control, independent, and dependent variable was not taken.
There were no independent variables, control groups or experimental groups.
5.4.4 Procedure
When each student arrived at the experiment they were shown to a room with a laptop. The
facilitator explained the process and gave the student a design problem in order to practice
thinking aloud. They were given ten minutes to solve the problem. The student was requested to
continuously verbalize their thoughts.
Next, the students were given a second training design problem, this time to solve
collaboratively with their partner. The goal was to get the students used to collaborating over
Skype and teach them how to verbalize their thoughts while simultaneously collaborating. A
total of five minutes was given to solve the problem. Students were told they were not expected
find a final solution, as this was only a warm up.
Finally, students were given the actual design problem, and told to come to a final
solution with their partner in 30 minutes. During this session, like the previous, they were asked
to concurrently think aloud during the entire process, except when talking to their partner. To
48

speak with their partner, they would push the push-to-talk button and say their statement across
Skype. A summary of the process is shown below in Figure 5-2.

Figure 5-2 Pilot experiment 1 process
5.4.5 Results
While the concurrent approach obtained protocols successfully and the designers were not
influenced by each other’s protocols, there were several problematic issues. First, it was
impossible for individuals to fully verbalize their thoughts when the other designer was talking to
them. It was too difficult to listen to what the other designer was saying while also trying to
verbalize their own thoughts. Secondly, and more importantly, working together via Skype and
an electronic sketchpad made the collaborative design process complicated and reduced
efficiency. It did not allow designers to collaborate as they could have in person. No doubt, the
development of better collaboration tools (like using actual digital sketchpads) and more time
spent on training would improve this approach.
As verbal protocols were discontinuous, no useable data on thought processes was
collected. Therefore, the results of this experiment solely contributed to the development of
future experimental methods. For example, it was discovered more extensive training on
verbalizing thoughts would be required in the future and good design problems were abstract and
49

simple. The imposed time constraints were also found to be restrictive as teams were rushing to
finish their design in the allotted time.
5.5 Pilot Experiment 2 – Retrospective Collaborative Protocol
Analysis
Retrospective protocol analysis took a different approach to solve challenges C1 and C2.
Subjects were allowed to collaborate in person in a natural environment while being videotaped.
After the session was complete, subjects watched the video and retrospectively verbalized their
thoughts occurring during that portion of the video. Retrospective protocols have been found to
produce similar results to concurrent protocols (Gero & Tang, 2001). Conducting the thinking
aloud after collaborating on the design problem allows the subjects to collaborate in a natural
environment and allows for continuous verbalization of their thoughts (solving C2). Also, as the
verbalizations occurred after the collaboration, it was impossible for the subject’s verbalizations
to impact their collaborators thoughts (solving C1).
5.5.1 Participants
Seven mechanical engineering students from the University of Southern California, all males,
were recruited for the study on a volunteer basis. All students had either just completed a
senior/graduate level course on design theory and methodology or were members of the
IMPACT lab, and thus had exposure to design methodology and collaborative design. However,
the participants were novice designers having less than a year of industry work experience.
While the students participating in the study represented many nationalities and had grown up in
different cultures, all were currently living in the Los Angeles area.
50

5.5.2 Task and Materials
The Biographical Inventory of Creative Behaviours (BICB) was given to participants to
determine their past creative experience. As mentioned earlier, there are multiple other tests to
measure past creative experiences such as the Creative Achievement Questionnaire (CAQ) and
the Creative Behavior Inventory (CBI). However, the BICB was chosen as it was found to be the
quickest of the three tests but still had comparable accuracy (Silvia et al., 2012). It has also been
used in several other design studies (Batey & Furnham, 2008; Furnham & Bachtiar, 2008). The
main disadvantage to the BICB is that while it measures the number of various activities in
which the individual has participated, it does not measure the frequency of those activities. As
this study was only interested in recent creative behaviors, the students were only asked to
include the creative behavior in the inventory if it had taken place in the past twelve months.
At the study, the students were presented with a design problem statement (given in full
as “problem 1” in the appendix) asking them to design a system or device that would reduce
traffic congestion. As traffic is a prevalent problem in Los Angeles, all the students were familiar
with this issue. In addition to the statement, students were given verbal instructions to go about
the design task as they normally would and to come to one final solution. The students were
given pens and paper to write down and sketch their ideas. The design process was documented
using a video camera and a microphone, which recorded directly to a computer. Two computers
with microphones were used to play back the design process video and record the retrospective
think aloud (to be discussed in detail later).
51

5.5.3 Experiment Design
In addition to testing a new experimental methodology, this study was designed to compare
groups to individuals. The students were randomly divided into two groups (although it was
ensured BICB scores within the two groups were similar, see below) to create the independent
variable: an experimental group which collaborated (two teams of two) and a control group,
working individually (three subjects). The dependent variables of this study were the number of
cases of collaborative prompting and the number of cases of individual memory stimulation. The
control variables in this experiment design were the design problem and general background of
the students. The same design problem was given to each student; all students had similar
mechanical engineering backgrounds with some exposure to design theory; and each lived in the
greater Los Angeles area. Figure 5-3 summarizes the design of the experiment.

Figure 5-3 Pilot experiment 2 design
5.5.4 Procedure
Before coming to the study, participants were given the Biographical Inventory of Creative
Behaviours (BICB) via an internet survey to measure past creative experience. The results of the
BICB were used to create control and experimental groups, such that the difference in average
52

BICB score between both groups was within one standard deviation. The BICB was also used in
the experimental condition, to ensure both teams had the same average BICB score.
When first arriving at the study, students were given training in verbalizing their
thoughts. This training session consisted of thinking aloud while working through several
problems. Each increased in difficulty, starting by asking the student to describe a simple process
and eventually working up to asking the student to think aloud during a practice design task.
Next participants in the experimental group were given a design problem to solve with
their partner, while individuals in the control groups were given a design problem to solve alone.
Participants were provided with a pen, paper, and the design problem statement. Both the control
and experimental groups were recorded on video as they worked through the problem.
Immediately after the completion of the design problem, the subjects viewed the recorded
video and were asked to retrospectively verbalize their thoughts while watching. The designers
in the experimental (collaborative) groups were moved to different locations while
retrospectively verbalizing, so they could do so in private. The retrospective verbalizations were
recorded for later transcription. While the control subjects could have done the more traditional
concurrent think aloud technique while going through the design problem, in order to ensure
similarity between the control and experimental groups, they performed retrospective thinking
aloud. The experiment procedure is illustrated in Figure 5-4.
53

Figure 5-4 Pilot experiment 2 procedure
5.5.5 Results and Analysis
It was found using the retrospective approach designers were able to collaborate naturally. Also,
the video provided adequate cues to the designers to remind them what they had been thinking
(designers were also allowed to look at their sketches which provided additional assistance in
remembering). Subjects reported that they were able to remember 90% or greater of their
thoughts for design processes which lasted under thirty minutes. One of the challenges this
methodology faced was that occasionally while designers were retrospectively thinking aloud,
they would slip into describing the task they were doing instead of describing their thoughts. To
correct this, the experimenter reminded the designer to verbalize their thoughts, not just their
actions.
In addition to experimental methodology feedback, the verbalizations were useable and
could be coded. All the proposed types of collaborative stimulation were observed to occur,
which implied this research was on the right track with both the experimental method and
analysis. It also provided feedback resulting in the modification of the coding scheme, as
54

originally there was a fifth type of collaborative stimulation, collaborative completion. However,
this was discovered to be a special case of seeding.
The experiment also produced results which allowed for the comparison of individuals
working alone to collaborative groups, as can be seen in Table 5-1 giving the average values for
each designer. There were three areas on which the comparison focused: individual memory
stimulation, prompting, and analogies. Individual memory stimulation is similar to prompting,
except it occurs when a designer’s own design entities stimulate memory retrieval. Prompting, as
was already mentioned, occurs when memory retrieval is stimulated by a collaborator’s design
entity. Analogies were defined as occurring when a solution from one problem (the source) was
used to solve a different problem (the target). Association are drawn between the source and
target problems, until the solution to the source problem is transformed to apply to the target
problem.
Experimental Group Control Group
Avg. BICB Score 7.5 11.3
Avg. Problem Time 25 35
Individual Stimulation:
Avg. Memory Stimulations
2.5 2.0
Avg. # Promptings 2.3 0
Avg. # Analogies 1.0 0.67
Table 5-1 Pilot experiment 2 results
The analysis process and coding scheme of the protocols will be explained in detail in the next
chapter. Discussion of the results will occur in the following chapter.
55

5.6 Summary
The two pilot experiments demonstrated that the retrospective methodology provided better data
and allowed for more natural design conditions. Therefore, it was decided to use the
retrospective collaborative think-aloud method in the full experiment, answering the
methodological research question. A summary of the pilot experiments is given in Table 5-2.

Key
Advantage
Key
Disadvantage
Usable
Method
Methodology
Feedback
Coding
Scheme
Feedback
Useful
Data
Concurrent
Method
Immediate
Activity
Reporting
Interrupted
Design
Process
No Yes N/A No
Retrospective
Method
Ability to
Collaborate
Naturally
Potential
Memory
Issues
Yes Yes Yes Yes
Table 5-2 Pilot experiment comparison
56

6 Evaluation of Collaborative Stimulation: Experiment,
Analysis, and Results
6.1 Introduction
This chapter begins by discussing the identified research questions, followed by an examination
of the experimental method used to answer the research questions and how the raw data was
refined through protocol analysis. Finally, the experimental results are presented.
6.2 Research Questions to be Answered
At the beginning of this work the following research questions were proposed.
Q1: How are creativity relevant thought processes stimulated by interactions between
designers?
Q2: What are the types of collaborative stimulation created by designers’ interactions?
Q3: How influential is each type of collaborative stimulation?
These questions were answered by the proposal of collaborative stimulation and collaborative
stimulation types in chapter 4. However, the proposed concepts need to be experimentally
substantiated. The proposed answer to Q1 can be verified experimentally by multiple individuals
identifying the existence of collaborative stimulation at the same points in collaboration (which
would mean collaborative stimulation can be identified consistently). The proposed types of
collaborative stimulation answering Q2 can be verified by having multiple individuals
identifying the existence of a specific type of collaborative stimulation at the same point in
design. Q3, exploring how influential each type of collaborative stimulation is, can be answered
57

empirically by exploring novelty, collaborative stimulation frequency, and patterns in the
stimulation of thought processes.
6.3 Experimental Methods
The subjects, tasks and materials, experiment design, and procedure were developed to verify or
provide answers to the research questions.
6.3.1 Subjects
Subjects for this experiment consisted of ten senior undergraduate students and masters level
graduate students in mechanical engineering at the University of Southern California, divided
into five groups of two. The students were semi-randomly assigned to groups. All students were
enrolled in engineering design classes and undertook group projects in those classes. Therefore,
they were familiar with participating in collaborative design and had been taught basic
engineering design methodologies. However, the participants were novice designers as all
claimed less than a year of industry work experience. The subjects were compensated by being
entered in a drawing for an iPod nano and gave consent when arriving at the study. The study
was reviewed and approved by the institutional review board.
6.3.2 Task and Materials
Before coming to the study the students were given the Biographical Inventory of Creative
Behaviours (BICB) to determine their past creative experience. Once at the study, the students
were presented with a design problem statement (given in full as Problem 2 in the appendix)
asking them to create a product that would securely store skateboards outside campus
classrooms. This device would prevent a frequent problem at USC: students stacking their boards
58

against classroom walls which are a nuisance and an eyesore. As all the subjects attended USC,
they were familiar with seeing skateboards on campus and were aware of the skateboard storage
issue.
In addition to the statement, students were given verbal instructions to go about the
design task as they normally would and to come to one final solution of which they would create
a sketch and write a description. The students were given pens and paper to write down and
sketch their ideas. The design process was documented using a video camera and a microphone,
which recorded directly to a computer. Two computers with microphones were used to play back
the design process video and record the retrospective thinking aloud.
6.3.3 Experiment Design
The experiment was primarily designed as an observational experiment to analyze cases of
collaborative stimulation and explore patterns in their occurrences. However, the teams average
BICB scores ranged from 1.5 to 14, which allowed average past creative experience to be used as
a predictor variable. The controlled variables were the design problem given to each team and
the general educational background of each student. The dependent variables were collaborative
stimulation frequency and mechanism percentages.

Figure 6-1 Experiment design
59

6.3.4 Procedure
Each designer participated in the experiment by first taking a Biographical Inventory of
Creative Behaviours (BICB) test, participating in think aloud training, working on the design
problem, and finally retrospectively verbalizing their thoughts. Each step of the experimental
procedure is summarized in Figure 6-2.

Figure 6-2 Experiment Process
6.3.4.1 BICB Test
Before coming to the study, participants were given the Biographical Inventory of Creative
Behaviours (BICB) via an online survey to determine their individual creative potential. The
groups were semi-randomly assigned (some groups were arranged due to schedule availability),
and given a time to come to the experiment.
6.3.4.2 Think Aloud Training
When first arriving at the study, participants were given training in verbalizing their thoughts.
The training started by asking the subject to verbalize in detail the process they went through to
come to the study. Next, they were given a brainteaser and asked to think out loud while solving
60

the problem. Finally, they were given a practice design problem, and told to solve it while
thinking out loud.
6.3.4.3 Design Problem
Designers were provided with pencil, paper, and the design problem statement asking them to
develop a device that would securely store skateboards eliminating the need to stack them up
against classroom walls. The designers were then video recorded as they collaboratively worked
through the design problem.
6.3.4.4 Retrospective Thinking Aloud
Immediately after the subjects completed the design problem, they were individually asked to
retrospectively verbalize their thoughts throughout the preceding design process. This was done
while watching a video of the design problem, which provided verbal and visual cues. If the
video moved too fast for the subject to provide a complete verbalization, they could pause the
video and complete their thought. The retrospective verbalizations were recorded in an audio file
for later transcription.
6.4 Applying Protocol Analysis
After the experiment was conducted, the goal was to observe creativity relevant generative
thought processes and the various types of collaborative stimulation occurring in the experiment.
The data from each experiment consisted of two audio files and a video file. To analyze these
files and identify the desired elements, protocol analysis was used.
61

6.4.1 Identifying Episodes and Segmenting
The first step in protocol analysis was to transcribe the results. This was done in such a
way that the timing of the two audio files and the video file could be compared to each other
(Figure 6-3). The collaborative dialog is on the left, followed by the two retrospective protocols
to the right.

Figure 6-3 Transcript, in episodes and segments
The second step in protocol analysis involved dividing the conversation into manageable,
code-able segments. This was started by dividing the transcript into episodes, which were
defined as a portion of the transcript related to the development of one concept (upper left Figure
6-3). The episodes were determined by the conversation dialog, and then applied to the
retrospective thinking aloud. After episodes had been created, they were divided into segments
(also in Figure 6-3). Segments are traditionally determined by breaks in the text and natural
pauses (Ericsson & Simon, 1993). They are used to break the transcript into single thoughts and
small components which can be coded. The third step in protocol analysis was to apply a coding
scheme to each segment, which was entered into the white space between the transcripts in
Figure 6-3.
62

6.4.2 Coding Scheme
The three elements requiring identification by the coding scheme in each segment were design
entities, thought processes, and collaborative stimulations. Each is described below.
6.4.2.1 Design Entities
A design entity was identified as a potential or partial solution having a form, function, and/or
behavior. Forms consist of the physical shape of an object. A function is the purpose an entity
serves as related to the problem. A behavior consists of how an entity interacts with its
environment. Anytime a form, function, or behavior was mentioned, it was classified as a design
entity. Initially, a design entity started out as only a partial solution, but later developed into a
full solution (Jin & Benami, 2010). Sometimes, design entities were accompanied by sketches,
which made them easier to identify. In one segment, there could be multiple design entities.
6.4.2.2 Thought Processes
After the design entities were identified, all the thinking processes occurring in the transcript
were identified. Thought processes relevant to the CTS model in design consisted of the
generative processes of memory retrieval, association, and transformation (Jin & Benami, 2010).
Generally, there is only one thought process per segment, although there are exceptions for some
special cases where the thought processes are directly related to each other. The definition of
each thinking process is mentioned below,
Memory Retrieval (MR): an experience or design entity which existed in the past is remembered.
Association (AS): connections are drawn between two design entities.
Transformation (TF): a design entity is altered or changed.
63

It should be noted, a detailed examination of the data will reveal two exploratory thought
processes were also coded. They were recorded for future work investigating collaborative
stimulation, and are not immediately relevant to this dissertation, and therefore are not included
in the coding scheme.
1. Solution Analysis (SA): a design entity is reevaluated for fitness to the solution.
2. Problem Analysis (PA): the problem statement is reevaluated.
6.4.2.3 Collaborative Stimulation Types
Next, the collaborative stimulation types were identified by examining how thinking processes
came about and if they could be attributed to collaborative stimulation. Most segments did not
have a collaborative stimulation associated with it, as they did not occur all the time. In a single
segment there was not more than one collaborative stimulation type, and sometimes a single
collaborative stimulation would span multiple segments. The definitions of each collaborative
stimulation type are listed below:
1. Prompting: An external design entity leads to memory retrieval. It can occur
collaboratively or non-collaboratively.
2. Seeding: A collaborator’s external design entity is internalized by the subject and
modified.
3. Correcting: A subject corrects their idea to make it acceptable because of a partner’s
question or challenge.
4. Clarifying: A subject senses their collaborator does not understand an idea, so they
further clarify it. The process of clarification leads to further development.
64

6.4.2.4 Coding Scheme Summarized
The coding scheme, as well as notations and transcript examples, is summarized by Table 6-1,
which was used by the investigators to code the transcripts. The coding notation is what would
appear in the white columns of Figure 6-3.
Name Abbr. Coding Notation Transcript Example
Design Entities DE
Function F F(make hole) Makes hole in wood
Structure S S(car) attached to a car
Behavior B B(moves) Moves up and down
Thought Processes TP
Memory Retrieval MR MR(DE(X)) I think a solution would be
(lists a preexisting solution)
Transformation TF TF(DE(X), expanded) If X was expanded
Association AS AS(DE(X), DE(Y)) Idea X is like Idea Y
Collaborative
Stimulation
CS
Prompting Pr Pr(DE(X),MR(Y)) X reminded me of Y
Seeding Se Se(DE(X),CP(DE(X));DE(X*)) X you proposed can be
modified to create X*
Correcting Co Co(DE(X), CP(S(X), DE(X*))) X can be modified to create
X*, which solves the issue
you brought up
Clarifying Cl Cl(DE(X), CP(X);DE(X*)) or
Cl(DE(X), CP(Y);DE(X*))
X works like this, but it can
be changed to X* or X works
like Y, which changes it to X*
Table 6-1 Coding Scheme
To ensure the coding scheme was accurate, it was checked by conducting intercoder
reliability on more than six percent of the data. Two coders would encode identical, but
randomly selected, portions of the transcript. The intercoder agreement was 85% for identifying
collaborative stimulation and 82% for identifying generative thought processes. In general, an
agreement above 70% means the coding scheme is sufficiently clear and defined (van Someren
et al., 1994).
65

6.4.3 Example Coding
To demonstrate how to apply the coding scheme, consider the related sections of collaborative
dialog and individual retrospective transcripts below, where two designers were discussing a
wall mounted skateboard rack. This is a small section of the dialog from episode 3. The entire
episode can be found in appendix 2. The numbers in the collaborative dialog transcript indicate
which designer was speaking.
Collaborative Dialog Transcript:
(1) Okay. Have you seen Parkside?...(2)Yeah… (1)They have those racks…(2)Yeah… (1) Well,
assuming like if they have those racks plus a locking device that you can just use like a padlock…
(2) Well, who with a skateboard carry around a pad lock? (2) What if it was ID card swipeable?
(2) Every USC student is going to have an ID card…
Collaborator 1 Retrospective Protocol Transcript:
At this pass, I got the pad lock, I think it's pretty good but-- yeah, a little frustrated by the idea,
the current idea, but its fine. I think the reason why I was going for locks is because it’s got less
involved I try to stay away from electronics because it makes things a lot more complex, and I
like to focus on the simple side of things..
Collaborator 2 Retrospective Protocol Transcript:
The other thing that I was just mentioning was talking about ID cards. While it’ll be a great
problem on campuses, what if you didn’t want it on a campus? One of the things that I was
thinking about when we’re going to talk about the locking mechanism is how in that convention
center back in Chicago I saw, almost like people lockup their coats in individual lock boxes and
they all had ID cards that were based—I was in a convention center located in a hotel and that’s
part of where the ID card idea came from but if it wasn’t located on college campus, how can
66

you monitor who has a card, who doesn’t have a card and what if someone’s visiting and brings
a skateboard?
The transcripts were then divided into segments, and after all the transcripts had been segmented,
they were coded. The results of these two operations for the transcript and coding are shown in
Table 6-2.

67

Collaborative Dialog Segments and Coding:
#
Time Segmented Dialog Coding
1 Episode 3: Racks and Security System
2 (1) Okay. Have you seen Parkside?...(2)Yeah…
(1)They have those racks…(2)Yeah…
(1) MR(S(parkeside B(has
S(racks))))
3 (1) Well, assuming like if they have those racks plus a
locking device that you can just use like a padlock…
(1) TF(S(racks AS(S(lock
device B(use S(padlock))))
4 (2) Well, who with a skateboard carry around a
padlock?
(2) SA(F(why( S(skateboard
user B(carry S(padlock)))
5 02:06 (2) What if it was ID card swipeable? (2) Se(TF( S(rack S(ID card
B(swipable)))) (below)
6 (2) Every USD student is going to have an ID card… (2) SA( S(usc student B(have
S(ID card))))
Collaborator 1 Retrospective Segments and Coding:
# Time Segmented Dialog Coding
1 Episode 3: Racks and Security System
2
3 At this pass, I got the pad lock, I think it's pretty good but S(padlock)
4 yeah, a little frustrated by the idea, current idea but its fine. SA(frustrate S(padlock)
5 02:15 I think the reason why I was going for locks is because it’s
got less involved I try to stay away from electronics
because it makes things a lot more complex and I like to
focus on the simple side of things..
SA(S(locks F(less
involved)F(stay away
S(electronics B(complex)))))
Collaborator 2 Retrospective Segments and Coding:
# Time Segmented Dialog Coding
1 Episode 3: Racks and Security System
5 03:10 The other thing that I was just mentioning was talking
about ID cards.
Se(TF(S(ID cards)))
6 While it’ll be a great problem on campuses, what if
you didn’t want it on a campus?
SA(F(great B(on S(campuses)
F(didn't want S(rack B(on
S(campus))
8 03:40 One of the things that I was thinking about when we’re
going to talk about the locking mechanism
Describe (S(lock mechanisms))
9 is how in that convention center back in Chicago I
saw, almost like people lockup their coats in
individual lock boxes and they all had ID cards that
were based—I was in a convention center located in a
hotel and that’s part of where the ID card idea came
from
Ms( S(lock
device),MR(S(convention
center S(people B(lockup
S(coats B(in S(individaul lock
boxes)S(people[they] B(had
S(ID cards)S(convention center
B(located in S(hotel)))))))
10 04:10 but if it wasn’t located on college campus, how can
you monitor who has a card, who doesn’t have a card
and what if someone’s visiting and brings a
skateboard?
SA(F(wasn't B(locates
S(college campus))) F(how
B(monitor S(ID cards)S(vistor
B(brins S(skateboard))))
Table 6-2 Segmented and coded transcripts
68

Images of the way the skateboard is locked are shown in Figure 6-4. The locking
mechanism (ID card reader or padlock) would be located at the front of the arm where the arrow
points.

Figure 6-4 Skateboard locking arm and lock location
In the example given above, it can be observed how the individual verbalization brings
additional information not revealed by the collaborative dialog. The dialog only specifies design
entities being discussed. However, the retrospective verbalization reveals the thinking process
(memory retrieval) creating the new entities, the design entity which stimulated the thinking
process (locking mechanism/padlock), and the type of collaborative stimulation involved
(prompting). It can also be observed how this retrospective verbalization goes into detail about
the memory which was retrieved. By using collaborative retrospective protocol analysis, internal
thoughts can be observed that conversation analysis would ignore.
However, there are some limitations to the method. In addition to research which shows
protocol analysis may not capture all information (Chiu & Shu, 2010), retrospective protocol
analysis also has presents the issue of memory accuracy. While subjects reported being able to
remember ninety percent or greater of their thoughts while using this method, it should be noted
there is no certain way to determine how much information is missing. It is hard to quantify how
large an issue memory recall may be but this is the only approach available to investigate
individual thought processes in the collaborative setting and provides more information than just
conversation analysis.
69

6.5 Analyzing the Novelty of Ideas
To answer the third research question regarding the influence of each type of collaborative
stimulation, it was necessary to measure the creativity of the ideas produced. The most relevant
way to measure creativity was to measure concept novelty, as it is both quantitative and can
measure each concept individually, not only the entire set of concepts produced. To quantify
novelty an approach derived from Shah, Smith and Vargas-Hernandez (2003) was used.
6.5.1 Quantifying Idea Novelty
First, each new concept (or design entity) had to be identified in order to quantify its novelty. A
new design entity was identified by examining all the design entities found through the coding
scheme and determining when each design entity was created. These design entities were put
into a list with a time stamp of their occurrence. In the list, it was also noted if the design entity
was created by a collaboratively stimulated thought processes and the number of times other
groups had identified the same design entity.
After each design entity was identified, they were then categorized into a series of hierarchy
levels: functions, physical principles, working principles, embodiment, and details. Functions are
extracted from the problem statement and are the concept requirements. All the possible
functions from the problem statement were first identified, and the rest of the hierarchy was then
grouped under each function. Physical principles are verbs which fulfill a function. Working
principles are nouns which are able to implement the physical principle, usually through
structures. Embodiment is how the structure is constructed. Details consist of many minute
aspects, such as materials or aesthetics.
70

The total novelty of a design entity, N, is measured by comparing the maximum number
of times a design entity can be repeated (or the number of teams) to the number of times it was
invented by each team (Equation 6-1).

Equation 6-1 Novelty equation
In the equation, the number of times a design entity occurs is subtracted by the maximum
number of times a design entity can be repeated (equivalent to the number of teams in the
experiment), and then divided by the same. Note that equation 1 is slightly modified from Shah’s
metrics as the authors were interested in the novelty of each design entity and not total novelty
over the entire process. Therefore, novelty is not aggregated at each hierarchy level. Also novelty
is not given in terms of ideas and their repetition within categories, but rather the repetition of
specific ideas, as it is hard to place design entities into meaningful categories being so specific.
Next is a normalization factor of 10, used to prevent small decimal results. Finally, p is the
weighting factor for each level of the hierarchy. A weighting factor is necessary as novelty in
broad working principles is more influential than novelty in small details (Shah et al., 2003).
Physical principles have a weight of 10, working principles a weight of 6, embodiments a weight
of 3, and details a weight of 1.
6.5.2 Example Novelty Analysis
The transcript used for the protocol analysis example will also be used as a novelty analysis
example. In the transcript, two concepts, both at the working principles level in the hierarchy,
were identified. The first was “padlock” and was a concept which was repeated by all the teams.
71

The second concept, “ID card swipeable”, was repeated by only one team. To compute novelty
scores for both concepts, the weighting factor (6 for working principles) was multiplied by the
number of maximum possible repetitions (5, as there were 5 teams) minus the number of times
the design entity was repeated (5 times), and then divided by the number of maximum possible
repetitions (5). For the “padlock” concept, this resulted in (5-5)/5*10*6 (as the concept was
repeated by all 5 teams) producing a novelty score of 0. For the “ID card swipeable” concept, the
novelty was computed by (5-2)/5*10*6 (as the concept was repeated by 2 teams ) giving a total
novelty score of 36.
6.6 Results
The five cases of collaborative design lasted an average of 22 minutes, ranging from about 10 to
40 minutes. From these five cases, there were 92 occurrences of collaborative stimulations,
which resulted in the stimulation of 161 generative thinking processes. Table 6-3 gives specific
numbers for each occurrence of collaborative stimulation and each thinking process stimulated.
Collaborative
Stimulation
Number Thinking Process Number
Prompting 30 Memory Retrieval 58
Seeding 25 Association 24
Correcting 20 Transformation 77
Clarifying 17

Table 6-3 Collaborative stimulation results
Obtaining an intercoder reliability of 85% for collaborative stimulation verifies the first
research question’s proposed answer, the existence of collaborative stimulation, by multiple
individuals being able to identify collaborative stimulation occurring at the same point in the
transcript using the proposed definition. The answer to the second research question, regarding
72

the types of collaborative stimulation, was verified by all the proposed types of collaborative
stimulation (prompting, seeding, correcting, and clarifying) being observed with a high
intercoder reliability.
In the analysis of the concepts each team generated, a total of 296 concepts were found.
The number of concepts generated at each level of the hierarchy and their average novelty are
given in Table 6-4.
Hierarchy Level
Number of
Concepts
Average
Novelty
Physical Principle 44 30.9
Working Principle 98 31.5
Embodiment 94 22.8
Detail 60 7.9
Table 6-4 Novelty results
Additional data comparing the teams, like BICB results, stimulation frequency, analogies,
and other data, was available from the experiment. The average BICB score of the participants
was 8.90 with a standard deviation of 5.84. The average time between two consecutive
collaborative stimulations was 1 minute and 7 seconds, with a standard deviation of 35 seconds.
This created an average frequency of 0.89 collaborative stimulations per minute. The average
number of analogies generated per team was 6 with a standard deviation of 5.1. Table 6-5
compares and contrasts the results of each team.

73

Team Number 1 2 3 4 5 Avg.
Avg. BICB Score 4.5 10.5 1.5 14 14 8.9
Time Spent (min) 28.8 9.4 33.0 17.5 10.5 19.9
Tot. Prompting 9 2 3 12 4 6
Tot. Seeding 4 4 2 12 3 5
Tot. Correcting 5 2 6 5 2 4
Tot. Clarifying 5 3 5 0 4 3.4
Tot. Collaborative Stim. 23 11 16 29 13 18.4
Stim. Frequency (1/min) 0.8 1.2 0.5 1.7 1.2 0.9
Tot. Analogies 14 1 4 3 8 6
Table 6-5 Team results
The raw results presented above will be compared and contrasted to each other in the
next chapter in order to establish patterns. These patterns will provide answers to the third
research question.

74

7 Observed Collaborative Stimulation Patterns
7.1 Introduction
The raw data results consisted of collaborative stimulation occurrences, thought process
occurrences, novelty of ideas generated, BICB scores of the subjects, and collaborative
stimulation frequency. But when compared and contrasted to one another, they created
interesting patterns. In exploring patterns, those between collaborative stimulation and cognitive
processes will first be examined, followed by the patterns between collaborative stimulation and
novelty. Then some other observations between BICB scores and collaborative stimulation
frequency will be explored. Concluding, work from the pilot experiment will be presented
comparing and contrasting individuals working alone to collaborators.
7.2 Collaborative Stimulation and Thought Processes
It was possible to examine how often a specific thought process would be stimulated from each
type of collaborative stimulation. Prompting (Figure 7-1 upper left) inspired memory retrieval all
the time, but infrequently inspired association or transformation. Conversely, seeding (Figure 7-1
upper right) and correcting (Figure 7-1 lower left) resulted in the stimulation of transformation
all of the time, but rarely an association or memory retrieval. Clarifying (Figure 7-1 lower right)
inspired all the generative thinking processes examined with a greater influence on memory
retrieval (whose stimulation resulted 40% more often than association or transformation through
clarifying).
75

There was also found to be primary and secondary stimulation of thinking processes. A
primary stimulation occurred if the collaborative stimulation directly stimulated a thinking
process, where-as a secondary stimulation occurred when a generative thinking process was born
out of the results of collaborative stimulation (i.e. the generative thinking process would not have
been possible if the collaborative stimulation did not occur). The secondary stimulation effect
was rather minor and did not occur for all types of thought processes.

Figure 7-1 Probability of collaborative stimulation stimulating specific thought processes
(MR=Memory Retrieval; AS=Association; TF=Transformation)
The error bars in Figure 7-1 display a 90% confidence interval. It should be noted that
collaborative stimulation and thinking processes do not have a one to one relationship, as one
collaborative stimulation can stimulate multiple thinking processes.
0%
20%
40%
60%
80%
100%
120%
MR AS TF
% of times Pr stimulates a
specific cogntive process
Primary
Secondary
0%
20%
40%
60%
80%
100%
120%
MR AS TF
% of times Se stimulates a
specific cogntive process
0%
20%
40%
60%
80%
100%
120%
MR AS TF
% of times Co stimulates a
specfic cogntive process
Primary
Secondary
0%
20%
40%
60%
80%
100%
120%
MR AS TF
% of times Cl stimulates a
specific cogntive process
76

It was also possible to explore collaborative stimulation by examining the thought
process. This takes the approach of examining what percent of all the collaboratively stimulated
thought processes were stimulated by each type of collaborative stimulation. Collaboratively
stimulated memory retrieval (Figure 7-2 left) occurred most frequently through prompting
(almost 60% of the time), occasionally through clarifying, and rarely through correcting. There
were no experimental observations of memory retrieval being stimulated through seeding.
Collaboratively stimulated association (Figure 7-2 center) was generally inspired by prompting
or clarifying, and less often by seeding and correcting. Secondary prompting also had a
significant role inspiring association. Collaboratively stimulating transformation (Figure 7-2
right) was most often stimulated (73% of the time) through seeding and correcting. Clarifying
was moderately influential, and prompting and secondary effects were minimal. Once again, the
error bars represent a 90% confidence interval.

Figure 7-2 Percent of thought processes stimulated by each collaborative stimulation type
(Pr=prompting; Se=seeding; Co=correcting; Cl=clarifying; Xx2=secondary)
0%
10%
20%
30%
40%
50%
60%
70%
80%
Pr Co Cl
Memory Retireval Stimulation %
Collaborative
Stimulaiton Type
0%
5%
10%
15%
20%
25%
30%
35%
Pr Se Co Cl Pr2
Association Stimulation %
Collaborative Stimulaiton
Type
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Pr Se Co Cl Pr2 Co2
Transformation Stimulation %
Collaborative Stimulaiton
Type
77

From the graphs above, it is evident recognizable patterns between collaborative
stimulation and thought processes occurred within the realms of statistical significance (90%
confidence intervals). This provides part of answer to the third research question, regarding the
influence of each type of collaborative stimulation, as more influential types of collaborative
stimulation will stimulate more thought processes. The results found in the graphs also align with
the findings of others. Prompting was most influential on memory retrieval, which is supported
by other research stating ideas from one collaborator will stimulate the memory of another
collaborator (Brown et al., 1998; Dugosh et al., 2000; Nijstad & Stroebe, 2006). Seeding showed
a strong relationship with transformation, thus aligning with aggregate models of group
creativity (Shalley & Perry-Smith, 2008; Taggar, 2002) and ideation methods like C-sketch
(Shah et al., 2001), which propose collaborators build on (or transform) one another’s ideas to
produce more creative outcomes. Correcting was also found to have a strong relationship with
transformation, aligning with work suggesting that critiquing and then transforming a design
from critiques is an important stage of the design process (Holsapple & Joshi, 2002).
Clarifying, mostly stimulated memory retrieval but also showed moderate relationships
with association and transformation. Initially, the finding may seem a bit surprising as one of the
key ways clarifying was expected to work was through analogies, which have been used explain
concepts (Goldschmidt, 2011). However, an analogy consists of a memory retrieval, association,
and then transformation (Jin & Benami, 2010), so why were all three not stimulated equally?
This could be explained by incomplete analogies and some analogies focusing on explanations,
not moving toward generating new ideas. This is supported by findings (see the later section in
this chapter on analogies), which found only some explanatory analogies would become
generative. The generative analogies are the only ones which would produce transformation and
78

association, therefore a majority of all analogies would only produce memory stimulation. This
aligns with the observed results in Figure 7-1.
While the graphs in Figure 7-1 and Figure 7-2 provide a lot of information regarding
relationships between collaborative stimulation and thought processes, the information can be
complex to absorb. The results can be simplified by constructing a relationship strength matrix
between the various types of collaborative stimulation and generative thought processes from the
observed patterns ( Table 7-1).
Collaborative Stimulation MR AS TF
Design
Entity
Inspired
Prompting ●●● ●

Seeding

● ●●●
Question
Inspired
Correcting ● ● ●●●
Clarifying ●●● ●● ●●
Table 7-1 Collaborative stimulation and thought processes relationship strength matrix
But of what practical use are these patterns? They provide implications for how to develop
methodologies which target a specific type of thought process. For example, if it was desired to
stimulate memory retrieval, which would probably be important at the beginning of the design
process when generating multiple concepts, then it would be advantageous to have
methodologies which focus on encouraging collaborators to stimulate each other through
prompting or clarifying. Later on in the design process, when the focus moves to developing a
concept, it would be useful to have methodologies which encourage seeding and correcting,
stimulating transformation. The implications of stimulating specific types of collaborative
stimulation will be explored in detail in the next chapter.
Strong ● ● ●
Moderate ● ●
Weak ●
79

One other use of patterns, is exploring what mechanisms tend to work for collaboratively
stimulating thought processes. It can be observed that question-inspired stimulation has a higher
likelihood of stimulating more thought process, and this is specifically true with regards to
transformation. Interestingly, there has been much work on encouraging design entity inspired
stimulation, (Michalko, 2001; Nijstad & Stroebe, 2006; Shah et al., 2001). However, much less
time has been spent on developing effective questioning methods. Perhaps this is because early
work regarding brainstorming demonized the act of questioning (Osborn, 1957).
7.3 Collaborative Stimulation and Novelty
Patterns could also be drawn between the concept novelty and collaborative stimulation, by
comparing the average novelty of ideas resulting from collaboratively stimulated thought
processes, and those individually stimulated (Figure 7-3). The error bars in Figure 7-3 represent
80% confidence intervals.

Figure 7-3 Stimulation and average novelty
0.00
5.00
10.00
15.00
20.00
25.00
30.00
35.00
Prompting Seeding Correcting Clarifying Individual
Stimulation
Average Concept Novelty
Type of Stimualtion
80

Surprisingly, Figure 7-3 shows design entities influenced by only individual stimulation
were just as novel, and in many cases more novel, than those influenced by collaborative
stimulation. It was anticipated ideas which a team builds together would be more novel.
However, exploring the distributions of ideas provided an explanation as to why individual
stimulation produced more novel ideas. Individually stimulated ideas were found to roughly
occur equally in all four hierarchy levels. However, collaboratively stimulated ideas only started
to occur at the lower three hierarchy levels (Figure 7-4). This means ideas collaboratively
stimulated had an inherent disadvantage when looking at the average concept novelty as they
only occurred at hierarchy levels with a lower weighting factor. On the other hand, the
individually stimulated ideas received a large novelty bonus by influencing many ideas which
had a 10x weighting factor.

Figure 7-4 Distribution of stimulated ideas
If novelty was examined at only one hierarchy level, weighting no longer becomes an
issue. In Figure 7-5, which explores only the working principle hierarchy level, design entities
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Physical
Principle
Working
Principle
Embodiment Detail
Stimulated Idea Distribution
Level of Abstraction
Individually
Stimulated
Collaboratively
Stimulated
81

influenced by collaborative stimulation were generally more novel than those influenced by only
individual stimulation. The only exception was the results for prompting.

Figure 7-5 Working principles average novelty
What is particularly fascinating is how the results presented in Figure 7-5 aligns with how
likely a collaborative stimulation is to produce a transformation as outlined in Table 7-1.
Seeding, clarifying and correcting which have strong and moderate relationships with
transformation produce the most novel ideas. Prompting, which had no relationship, produced
the least novel ideas. This finding aligns with the definition of novelty, which explains it to be
most influenced by the thought process of transformation (Shah et al., 2003) and by discoveries
in creative cognition finding a connecting between novelty and transformation (Jin & Benami,
2010). Promptings poor performance in novelty also makes sense as prompting generally results
in memory retrieval. When an idea is a retrieved memory, it occurs because a designer observed
it before, and therefore other designers are likely to have observed the same concept. As novelty
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
Prompting Seeding Correcting Clarifying Individual
Stimulation
Average Working Principle Novelty
Type of Stimualtion
82

is measured by comparing how many other designers have the same idea, those which come
from memory are likely to be more common.
The concepts produced by collaboratively stimulated cognitive processes can be further
broken down by hierarchy levels to observe how many ideas collaborative stimulation influenced
at each level (Figure 7-6).

Figure 7-6 Collaborative stimulation and hierarchy levels
Combining the number of concepts influenced by collaborative stimulation at each
hierarchy level and the novelty of each concept, a graph could be developed which compares the
average novelty of concepts influenced by each type of collaborative stimulation (and no
stimulation) at each hierarchy level (Figure 7-7). Once again, the error bars represent 80%
confidence intervals. It should be noted in this graph, as it compares novelty at each hierarchy
level, the weighting factor has been removed, which makes 8 the maximum possible novelty
score.
0
2
4
6
8
10
12
14
16
18
Physical
Principle
Working
Principle
Embodiment Detail
Number of Concepts
Hierarchy Level
Prompting
Seeding
Correcting
Clarifying
83

Figure 7-7 Average concept novelty per hierarchy level
Both seeding and correcting performed well in all cases, and clarifying does very well at
the working principle level. The data at the detail level attained the maximum level of novelty (as
details are rarely repeated in by just 5 groups), and therefore while ideas influenced by
collaborative stimulation were slightly more novel than ideas influenced by individual
stimulation, this information is not very meaningful. What is specifically interesting is that ideas
stimulated by correcting and seeding are not seen to occur until after the physical principle level
(Figure 7-6), but they produced the ideas with the highest average novelty at all the other levels
(Figure 7-7).
The findings presented in this chapter have provided further information regarding the
influence of collaborative stimulation and have demonstrated seeding, correcting, and clarifying
are the most effective for influencing novelty. However, just because an idea is not novel does
not mean it is a poor solution to the problem, but novel ideas tend to produce some of the
greatest gains for those in industry giving a company a competitive advantage.
0
1
2
3
4
5
6
7
8
9
PP WP EM DL
Average Concept (unweighted)
Novelty

Hierarchy Level
Ms
Se
Co
Cl
None
84

7.4 Collaborative Stimulation and Analogies
The second pilot experiment and the full experiment both produced results regarding analogies.
In the pilot experiment, each collaboration group had an average of 1.0 analogies per designer,
where-as the individuals had an average of 0.67 analogies per designer. In the full experiment,
there were anywhere from 0.5 to 7 analogies per designer, with an average of 3. Unfortunately,
due to the limited number of data points in both experiments and the high standard deviation, no
statistically significant data was obtained. However, some interesting patterns were observed in
the full experiment.
There are several ways to categorize analogies. First, there are surface analogies and
structural analogies. Surface analogies have a source and target which are similar, except for
perhaps a few surface components. Structural analogies have a source and target which are
wildly different, and tend to only be made by experts. Of the two, structural analogies tend to be
the most powerful, and create the most novel ideas (Novick, 1988). The types of analogies
occurring in the experiment were divided into those occurring under the influence of
collaborative stimulation, versus those not involving stimulation or “none”. Interestingly, the
collaboratively stimulated analogies tended to be surface, and most of the structural analogies did
not involve stimulation (Figure 7-15).

85

Figure 7-8 Structural and surface analogies
Second, there are explanatory analogies and inventive analogies. Explanatory analogies
are used to explain a concept, whereas inventive analogies are used to find a new solution to a
problem. Inventive analogies are the only ones which develop new concepts to solve a problem
(Goldschmidt, 2011). Once again, the strongest analogies occurred outside the influence of
collaborative stimulation. In fact, most of the explanatory analogies involved collaborative
stimulation, whereas most of the inventive analogies did not involve collaborative stimulation
(Figure 7-15).

Figure 7-9 Inventive and explanatory analogies
The analogies influenced by collaborative stimulation could be attributed to specific
stimulation types. Prompting had the largest influence on the stimulation of analogies, and
86

clarifying had a secondary input. All of the analogies clarifying influenced were explanatory, as
expected.

Figure 7-10 Analogies and collaborative stimulation type
However, an interesting effect was discovered looking at the clarifying analogies in
detail. While all clarifying analogies were explanatory, 80% them became inventive. The
analogy would start by the collaborator using it to explain an idea to his partner, but then midway
through the explanation the collaborator would realize the analogy also held some transformative
implications. Then the analogy would become inventive as it generated new concepts.
7.5 BICB Scores and Stimulation Frequency
One other pattern explored were relationships between past creative experience of the designers
and collaborative stimulation. Past creative experience was measured by BICB scores, and was a
predictor variable in the collaborative stimulation verification experiment. The team average
BICB score, ranging from 1.5 to 14, and the frequency of collaborative stimulation, ranging from
0.5 to 1.7 stimulations per minute, had a strong correlation of 0.94 (Figure 7-11).
87

Figure 7-11 Team avg. BICB vs. stimulation frequency
Similar results were also observed on the individual level. Each individual’s BICB and
frequency of collaborative stimulation also had a strong correlation of 0.89 (Figure 7-12).

Figure 7-12 Individual BICB vs. stimulation frequency
These results signify a broad variety of past creative experiences has a strong positive
correlation with stimulation frequency. While correlation is not necessarily causation, these
results, on both the individual and team levels, would suggest individuals with high diversity in
creative backgrounds are either stimulated more easily or better stimulate their collaborator. Of
88

course, both effects could be at work. This theory aligns with research which has revealed
individuals with diverse creative backgrounds to be more effective in collaboration (Perry-Smith
& Shalley, 2003; West, 2002).
As mentioned earlier, collaborative stimulation occurs through the mechanisms of shared
design entities or questions. In each individual, the percentage of stimulation by a design entity
(prompting and seeding) versus a question (clarifying and correcting) was an average of 55%
with a standard deviation of 26%. This ratio, or the percent of time a student was stimulated by a
design entity, was compared to each designer’s BICB score and a very weak correlation of 0.31
was found (Figure 7-13).

Figure 7-13 Percent Design Entity Inspired
A weak positive correlation was found between BICB scores and the percent of time a
student was stimulated by the design entities their collaborator invented. While a positive
correlation should be expected, it was weaker than anticipated because of past research results. A
team’s diversity in background results in each team member being stimulated differently than
their collaborator by the same concept (West, 2002). This would lead to more new ideas being
produced through collaborative stimulation if the team had a high diversity of creative
experiences. The same theory has been proposed from a network perspective: that a team’s
89

creative potential comes from the members’ diverse connections outside the team (Perry-Smith
& Shalley, 2003). Therefore, this research would imply that as diversity of past creative
experience increases (higher BICB scores), there should be a shift to more design entity inspired
stimulations.
However, the results suggest it is not only design entity inspired stimulation which is
correlated with past creative experiences. Past creative experiences appear to increase the
frequency of both question and design entity inspired collaborative stimulation, shown by an
increased frequency of collaborative stimulation but only a weak increase in the percentage of
design entity inspired stimulation correlated with BICB scores. Studies from interpersonal
congruence provide theories which may explain why both question and design entity inspired
stimulation increased about equally.
Work in the area of interpersonal congruence shows diversity encourages creative
collaboration as long as interpersonal congruence (how well group members can understand each
other’s perspectives) is high (Polzer et al., 2002). Since the engineering students had similar
educational backgrounds and were working on technical projects in their area of expertise, it was
expected interpersonal congruence would be high. This would result in greater creativity (or
more stimulation) for teams with greater diversity in creative experiences (and also higher BICB
scores). It could also by hypothesized that team members with diverse creative backgrounds may
actually have the ability to have higher interpersonal congruence levels. A diverse creative
background provides more areas for team members to relate to their collaborator’s perspective.
This mean they would be more open to a design entity being proposed or a question being asked,
which would lead to greater collaborative stimulation. A limitation to these results should be
90

noted: personality inventories were not included as part of the study, but will be included in
future studies.
7.6 Individual and Collaborative Stimulation of Memory Retrieval
While there were a number of points investigated from the first pilot experiment, because of the
small sample size the only somewhat significant data (85% confidence) occurred regarding
memory retrieval, which is one of the most common processes.
In analyzing the transcripts, it was found that memory retrieval was stimulated by design
entities through both the collaborative stimulation of prompting and individual memory
stimulation. The experimental condition averaged 2.25 promptings (collaborative stimulations)
of memory retrieval per designer with a standard deviation of 1.7 and 2.50 individual
stimulations per designer with a standard deviation of 1.29. The difference between these two is
not statistically significant. The results are shown in Figure 7-14, with 70% confidence intervals.

Figure 7-14 Experimental stimulation breakdown
In comparing the control group to the experimental group (Figure 7-15), it was observed
the control group had 2.0 cases of design entities stimulating memory retrieval per person with a
91

standard deviation of 1.00, whereas the experimental group had 4.75 cases of design entities
stimulating memory retrieval per person with a standard deviation of 2.21. The results were
found to have a significance level of p = 0.15.

Figure 7-15 Collaborative stimulation: control vs. experimental
The error bars above mark 70% confidence intervals. While it is evident there is a
difference between the experimental and the control group, one of the first questions raised is,
“Did the experimental group have any advantages besides collaboration?” In actuality, the
experimental group had two disadvantages with regards to BICB scores and time spent on the
problem.
The experimental group had an average BICB score of 7.5 while the control group had an
average BICB score of 11.3. Because of the small sample size, it was impossible to obtain the
same average BICB score in the control and experimental groups. Therefore it was decided to
allow the BICB score to weigh in favor of the control group, as it would be undesirable to negate
positive implications for collaboration by BICB scores favoring the experimental group. The
experimental group spent an average of 25 minutes on the problem (ranging from 21 to 30
minutes), while the control group spent an average of 35 minutes on the problem (ranging from
92

22 to 43 minutes). This means there was a significant difference in the period of stimulation of
memory retrieval, as the experimental group had an average of 5.4 minutes between each
stimulation whereas the control group had an average of 17.5 minutes between each stimulation.
Therefore, it is evident the effect of collaboration on stimulation is powerful.
Fascinatingly, while prompting was a key element in making the stimulation of memory
retrieval more likely in the collaborative setting than the individual one, the number of individual
memory stimulations was also greater. There were 2.5 individual stimulations per designer in the
collaborative setting, whereas in the individual there were only 2.0 stimulations per designer.
This would imply that collaboration not only has a direct effect on stimulation (through
prompting), but that there is also an indirect effect on individual stimulation. Similar
observations of collaboration’s indirect stimulating effect were made by Dugosh, et al. (2000).
But why does collaboration lead to greater stimulation of memory retrieval? An
explanation could be that prompting reduces design fixation. When a collaborator shares a design
entity, it may stimulate an individual, pushing them away from their fixated perspective and
causing them to remember new perspectives. This theory would be supported by creative
cognition work which has found uncommon (Perttula & Sipilä, 2007) and immature (Jin &
Benami, 2010) stimuli result in less fixation. Instead of a third party providing a stimulus as has
occurred in past research (e.g. Chan et al., 2011), the collaborator provides the stimulating design
entity which results in prompting. While collaborating may assist in reducing fixation on the
individual level, caution must be exercised to ensure fixation does not occur on the team level,
also known as group think (Esser, 1998).
93

8 Implications for Collaboration and Recommendations
8.1 Introduction
In the past chapter, a number of interesting patterns and findings were discussed. These observed
patterns and findings together provide an important base for developing useful applications. This
chapter discusses the applications and implications from the results presented in the previous
chapter. They are focused on the development of questioning techniques, encouraging early
collaborative stimulation, creative diversity training, and enhancement of memory retrieval
stimulation.
8.2 Development of Questioning Techniques
As was mentioned in the results, while there is a lot of work highlighting the effects of design
entity inspired stimulations of prompting and seeding (e.g. Brown et al., 1998; Pirola-Merlo &
Mann, 2004), the effects of question inspired collaborative stimulation are not explored in detail.
Perhaps this is because questioning is discouraged in idea generation for fear of increasing social
inhibitions, particularly applying to brainstorming (Osborn, 1967).
However, from a collaborative stimulation perspective, question inspired stimulation is
related to more thought processes through stronger connections than design entity inspired
stimulation (as can be observed in Table 7-1). In particular, clarifying, is quite powerful as
unlike any of the other types of collaborative stimulation, moderate and strong relationships exist
stimulating all three generative thought processes. Additionally, from a novelty perspective,
correcting and clarifying produced the most novel ideas at the working principles hierarchy level
94

and are a major contributor at the embodiment level. This suggests questioning can be more
effective than the presentation of design entities in the collaborative setting.
Unfortunately question inspired stimulation occurred much less frequently (Table 6-5)
than design entity inspired stimulation. While there have been many methods which suggest way
of increasing design entity inspired collaborative stimulation, like sharing design notebooks
(Michalko, 2001), sharing sketches (Shah et al., 2001), or providing stimulus to individuals
(Nijstad & Stroebe, 2006), there is not much work addressing effective questioning methods.
Those which do tend to be vague, focusing on other issues like analogies (Goldschmidt, 2011) or
ontological design (Holsapple & Joshi, 2002). Therefore, it is needful for interventions to be
developed which encourages effective questioning within collaboration. Effective interventions
for encouraging questioning should:
 Assert questioning is not negative and make the stimulation benefits of
encouraging clarifying and correcting clear to designers.
 Promote the over clarification of ideas and provide an environment which
encourages designers to challenge each other’s concepts with encouraging,
constructive criticism.
 Still provide time for idea generation without questioning, in case social
inhibitions are a major inhibitor to the group.
8.3 Encouraging Early Collaborative Stimulation
Seeding, correcting and clarifying were found to be the most influential collaborative
stimulations increasing idea novelty. But, from examining Figure 7-4 and Figure 7-6, it can be
95

observed that in the natural design process they occur more often at lower hierarchy levels, with
most seeding, correcting and clarifying occurring during embodiment. It would be beneficial to
encourage seeding, correcting and clarifying at higher hierarchy levels where they have the
opportunity for greater influence.
Both systematic (Pahl & Beitz, 1996) and axiomatic (Suh, 2001) approaches to design
explain the process as beginning with high level concepts, and then working its way down to
details. In the experiment, the design problem statement defined the problem/customer needs and
even some functions. Therefore, the design did not start until defining functions and proposing
physical principles. Figure 7-4 appears to show a delay between when the design problem starts
(functions) and when collaboration starts (working principles), creating a zone where a person is
only individually stimulated (see Figure 8-1).

Figure 8-1 When design and collaboration start
The individual zone are hypothesized to occur when the designer proposes design entities
at the point where the design problem starts (like a physical principle which is individually
stimulated), however the idea is not collaborated on as designers move from the uncertain
96

abstract to the certain concrete as quickly as possible. Rather, additional ideas are built on the
design entity at lower, more concrete abstraction levels. Therefore, in order to encourage
collaborative stimulation at higher abstraction levels, it is necessary to start collaboration in the
initial stages of defining the problem.
Another approach besides starting the design earlier would be to decrease the size of the
individual zone. This could possibly be accomplished by developing interventions which focus
on encouraging seeding, correcting, and clarifying early in the design process. Interventions
which work at an abstract level perform the best and avoid fixation (Jin & Benami, 2010).
The nature of an intervention encouraging correcting and clarifying was already discussed
in the previous section. Seeding could be encouraged by methods which get collaborators to
build on each other’s ideas. In fact, Shah’s method, mentioned earlier, heavily uses this type of
collaborative stimulation with great success (Shah et al., 2001). While Shah’s method focuses on
sketches, there are other opportunities for interventions which would encourage building on
ideas verbally or with physical prototypes. Effective interventions to increase concept novelty
should:
 Promote seeding, clarifying and correcting early in the design process when
physical and working principles are being developed.
 Encourage designers to build on each other’s ideas verbally, visually and
physically and challenge each other’s concepts with encouraging, constructive
criticism.
 Convince designers to spend more time in abstract concept generation.
In addition to the interventions, it is necessary designers start on the design problem at the
97

highest abstraction level possible.
8.4 Creative Diversity Training
From the findings regarding BICB scores and stimulation frequency, as well as previous
research, it is evident diverse past creative experiences provide triggers for increased
collaborative stimulatio. Unfortunately, often times current engineering design education does
not provide or encourage rich and diverse creative experiences. While many design classes have
a component training engineers in the creative thinking process (Cropley & Cropley, 2000;
Stouffer, Russell, & Oliva, 2004), there is a lack of classes encouraging students to explore
diverse areas of creative activity (Richards, 1998). If a student participated in only the activities
most engineering design programs provide and encourage, their creative activity would be
represented as only a seven or eight on the BICB. But designers in the experiment who were
stimulated most frequently had scores of more than double this number. From a collaborative
stimulation perspective, there is a dire need for engineering design education to provide a greater
diversity of creative experiences.
One of the first lessons others have noted is that educating engineering students to be
creative, especially in areas of psychology and the studio arts, broadens the student’s
perspectives (Richards, 1998). Focusing on only a single area in education can lead to low
interpersonal congruence, which results in poor team performance (Polzer et al., 2002). While
expertise in their area of training is a requirement, focusing on just a single expertise can lead to
arrogance and the false belief that a purely engineering perspective is most important. It is
helpful to accompany training in diverse creative experiences by having engineering students
work on multi-disciplinary teams including students from outside the field of engineering
98

(Richards, 1998). This can assist in understanding the value of perspectives from different
disciplines.
However, training engineering students in a diverse set of creative experiences is not
without its challenges. While it would be advantageous to design new or modify current classes
to incorporate training in diverse creative experiences, often there is a lack of resources. Finding
the right faculty to design and teach such a course is a challenge. Additionally, getting current
faculty within the engineering department to accept the new courses can be even more
challenging, as students’ course loads are already full of technical courses and many faculty do
not see the value in creativity exercises for engineers. Even when such classes are successfully
created, a key challenge will be creating long lasting change in the students. Current studies
caution courses on creativity may only produce short term results, not long lasting results
impacting the student’s career (Cropley & Cropley, 2000). To effectively overcome these
challenges, courses should be developed which:
 Train engineering students in a diverse set of creative experiences, from acting to
painting.
 Have dedicated instructors behind them who are will to push through whatever
obstacles needed to establish such a course.
 Establish long-term habits in engineering students to continue participation in
creative activities.
Of course, efforts to create such a course should start as a pilot study, to ensure training
in diverse creative activities indeed is effective.
99

8.5 Enhancing Memory Retrieval Stimulation
A few implications for collaborative methodologies encouraging creativity can be drawn from
the results of the second pilot experiment. First, it is important for designers to have full
exposure to each other’s ideas. As prompting was stimulated by external design entities, it is
important for designers to ensure any internal design entities are shared and become external.
Secondly, memory retrieval is more frequently stimulated by design entities in the group setting
than the individual setting (Figure 7-15). Therefore, it is important designers generate concepts
together, or at least in a way ideas can be shared. For example, methods like electronic
brainstorming can be used to reduce social inhibitions and procedural issues (Gallupe et al.,
1992) or only sharing the concepts momentarily has provided positive design outcomes (Perttula
et al., 2006). Thirdly, the method should avoid fixation on the team level, or groupthink (Esser,
1998), which occurs when the entire team begins to think within a limited set of solutions. In
design experiments, groups have been observed to especially fixate when a poor example is
provided (Fu et al., 2010). This can be avoided by the presence of an outside “watchdog” attuned
to identifying groupthink.
100

9 Contributions and Future Work
A number of useful insights and implications were discussed in the previous chapter. In this
section, the various contributions from this dissertation and subsequent future work are laid out.
9.1 Contributions
This work has made contributions through three key areas: the CTS model and identification of
collaborative stimulation, developing retrospective protocol analysis, and a foundation and
recommendations for creative collaborative design.
9.1.1 CTS Model and Collaborative Stimulation
This work has contributed to the creative cognition and group creativity approaches to design by
proposing the CTS model, extending creative cognition to collaboration. The CTS model
contributes to current approaches by proposing that collaboration impacts thought processes
through the sharing of design entities. It also defines the concepts of internal generation,
external generation, internal design entities, external (shared) design entities, individual
stimulation, and most importantly, collaborative stimulation.
The primary contribution of this work is the proposal of collaborative stimulation and the
identification of various types of collaborative stimulation. Collaborative stimulation was
proposed to occur through two mechanisms: design entities and questions. Design entity initiated
collaborative stimulations were prompting and seeding, whereas question initiated stimulations
were correcting and clarifying. Patterns were then empirically found between the various types
of collaborative stimulation and thought processes, as displayed in Table 7-1. The patterns and
101

mechanisms contribute to creative cognition and group creativity in design by providing a better
understanding of how thought processes are influenced by collaborative interactions.
9.1.2 Retrospective Protocol Analysis
In order to observe thought processes and collaborative stimulation, it was necessary to have a
method which would identify thinking occurring during collaboration. This was accomplished by
adapting current protocol analysis methods to create retrospective protocol analysis. The method
worked by videotaping the design process, and then immediately after the design, the
collaborators would watch the video and verbalize what they were thinking.
While the method was successfully used for studying the design process, it could be used
in a variety of applications in cognitive science. Therefore, the development of the retrospective
protocol analysis experimental approach not only contributes to design theory and methodology,
but also a wide variety of cognitive studies involving collaboration.
9.1.3 Recommendations for Creative Collaborative Design
The development of better design methods provided the initial motivation for this work. In
chapter 8, a number of suggestions were given in regards to collaborative stimulation, which can
result in more creative design. The suggestions are summarized below:
Designers should be encouraged to ask good questions, as question initiated stimulations show
strong patterns with multiple thought processes and novelty.
Designers should build on each other’s ideas, as seeding was found to create highly novel ideas.
Teams should start working together at the problem definition stage of a design, so collaborative
stimulation will occur at high hierarchy levels.
102

Designers should have a diverse set of creative experiences in order to make them effective
creative collaborators.
Designers should work together and be fully exposed to each other’s ideas when generating
concepts in a way that does not promote groupthink.
These suggestions are specifically useful to those within the field of design theory and
methodology who are looking at ways to encourage group creativity. They are also beneficial for
those who are creating collaborative methods unrelated to creativity, but want to ensure they do
not negatively impact creativity, which can be accomplished by following the guidelines.
9.2 Future Work
While this research has made many valuable contributions, there are a number of areas that were
beyond the scope of this dissertation’s time frame. Key areas for expansion are collaborative
stimulation & the GSP model, experimental methods, and interventions & industry applications.
Further research in each of these areas could greatly increase this work's contributions.
9.2.1 Collaborative Stimulation and GSP Model
This work only explored the collaborative stimulation of generative thought processes. The
collaborative stimulation of exploratory (convergent) thought processes, consisting of problem
analysis and solution analysis, should also be examined in detail. Various types of collaborative
stimulation for exploratory thought processes need to be identified, as well as their impact. It is
expected exploratory processes will be stimulated through the mechanisms of both design
entities and questions. However, there may be some additional mechanisms of stimulation yet to
be discovered for exploratory thought processes.
103

There are also a number of areas of the collaborative thought stimulation model which
could use further development. First, the entire cycle beyond collaborative stimulation should be
verified by identifying the cycles and confirming the validity of the definitions of internal
generation, external generation, internal design entities, and external design entities. Specific
internal and external design operations, and how each operation influences collaboration should
also be identified.
After the rest of the details of the CTS model have been verified, there is a need to extend
the model to account for the different perspectives of each collaborator. Specifically, it is
important to understand how the various perspectives of the shared design entity space influence
the design process. Finally, social inhibitions' and procedural issues' influence cycles should be
explored to determine if they alter the model at all.
9.2.2 Experimental Methods and Analysis
In addition to all the collaborative cases analyzed in this work, the collaborative
stimulation verification experiment also had a total of seven individuals who worked alone on
the same design problem. While most of this data has been transcribed, it has not yet been coded.
It would be beneficial to code this data, and then explore patterns between individuals and
groups.
Follow-up experiments should also be conducted involving more subjects for greater
statistical significance of data regarding patterns and novelty. These experiments should compare
and contrast collaborative and non-collaborative cases. Also, the individuals involved should be
evaluated by the Raven, mechanical reasoning, personality, and general reasoning tests beyond
just the BICB, to better understand the many different factors influencing collaborative
104

stimulation. It would also be beneficial to run an experiment in which retrospective protocol
analysis is contrasted to standard protocol analysis in the individual case providing insights into
the performance of retrospective protocol analysis.
After experiments with more subjects have been completed, there is need to experiment
with groups of greater than two. While the results for dyads can be used as a baseline to
understand how larger groups will be influenced by collaborative stimulation, no doubt some
changes in patterns would be observed. Also, there is a need to explore how patterns and the
CTS model may change if applied to industrial professionals rather than relatively inexperienced
students. This can be accomplished by performing the same experiment with both students and
professionals, and then comparing and contrasting the similarities and differences.
9.2.3 Interventions and Industry Applications
The implications from this research provide the knowledge for developing a number of
interventions to encourage creativity from collaboration. The first step for creating those
interventions is to identify how the implications can be applied. Some specific approaches may
be:
1. Develop a mini-course on effective constructive questioning methods in order to instruct
and equip collaborators to ask questions which result in stimulation.
2. Train design engineers in diverse creativity activities in an attempt to raise their
frequency of stimulation.
3. Test whether collaborative involvement in the early stages of defining a problem
increases collaborative stimulation at higher levels of abstraction. Also, the scenario
should be tested in which designers receiving a problem statement are told to entirely
105

reformulate the problem. Does this improve collaborative stimulation at high abstraction
levels?
4. Include questioning during a brainstorming session to increase the kinds of collaborative
stimulation occuring, thus possibly increasing the effectiveness of brainstorming.
The next step would be to test these devised methods with an experimental and control group, in
which the control group proceeds with the design using a traditional method. The goal of these
experiments should be to observe whether the experimental group using the intervention
produces a greater quantity, quality, variety and/or novelty of ideas than the experimental group.
As the experiments are likely to start in the academic setting, after the interventions have been
refined, they should also be tested and modified for effectiveness in industry.
106

10 Bibliography
Altshuller. (1984). Creativity As an Exact Science (1st ed.). CRC Press.
Amabile, T. (1982). Social Psychology of Creativity: A Consensual Assessment Technique.
Journal of Personality and Social Psychology, 43(5), 997–1013.
Amabile, T., Conti, R., Coon, H., Lazenby, J., & Herron, M. (1996). Assessing the Work
Environment for Creativity. The Academy of Management Journal, 39(5), 1154–1184.
Batey, M., & Furnham, A. (2008). The relationship between measures of creativity and
schizotypy. Personality and Individual Differences, 45(8), 816–821.
doi:10.1016/j.paid.2008.08.014
Benami, O., & Jin, Y. (2002). Creative stimulation in conceptual design. Proceedings of ASME
DETC/CIE, Montreal, QC, Canada: DETC2002/DTM-34023, 2(1).
Brereton, M., Cannon, D., Mabogunje, A., & Leifer, L. J. (1997). Collaboration in Design
Teams: How Social Interaction Shapes the Product. In N. Cross, H. Christiaans, & K.
Dorst (Eds.), Analysing Design Activity (1st ed.). Wiley.
Brown, V., Tumeo, M., Larey, T. S., & Paulus, P. B. (1998). Modeling Cognitive Interactions
During Group Brainstorming. Small Group Research, 29(4), 495 –526.
doi:10.1177/1046496498294005
Carson, S. H., Peterson, J. B., & Higgins, D. M. (2005). Reliability, Validity, and Factor
Structure of the Creative Achievement Questionnaire. Creativity Research Journal,
17(1), 37–50. doi:10.1207/s15326934crj1701_4
Catford, L. R., & Ray, M. L. (2004). The Path of the Everyday Hero: Drawing on the Power of
Myth to Meet Life’s Most Important Challenges. Forestville, Calif.: Creative Quest Pub.
107

Chan, J., Fu, K., Schunn, C., Cagan, J., Wood, K., & Kotovsky, K. (2011). On the Benefits and
Pitfalls of Analogies for Innovative Design: Ideation Performance Based on Analogical
Distance, Commonness, and Modality of Examples. Journal of Mechanical Design,
133(8), 081004–081004. doi:10.1115/1.4004396
Chiu, I., & Shu, L. (2010). Potential Limitations of Verbal Protocols in Design Experiments.
Presented at the ASME International Design Engineering Technical Conferences &
Computer and Informaiton Engineering Conference, Montreal, Quebec, Canada: ASME.
Chusilp, P., & Jin, Y. (2006). Impact of Mental Iteration on Concept Generation. Journal of
Mechanical Design, 128(1), 14–25. doi:10.1115/1.2118707
Cropley, A. J. (2000). Defining and measuring creativity: Are creativity tests worth using?
Roeper Review, 23(2), 72–79.
Cropley, D. H., & Cropley, A. J. (2000). Fostering Creativity in Engineering Undergraduates.
High Ability Studies, 11(2), 207–219. doi:10.1080/13598130020001223
Cross, N., Christiaans, H., & Dorst, K. (Eds.). (1996). Analysing design activity (Vol. 11, p. 463).
Chichester: Wiley.
Cross, N., & Clayburn Cross, A. (1995). Observations of teamwork and social processes in
design. Design Studies, 16(2), 143–170. doi:10.1016/0142-694X(94)00007-Z
De Bono, E. (1970). Lateral Thinking: Creativity Step by Step. Harper & Row.
Diehl, M., & Stroebe, W. (1987). Productivity loss in brainstorming groups: Toward the solution
of a riddle. Journal of Personality and Social Psychology, 53(3), 497–509.
doi:10.1037/0022-3514.53.3.497
Dong, A. (2005). The latent semantic approach to studying design team communication. Design
Studies, 26(5), 445–461. doi:10.1016/j.destud.2004.10.003
108

Dong, A., Kleinsmann, M. S., & Deken, F. (2013). Investigating design cognition in the
construction and enactment of team mental models. Design Studies, 34(1), 1–33.
doi:10.1016/j.destud.2012.05.003
Dugosh, K. L., Paulus, P. B., Roland, E. J., & Yang, H.-C. (2000). Cognitive stimulation in
brainstorming. Journal of Personality and Social Psychology, 79(5), 722–735.
doi:10.1037/0022-3514.79.5.722
Ericsson, K. A., & Simon, H. A. (1993). Protocol Analysis: Verbal Reports As Data. Mit Press.
Esser, J. K. (1998). Alive and Well after 25 Years: A Review of Groupthink Research.
Organizational Behavior and Human Decision Processes, 73(2–3), 116–141.
doi:10.1006/obhd.1998.2758
Finke, R. A., Ward, T. B., & Smith, S. M. (1992). Creative cognition: Theory, research, and
applications. Cambridge, MA: MIT press.
Fu, K., Cagan, J., & Kotovsky, K. (2010). Design Team Convergence: The Influence of Example
Solution Quality. Journal of Mechanical Design, 132(11), 111005–111005.
doi:10.1115/1.4002202
Furnham, A., & Bachtiar, V. (2008). Personality and intelligence as predictors of creativity.
Personality and Individual Differences, 45(7), 613–617. doi:10.1016/j.paid.2008.06.023
Gallupe, R. B., Dennis, A. R., Cooper, W. H., Valacich, J. S., Bastianutti, L. M., & Nunamaker,
J. F. (1992). Electronic Brainstorming and Group Size. The Academy of Management
Journal, 35(2), 350–369. doi:10.2307/256377
Gero, J. S., & Mc Neill, T. (1998). An approach to the analysis of design protocols. Design
Studies, 19(1), 21–61. doi:10.1016/S0142-694X(97)00015-X
109

Gero, J. S., & Tang, H.-H. (2001). The differences between retrospective and concurrent
protocols in revealing the process-oriented aspects of the design process. Design Studies,
22(3), 283–295. doi:10.1016/S0142-694X(00)00030-2
Glynn, S. M., & Takahashi, T. (1998). Learning form Analogy-Enhanced Science Text. Journal
of research in science teaching, 35(10), 1129–1149.
Goldschmidt, G. (1997). The Designer as a Team of One. In N. Cross, H. Christiaans, & K.
Dorst (Eds.), Analysing Design Activity (1st ed.). Wiley.
Goldschmidt, G. (2011). Avoiding Design Fixation: Transformation and Abstraction in Mapping
from Source to Target. The Journal of Creative Behavior, 45(2), 92–100.
doi:10.1002/j.2162-6057.2011.tb01088.x
Gough, H. G. (1979). A creative personality scale for the Adjective Check List. Journal of
Personality and Social Psychology, 37(8), 1398–1405.
doi:http://dx.doi.org.libproxy.usc.edu/10.1037/0022-3514.37.8.1398
Hocevar, D. (1979). The Development of the Creative Behavior Inventory (CBI). Retrieved from
http://www.eric.ed.gov/ERICWebPortal/detail?accno=ED170350
Holsapple, C. W., & Joshi, K. D. (2002). A collaborative approach to ontology design. Commun.
ACM, 45(2), 42–47. doi:10.1145/503124.503147
Jin, Geslin, M., & Lu, S. C.-Y. (2007). Impact of Argumentative Negotiation on Collaborative
Engineering. CIRP Annals - Manufacturing Technology, 56(1), 181–184.
doi:10.1016/j.cirp.2007.05.043
Jin, Y., & Benami, O. (2010). Creative patterns and stimulation in conceptual design. AI EDAM,
24(Special Issue 02), 191–209. doi:10.1017/S0890060410000053
110

Kvan, T. (2000). Collaborative design: what is it? Automation in Construction, 9(4), 409–415.
doi:10.1016/S0926-5805(99)00025-4
Lamm, H., & Trommsdorff, G. (1973). Group versus individual performance on tasks requiring
ideational proficiency (brainstorming): A review. European Journal of Social
Psychology, 3(4), 361–388. doi:10.1002/ejsp.2420030402
Latané, B., Williams, K., & Harkins, S. (1979). Many hands make light the work: The causes
and consequences of social loafing. Journal of Personality and Social Psychology, 37(6),
822–832. doi:10.1037/0022-3514.37.6.822
McGrath, C., & Krackhardt, D. (2003). Network Conditions for Organizational Change. The
Journal of Applied Behavioral Science, 39(3), 324–336.
Michalko, M. (2001). Cracking creativity: The secrets of creative genius. New York, NY:
Random House Digital, Inc..
Moreland, R. L. (2010). Are Dyads Really Groups? Small Group Research, 41(2), 251–267.
doi:10.1177/1046496409358618
Mullen, B., Johnson, C., & Salas, E. (1991). Productivity loss in brainstorming groups: A meta-
analytic integration. Basic and applied social psychology, 12(1), 3-23.
Nijstad, B. A., & Stroebe, W. (2006). How the Group Affects the Mind: A Cognitive Model of
Idea Generation in Groups. Personality and Social Psychology Review, 10(3), 186–213.
doi:10.1207/s15327957pspr1003_1
Novick, L. R. (1988). Analogical transfer, problem similarity, and expertise. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 14(3), 510–520.
doi:10.1037/0278-7393.14.3.510
111

O’Quin, K., & Besemer, S. P. (2006). Using the Creative Product Semantic Scale as a Metric for
Results-Oriented Business. Creativity and Innovation Management, 15(1), 34–44.
doi:10.1111/j.1467-8691.2006.00367.x
Oman, S. K., Tumer, I. Y., Wood, K., & Seepersad, C. (2013). A comparison of creativity and
innovation metrics and sample validation through in-class design projects. Research in
Engineering Design, 24(1), 65–92. doi:10.1007/s00163-012-0138-9
Osborn, A. F. (1957). Applied Imagination: Principles and procedures of creative
problemsolving. New York: Charles Scribner’s Sons.
Pahl, G., & Beitz, W. (1996). Engineering Design: A Systematic Approach. London; New York:
Springer.
Perry-Smith, J. E., & Shalley, C. E. (2003). The Social Side of Creativity: A Static and Dynamic
Social Network Perspective. The Academy of Management Review, 28(1), 89–106.
Perttula, M. K., Krause, C. M., & Sipilä, P. (2006). Does idea exchange promote productivity in
design idea generation? CoDesign, 2(3), 125–138. doi:10.1080/15710880600797942
Perttula, M. K., & Sipilä, P. (2007). The idea exposure paradigm in design idea generation.
Journal of Engineering Design, 18(1), 93–102. doi:10.1080/09544820600679679
Pirola-Merlo, A., & Mann, L. (2004). The relationship between individual creativity and team
creativity: aggregating across people and time. Journal of Organizational Behavior,
25(2), 235–257. doi:10.1002/job.240
Polzer, J. T., Milton, L. P., & Swann, W. B. (2002). Capitalizing on Diversity: Interpersonal
Congruence in Small Work Groups. Administrative Science Quarterly, 47(2), 296–324.
doi:10.2307/3094807
112

Reis, S. M., & Renzulli, J. S. (1991). The Assessment of Creative Products in Programs for
Gifted and Talented Students. Gifted Child Quarterly, 35(3), 128–134.
doi:10.1177/001698629103500304
Richards, L. (1998). Stimulating creativity: teaching engineers to be innovators. In FIE ’98.
(Vol. 3, pp. 1034 –1039). Presented at the 28th Annual Frontiers in Education
Conference. doi:10.1109/FIE.1998.738551
Ritchey, T. (1998). General Morphological Analysis: A general method for non-quantified
modelling. Presented at the 16th Euro Conference on Operational Analysis.
Rohrbach, B. (1969). Creative by rules—method 635, a new technique for solving problems.
Absatzwirtschaft, 12, 73–75.
Sarmiento, J. W., & Stahl, G. (2008). Group Creativity in Interaction: Collaborative Referencing,
Remembering, and Bridging. International Journal of Human-Computer Interaction,
24(5), 492. doi:10.1080/10447310802142300
Schunn, C. D. (2010). Chapter 6 - From Uncertainly Exact to Certainly Vague: Epistemic
Uncertainty and Approximation in Science and Engineering Problem Solving. In Brian
H. Ross (Ed.), Psychology of Learning and Motivation (Vol. Volume 53, pp. 227–252).
Academic Press. Retrieved from
http://www.sciencedirect.com/science/article/pii/S0079742110530068
Shah, J., Smith, S. M., & Vargas-Hernandez, N. (2003). Metrics for measuring ideation
effectiveness. Design Studies, 24(2), 111–134. doi:10.1016/S0142-694X(02)00034-0
Shah, J., Smith, S., Vargas-Hernandez, N., Gerkens, D., & Wulan, M. (2003). Empirical Studies
of Design Ideation: Alignment of Design Experiments With Lab Experiments. Presented
at the 15th International Conference on Design Theory and Methodology. Retrieved from
113

http://asmedl.org.libproxy.usc.edu/getabs/servlet/GetabsServlet?prog=normal&id=ASME
CP00200337017b000847000001&idtype=cvips&gifs=yes
Shah, J., Vargas-Hernandez, N., Summers, J., & Kulkarni, S. (2001). Collaborative Sketching
(C-Sketch) - An Idea Generation Technique for Engineering Design. The Journal of
Creative Behavior, 35(3), 168–198.
Shalley, C. E., & Perry-Smith, J. E. (2008). The emergence of team creative cognition: the role
of diverse outside ties, sociocognitive network centrality, and team evolution. Strategic
Entrepreneurship Journal, 2(1), 23–41. doi:10.1002/sej.40
Sauder, J., Lian, E., & Jin, Y. (2013). The Effect of Collaborative Stimulation on Design
Novelty. Proceedings of the ASME 2013 International Design Engineering Technical
Conferences & Computers and Information in Engineering Conference, Portland, OR
Sauder, J., Lian, E., Wang, B., & Jin, Y. (2012). Collaborative Stimulation in Design: A
Retrospective Protocol Analysis Approach. Design Creativity Workshop, College
Station, TX
Sauder, J., & Jin, Y. (2010). Modeling Collaborative Design Creativity - An Interactive
Cognition Approach. Proceedings of 3rd Design Creativity Workshop. Stuttgart,
Germany
Sauder, J., & Jin, Y. (2012). Collaborative stimulation of memory retrieval in creative design.
Proceedings of the 2nd International Conference on Design Creativity 2012. Glasgow,
UK
Sauder, J., & Jin, Y. (2013). Training the Participatory Renaissance Man: Past Creative
Experiences and Collaborative Design. Proceedings of the ASME 2013 International
114

Design Engineering Technical Conferences & Computers and Information in Engineering
Conference, Portland, OR
Sauder, J., & Jin, Y. Collaborative stimulation of memory retrieval in design. International
Journal of Design Creativity and Innovation (In Press). Taylor & Francis
Silvia, P. J., Wigert, B., Reiter-Palmon, R., & Kaufman, J. C. (2012). Assessing creativity with
self-report scales: A review and empirical evaluation. Psychology of Aesthetics,
Creativity, and the Arts, 6(1), 19–34.
doi:http://dx.doi.org.libproxy.usc.edu/10.1037/a0024071
Sonnenburg, S. (2004). Creativity in Communication: A Theoretical Framework for
Collaborative Product Creation. Creativity and Innovation Management, 13(4), 254–262.
Sosa, M., & Marle, F. (2010). Forming Creative Teams: An Experimental Approach. Presented
at the ASME IDETC/CIE 2010, Montreal, Quebec, Canada.
Stempfle, J., & Badke-Schaub, P. (2002). Thinking in design teams - an analysis of team
communication. Design Studies, 23(5), 473–496. doi:10.1016/S0142-694X(02)00004-2
Stouffer, W. B., Russell, J., & Oliva, M. (2004). Making The Strange Familiar: Creativity and
the Future of Engineering Education. Presented at the 2004 American Society for
Engineering Education Annual Conference & Exposition, American Society for
Engineering Education.
Suh, N. P. (1990). The Principles of Design. New York: Oxford University Press.
Suh, N. P. (2001). Axiomatic Design: Advances and Applications. New York: Oxford University
Press.
115

Suwa, M., Purcell, T., & Gero, J. (1998). Macroscopic analysis of design processes based on a
scheme for coding designers’ cognitive actions. Design Studies, 19(4), 455–483.
doi:10.1016/S0142-694X(98)00016-7
Sundholm, H., Artman, H., & Ramberg, R. (2004). Backdoor Creativity: Collaborative Creativity
in Technology Supported Teams. In COOP (pp. 99-114). Amsterdam, The Netherlands:
IOS Press.
Taggar, S. (2002). Individual Creativity and Group Ability to Utilize Individual Creative
Resources: A Multilevel Model. The Academy of Management Journal, 45(2), 315–330.
doi:10.2307/3069349
Tang, H. H., Lee, Y. Y., & Gero, J. S. (2011). Comparing collaborative co-located and
distributed design processes in digital and traditional sketching environments: A protocol
study using the function–behaviour–structure coding scheme. Design Studies, 32(1), 1–
29. doi:10.1016/j.destud.2010.06.004
Taylor, D. W., Berry, P. C., & Block, C. H. (1958). Does Group Participation When Using
Brainstorming Facilitate or Inhibit Creative Thinking? Administrative Science Quarterly,
3(1), 23–47. doi:10.2307/2390603
Terrenghi, L., Fritsche, T., & Butz, A. (2006). The EnLighTable: Design of Affordances to
Support Collaborative Creativity. In A. Butz, B. Fisher, A. Krüger, & P. Olivier (Eds.),
Smart Graphics (Vol. 4073, pp. 206–217). Berlin, Heidelberg: Springer Berlin
Heidelberg. Retrieved from
http://www.springerlink.com.libproxy.usc.edu/content/43387012441622p8/
116

Torrance, E. P. (1974). Torrance Tests of Creative Thinking: Verbal Tests, Forms A and B,
Figurals Tests, Forms A and B: Norms-technical Manual. Personel Press/Ginn, Xerox
Education Company.
Van der Lugt, R. (2000). Developing a graphic tool for creative problem solving in design
groups. Design Studies, 21(5), 505–522. doi:10.1016/S0142-694X(00)00021-1
Van Someren, M. W., Barnard, Y., & Sandberg, J. (1994). The Think Aloud Method: A Practical
Guide to Modelling Cognitive Processes. Academic Press, Inc., 525 B Street, Suite 1900,
San Diego, CA 92101 ($48). Retrieved from
http://www.eric.ed.gov/ERICWebPortal/detail?accno=ED399532
Vargas-Hernandez, N., Shah, J., & Smith, S. M. (2010). Understanding design ideation
mechanisms through multilevel aligned empirical studies. Design Studies, 31(4), 382–
410. doi:10.1016/j.destud.2010.04.001
Vyas, D., & Nijholt, A. (2009). Collaborative Practices That Support Creativity in Design.
Retrieved from http://doc.utwente.nl/67550/
Warr, A., & O'Neill, E. (2005, April). Understanding design as a social creative process. In
Proceedings of the 5th conference on Creativity & cognition (pp. 118-127). New York,
NY: ACM. doi:http://doi.acm.org.libproxy.usc.edu/10.1145/1056224.1056242
West, M. A. (2002). Sparkling Fountains or Stagnant Ponds: An Integrative Model of Creativity
and Innovation Implementation in Work Groups. Applied Psychology, 51(3), 355–387.
doi:10.1111/1464-0597.00951
Wilde, D. (2010, February). Personalities Into Teams. mechanical engineering, (February 2010).
Retrieved from
http://memagazine.asme.org/Articles/2010/february/Personalities_Into_Teams.cfm
117

Wiltschnig, S., Christensen, B. T., & Ball, L. J. (2013). Collaborative problem–solution co-
evolution in creative design. Design Studies. doi:10.1016/j.destud.2013.01.002
Woodman, R. W., Sawyer, J. E., & Griffin, R. W. (1993). Toward a Theory of Organizational
Creativity. The Academy of Management Review, 18(2), 293–321. doi:10.2307/258761
118

11 Appendix A- Experimental Material
11.1 Problem 1
In Los Angeles, the freeway system is way too crowded during rush hour. Unfortunately, the
sprawling nature of Los Angeles is not friendly for public transit systems, so people need cars.
Design some type of system that can be integrated into either vehicles or the freeway (or both)
which will reduce rush hour traffic. If you choose to integrate it into vehicles, it must work even
if not all vehicles have this system.
11.2 Problem 2
Skateboards are one of the most popular forms of transportation at USC. Unfortunately though,
when students come to class, the only current method for skateboard storage is to line them up
against the wall. However, this has the potential to mark up the wall and skateboards can fall
over in a domino effect if one is accidently bumped. A larger problem is that in large lecture
halls, where there are often 2-3 rows of skateboards stacked up against the back wall. With so
many boards, it can be hard to find yours, or even worse, it provides the opportunity for someone
to steal one unnoticed. Design a device which will safely and securely hold skateboards while
students are in class. This device could either be located in the hallway or outside the building,
but not in the class room due to space constraints.
119

12 Appendix B- Protocol Analysis Examples
12.1 Raw Transcripts Divided by Episodes
12.1.1 Collaborative Dialog
Episode 3: Racks and Security System
(1) Okay. Have you seen parkside?...(2)Yeah… (1)They have those racks…(2)Yeah… (1) Well,
assuming like if they have those racks plus a locking a device that you can just use like a pad
lock… (2) Well, who with a skateboard carry around a pad lock? (2) What if it was ID card
swipeable? (2) Every USD student is going to have an ID card… (1)I feel like it would be
cheaper to use a pad lock… (2)Probably… (2) You know, people carry locks into the gym but
they do not carry a lock for their skateboard. (1) Okay. That’s fine. (2) I do ride a bike. I do
ride a bike. I don’t ride a skateboard… (1)Yeah… (2)I don’t know. (2) I mean I have no
problems with locking up my bike but it’s almost like I might have problems locking up my
skateboard… (1) Okay… (2) Locking up with a pad lock like why are going to bring a pad lock
when you’re riding a skateboard… (1) Just put it in your backpack… (2) What if they don’t have
a backpack?... (1) Well, then if you don’t own it, you don’t get to lock it just sits there (2)
You're just out of luck so well…(1) Well then, if you’re going to carry, I guess, [Unclear]. (2)
Okay. [Unclear]…
Episode 4: Discussion of how Device Holds
(2)Yeah. So we’re just designing a device that safely and securely holds skateboards… (1) Well,
for me from this side it would be like something like this. More like, there’s like slats. You like
slide it into… (2) Oh, like the insides… (1) Right… (2) Okay…(1) Yeah. Something like each
120

one you would put slide like that… (2) More like on the front face of it, you’re going to have like
a huge—okay. That’s the ugly spots that I’m drawing… (1) Like, from the front of looking in?
(2) Yeah. Like, if you have a skateboard in here, it goes like this, right?... (1) Right… (2) Like,
with the wheels. Okay. Then, there’s some on top of it. Where does the locking mechanism
lock it into?...
Episode 5: Swinging Arm
(1) Okay. So, from the side view of the arm—so here’s your side view, right? …and here’s
where your skateboard is. This pretty much have wheels here, wheels here, and then your
skateboard is here like this. So, in between here and up here, ideally it would be back somehow.
Then we go back kind of staggering the next steps. But then here… (2) Why do we need to be
staggered?... (1) So that, your not… (2) I mean, you’re still going to have the wheel dimension
problems with like the width of the skateboard is, the wheel, plus the height of the board
deflection is…That’s right. (2) What if with this, what if you just have, if they were all the same
length, could you have something that went across like this and then could swing down in front
and pin it right here? (1) I was thinking something like some type of like a bar like this on a
hinge. Then, you would pull it… (2) And lock into place. (1)Yeah. It would pull itself. Like
here, you see it here, you slide it in and you would pull it down and then you lock it.
12.1.2 Collaborator 1 Retrospective
Episode 3: Racks and Security System
At this pass, I got the pad lock, I think it's pretty good but-- yeah, a little frustrated by the idea,
current idea but its fine. I think the reason why I was going for locks is because it’s got less
involved I try to stay away from electronics because it make things a lot complex and I like to
121

focus on the side of things.. Here, I’m concentrating on how bad a drawer I am. That's why it’s
taking me longer time.
Episode 4: Discussion of how Device Holds
But I’m still pretty much drawing from what I remember from the parkside racks. So pretty
much just drawing from memory and thinking about the physical like motion of putting
skateboards in and being secure, stuff like that. At this point, I’m actually kind of surprised
because I wasn’t thinking about like a box....a box like apparatus I guess or storage container.
That was actually pretty surprising to me and I'm just thinking as far effeiceny goes, it makes
sense and just to account for a certain amount of space tends to free racks and plus, a more self-
contained. It can be locked up and moved and stuff like that.
Episode 5: Swinging Arm
Here, I'm drawing my first, I think top down one. Well, it's pretty much thinking about
beginning for a locking mechanism. I'm just pinning the board down into the shelf. I'm not really
sure where I got his idea from but it just appealed to me.When Collab 2 brought the idea of not
having them staggered back like that not surprised but it made sense to me because I realize I had
been thinking more aesthetically pleasing as oppose to space efficiency or just like mechanically
efficient. I pretty opened to that change.
12.1.3 Collaborator 2 Retrospective
Episode 3: Racks and Security System
The other thing that I was just mentioning was talking about ID cards. While it’ll be a great
problem on campuses, what if you didn’t want it on a campus?.. You keep mentioning anything
where any ideas came, too.…Okay. (experimenter) One of the things that I was thinking about
122

when we’re going to talk about the locking mechanism is how in that convention center back in
Chicago I saw almost people lockup their coats in individual lock boxes and they all had ID
cards that were based—I was in a convention center located in a hotel and that’s part of where
the ID card idea came from but if it wasn’t located on college campus, how can you monitor who
has a card, who doesn’t have a card and what if someone’s visiting and brings a skateboard?
Episode 4: Discussion of how Device Holds
The other thing that we are considering when we’re considering when we’re drawing these
diagrams was how many skateboards could potentially fit in a rack…[Unclear] mentioned
anything you personally [Unclear].
Episode 5: Swinging Arm
Right here I was thinking about why were they ascend when it would just take out more floor
space which has to do with the amount of room in a hallway because while it’d be cool to just
put them outside, if I had a skateboard, I would prefer to put it in a hallway where it was closer
and where there just weren’t a hundred people of walking by every single ten minutes or every
hour. Right here, I was trying to think of something that could physically lock into the back of
the skateboard that could be more secure and unreachable especially if you nested them modules
up against each other…and this just comes from the fact, I mean, I’ve had a bike stolen before
and while it was locked to something, sometimes it really just doesn’t matter. You need
something that should be inaccessible. Right here, we’re trying to discuss feasibility of this.
123

12.2 Segmented and Coded Episodes for Episodes 3-4
12.2.1 Collaborative Dialog
#
Time Segmented Dialog Coding
1 Episode 3: Racks and Security System
2 (1) Okay. Have you seen
parkside?...(2)Yeah… (1)They have those
racks…(2)Yeah…
(1) MR(S(parkeside B(has S(racks))))
3 (1) Well, assuming like if they have those
racks plus a locking a device that you can
just use like a pad lock…
(1) TF(S(racks AS(S(lock device B(use
S(padlock))))
4 (2) Well, who with a skateboard carry
around a pad lock?
(2) SA(F(why( S(skateboard user B(carry
S(padlock)))
5 02:06 (2) What if it was ID card swipeable? (2) Se(TF( S(rack S(ID card B(swipable))))
(below)
6 (2) Every USD student is going to have an
ID card…
(2) SA( S(usc student B(have S(ID card))))
14 (1)I feel like it would be cheaper to use a
pad lock… (2)Probably…
(1) SA(F(cheaper B(use S(padlock))) (2)
SA(agreement)
15 (2) You know, people carry locks into the
gym but they do not carry a lock for their
skateboard.
(1) S(people B(carry S(locks) to S(gym
F(not B(carry S(lock)) for S(skateboard)))
16 (1) Okay. That’s fine. (1) SA(Agreement)
17 (2) I do ride a bike. I do ride a bike. I
don’t ride a skateboard…
(2) PA(B(ride S(bike)) F(don't B(ride
S(skateboard))))
18 (1)Yeah… (2)I don’t know. (2) PA(F(no problems B(locking
S(bike))F(have problems B(locking
S(skateboard)))))
19 02:36 (2) I mean I have no problems with locking
up my bike but it’s almost like I might have
problems locking up my skateboard… (1)
Okay…
SA(B(locking with S(pad lock)) F(why
B(bring S(padlock B(riding
S(skateboard))))
20 (2) Locking up with a pad lock like why
are going to bring a pad lock when you’re
riding a skateboard…
(1) Co(TF(B(put S(pad lock) in
S(backpack))))
21 (1) Just put it in your backpack… (2) SA(S(user B(doesn't have
S(backpack)))
22 (2) What if they don’t have a backpack?... (1) TF(B(don't own S(padlock [it]))
S(skateboard B(sits S(there)))
23 (1) Well, then if you don’t own it, you
don’t get to lock it just sits there
(2) SA(S(user(F(out of luck)))
24 (2) You're just out of luck so well… (1) SA(S(user B(carry S(padlock
[implied])))
25 (1) Well then, if you’re going to carry, I
guess, [Unclear].

124

26 (2) Okay. [Unclear]…
27 Episode 4: Discussion of how Device
Holds
SA(S(device F(safely, securely B(holds
S(skateboards)))`
28 (2)Yeah. So we’re just designing a device
that safely and securely holds
skateboards…
(1) Description S(side B(like S(this
[drawing])) B(like S(slats) B(slide
S(skateboard) into)))
29 03:06 (1) Well, for me from this side it would be
like something like this. More like, there’s
like slats. You like slide it into…
(2) Describe B(like S(insides)) (2
S(skateboard B(is S(here)))
30 (2) Oh, like the insides… (1) Right… (2)
Okay…
(1) Describe S(skateboard [each one]
B(slide))
31 (1) Yeah. Something like each one you
would put slide like that…
(2) SA(S(front face B(huge))
33 (2) More like on the front face of it, you’re
going to have like a huge—okay. That’s
the ugly spots that I’m drawing…
(1)Sketch question
34 (1) Like, from the front of looking in? (2) Describe B(have S(skateboard B(in
S(slot [here]) B(horizontle [like this])) (1)
SA(agreement)
35 (2) Yeah. Like, if you have a skateboard in
here, it goes like this, right?... (1) Right…
(2) Describe S(wheels) S(skateboard[some
B(on top S(skateboard [it]))) SA(S(where
S(locking mechanism B(lock)))
36 (2) Like, with the wheels. Okay. Then,
there’s some on top of it. Where does the
locking mechanism lock it into?...

12.2.2 Collaborator 1 Retrospective
# Time Segmented Dialog Coding
1 Episode 3: Racks and Security System
2
3 At this pass, I got the pad lock, I think it's pretty good but-- S(padlock)
4 yeah, a little frustrated by the idea, current idea but its fine. SA(frustrate S(padlock
[idea])
5 02:15 I think the reason why I was going for locks is because it’s
got less involved I try to stay away from electronics because
it make things a lot complex and I like to focus on the side of
things..
SA(S(locks F(less
involved)F(stay away
S(electronics
B(complex)))))
15 02:45 Here, I’m concentrating on how bad a drawer I am. That's
why it’s taking me longer time.
Process
27 Episode 4: Discussion of how Device Holds
28 03:15 But I’m still pretty much drawing from what I remember
from the parkside racks. So pretty much just drawing from
memory and thinking about the physical like motion of
putting skateboards in and being secure, stuff like that.
Describe (S(parkside
racks)B(motion putting
S(skateboards) in))
32 03:45 At this point, I’m actually kind of surprised because I wasn’t
thinking about like a box....a box like apparatus I guess or
storage container. That was actually pretty surprising to me
Describing what she was
surprised it was not
125

and I'm just thinking as far effeiceny goes, it makes sense
and just to account for a certain amount of space tends to
free racks and plus, a more self-contained. It can be locked
up and moved and stuff like that.

12.2.3 Collaborator 2 Retrospective
# Time Segmented Dialog Coding
1 Episode 3: Racks and Security System
5 03:10 The other thing that I was just mentioning was talking
about ID cards.
Se(TF(S(ID cards)))
6 While it’ll be a great problem on campuses, what if you
didn’t want it on a campus?..
SA(F(great B(on
S(campuses) F(didn't want
S(rack B(on S(campus))
8 03:40 One of the things that I was thinking about when we’re
going to talk about the locking mechanism
Describe (S(lock
mechanisms))
9 is how in that convention center back in Chicago I saw
almost people lockup their coats in individual lock boxes
and they all had ID cards that were based—I was in a
convention center located in a hotel and that’s part of
where the ID card idea came from
Ms( S(lock
device),MR(S(convention
center S(people B(lockup
S(coats B(in S(individaul
lock boxes)S(people[they]
B(had S(ID
cards)S(convention center
B(located in S(hotel)))))))
10 04:10 but if it wasn’t located on college campus, how can you
monitor who has a card, who doesn’t have a card and
what if someone’s visiting and brings a skateboard?
SA(F(wasn't B(locates
S(college campus))) F(how
B(monitor S(ID
cards)S(vistor B(brins
S(skateboard))))
27 Episode 4: Discussion of how Device Holds
28 The other thing that we are considering when we’re
considering when we’re drawing these diagrams was how
many skateboards could potentially fit in a rack…
Describing

Abstract (if available)

Abstract In both design education and industry it is often assumed that collaboration encourages creativity, although research in brainstorming has shown that this is not always the case. There is an opportunity to develop more effective collaborative methods, based on research. This work lays the groundwork for new methods, by extending creative cognition to group creativity by proposing an interactive cognition approach. Specifically, designers' interactions, through questions and shared design entities, stimulate creativity relevant thought processes. Various types of collaborative stimulation are identified, namely, prompting, seeding, correcting, and clarifying. An experiment using collaborative retrospective protocol analysis was developed to determine if the hypothesized types of collaborative stimulation exist and how influential each type was by exploring stimulation patterns. Patterns were found between collaborative stimulation and thought processes, novelty of ideas, and analogies. Specifically, prompting had a strong relationship with memory retrieval, seeding and correcting had strong relationships with transformation, and clarifying had moderate to strong relationships with memory retrieval, association, and transformation. Also, seeding and correcting were found to create the most novel ideas. Additional patterns were found between the past creative experience of individuals and stimulation frequency, and designers were found to be stimulated more often in collaborative than individual settings. The implications of these patterns for developing new methods to promote group creativity are discussed.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

A social-cognitive approach to modeling design thinking styles

PDF

Managing functional coupling sequences to reduce complexity and increase modularity in conceptual design

PDF

Analysis of student cyclic responses and tri-engagement levels in a global learning environment

PDF

Decoding information about human-agent negotiations from brain patterns

PDF

Computer aided visual analogy support (CAVAS) for engineering design

PDF

Design, modeling and analysis of piezoelectric forceps actuator

PDF

Extraction of preferential probabilities from early stage engineering design team discussion

PDF

Contingency handling in mission planning for multi-robot teams

PDF

A meta-interaction model for designing cellular self-organizing systems

PDF

A synthesis approach to manage complexity in software systems design

PDF

Assessing therapeutic change mechanisms in motivational interviewing using the articulated thoughts in simulated situations paradigm

PDF

Wake modes of rotationally oscillating circular cylinder in cross-flow and its relationship with heat transfer

PDF

AI-driven experimental design for learning of process parameter models for robotic processing applications

PDF

Improving stimulation strategies for epiretinal prostheses

PDF

Document understanding based design support (DocUDS) for augmenting engineering design

PDF

Stimulation strategies to improve efficiency and temporal resolution of epiretinal prostheses

PDF

Modeling dyadic synchrony with heterogeneous data: validation in infant-mother and infant-robot interactions

PDF

Child's play: designing play spaces that support healthy cognitive development in children

PDF

Manipulation of RGCs response using different stimulation strategies for retinal prosthesis

PDF

A virtual reality exergaming system to enhance brain health in older adults at risk for Alzheimer’s disease

Asset Metadata

Creator Sauder, Jonathan Frank (author)

Core Title Collaborative stimulation in team design thinking

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Mechanical Engineering

Publication Date 11/19/2013

Defense Date 09/20/2013

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag creative cognition,group creativity,interactive cognition,OAI-PMH Harvest,teams,thought processes

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Jin, Yan (committee chair), Madigan, Stephen (committee member), Shiflett, Geoffrey R. (committee member)

Creator Email jsauder@live.com,jsauder@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-348095

Unique identifier UC11296627

Identifier etd-SauderJona-2164.pdf (filename),usctheses-c3-348095 (legacy record id)

Legacy Identifier etd-SauderJona-2164.pdf

Dmrecord 348095

Document Type Dissertation

Format application/pdf (imt)

Rights Sauder, Jonathan Frank

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA