University of Southern California Dissertations and Theses

WikiWinWin: a Wiki-based collaboration framework for rapid requirements negotiations

(USC Thesis Other)

WikiWinWin: a Wiki-based collaboration framework for rapid requirements negotiations

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content 1

WIKIWINWIN – A WIKI-BASED COLLABORATION
FRAMEWORK FOR RAPID REQUIREMENTS
NEGOTIATIONS
by
Di Wu

A Dissertation Presented to the
FACULTY OF THE GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(COMPUTER SCIENCE)

December 2014

2

Acknowledgements
First and foremost, my deepest gratitude goes to my advisor Dr. Barry Boehm, who
has guided and supported me through my entire Ph.D. journey. He inspired me to
finish and strive for the best after my extended leave from the program. I would not
have completed my defense without him standing by me.
I want to express my gratitude to my family who have nurtured and supported me
since childhood. To my grandmother, I miss her every day. To my grandfather, who
taught me to always aim high. To my son, Andrew, who makes this journey
challenging and yet rewarding.
Last but not least, to my husband, Steve. This journey would not be the same without
you. I am grateful for everything we have.

3

Table of Contents
Acknowledgements................................................................................................ 2
Table of Contents ................................................................................................... 3
List of Figures ......................................................................................................... 5
List of Tables .......................................................................................................... 7
Chapter 1: Introduction ......................................................................................... 9
1.1 Motivation ......................................................................................................... 9
1.2 Terms and Definitions ...................................................................................... 15
1.3 The problems .................................................................................................... 16
1.4 Research Questions and Hypotheses .............................................................. 19
1.5 Research Contribution ..................................................................................... 21
1.6 Dissertation Outline ......................................................................................... 23
Chapter 2: Background and Related Work ..........................................................24
2.1 Requirements Elicitation and Negotiation ..................................................... 24
2.2 Win-Win Negotiation Model and Equilibrium Theory ................................. 28
2.3 VBSSE Framework ............................................................................................ 30
2.4 WinWin Support Systems and EasyWinWin .................................................. 32
2.5 Wiki Collaboration Technology ....................................................................... 35
2.6 Successful Practice - Shaper Role .................................................................... 37
2.7 Top-Level Dependability Framework ..............................................................39
2.8 USC Software Engineering Course .................................................................. 41
Chapter 3: WikiWinWin Framework ....................................................................42
3.1 Overview .......................................................................................................... 42
3.2 Negotiation Process ......................................................................................... 46
3.3 Usage Scenarios and Roles .............................................................................. 50
3.4 Dependability Attributes Classification .......................................................... 55
Chapter 4: Research Methodology ...................................................................... 56
4.1 Overall Strategy ............................................................................................... 56
4.2 Subjects ............................................................................................................ 58
4

4.3 Experiment Design ........................................................................................... 61
Chapter 5: Evaluation and Analysis ..................................................................... 69
5.1 WikiWinWin Usage Results ........................................................................... 69
5.2 WikiWinWin Qualitative Results ................................................................... 80
5.3 Stakeholder/ Value Dependency Results ........................................................ 83
5.4 Discussion ........................................................................................................ 92
5.5 Threats to Validity ........................................................................................... 98
Chapter 6: Conclusions and Future Work ......................................................... 100
Bibliography ........................................................................................................ 102

5

List of Figures
Figure 1: 2013 CHAOS Summary ...................................................................................... 10
Figure 2: 2012 Mckinsey Survey Summary ...................................................................... 10
Figure 3: Model-clash Spider Web .................................................................................. 13
Figure 4: ICM Activity and Level of Effort ...................................................................... 17
Figure 5: Win-Win Negotiation Model .......................................................................... 29
Figure 6: VBSSE 6-step Process Framework ................................................................... 31
Figure 7: WikiWinWin Big Picture ................................................................................ 42
Figure 8: Negotiation Process and Tool Support ........................................................... 47
Figure 9: Negotiate WIOA .............................................................................................. 48
Figure 10: Shaper Adds Value in WikiWinWin .............................................................. 51
Figure 11: Initial Brainstormed Ideas ............................................................................... 52
Figure 12: Win Condition Organized by Shaper ............................................................. 52
Figure 13: Issue Logged by Shaper ................................................................................... 53
Figure 14: Option Proposed by PKC ................................................................................54
Figure 15: WIOA ...............................................................................................................54
Figure 16: Fall 2007 Outcome ~ Use by Team ................................................................. 71
6

Figure 17: Fall 2008 Outcome ~ Use by Team ................................................................. 71
Figure 18: Fall 2009 Outcome ~ Use by Team ................................................................ 72
Figure 19: Fall 2007 Outcome ~ Use by Shaper .............................................................. 72
Figure 20: Fall 2008 Outcome ~ Use by Shaper .............................................................. 73
Figure 21: Fall 2009 Outcome ~ Use by Shaper .............................................................. 73
Figure 22: Fall 2007 Outcome ~ Number of Days Used by Team ................................. 74
Figure 23: Fall 2008 Outcome ~ Number of Days Used by Team ................................. 74
Figure 24: Fall 2009 Outcome ~ Number of Days Used by Team .................................. 75
Figure 25: Fall 2007 Outcome ~ Number of Days Used by Shaper ............................... 76
Figure 26: Fall 2008 Outcome ~ Number of Days Used by Shaper .............................. 76
Figure 27: Fall 2009 Outcome ~ Number of Days Used by Shaper ................................ 77
Figure 28: Fall 2007 Outcome ~ Number of Days Creating Artifacts ........................... 78
Figure 29: Fall 2008 Outcome ~ Number of Days Creating Artifacts ........................... 78
Figure 30: Fall 2009 Outcome ~ Number of Days Creating Artifacts ........................... 79
Figure 31: Project Package Grade Loss in Each Year ...................................................... 79
Figure 32: Student Critiques on WikiWinWin ............................................................... 81
Figure 33: Client Evaluation Grade Loss ........................................................................ 82
7

List of Tables
Table 1: Win-Lose = Lose-Lose ......................................................................................... 11
Table 2: Summary of EasyWinWin Fall 2006 Critiques ................................................. 34
Table 3: Wiki Shapers Add Business Value ..................................................................... 38
Table 4: Top-level Dependency Framework .................................................................. 40
Table 5: WikiWinWin Collaboration Scenarios ............................................................ 49
Table 6: Classification of Dependability Attributes ....................................................... 55
Table 7: Summary of projects ......................................................................................... 60
Table 8: Fall 2007 Early Usage vs. Milestone Usage ...................................................... 70
Table 9: Fall 2008 Early Usage vs. Milestone Usage ...................................................... 70
Table 10: Fall 2009 Early Usage vs. Milestone Usage ..................................................... 70
Table 11: Summary of WikiWinWin Shortfalls from Student Critiques ....................... 82
Table 12: Mean Rating in Multi-media Projects ............................................................. 84
Table 13: Correlation in Multi-media Projects ............................................................... 84
Table 14: Dependability Rankings in Multi-media Projects .......................................... 85
Table 15: Mean Rating in Community Service Projects ................................................. 87
Table 16: Correlation in Community Services Projects ................................................. 87
8

Table 17: Dependability rankings in community services projects ............................... 88
Table 18: Role-play Representativeness in Fall 2008 Experiment ................................. 89
Table 19: Role-play Representativeness in Fall 2009 Experiment ................................. 89
Table 20: Summary of WikiWinWin Changes .............................................................. 94
Table 21: Changes in Experiment Settings ..................................................................... 95
9

Chapter 1: Introduction
1.1 Motivation
1.1.1 Win-Lose = Lose-Lose
A major challenge in the development of software intensive systems is to define
requirements to satisfy the different needs of its multiple inter-dependent
stakeholders. Without win-win requirements, software projects often suffer
unsatisfactory outcomes. CHAOS research [Standish, 2013] by Standish Group
reported that as much as 24% of IT projects were canceled before completion. About
half of the projects were challenged to deliver on time, on budget, see Figure 1, with
required features. The majority of projects spent more time and money than
budgeted and yet still failed to deliver expected benefits. Another recent study by
Mckinsey [Mckinsey, 2012] revealed similar results: 45% experienced cost overrun, 7%
were behind schedule, and 56% delivered less functionality than expected. The cost of
failures and rework are significant: typical budget of large projects in the study is
more than $15 million, not to mention lost opportunities. The top factors challenging
project success in both studies clearly indicate that projects that lack stakeholder
involvement and convergence perform poorly, see Figure 2.
10

Figure 1: 2013 CHAOS Summary

Figure 2: 2012 Mckinsey Survey Summary
What frequently happens in software projects is a win-lose situation, where some key
stakeholders are either left out or put in a lose position and decide not to participate,
which eventually turns other key stakeholders from “winners” into “losers”. As
illustrated in Table 1, if developer and customer deliver a quick and sloppy product to
11

user, even though it satisfy a contract of delivering on-time, within budget, but if the
product cannot support users with their operational tasks, it is not a satisfactory
solution to user. The consequences are developer loses reputation and customer loses
business. In the end, nobody wins. Conversely, if developer and user throw in too
much “bells and whistles” in the requirements, it will potentially drive customer out
of funding, which in turn, will not give the capabilities that user really needs and
developer will not achieve the benefit if the product was delivered successfully. In the
end, another lose-lose situation. The real world situations are obviously more
complex than what is shown in here, the lesson does come across: nobody wins in this
kind of win-lose situations.
The win-win project management theory [Boehm and Ross, 1989] suggests that the
project can only be successful if and only if the success critical stakeholders can
collaborate and make everyone a winner.
Proposed Solution “Winner” Loser
Quick, cheap, sloppy product Developer & Customer User
Lots of “bells and whistles” Developer & User Customer
Driving too hard a bargain Customer & User Developer
Table 1: Win-Lose = Lose-Lose

12

1.1.2 Challenges in achieving win-win
Although the theory has long established, achieving the win-win outcome in practice
remains a major challenge. Such challenge is driven by:
Multi-stakeholders with diverse expectations and interests.
Different stakeholders – users, customers, managers, developers, and maintainers
come to a project with diverse perspectives, backgrounds, abilities and expectations.
It is challenging to define feasible and mutually satisfactory requirements that
accommodate all stakeholders’ goals and expectations.
Stakeholders have little understanding of each other’s principles and
practices.
Software projects involve multi-disciplinary stakeholders. There are designers, users,
customer, maintainer, developers, etc. Each type of stakeholders has its own mindset,
language, and expertise. It is challenging to rapidly understand each others’ “thought
world”. In his seminal work on The Two Cultures [Snow, 1959], C. P. Snow found that
science and technology policymaking was extremely difficult because it required the
combined expertise of both scientists and politicians, whose two cultures had little
understanding of each other’s principles and practices.
Stakeholders’ needs are often in conflict and their reconciliation needs to be
built into the nature of software projects.
Figure 3 shows the classic model-clash spider web [Boehm et al. 2000]. It is the result
from analyzing one of the multi-million dollar failed projects: the Bank of America
13

Master Net project. The project was cancelled after 48 months and an expenditure of
$88M, with no adequate capability. The spider web diagram demonstrates the four
most common stakeholder classes in software development: users, acquires,
maintainers, developers, and their needs that are often in conflict. For example, the
users want many features, which conflicts with the acquirers’ limited budget and
schedule, and with the developers’ ease of meeting budget and schedule. The
developers want freedom choice of COTS, which may not be compatible with
maintainers’ need of application compatibility, etc. When these kinds of conflicts are
not mediated from the beginning, the result is often unsatisfactory.

Figure 3: Model-clash Spider Web

Rapid changes of COTS, competitive threats and opportunities.
Finally, changes in COTS releases, competitive threats, market opportunities, etc.
requires the ability to quickly renegotiate a new solution when previous agreements
are no longer mutually satisfactory. Having the mechanism for multi-disciplinary
14

stakeholders to more quickly understand each other and to converge on mutually
satisfactory solutions with respect to emergent changes is a key for success [Boehm et
al. 2014].
15

1.2 Terms and Definitions
Win-Win Negotiation: is a negotiation framework based on the win-win principle.
EasyWinWin: previous groupware support system for win-win negotiation. It was
based on client/server architecture and had a facilitated negotiation process.
WikiWinWin: an implementation of win-win negotiation based on Wiki-based
collaboration.
Dependability: to what extent does the system satisfy the value propositions that its
stakeholders are depending on.
16

1.3 The problems
In a nutshell, the nature of the problem for this research is summarized as the
following:
Requirements definition needs to be based upon stakeholder negotiation
We have seen that if a success critical stakeholder is put in a lose situation, he/ she
will generally refuse to cooperate, eventually resulting other stakeholders to lose as
well. Therefore, stakeholder satisificing occurs in consensus building, where
stakeholders work together toward a solution that everyone can accept, even if it may
not be their ideal. The best place for such consensus building is in negotiation.
Traditional approach to requirements has limitations
Traditionally, requirements have served as basis for contract selection and payment
award. As such, they are expected to be complete, consistent, precise, and testable
specifications. Existing requirements techniques reviewed in Chapter 2 are generally
done well at documenting requirements specifications, but are generally not done
well at making it easy for stakeholders to participate.
Further, as systems are increasingly network-centric and collaboration-intensive, the
traditional approach of pre-specifying requirements has difficulties to cope with the
evolving nature of requirements [Pew and Mavor, 2007], which include emergent
requirements, rapid change, and reusable components. In situations when trade-offs
and their implications among various needs such as cost, schedule, performance, and
capabilities are not well understood, it is better to base requirements upon
17

negotiation where stakeholders reach mutually agreeable prioritized capabilities and
ranges of satisfactory quality attributes.
No one size fits all solution
Requirements negotiation happens in the concurrent context of system development,
as illustrated in Figure 4, it requires complementary practices. For example,
understanding needs and negotiating commitments are major activities in the
exploration phase and valuation phase, doing it well involves a considerable amount
of activities in system scoping, envisioning opportunities, architecting solutions, and
evaluating alternatives. Identifying what activities and techniques are supportive of
win-win requirements negotiation is a key aspect of this research.

Figure 4: ICM Activity and Level of Effort

18

Need for client-friendly support system
The previous EasyWinWin system [Boehm and Kitapci, 2006] had been used at USC
graduate Software Engineering classes from 1999 to 2006. According to its usage in
over 150 projects, EasyWinWin had been very good in capturing initial requirements.
However, it had been less easy to adapt to the evolving nature of requirements, and to
provide the flexibility for stakeholders, especially the client, to update the win
conditions and agreements as a project proceeds. More client-friendly solutions were
increasingly needed for future requirements negotiation support systems.
19

1.4 Research Questions and Hypotheses
Research Question 1. Can a Wiki-based collaboration framework (WikiWinWin) be
used to support more effective requirements negotiation? If so, will it make a
difference in project outcome?
Hypothesis 1. There is a lack of correlation between various usage aspects of
WikiWinWin and project outcome aspects, as compared with previous experience
with EasyWinWin, in terms of the projects’ loss in team grade.
The following sub-hypotheses are formulated to test each usage aspect:
H1.1. Higher or lower numbers of tool usage transactions by team has no correlation
with project outcome.
H1.2. Higher or lower numbers of tool usage transactions by shaper has no correlation
with project outcome.
H1.3. Higher or lower numbers of team usage days has no correlation with project
outcome.
H1.4. Higher or lower numbers of shaper usage days has no correlation with project
outcome.
H1.5. Higher or lower numbers of days creating artifacts has no correlation with
project outcome.
H1.6. Using WikiWinWin will not improve project outcome (as compared to using
EasyWinWin).
20

Research Question 2. How do top-level dependability attributes of a software
system developed by USC software engineering projects satisfy by stakeholder class?
Hypothesis 2.1. Different classes of stakeholders do not have different priorities on
quality needs.
Hypothesis 2.2. Different classes of stakeholders do not have different priorities on
functional needs.
21

1.5 Research Contribution
The contributions provided by this research are the following:
 A framework and tool support for using wikis to involve interdisciplinary
stakeholders in participating requirements negotiation, bridge the gap of
inadequate tool support in previous researches.
 Patterns of use that tend to lead to successful outcomes.
 Validation of shaper role as a success factor for effective wiki-based
negotiation.
 Complementary techniques to requirements negotiation, such as pre-
negotiation team building, mutual learning, etc.
 Quantitative information about different classes of system stakeholders’
dependability priorities to extend the top-level dependability framework. The
resulting stakeholder/ value priorities lists have been valuable for subsequent
project teams developing similar software to identify and negotiate more
complete win-win requirements.
Below is the list of publications during the course of this research:
Di Wu, Qi Li, Mei He, Barry Boehm, Ye Yang, Supannika Koolmanojwong, “Analysis
of Stakeholder/ Value Dependency Patterns and Process Implications: A Controlled
Experiment”, Proceedings of the 43rd Annual Hawaii International Conference on
System Sciences, January 2010 (Awarded Best Paper)
22

Di Wu, Da Yang, Barry Boehm, “Finding Success in Rapid Collaborative Requirements
Negotiation using Wiki and Shaper”, Proceedings of the 43rd Annual Hawaii
International Conference on System Sciences, January 2010 (Best Paper Nominee)
Di Wu, Da Yang, Supannika Koolmanojwong, Barry Boehm, “Experimental Evaluation
of Wiki Technology and the Shaper Role in Rapid Interdisciplinary Requirements
Negotiation”, Proceedings of the 42nd Annual Hawaii International Conference on
System Sciences, January 2009
Da Yang, Di Wu, Supannika Koolmanojwong, A. Winsor Brown, Barry Boehm,
“WikiWinWin: A Wiki Based System for Collaborative Requirements Negotiation”,
Proceedings of the 41st Annual Hawaii International Conference on System Sciences,
January 2008

23

1.6 Dissertation Outline
The organization of this dissertation is as follows:
Chapter 2 presents the background information of the research and summarizes the
related work.
Chapter 3 describes the WikiWinWin framework.
Chapter 4 discusses the research methodology and the experiment context.
Chapter 5 presents the evaluation results and a discussion.
Chapter 6 summarizes the contributions and proposes directions for future work.
24

Chapter 2: Background and Related
Work
2.1 Requirements Elicitation and Negotiation
Requirements elicitation involves discovering objectives and motivations from
different stakeholders’ perspectives, understanding the domain, identifying system
boundaries, exchanging information, capturing requirements and their priorities, and
converging on a set of requirements mutually acceptable to the stakeholders.
Basic techniques, such as interviews, observation, scenarios, workshops, focus groups,
prototypes, etc. have been established for many years. Several surveys [Davis, et al.
2006] [Couglan and Macredie, 2002] found that no apparent advantage in any one
technique over structured interviews, but appropriate combination of techniques may
be more effective, as evidenced in [Sutcliffe, et al. 2011]. For instance, scenarios and
prototypes are a frequent combination to elicit design improvements, in addition to
interviews and workshops. A selection framework based on combination of criteria
including domain, requirements, stakeholders, organizations, etc. is proposed in
[Hickey and Davis, 2004].
In model-based techniques, requirement models are recorded either as diagrams or
modeling languages. Such techniques range from simple use cases and scenarios
[Hickey and Davis, 2004], to goal-oriented formal analysis, such as KAOS
25

[Lamsweerde and Letier, 2000] and i* [Giorgini et al. 2011], to a UML model-based
framework [Fernández et al. 2010]. Natural language representations, such as scenario
lists, table templates, or informal diagrams, are more accessible for end-users. On the
other hand, formal languages used by i* and KAOS, action-object use case scripts, and
complex diagrammatic notations, are difficult for non-technical users to understand.
Tool support for elicitation can be categorized as: a) natural language tools, b) model-
based tools, c) collaboration tools.
Natural language tools do not support direct elicitation from stakeholders. Instead
they are used for processing requirements documents. Some have been integrated
with a domain ontology [Kaiya and Saeki, 2006] for checking model consistency,
detecting conflicts, etc. More powerful linguistic tools [Yang et al. 2012] can detect
and remove ambiguities and inconsistencies in requirements texts. Text-mining tools
[Duan et al. 2009] have been used for categorizing and grouping similar
requirements.
Most requirement models have supporting tools, i.e. checkers and reasoners. Such
tools use formal specification languages and rely on models to perform checking and
reasoning. Examples include SPIN for goal-oriented requirements [Cheng et al. 2009],
GRAIL for KAOS specifications [Darimont et al. 1997], and Tropos specification for i*
[Horkoff and Yu, 2010].
From a social perspective, tools have been developed to support human collaboration
for requirements elicitation. Some utilize crowd sourcing over the Internet [Chi et al.
26

2010][ Greenwood et al. 2012], others use social networks to support distributed
collaboration among requirements analysts [Marczak and Damian, 2009].
Beyond automated solutions for detecting and addressing conflicts in requirements
documents, the stakeholder negotiation approach for reaching agreeable
requirements emphasizes people considerations and trade-offs. Particularly, the Win-
Win negotiation systems are developed for supporting stakeholder Win-Win
negotiations, which involve a set of principles, practices, and tools that enable a set of
interdependent stakeholders to work out a mutually satisfactory (win-win) set of
shared commitments. The Win-Win approach is consistent with the Harvard
negotiation principle, “Getting to Yes” [Fisher and Ury, 1981].
Empirical evidence of distributed stakeholder negotiation includes Damian’s
comparison [Damian, 2003] of productivity in face-to-face and distributed
negotiation, exploration of using synchronous and asynchronous media to facilitate
communication [Mallardo et al. 2007][ Arthi, 2009], and USC real-client software
engineering projects using EasyWinWin.
Key conclusions from the above review are:
 There is no one size fits all solution. Appropriate combination of elicitation
techniques can be complementary.
 Existing methods and tools have limitations in coping with emergent
challenges of requirements, as discussed in Chapter 1. Both interviews and
workshops are limited by sample size, sessions and duration. They also
depend on the skills of the facilitator/ analyst to be effective. Observation is a
27

passive approach because end users do not participate directly, so it is difficult
to extract information from them. Modeling approaches that using formal
languages and complex notations are difficult for end-users to understand.
Natural language tools are towards processing requirements documents
processing, as opposed to directly supporting stakeholder involvement.
Prototypes are effective in engaging users, but difficult to translate into
documented requirements.
 Stakeholder negotiation is often needed, such as Win-Win negotiation
approach to getting stakeholders involved in electing needs and negotiating
trade-offs.
28

2.2 Win-Win Negotiation Model and Equilibrium
Theory
The Win-Win negotiation model consists of four types of artifacts – Win Condition,
Issue, Option, and Agreement (WIOAs).
Win Condition – captures individual stakeholders’ desired objectives.
Issue – captures conflicts between win conditions and their associated risks and
uncertainties.
Option – identifies candidate solutions to resolve an issue.
Agreement – captures shared commitment of stakeholders with regard to accepted
win conditions or adopted options.
The win-win equilibrium theory [Lee, 1996] links win conditions, issues, options and
agreements, as shown in Figure 5. It establishes a win-win equilibrium state where all
win conditions are covered by agreements and all issues are resolved by options
covered by agreements, and shows how this state is reached from other states. When
a stakeholder enters a Win Condition, either the other stakeholders all agree with it,
in which case it becomes an Agreement; or at least one stakeholder has a conflict with
it, in which case an Issue is recorded and the equilibrium is lost. Stakeholders then
propose Options to resolve the Issue and regain the equilibrium by having an
acceptable Option resolve the Issue and become an Agreement.

29

Figure 5: Win-Win Negotiation Model
30

2.3 VBSSE Framework
The Value-Based Systems and Software Engineering (VBSSE) theory and process
[Boehm and Jain, 2006] provides a framework for stakeholders to better understand
each others’ value propositions, and to effectively collaborate to achieve mutually
satisfactory enterprise outcomes. The VBSSE theory rests on two primary theorems.
The Enterprise Success Theorem, “Your enterprise will succeed if and only if it makes
winners of its success-critical stakeholders,” provides necessary and sufficient
conditions for a successful enterprise such as a software-intensive systems project.
The Enterprise Success Realization Theorem provides sufficient but not necessary
conditions for a project to succeed:
 Identifying all of the success-critical stakeholders (primarily involves
dependency theory)
 Understanding how the success-critical stakeholders want to win (primarily
involves utility theory)
 Having the success-critical stakeholders negotiate win-win product and
process plans (primarily involves negotiation theory and other aspects of
decision theory)
 Adaptively controlling progress toward a success-critical stakeholder win-win
outcome (primarily involves adaptive control theory).
The conditions are not necessary because some lucky projects may violate all of the
conditions but are saved in the end by the emergence of a new COTS product or the
like.
31

The Enterprise Success Theorem and the four Enterprise Success Realization
Theorem conditions establish not only a theory, but also a 4+1 process architecture
and a 6-step process for achieving a successful outcome, as shown in Figure 6. This
process has been integrated with the spiral model of system and software
development and evolution and its next-generation successor, the Incremental
Commitment Model [Boehm et al. 2014].
Utility Theory
Theory W:
SCS Win-Win
Decision Theory
Dependency
Theory
Control Theory
5a, 6c. State measurement,
prediction, correction;
Milestone synchronization
4b. Investment analysis,
Risk analysis
1. Protagonist goals
3a. Solution exploration
6. Risk, opportunity,
change management
4b, 6b.
Prototyping
2a. Results Chains
3b, 4b, 6b. Cost/schedule/
performance tradeoffs
2. Identify SCSs
3b, 6a. Solution
Analysis
4b, 6b. Option, solution
development & analysis
4a. SCS
expectations
management
3. SCS Value
Propositions
(Win conditions)
SCS: Success-Critical Stakeholder
5, 6c. Refine, Execute,
Monitor & Control Plans
4. SCS Win-Win
Negotiation

Figure 6: VBSSE 6-step Process Framework
32

2.4 WinWin Support Systems and EasyWinWin
2.4.1 Summary of WinWin support systems
Since 1994, four generations of collaboration support systems were developed at USC
to cope with the software requirements negotiation issues. EasyWinWin is the most
recent solution prior to the emergent wiki technology. The collaboration
technologies, negative results and success factors from previous tool generations have
driven evolution of these requirements negotiation support systems. We learned a
number of lessons [Boehm et al. 2001] from developing the four generations of
WinWin systems:
 Define a repeatable process and negotiation model.
 Incorporate facilitation and collaboration technique.
 Preliminary team building and concurrent prototyping are also useful.
 Make use of unobtrusive and flexible tools.
 Focus on ease of use to foster stakeholder involvement.
 Provide a robust infrastructure.
 Support for multiple modes of negotiation.
Early empirical study [Boehm and Egyed 1998] of using WinWin systems identified
some usage patterns correlated with better results.
 Non-sequential negotiation approach - project teams followed a non-
sequential approach tended to achieve better results than teams used a
waterfall approach. In the non-sequential approach, teams started passing
33

agreements as soon as a few win conditions were entered, even though some
passed agreements might need revisit later on. In the waterfall approach,
agreements were created only after most if not all win conditions were
entered.
 Efficiency in producing negotiation artifacts - project teams tended to get
good results if they are efficient at producing negotiation results.
 Team experience - people satisfaction correlated positively with team
experience and team success.
 Complementary practice - when WinWin requirements negotiation was
accompanied by better preparation and concurrent prototyping, the outcome
was better.
In terms of integrating Win-Win negotiation with other requirements management
systems, Kitapci [Kitapci, 2007] proposed a hybrid framework for improving quality of
requirements specifications from Win-Win agreements. We are seeking to improve
on the existing WinWin system with a more client friendly solution. And previous
lessons learned provide useful references for developing and evaluating such solution.
2.4.2 EasyWinWin
EasyWinWin is the fourth generation implementation for the WinWin requirements
negotiation approach based on a Group Support System. Compared with the earlier
three generations of WinWin requirements negotiation support system, EasyWinWin
is more robust, easier to use, and has a more friendly user interface. EasyWinWin has
been used in more than 150 real world projects of various domains, e.g., digital
34

libraries, e-marketplaces, and video data processing. It has been very good in
capturing initial requirements in an informal way, but it is not easy for clients with
moderate computer skills to update content, preserve revision history, share various
requirements related information, or support higher level of constructive customer
engagement. Table 2 summarizes the strength and limitations of EasyWinWin based
upon the user feedbacks from fall 2006.
Category Feedback
Strength Support WinWin
negotiation
• Brainstorming helps gathering win conditions
• Keep negotiation focused
• Useful in resolving conflicts
• Prioritization is useful
Support
communication
• Useful in communicating with remote client
Limitation Tool (ease of use,
robustness, archive
history)
• Difficult for client to use
• Not easy for new users without sufficient training
• Used only once, should use more after initial
negotiation
• Tool is not robust, time-consuming to use
Learning • Initial learning curve took time to overcome
• Better to provide detailed guideline
Preparation • Should provide training to clients
• Client and dev team should have several
engagements before WinWin negotiation
Table 2: Summary of EasyWinWin Fall 2006 Critiques
35

2.5 Wiki Collaboration Technology
The word “wiki” means “quick” in the Hawaiian language. Wiki technology allows
multiple people to edit the same document without overwriting each other’s changes,
and has the advantages of version control and tracking contributions. In Wiki a
contributor is able to access a page and edit it by clicking on the edit button and later
clicking on a save button. The changes are immediately visible for others to see and
further modify upon publication.
A wiki has several key characteristics, as listed below, which make it different from
the other widely used collaborative technologies. Majchrzak discussed using Wiki as
a knowledge management tool [Majchrzak et al. 2013]. She identified several key
characteristics, which make it different from the other widely used collaborative
technologies:
 Discussion in Wiki is based on importance and organization of content, not
based on chronological order.
 The complete version control in Wiki allows a participant to roll back to any
previous version regardless who made the changes.
 Any participant is able to access and edit a Wiki page with a click of a button.
 Once changes on a page are completed, it is immediately available for others
to view and further modify.
Wiki, as a lightweight documentation and collaboration platform, has demonstrated
its capability in improving productivity, collaboration, and innovation in many
different domains. The Wikipedia, built upon the MediaWiki engine, is one of the
36

most well known cases. It’s an online encyclopedia, collaboratively written by a group
of about 50,000 volunteers. TWiki is another popular wiki engine used by
corporations. Yahoo uses TWiki internally to manage documentation and project
planning; Disney uses TWiki as the central resource for ideas, notes, how to’s, specs,
and brainstorming; and SAP uses TWiki for multi-team collaboration.
Wiki has also been applied to support various requirement engineering activities. The
Fraunhofer Institute uses wiki technology to facilitate stakeholder participation. They
developed SOPWiki [Ras, 2009] to guide stakeholders in editing, grouping, and
structuring requirements. ReqWiki [Sateli et al. 2012] is integrated with natural
language processing to improve requirements specification documents. OntoWiki
[Riechert and Berger, 2009], employ semantic Web technologies to express
requirements semantics. ShyWiki [Solis and Ali, 2011] adopts spatial hypertext to
support graphical based representation. SmartWiki [Knauss et al. 2009] supports the
usage of semantic forms to generate forms for templates. The collaborative platform
of Wiki led us to explore the wiki technology and develop a Wiki-based collaborative
requirements negotiation system (WikiWinWin).
37

2.6 Successful Practice - Shaper Role
The users of wiki are encouraged to contribute content to the web pages and modify
others' pages. There are two types of contributions in using wiki: 1) contributing
knowledge to the wiki; 2) contributing primarily by refactoring the works of others,
also known as shaping. Participants who primarily contribute their personal
knowledge are called Personal Knowledge Contributors (PKC). Participants who
perform the role of shaping are called shapers. Researches [Majchrzak et al. 2013]
identified the shaper role as a critical success factor in wiki-based collaboration.
Shaping is defined as “the act of integrating, distilling, refactoring, identifying areas of
convergence and discrepancies, identifying topics receiving little attention in the
community, and significantly rewriting the contributions of others” [Majchrzak et al.
2006]. Below is an example of a shaper: (75-person software engineering group at a
multi-billion dollar tech company)
“I spend up to two hours a day working on the wiki. Much of this time I reorganize
other people’s materials, rename pages, create new links on the home page, or
restructure the home page. Benefits aren’t to mean personally, but they help the
group collaborate more effectively. They can find things easier” [Majchrzak et al.
2006].
In her study, Majchrzak also explained shapers’ value to a wiki community, as shown
in Table 3. The key points here are that shapers and knowledge contributors have
different levels of expertise depth. Knowledge contributors have more in-depth
expertise; they are more likely to contribute to the wiki. However, shapers, along with
38

transactive systems such as the Win-Win framework, are more effective in using the
wiki to facilitate stakeholder convergence and to identify business opportunities.
Thus they can help the group to collaborate more effectively.

Table 3: Wiki Shapers Add Business Value
39

2.7 Top-Level Dependability Framework
Identifying a system’s desired quality attributes is another challenging task in
requirements elicitation. Fortunately, some research results such as Table 4 [Boehm
et al. 2004], shows the top-level stakeholder/ value metric in the framework identified
in the USC 2004 NASA high dependability computing study. This framework provides
a top level baseline for how the major quality attributes vary by system stakeholder.
The key elements in the framework are: A definition of major classes of success
critical stakeholders; a classification of dependability attributes; and dependency
strength on these attributes for each class of stakeholders. The dependability
attributes in Table 4 are essentially the same as the attributes frequently associated
with software quality. Most of the relationships are hypotheses yet to be tested
quantitatively. In this research, we try to empirically test them and extend the
dependability attributes to functional requirements based on our project
characteristics.
40

Table 4: Top-level Dependency Framework
41

2.8 USC Software Engineering Course
The experiments of this research were conducted in the context of the USC software
engineering project course. This course (CSCI577ab) is the keystone two-semester
team project graduate software engineering course sequence at USC. Graduate
students form teams of 6 on-campus students and 2 off-campus students to develop
real-client software system projects. Off-campus students usually work full-time and
have a lot of experience in the software engineering field, while the majority of on-
campus students come directly from undergraduate programs and have very little
work experience. In each year, generally, we have about 150 students enrolled in the
class, 20 clients and 20 project teams. The project clients come from USC-
neighborhood small business, non-profit, local government, and campus
organizations needing improved e-services applications. Most of the projects are web
applications or database applications with the scope of implementable and deployable
within 24 weeks. Compared to fall 2007, there were increased number of COTS/ web
services intensive projects in fall 2008 and fall 2009.
42

Chapter 3: WikiWinWin Framework
3.1 Overview
WikiWinWin is a groupware framework and tool that leverages the wiki technology,
adopts a successful wiki practice – the shaper role, combines with existing theories and
stakeholder/ value metrics, to help interdisciplinary stakeholders involved in the software
project to more rapidly understand each other and more easily negotiate mutually
satisfactory requirements. Figure 7 illustrates the big picture of the framework. The major
components in this framework are reviewed in Chapter 2: underlying theories, enabling
technology, shaper role, and stakeholder/ value metric. Below we summarize the
characteristics of WikiWinWin.

Figure 7: WikiWinWin Big Picture

43

Easy accessible platform
Wiki technology can support large numbers of users to collaborate. Its web-based
platform allows distributed team members to use WikiWinWin at anytime and from
anywhere. Individuals can add ideas to Wiki pages and content is immediately available
for others to see and further update on it. It saves stakeholders a lot of time and provides
convenience by reducing verbal meetings or telecommunications.
Maintain the WIOAs
The relationship between WIOA artifacts that are created in WinWin negotiation must be
maintained throughout the system development process. Examples are the relationships
between issues and their win conditions or options. In WikiWinWin, we organize the
WIOAs as topics and define metadata to express the relational semantics among them.
Guidance for negotiation process
It is difficult for stakeholders to collaborate and conduct WinWin negotiation without
clear guidance of the negotiation process. The WikiWinWin process defines clear steps to
guide the WinWin requirements negotiation, and provides a clear structure to organize
requirements and related information. WikiWinWin also includes boundary objects to
facilitate the collaborative learning of stakeholders, e.g. the general terminologies in
WinWin negotiation and the win condition taxonomy.

44

Support of synchronous collaboration
Information in the wikis is organized by topics. When more than two people edit the same
topic at the same time, there will be a conflict. The strategy in TWiki to resolve such
conflict is to lock the page until the first person finishes editing. It is not easy for more
than one person to edit the same page to exchange ideas. And wiki is generally only used
for asynchronous collaboration. However, requirements negotiation usually involves
synchronous collaboration. We found several ways to resolve this issue. First, there is
hardly any chance of conflict when more than one person uses the “comment” function to
append information to a page during synchronous collaboration. And we can organize the
information into different topics to reduce the chance of conflict. The information from
different topics can also be automatically collected into the same page. To some extent,
people can exchange ideas upon the same web page. A shaper will be a person who plays a
major role in ensuring the integrity and consistency. With the various forms,
WikiWinWin will automatically and correctly organize the various inputs from users into
the corresponding WIOA topics and maintain the relationships among them.
Versioning across several pages
Since wikis only provide versioning on the basis of a single page, defining releases based
on a certain status of several pages (for example, a baseline) can become laborious.
WikiWinWin provides features to support cross-page versioning and baselining. It can
collect the links to the current version of all the requirements related topics into a certain
page and thus represent a release and baseline of the requirements negotiation.

45

Preserve negotiation history
In WikiWinWin, any stakeholder can contribute to any win conditions, issues, options,
terms and ideas, and the editing history is preserved with individual wiki page versions.
Milestone negotiation records can be preserved with cross-page versioning.
Balance waterfall and iterative approach
WikiWinWin enables concurrent passing of agreements with identifying and evaluating
new win conditions. Project teams may revisit passed agreements by retrieving previous
negotiation records.
Shaper facilitated collaboration
The shaper plays an important role WikiWinWin negotiation. During negotiation
meetings, the shaper is a facilitator for meeting discussions. Shapers will capture initial
win conditions based on discussed ideas. Then other stakeholders can review and vote for
agreements. In asynchronous collaborations, shapers can configure the WikiWinWin
system and let it automatically notify every participant with the recent changes in
WikiWinWin. Shapers will set a negotiation deadline and send out emails asking
participants to confirm with the options and agreements. Shaper also is responsible for
ensuring the WinWin equilibrium that all the win conditions are covered by agreement
and there is no outstanding issue.
46

3.2 Negotiation Process
Figure 8 overviews the WikiWinWin activities and Figure 9 elaborates the negotiation
steps. The major activities in negotiating WIOA (Win condition, Issue, Option, and
Agreement) include:
 Brainstorm/ identify stakeholders’ win conditions
 Agree on win conditions or identify issues;
 Provide and negotiate options
 Reach agreements
 Prioritize agreements
These activities can be carried out in two different collaboration modes: synchronous
mode (negotiation meeting) and asynchronous mode (off-line, self-facilitated
negotiation). Table 5 elaborates them with scenarios.
The negotiation meetings focus on providing win conditions and inventing options to
resolve issues. By having participants brainstorm and stimulate each others’ ideas in a
group setting, the teams are likely to produce more and better ideas than brainstorming
individually. Co-located team members participate in person. Remote team members
participate through web conference or telephone.
During the off-line negotiation, Participants focus on reviewing and discussing ideas.
Activities that express an individual’s preference, such as prioritization, also happen in the
off-line negotiation. Participants are not required to meet together. Typically they use the
WikiWinWin tool at their convenience with some degree of coordination among them.
The WikiWinWin tool has some features for supporting coordination, such as displaying
47

recently changed topics, sending email notifications about daily changes. Participants also
use telephone, email, messaging, etc. to supplement communication.

Figure 8: Negotiation Process and Tool Support

48

Figure 9: Negotiate WIOA

49

Mode Used in Issue & Solution Scenarios
Synchronous Face to face
negotiation
meetings;
teleconference
Issue: editing conflict when more than one person
edits the same page at the same time

Solution:
• Organize information into different pages to
reduce the chance of conflict
• Oral discussion, at the same time, a facilitator
(usually shaper) records discussion in the tool,
participants see the results

• Stakeholders participate in initial brainstorming meeting to
identify win conditions
• Subsequent meeting discussion on win conditions after
shaper rework the brainstorming results, identify agreed
upon win conditions and issues
• Subsequent meeting discussion on options for resolving
issues
Asynchronous Off-line,
self-use
Issue: coordination

Solution:
• Configure the tool to automatically send out
notification
• Display recent updates in the tool
• Shaper in charge: set deadline, periodically check
discussion progress in the tool, use emails to
inform stakeholders
• Shaper reworks brainstorm results (Remove duplication,
ambiguity, categorize); gather glossary
• Stakeholders review and signify agreement on win
conditions by posting signed comments in the tool
• Stakeholders can also submit new win conditions in the tool
• Shaper identify point of agreement or identify conflict as an
issue based on stakeholders’ inputs
• Stakeholders submit and review options in the tool
• Shaper checks equilibrium status after an issue is resolved
• Shaper prepares a prioritization page for each stakeholder
• Stakeholders prioritize agreements
Table 5: WikiWinWin Collaboration Scenarios
50

3.3 Usage Scenarios and Roles
In WikiWinWin, all stakeholders will play as PKCs and optionally as Shapers in expressing
their ideas, win conditions, and shared knowledge. A facilitator, who could be an
appropriately skilled stakeholder, will serve to initialize the WinWin process and
participate as a shaper, who will encourage PKCs to express their ideas, moderate the
negotiation process, and help to shape stakeholders’ inputs.
Adopting the successful shaper practice is likely to bring a number of benefits to
requirements negotiations:
 By focusing on similarity and discrepancy among win conditions, the shaper helps
focus negotiations towards reaching agreements.
 Allows participants without software negotiation expertise to contribute since
experienced wiki users/ requirements engineers can take on shaper role.
 By identifying missing information, and inactive pages, the Shaper helps ensure
stakeholders ‟ inputs are recognized.
 Shapers coordinate collaboration tasks in asynchronous situations, such as
monitoring changes and informing relevant stakeholders to participate.
 Reducing overhead by having the shaper take care of facilitation tasks instead of
having every negotiator repeating the effort.
Figure 10 illustrates an example. The top screen shows the initial ideas from stakeholders,
a few are pointing to a prospective content management system. So the shaper organizes
these similar ideas into a win condition, as shown in the middle screen. What happens
next is that stakeholders engage a further discussion on the win condition, shown in the
51

bottom screen. Shaper will continue to steer discussion by identifying points of
convergence and points of conflict.

Figure 10: Shaper Adds Value in WikiWinWin
We discuss four negotiation activities to illustrate how the WikiWinWin tool was used
and what were performed by the PKC role and the shaper role.
Scenario 1: Brainstorming
The purpose of the brainstorming meeting is to produce as many win condition ideas as
possible. To keep the brainstorming focused, a shaper acted as facilitator. The tool
provided a pre-defined negotiation topic taxonomy that corresponds to categories in the
requirements documentation. With the shaper guidance, PKCs brainstormed their win
conditions one topic at a time. Originally we planned to have all PKCs type their own
ideas in the tool, but we ran into the wiki page editing conflict problem when we
conducted the training. So instead of having all PKCs using the tool to submit ideas, co-
52

shaper recorded every idea into the tool as PKCs spoke out their ideas. Figure 11 shows an
example of initial ideas surfaced at the brainstorming meeting. As circled in the figure,
several of these ideas point to a prospective agreement: using a COTS candidate – Joomla.

Figure 11: Initial Brainstormed Ideas
Scenario 2: Converge/ Identify Issue
The resulting ideas from brainstorming were captured as comment under each topic
category in the tool. Shaper then crafted the initial win conditions based on those ideas.
The tool provided a form-based interface for submitting a win condition. When going
through the ideas, shapers combined ideas indicating similar meanings. If an idea covered
several aspects, shapers split it into separate win conditions. Shapers also re-categorized
ideas and rewrote descriptions to make the win condition more understandable. Shapers
also collected terms defined in the project context and added them to the glossary. Figure
12 shows shaper proposed a content management win condition by combining the similar
ideas.

Figure 12: Win Condition Organized by Shaper
53

After the shaper reworked the ideas into win conditions, PKCs reviewed them
individually. If a PKC agreed with the win condition described by the shaper, he added an
agreement comment in the comment box. Otherwise, he put his opinions in the comment
box. If a PKC felt a win condition description produced by a shaper did not capture her
idea, she could also edit it. Additionally, PKCs could also submit new win conditions.
Shapers came back to the tool to check PKCs ‟ comments periodically. When the shaper
saw all PKCs ‟ agreement comment on a win condition, he changed the win condition
status to an Agreement. If he saw conflicting/ opposing comments, he submitted an
outstanding issue. Figure 13 shows a shaper submitted an issue of inflexibility based on
discussion.

Figure 13: Issue Logged by Shaper
Scenario 3: Invent Options
Most of the teams had a second negotiation meeting to discuss issues and options within a
few days after the first meeting. Some teams did not finish converging win conditions, so
they discussed the remaining win conditions at the meeting. The meeting was carried out
54

in the same fashion as the brainstorming meeting, except a shaper used the issues logged
in the tool as the discussion thread. PKCs discussed possible options for each issue and
the co-shaper recorded them in the tool. During the discussion, participants often agreed
on which options to adopt. Figure 14 shows a PKC proposed an option after further
investigating the proposed COTs candidate.

Figure 14: Option Proposed by PKC
Scenario 4: Reach Agreement
Because project teams ‟ WinWin negotiation took place concurrently with prototyping,
exploring COTs, and analyzing alternatives, we did not expect the teams to address all the
issues and options by the end of the second meeting. After the meeting, PKCs continued
identifying options to close remaining issues. In this step, shapers periodically checked the
discussion progress in the tool and tagged the adopted option as agreement. Figure 15
shows the tree view of the agreements.

Figure 15: WIOA

55

3.4 Dependability Attributes Classification
In order to analyze the dependency relationship, we extended attributes from the Top-
Level Dependability Framework reviewed in Chapter 2, and defined three sets of
attributes: level of service goals, functional tasks, and project activities, as shown in Table
6. The levels of service attributes were compatible with the corresponding dependability
attributes in Table 4. Functional task attributes were added according to 14 projects’
specific operational context. Project activities were extended from cost and schedule to
more project related activities such as requirements of transition, deployment etc.
1.Functional Tasks

2.Level of Services

3.Project Activities

Administration support Correctness User interface standards
Financial support Availability Hardware interface
Query Security Communications interface
Display Privacy Other software interface
Update Interoperability Course required process& tools
Storage Usability Other development tools
Tracking Performance Language
Scheduling Evolvability Computer hardware
Notification Reusability Computer software
Automation Concurrency Computer communication
Social networking Deployment
Scan and conversion Transition
Status message Support environment
Usage analysis Budget
Archive Schedule
Validation
Feedback& evaluation
Advertising
Others
Table 6: Classification of Dependability Attributes
56

Chapter 4: Research Methodology
4.1 Overall Strategy
To validate WikiWinWin framework solution, we use multiple real-client team projects
from the USC graduate software engineering course (CS577) as the experimental test bed.
We inject the WikiWinWin framework into the course each year, where we instruct all the
project teams to use WikiWinWin to conduct their requirements negotiations.
There are a total of five experiments conducted in CS577 from fall 2007 to fall 2009. Three
out of five experiments are negotiation experiments conducted in the project inception
phase in fall 2007, fall 2008 and fall 2009. The remaining two experiments are role-played
experiments conducted in the project elaboration phase in fall 2008 and fall 2009. Major
activities in the inception phase include WinWin negotiation and top-level requirements
definition. Key effort in the elaboration phase is architecture definition with concurrent
elaboration on other system artifacts. Because all project teams use the WikiWinWin tool
for their WinWin negotiation, we evaluate the various tool usage aspects in the
negotiation period. Later in the elaboration phase, since stakeholders already had
WinWin negotiation and reached agreement about requirements, it provides assurance
for understanding dependability priorities.
We use multiple criteria to evaluate the outcomes of using WikiWinWin. These include
various WikiWinWin usage results among multiple teams in each year and correlation to
project grades; client and developer satisfaction via evaluations and critiques. The
experiment will be repeated in multiple years with adjustment made according to prior
57

year’s feedbacks. Repetitive experiments will give robust testing on the hypotheses; also
allow us to see whether the adjustment made from year to year is effective. If results
obtained from using multiple criteria, testing on multiple projects, repeating multiple
rounds, all point to the same direction, it will give reasonable evidence to accept or reject
the validity.
58

4.2 Subjects
USC’s annual graduate-level, real-client, team-project software engineering course
provides a valuable opportunity to evaluate the WikiWinWin solution. The project teams
are formed by first year graduate students and project clients. Typically each team consists
of 6 on-campus students, 2 off-campus students, and 1 client representative. Off-campus
students usually work full-time and have a lot of experience in the software engineering
field. Average industry experience of these off-campus students is at least five years. The
majority of on-campus students come directly from undergraduate programs with some
work experience. Although team composition generally stays the same, the quality of
students may be different from year to year. The project clients come from USC-
neighborhood small business, non-profit, local government, and campus organizations
needing improved e-services applications. Table 5-3 lists the projects used in fall 2007, fall
2008, and fall 2009 experiments.
S# Year Project Type
1 2007 Box Office Database System Stand-alone
2 2007 Online Peer Review System Web-based
3 2007 Thai CDC Software Tool Web-based
4 2007 USC COINCOMO Cost model
5 2007 Crystal Stairs Dashboard Stand-alone
6 2007 Family & Homeless Well-Being Project Web-based
7 2007 BTI Appraisal Project Stand-alone
8 2007 LAMAS Customer Service Application Web-based
9 2007 Web-based service for TPC Foundation Web-based
10 2007 REEO Database Web-based
11 2007 BID Review System Stand-alone
12 2007 THSA Website Project Web-based
59

13 2007 EEO Section Database Stand-alone
14 2007 Conference Room Reservation System Web-based
15 2007 Proctor and Test Site Tracking System Web-based
16 2007 E-Mentoring Program Web-based
17 2007 Social Networking Tools for Librarians Web-based
18 2007 EZSIM II Web-based
19 2007 Los Angeles County Generation Web Initiative Web-based
20 2007 Development of Hunter-Gatherer Database Web-based
21 2008 Master Pattern Web-based
22 2008 Housing Application Tracking System Web-based
23 2008 Hunter-gather interactive research data Web-based
24 2008 The IGM on-line Art Gallery Web-based
25 2008 EZBay Stand-alone
26 2008 AAA Petal Pushers Remote R&D Web-based
27 2008 Information Organization System Web-based
28 2008 The Roots of Inspiration Web site Web-based
29 2008 Web-based service for TPC Foundation Web-based
30 2008 ACME Research Engine for USC-WB Archives Web-based
31 2008 The Virtual Assistant Living and Education Program Stand-alone
32 2008 Database for III, Inc. Stand-alone
33 2009 Online DB Support for CSCI 511 Web-based
34 2009 SHIELDS for Family Web-based
35 2009 Theater Stage Manager Program Web-based
36 2009 Growing Great Online Web-based
37 2009 SPC Website Automation Enhancement Web-based
38 2009 VALE Information Management System Web-based
39 2009 LANI D-Base Web-based
40 2009 FreeHelpList.org Web-based
41 2009 Early Medieval East Asian Timeline Web-based
42 2009 BHCC Website Development Web-based
43 2009 Client Case Management Database Web-based
60

44 2009 Website Development & Improvement Web-based
45 2009 Healthcare The Rightway Web-based
46 2009 AROHE Web Development Web-based
*gray indicates COTS/ web services intensive
Table 7: Summary of projects
61

4.3 Experiment Design
4.3.1 WikiWinWin Negotiation Experiment
Objectives - The experiment aims to evaluate the effectiveness of WikiWinWin tool and
process at supporting rapid requirements negotiation among stakeholders.
Configuration - The WikiWinWin tool is configured to support multi-project concurrent
negotiation activities. To avoid interference between project teams, each project team was
given its own negotiation space in the WikiWinWin tool. We used the group access
control to grant access privilege to participants. Participants can view and edit topics in
their own negotiation space.
Roles - We defined two types of roles in WikiWinWin – Knowledge Contributor (KC) and
Shaper. The WinWin negotiation participants are students, client representatives, and
other key stakeholder representatives, e.g. users. All participants played the KC role. In
addition, two students on each team performed the shaper role: usually one full-time, on-
campus student and one part-time, off-campus working-professional student. The off-
campus shaper was generally the lead shaper.
Training - We provide training to both students and clients. All students received some
training on the general WinWin approach and tutorials on using the tool. In addition, the
shapers had additional one hour training on the shaper tasks. We provide one training
session to clients.
Activities - The activities performed by the project teams include pre-negotiation mutual
learning, negotiation meetings, and off-line negotiation.
62

Before the project teams formally start the initial WinWin negotiation, stakeholders
participate in mutual learning activities including a classroom workshop engaging
collaborative learning, site visits, and early team meetings which focus on understanding
project backgrounds and benefits. The purpose of these activities is to help stakeholders
with different backgrounds quickly build shared understanding among each other and set
a collaborative context for the up-coming negotiations.
The negotiations are carried out in both meetings and off-line situations. During the
meetings, participants brainstorm win conditions, discuss issues and options. Aside from
meetings, participants are free to use the tool at their convenience, as often as needed, for
reviewing, commenting, adding, and prioritization. Teams are required to do concurrent
prototyping along with negotiating requirements.
The negotiation results provide basis for developing other project artifacts. For example,
the prioritized agreements transform into formal requirements; the unresolved issues
represent risks that should be managed by the risk mitigation strategy; criteria are
provided for evaluating COTS products, etc.
4.3.2 Dependability Prioritization Experiment
Objectives - The experiment aims to gain understanding of stakeholder/ value
dependence patterns using real-client projects being developed at USC graduate software
engineering course. More specifically, we want to 1) identify the primary dependability
attributes in three categories: functional tasks, level of services, and project related
activities; 2) evaluate the hypotheses of dependency relationships shown in Table 1.
63

Preparation - Prior to the experiment, all teams conducted win-win negotiations with
real client representatives and agreements were reached. To make sure the experiment is
under control, we also gave students two tutorials, one is for teaching students how to
role-play stakeholders in step 1, the other is for teaching students how to rate
meaningfully in step 3.
Step 1: Define major classes of success critical stakeholders (SCSs) and identify role
players.
We focused on three main types of stakeholders in the software engineering process:
client, user, and maintainer. All three types of stakeholders prioritize from the perspective
of business importance, but often the clients were not able to provide end-user and
maintainer participants. Developer prioritization was not included in this study because it
was done from a different value perspective: ease of implementation.
We instructed students to role play client, user, and maintainer in the experiment. We
will compare the results between real-client and role-play client. If the results show a
strong correlation, it will provide some confidence for accepting results from role-play
users and role-play maintainers.
To make sure that the experiment is under control, in this step we teach students how to
role-play: A challenge in this experiment is to identify role players who are able to
prioritize from the perspective of other stakeholders. We suggested to teams to identify
role players based on who best understands the type of stakeholder. For example, the
person who works on the operational concept could be a good candidate to role play user.
We also provided some value proposition hints for the role players to consider. For
64

example, the role play client may consider on time, within budget, functions are correctly
implemented, and are interoperable with existing system, etc.
Step 2: Extend the classification of dependability attributes.
In order to analyze the dependency relationship, we extended attributes from Table 1, and
defined three sets of attributes: level of service goals, functional tasks, and project
activities, as shown in Table 8. The levels of service attributes were compatible with the
corresponding dependability attributes in Table 1. Functional task attributes were added
according to the 14 projects’ specific operational context. Project activities were extended
from cost and schedule to more project related activities such as requirements of
transition, deployment etc.
Step 3: Rate dependency strength on these attributes.
The WikiWinWin tool [36] has the capability of providing a rating page for each
participant. The rating page displays all the win-win agreements to be rated as shown in
Fig. 1. All role players and real-client representatives rated on business importance on a
scale of 1 to 9:
1-Not Worth Having, 3-Want to Have,
5-Could Have, 7-Should Have, 9-Must Have
Use of intermediate values between two adjacent judgments (2, 4, 6, 8) was allowed. A
participant might bypass rating on an agreement if he/ she felt it did not have direct
significance. Participants were encouraged to provide comment for their ratings.
In this step, we teach students how to rate meaningfully: It is likely that a participant
would simply treat every agreement as equally important, e.g. all rated as “Must Have”. To
65

minimize such instances, we encourage participants to consider the following factors
when giving their priority ratings:
 Whether this is a core function: a basic function that a system must have. e.g. user
administration is a must have function for most of the projects
 Whether this function depends on any other functions: if a function is rated
highly, the functions that it is strongly dependent on should be at least that high.
Step 4: Data collection and consolidation.
After all participants submitted their ratings, the results were reviewed for completeness.
Projects that didn’t have any priority ratings from the real client representative were
excluded.
Next, every rated agreement was classified based on the list of dependability attributes in
Table 3. If duplicate ratings were given on one agreement, it counted as one rating. We
calculated the following on each attribute:
a) Mean rating from each participant - take the average of all his/ her ratings
belonging to that attribute.
b) Mean rating from each project – for participants playing the same role, average
their results from a).
c) Mean rating from each domain – for projects belong to the same domain, average
results from b).
d) At the end of step 4, we had the dependency ratings per each stakeholder class and
domain.
66

4.3.3 Data collection
To answer the research questions in Chapter 1, both quantitative and qualitative data are
collected. The quantitative data include various patterns of tool use by each project team
and graded project artifacts. Qualitative data include feedbacks from student critiques and
client evaluations.
Tool use – WikiWinWin tool use is measured by numbers of transactions, and number of
days the tool is used. It includes use by both client and developers on each project team.
The data source is the wiki log files. Wiki logs time-stamped transactions with user
identity when users clicked on ‘save’. We obtained these usage data directly from the wiki
log files. The usage data was collected at two places: at the end of the 12-day negotiation
period, and at the end of the inception phase about three weeks later. Because there was a
difference in the number of calendar days in each year, using a fixed time frame - the 12-
day negotiation period was preferred for keeping consistency. In addition, the use
measured at the end of the inception phase was likely to be affected by the assignment
deadline.
Project outcome - project outcome or success defined as having a perfect grade on the
project package, which includes prototypes and documents for the operational concept,
requirements, architecture and plans. The loss in grade indicates shortfalls in artifact
quality such as thoroughness, clarity, et al. The graded project artifacts were measured by
the project package grade at the end of the inception phase in the Lean MBASE/ ICSM
(Incremental Commit Spiral Model) process used in the course.
(http://greenbay.usc.edu/ICSM/index.htm)
67

Tool success - tool success is measured by numbers of positive and negative comments in
student critiques. The critiques and evaluations were collected at the end of the semester.
Metrics - We perform regression to test the relationship between the various usage
metrics and outcome metric. Since we use loss in project grade, the correlation coefficient
should be negative. The log () was used in some cases because the data reflected an
exponential distribution. A significance level below 0.05 indicates strong support for
rejecting a hypothesis.
Incomplete projects and exception cases are not used in the analysis. Two teams in fall
2008 were excluded from correlation testing because their project packages were missing
operational concept artifact and requirements artifact.
Usage Metrics:
 Number of use by team. This is the total number of submitted transactions by
everybody in the project, including clients. (H1.1)
 Number of use by shaper. This is the total number of submitted transactions only
by shaper(s) in the project. (H1.2)
 Number of days used by team. This is the total number of calendar days that the
tool is used at least once by a project team. (H1.3)
 Number of days used by shaper. This is the total number of calendar days that the
tool is used at least once by shaper(s) in the project (H1.4).
 Number of days creating artifacts. This is the total number of calendar days that at
least one win condition, issue, option, or agreement was created. (H1.5)

68

Outcome Metrics:
 Number of points lost in project package grade.
 Number of points lost in client evaluations.
 Number of positive and negative comments of WikiWinWin in student critiques.
69

Chapter 5: Evaluation and Analysis
5.1 WikiWinWin Usage Results
5.1.1 Early usage vs. milestone usage
As mentioned in Chapter 4, we collected tool usage at the end of the 12-day negotiation
period (early usage) as well as at the end of the inception phase (milestone usage). To
illustrate whether early usage or milestone usage is more appropriate for testing the
hypotheses, Table 11 – Table 13 show the Pearson correlation coefficients among team early
usage, team milestone usage, and loss in project grade.
Cohen and Holliday (Cohen and Holliday 1982) suggest the following rule of thumb to
interpret the Pearson’s coefficient: 0.19 and below is very low correlation; 0.20 to 0.39 is
low correlation; 0.40 to 0.69 is modest correlation; 0.70 to 0.89 is high correlation; and
0.90 to 1 is very high correlation.
In all three years, team early usage and team milestone usage are correlated, however,
early usage has a stronger correlation to the loss in project grade than the corresponding
milestone usage: (-0.61) vs. (-0.49) in fall 2007, (-0.60) vs. (-0.39) in fall 2008, and (-0.92)
vs. (-0.89). A possible reason is that some teams rushed with last-minute, low quality use
when the assignment deadline was approaching. This comparison supported our
preference of choosing usage data measured at the end of 12-day early period for the
subsequent hypothesis testing. It also provided evidence suggesting relationships between
quality early usage and project outcome.
70

Milestone usage Early usage Log(loss)
Milestone usage 1 0.67 -0.49
Early usage 1 -0.61
Log(loss) 1
Table 8: Fall 2007 Early Usage vs. Milestone Usage
Milestone usage Early usage Log(loss)
Milestone usage 1 0.69 -0.39
Early usage 1 -0.60
Log(loss) 1
Table 9: Fall 2008 Early Usage vs. Milestone Usage
Milestone usage Early usage Log(loss)
Milestone usage 1 0.83 -0.89
Early usage 1 -0.92
Log(loss) 1
Table 10: Fall 2009 Early Usage vs. Milestone Usage
5.1.2 Hypothesis Support
Hypothesis 1. There is a lack of correlation between various usage aspects of
WikiWinWin and project outcome aspects.
H1.1. Higher or lower numbers of tool usage by team has no correlation with project
outcome.
Figure 16 illustrates a linear relationship between log (loss in grade) and number of tool
usage by team, fitted with fall 2007 data. The coefficient of determination (R
2
) has value of
0.3672. A similar relationship is also confirmed with fall 2008 and fall 2009 data, as shown
in figure 17 and figure 18.
71

There are two points in Figure 17 that are far from the fitted regression line: (63, 0.641) and
(254, 1.307). The team who had a smaller amount of usage (63 saved versions) had used the
tool more frequently (7 out of 12 days), compared to the other team (254 saved versions)
but only used the tool 4 days in the same period. This illustrates another factor: the
frequency of use (tested in H1.3). H1.1 is rejected at the 0.004 level in 2007, and at the
0.003 level in 2009, but close at the 0.06 level in 2008.

Figure 16: Fall 2007 Outcome ~ Use by Team

Figure 17: Fall 2008 Outcome ~ Use by Team

72

Figure 18: Fall 2009 Outcome ~ Use by Team
H1.2. Higher or lower numbers of tool usage by shaper has no correlation with project
outcome.
The relationship between log (loss in grade) and number of tool usage by shaper(s) is
confirmed by all three years’ data, shown in Figure 19 – figure 21. Both (R
2
) value and p-
value have improved in fall 2009. With respect to the p <= 0.05 criterion for hypothesis
rejection, H1.2 is rejected in 2007 and 2009, and comes close to rejection in 2008.

Figure 19: Fall 2007 Outcome ~ Use by Shaper

73

Figure 20: Fall 2008 Outcome ~ Use by Shaper

Figure 21: Fall 2009 Outcome ~ Use by Shaper
H1.3. Higher or lower numbers of team usage days has no correlation with project
outcome.
Figure 23 and figure 24 show the relationship between log (loss in grade) and number of
days each team used the WikiWinWin tool, fitted on fall 2008 data and fall 2009 data
respectively. There is a clear linear correlation with the coefficient of determination (R
2
)
value of 0.731 and 0.476. The fit on fall 2007, as shown in Figure 22, didn’t show significant
linear relationship and the coefficient of determination (R
2
) value is only 0.14. With
74

respect to the p <= 0.05 criterion for hypothesis rejection, H 1.3 is rejected at 0.002 level in
2008, and comes close to rejection at 0.06 level in 2009, but is not rejected at the 0.10 level
in 2007.

Figure 22: Fall 2007 Outcome ~ Number of Days Used by Team

Figure 23: Fall 2008 Outcome ~ Number of Days Used by Team

75

Figure 24: Fall 2009 Outcome ~ Number of Days Used by Team
H1.4. Higher or lower numbers of shaper usage days has no correlation with project
outcome.
The relationship between log (loss in grade) and number of shaper usage days is also
confirmed with fall 2008 data and fall 2009 data, as shown in Figure 26 and Figure 27.
After excluding one outlier in fall 2008 data, the (R
2
) value is improved from 0.28 to 0.48.
Fall 2009 shows similar trends as Fall 2008. When testing on Fall 2007, shown in Figure 25,
the result was not significant, R
2
value is only 0.11. H 1.4 is rejected at the 0.04 level in
2008, and at the 0.05 level in 2009, but is not rejected at the 0.14 level in 2007.

76

Figure 25: Fall 2007 Outcome ~ Number of Days Used by Shaper

Figure 26: Fall 2008 Outcome ~ Number of Days Used by Shaper
77

Figure 27: Fall 2009 Outcome ~ Number of Days Used by Shaper
H1.5. Higher or lower numbers of days creating artifacts has no correlation with project
outcome.
Figure 29 and Figure 30 show a clear linear relationship between log (loss in grade) and
number of days spent in creating negotiation artifacts in Fall 2008 and Fall 2009.
Coefficient of determination (R
2
) value of 0.596 and 0.64 indicate that teams which spent
only one or two days on creating negotiation artifacts had more shortfalls in their project
packages than the teams who evolved their artifacts over the negotiation period. H1.5 is
rejected at the 0.009 in 2008, and at the 0.017 level in 2009, but is not rejected at the 0.17
level in 2007.
78

Figure 28: Fall 2007 Outcome ~ Number of Days Creating Artifacts

Figure 29: Fall 2008 Outcome ~ Number of Days Creating Artifacts
79

Figure 30: Fall 2009 Outcome ~ Number of Days Creating Artifacts
H1.6. Using WikiWinWin will not improve project outcome (as compared to using
EasyWinWin).
Figure 31 shows project package grade loss at Milestone 1. For all three years that
WikiWinWin was used, grade loss was smaller compare to the year that EasyWinWin was
used. And fall 2009 has the least loss. The improved outcome was confirmed by T-test
results, when comparing each year that WikiWinWin was used, vs. fall 2006. Thus H1.6
was rejected.

Figure 31: Project Package Grade Loss in Each Year
80

5.2 WikiWinWin Qualitative Results
5.2.1 Student Critiques
Figure 32 shows improved satisfaction measured by positive and negative counts of
WikiWinWin tool in student critiques.
One observation is that in all three years there is a high consensus on the usefulness of
WikiWinWin tool. The most positive aspects of the usefulness include:
 Involve all stakeholders to collaborate
 Gather, communicate, and better understand of needs
 Prioritize agreements
 Eliminate paper work
 Allow individual use at convenience
 Keep history for tracking and future reference
 Generate border and deeper requirements
In fall 2007, the number of negative counts was much higher than the number of positive
counts. In fall 2008 and fall 2009, we got more positive comments than negative
comments. The ratio of positive comments vs. negative comments has improved from 0.81
in fall 2007, to 1.06 in fall 2008, to 1.51 to fall 2009. This indicates that we are seeing
positive effects from implemented improvements (further discussion in Section 5.4). By
fall 2009, tool robustness was no longer a concern by users. Tool efficiency was also
further improved from fall 08 to fall 09 with features like prioritization template and
automated report generation.
81

However, usability and making it easy to use remain major needed improvements. Key
shortfalls include: some clients could not use it independently; difficult to modify
artifacts; not straightforward for adding issues and options; and need simplified editing
and navigation. Contrary to our assumption, users didn’t think the default wiki markup
edit was easy and convenient. They preferred a more intuitive WYSIWYG (What You See
Is What You Get) editor.

Figure 32: Student Critiques on WikiWinWin
Fall 2007 Negative counts = 103
Easy to use 37
UI 32
Easy to learn 18
Efficiency 12
Robustness 4
Fall 2008 Negative counts = 54
Easy to use 20
UI 25
Easy to learn 4
Efficiency 5
Robustness 0

82

Fall 2009 Negative counts = 77
Easy to use 21
UI 27
Easy to learn 14
Training 13
Notification 2
Table 11: Summary of WikiWinWin Shortfalls from Student Critiques
5.2.2 Client Evaluations
The grade loss of client evaluation measured at the end of semester was summarized in
Figure 33. Overall, there was no significant improvement comparing to the year of using
EasyWinWin. Although fall 2007 and fall 2009 showed higher overall client satisfaction,
fall 2008 was not confirming the trend. Also, neither 2008 nor 2009 had the uniform
satisfaction, as indicated by the high standard deviation. Client evaluation survey contains
a number of questions that are intended to survey different aspects of the project and the
software engineering course; therefore, it is less conclusive on the effect of using
WikiWinWin.

Figure 33: Client Evaluation Grade Loss

83

5.3 Stakeholder/ Value Dependency Results
5.3.1 Stakeholder/ Value Dependency in Fall 2008 Experiment
The 6 projects in fall 2008 are those from the multi-media archive domain, chosen to
provide a relatively uniform sample. From Table 12 we can see that the average ratings
from different stakeholders are all around 7 which mean “Should Have”. For one thing, the
six projects are all new projects, the initial requirements are generally the core
requirements with few gold-plating requirements, so the average ratings for all
requirements are “Should Have”.
Another consideration is that the rating given by a participant may be subjective to his/
her own interpretation of the rating criteria. In this way, it might cause non-uniform
understanding of rating criteria which might finally threaten the validity of the
experiment. However, in our experiment, similar average ratings with small standard
deviation from different stakeholders might imply the uniform understanding of rating
criteria from different stakeholders, which means that different stakeholders understand
the rating “Must Have”, “Should Have”, “Want to Have” “Not Worth Having” the same
way, which is also the expected result since we give students tutorials on how to rate
meaningfully.
However, having a uniform understanding of rating criteria doesn’t mean that different
stakeholders have the same value dependency pattern. We did a correlation analysis on
the stakeholders’ ratings of 39 attributes listed in Table 6, and the result is shown in Table
13. We can see that except the client and the role play client’s highest correlation (0.53),
correlations among other stakeholders are low. Especially the correlation between role
84

play maintainer and real client is as low as 0.05, which implies that different stakeholders
have different value dependency patterns. Another interesting finding is that the client
and role-play user have a relatively high correlation (0.44). This might imply that client’s
rating sometimes included the user perspective since the client may also be the users
themselves.
To explore what the value dependency patterns for each type of stakeholder are, we sort
the average ratings on attributes from each type of stakeholder according to the ratings,
and list the top 10 attributes with the highest ratings and the last 10 attributes with the
lowest ratings for each type of stakeholder in Table 14. The number 1,2,3 before each
attribute stands for that whether the attribute belong to Functional Tasks, Level of
Services or Project Activities.
*User *Maintainer Client * Client
Average Ratings 7.16 6.75 7.22 6.74
Standard Deviation 1.53 1.35 1.44 1.37
*-role play stakeholders
Table 12: Mean Rating in Multi-media Projects
*User *Maintainer Client *Client
*User 1.00
*Maintainer 0.32 1.00
Client 0.44 0.05 1.00
*Client 0.29 0.33 0.53 1.00
Table 13: Correlation in Multi-media Projects
*: Role- play stakeholders
1: This attribute belongs to Functional Tasks
2: This attribute belongs to Level of Services
3: This attribute belongs to Project Activities
: The ratings decreases downwards, for the Top 10 part, it means the first one is the most important; for the
Last 10 part, it means the last one is the least important

85

Top 10  Last 10  Top 10  Last 10 
*User Client

1. Others 3. Deployment

2.Security
3.Computer
hardware
3.Support
environment
3. Computer
software

2.Correctnes
3.Computer
communication
1. Query 2. Security
3.Support
environment
1. Display
2. Availability
3. Other software
interface

1. Usage analysis 3. Budget
1. Automation 3. Schedule
1.Administration
support
3. Deployment
3.Computer hardware 2. Interoperability

1. Notification 1. Scheduling
2. Correctness 1. Notification

1. Query 3. Computer software
2. Performance 3. Language

1. Others
3. Course required
process& tools
1. Update 3. Budget

1. Automation
3. User interface
standards
3. Transition
3. Course required
process& tools

3.Transition 3. Language
*Maintainer *Client

2. Security 2. Performance

1. Scheduling
3. User interface
standards
3. Transition 1.Others

2.Security 1.Others
2. Availability 2. Usability

2.Correctnes
3.Support
environment
1. Query 2. Interoperability

3.Budget
3.Computer
communication
2. Correctness 1.Notification
1.Administration
support
3.Computer software
3.Computer software
3.Course required
process& tools

3.Schedule 3.Deployment
3.Deployment 1.Usage analysis

1. Usage analysis
3.Computer
hardware
3.Language
3.Computer
communication

2.Availability
3. Course required
process& tools
3.Schedule 3.Budget

1. Storage
3. Other software
interface
3.Computer
hardware
3.Other software
interface

2.Usability 3. Language
Table 14: Dependability Rankings in Multi-media Projects
86

5.3.2 Stakeholder/ Value Dependency in Fall 2009 Experiment
The 8 projects in fall 2009 represents community service domain. Table 15 shows that the
average requirements ratings from role-play user, real client, and role-play client are
around 7, which equivalent to “Should Have”. This is similar to the phenomenon in Table
12.
However, the role-play maintainer’s average rating is around 6, relatively lower to the
other stakeholders. Upon further exam ratings in each category, we find that there is a
clear difference on prioritizing functional tasks between role-play maintainer and all the
other stakeholders: an average rating of 6 from role-play maintainer vs. nearly 7.5 from all
the other stakeholders. The relative lower rating from role-play maintainer may due to the
perspective that maintainer’s responsibility is to provide cost-effective support after
development, rather than directly perform on system functionalities.
Observing standard deviation in Table 15 indicates that the understanding of requirements
among role-play users and role-play maintainers in the 8 community services projects are
more spread out than the understanding of requirements among real-clients and role-play
clients. One possible explanation is that in general, student teams have a lot interaction
with their client representatives, while most of the teams do not have real user or real
maintainer representatives.
Table 16 shows the correlation among different stakeholders’ priorities in 8 community
services projects. Similar to the multi-media group, the real client and role-play client
have the highest correlation among requirements priorities (0.57). This indicates some
synergy on mutual learning between student teams and real clients. Again, correlations
87

among other stakeholders are fairly low, which implies that these stakeholders do not
depend on value attributes in the same way.
Table 17 lists the most important and least important dependability attributes in the
community services projects. Similar to Table 13, the top 10 attributes have the highest
ratings and the last 10 attributes have the lowest ratings from each type of stakeholder.
The number 1,2,3 before each attribute stands for that whether the attribute belongs to
Functional Tasks, Level of Services or Project Activities.
*User *Maintainer Client * Client
Average Ratings 6.8 6.3 7.5 7.4
Standard Deviation 1.84 1.89 0.79 1.17
Table 15: Mean Rating in Community Service Projects
*User *Maintainer Client *Client
*User 1.00
*Maintainer 0.26 1.00
Client 0.17 0.20 1.00
*Client 0.24 -0.06 0.57 1.00
Table 16: Correlation in Community Services Projects

*: role- play stakeholders
1: This attribute belongs to Functional Tasks
2: This attribute belongs to Level of Services
3: This attribute belongs to Project Activities
: The ratings decreases downwards, for the Top 10 part, it means the first one is the most important; for the
Last 10 part, it means the last one is the least important
88

Top 10  Last 10  Top 10  Last 10 
*User Client

1. Update
3. Other software
interface

2. Performance 1. Display
2. Correctness 1. Archive

3. Transition 1. Scheduling
2. Transition
3. Computer
software

2. Concurrency 3. Language
1. Social networking 2. Evolvability

2. Interoperability
3. Course required
process & tools
1. Display 3. Budget

1. Advertising
3. Computer
hardware
2. Performance 1. Status message

1. Query 3. Schedule
1. Query 3. Schedule

2. Security 2. Evolvability
2. Availability
3. Other
development tools

1. Usage analysis
3. Other
development tools
2. Security 3. Language
1. Validation/
Administration
support
1. Social networking
1. Feedback/
Tracking/
Automation
3. Course required
process & tools

2. Correctness/
3. Budget
1. Storage
*Maintainer *Client

3. Support
environment
1. Query

2. Availability
1. Scheduling/
advertising/
notification
3. Transition 1. Validation

1. Validation
3. Course required
process & tools
2. Evolvability 3. Schedule

3. Deployment 1. Automation
3. Other development
tools
1. Financial support

3. Budget 2. Security
1. Archive 1. Advertising

1. Usage analysis 3. Language
2.Concurrency 1. Scheduling

2. Correctness 3. Schedule
2.Availability 3. Budget

2. Interoperability 1. Storage
1. Administration
support
1. Storage

1. Financial support
3. Computer
hardware
2. Performance 1. Notification

2. Performance 1. Display
2. Interoperability
3. Course required
process & tools

1. Query
3. Other
development tools
Table 17: Dependability rankings in community services projects

89

5.3.3 Role-play Representativeness Checking
Due to the unavailability of real users and maintainers, we train students to role-play
these stakeholders. This section discusses the representativeness of using role-players in
the study.
Project P1 P2 P3 P4 P5 P6
Correlation 0.48 0.92 0.78 0.64 0.36 0.40
Correlation
w/o top 2
differences 0.60 0.92 0.94 0.84 0.63 0.45
Table 18: Role-play Representativeness in Fall 2008 Experiment
Project P7 P8 P9 P10 P11 P12 P13 P14
Correlation 0.34 0.73 0.78 0.02 0.62 0.41 0.76 0.25
Correlation
w/o top 2
differences 0.50 0.87 0.85 0.36 0.87 0.62 0.89 0.69
Table 19: Role-play Representativeness in Fall 2009 Experiment
Table 13 shows that within 6 multi-media projects, the correlation between real clients and
role-play clients is the largest (0.53) among all stakeholders. It also can be seen that for the
most part, the role-players priority ratings could have been used as representative client
ratings, but with some significant exceptions. However, correlation coefficient 0.53 only
shows real clients and role-play clients’ rating are intermediately correlated overall. To
further check the representativeness of role players, we track back to each project to get
the respective correlations between real clients and role-play clients in the 6 multi-media
projects. The correlation values in Table 18 show that three of the six projects have
correlations above 0.6 for all ratings, and net five of the six have ratings above 0.6 when
the top two differences from each project are excluded. Especially for project 2, 3, 4, all
three are above 0.8, which means highly correlated.
90

Table 19 shows correlation between real clients and role-play clients within 8 community
services projects. We can see that 4 of the 8 projects have correlations above 0.6, and net 6
of the 8 have ratings above 0.6 when the top two differences from each project are
excluded. Especially for project 8, 9, 11, 13, all four have high correlations above 0.85. This
indicates that students on these teams did a good job in mutual learning with their clients.
An exception is project 10 with the lowest correlation between real client and role-play
client priorities (0.02). The reason we found for this low correlation in P10 is that intensive
defects or issues introduced in the WinWin negotiating process. Based this correlation
analysis, we verified their problems and gave them suggestion for further improvement for
negotiation.
One explanation of the strong correlations is that the clients and developers have
participated in several mutual-learning activities before and during the win-win
negotiations. These include goal-setting exercises, visits to clients’ work sites, and
development of operational scenarios, use cases, and benefits chains linking initiatives to
desired outcomes. Without these, the correlations would likely have been considerably
lower. Overall, the role-players’ ratings tend to be somewhat lower, with some exceptions.
Across all 6 multi-media projects and all attributes, the average rating for clients is 7.22
and 6.74 for client role-player as shown in Table 14. Among 8 community services projects,
the difference between average real client rating and role-play client rating is fairly close
(7.5 vs. 7.4) in Table 17.
Some further observations of multi-media project role-players’ ratings in Table 14 are:
clients’ major concerns such as system secure information, correct results, and availability
91

were well identified, but there were some understanding differences on transition
activities and support environment.
Ratings in Table 17 reflect that role-play clients in the 8 community services projects have
the same priorities at the high level with their real clients: functional tasks have the
highest priority, followed by desired levels of services, development activities are relatively
low priority. Role-players also well identified several clients’ high priorities in terms of
correctness, performance and interoperability with existing system. Some inconsistencies
in detail exist, such as, role-play client over concerned with system availability but
overlooked the needed information security. In both cases, the most obvious difference
from real client is that role-play client rates budgets higher, and this might be explained
by the reason that students are taught to think from client perspectives that they usually
consider about the budget but neglecting the real situation for these projects, which is
that clients don’t pay students for working on the projects.
92

5.4 Discussion
5.4.1 WikiWinWin Success usage factors
Among the various results presented in Section 5.1, we discuss the most significant usage
factors that tend to influence the project outcomes and their implications.
Use it early - In our data, early use (during the 12 day negotiation period) had a higher
correlation to project grade. This implies that the requirements negotiation process and
support tool need to facilitate early stakeholder involvement in the negotiation.
Use it more - Higher tool usage tended to correlate with better grades. This implies
better outcomes can be achieved when all the project stakeholders are involved and
contribute to the negotiation.
Use it frequently - Frequent use - Frequent tool usage by project teams tended to
correlate with better grades. This indicates using requirements negotiation tool is not a
one-shot activity. When stakeholders are involved throughout the negotiation, better
outcomes tend to be achieved.
Shaper use - Higher and frequent shaper usage tended to correlate with better grades.
This implies that the shaper is a success factor. It is important to make sure teams have
capable and responsible shapers.
Evolve artifacts - Duration of creating artifacts tended to correlate with project grade.
This suggests the need of continue evolving negotiation artifacts throughout the
negotiation period.

93

5.4.2 Changes in WikiWinWin
Each year, we made changes to the WikiWinWin tool based on the experiment experience
and subjects’ feedback from previous year. In addition, we also made some adjustments to
the front-end learning activities and instructions. These changes are summarized in Table
23, and are explained in below. These changes showed improvements in the results
described in Section 5.1.
Pre-negotiation team building - Finding the right set of people for the project is critical
for success. Usually on-campus students had chances to mingle with others. Off-campus
students, on the other hand, didn’t have much interaction with the rest of the class. Based
on feedback from fall 2007, we organized a classroom social as well as used the course
discussion board to facilitate team building in fall 2008.
WikiWinWin tool – In fall 2007, shaper had to do a lot of manual work in WikiWinWin,
such as creating voting pages for stakeholders and assigning identifiers when new artifacts
were added. In fall 2008, these tasks were automated and substantially reduced shaper’s
preparation effort. We also resolved bugs that caused some usage frustration in previous
year. In fall 2009, we further improved on efficiency and usability of using WikiWinWin
tool by displaying color coded artifact status, adding overview pages for each type of
negotiation artifacts, and adding shortcut navigation links.
Site visits - Another lesson we learned in fall 2007 is that students need to have a better
understanding of the operational concept of their clients’ organization, for which the new
system is designed. In fall 2008, we added a site visit assignment as another front-end
mutual learning activity. Students are asked to visit their client’s organization and
94

produce a report of learning in terms of organizational objectives, shortfalls in the current
operation, and major needs for improvement.
Training - In fall 2008 and fall 2009, we revised tutorials by adding negotiation process
diagrams and step by step examples. We had a separate guideline for shaper in fall 2009
that focused on shaper tasks in WikiWinWin. We also added hands-on practice to the
homework assignment.
Shaper assignment - We found in fall 2007 that having off-campus shaper-facilitated
meetings was not fully effective. In some teams, two shapers overwrote each other’s work
in the wiki. In fall 2008, we let the off-campus shaper to be the lead shaper on shaping
wiki content. The on-campus shaper was primarily the facilitator of negotiation meetings,
but also helped the lead shaper in shaping overlaps and issues among stakeholder inputs.
Category Fall 2008 Fall 2009
Tool  Automatically generate voting
template from agreements.
 Automatically generate artifact
identifier.
 Simplify taxonomy categories.
 Added site map.
 Support tagging multiple
contributors.
 Added overview pages for
win conditions and issues.
 Added navigation links to
recently changed pages.
 Use color code to display
status of negotiation
artifacts.
Process  Deferred prioritization to point of
agreement.
 Shaper assignment
No significant changes from
Fall 2008
Training  Tutorial
 Guideline
 Tutorial, shaper training
 Guideline
 Assignment
Activities  Pre-negotiation team building
 Site visits
No significant changes from
Fall 2008
Table 20: Summary of WikiWinWin Changes

95

5.4.3 Changes in Experiment Settings
In addition to the intended changes to WikiWinWin instruments, there were some
significant changes in the software engineering course, summarized in Table 24. These
changes were not controlled by our experiment, and confounded project outcomes to
some extent. As a result, we cannot fully compare class-wise project performance between
each consecutive year.
Process model - In fall 2007, project teams followed LeanMBASE Model. In fall 2008, we
introduced several process and product improvements via our new Incremental
Commitment Model (ICM). Fall 2009 continued with using ICM.
Process guideline - In fall 2008, we also experimented with using electronic process
guidelines developed using the IBM Rational Method Composer toolset.
Project applications – Compared to fall 2007, there were a higher percentage of projects
in fall 2008 and fall 2009 that were web service/ COTS intensive.
Category Fall 2007 Fall 2008 Fall 2009
Process
model
LeanMBASE Incremental Commitment
Model
Incremental Commitment
Model
Process
guideline
Paper Electronic Electronic
Project
Applications
Traditional Several web service/COTS
projects
Several web service/COTS
projects
Table 21: Changes in Experiment Settings
5.4.4 Quantitative Extension of Top-level Dependability Table
One of the main objectives of the study is to obtain more quantitative information than
Table 4 in section 2 about the relative priority of different classes of system stakeholders
with respect to various system dependability attributes. The study has some limitations in
96

that the clients prioritizing the dependability attributes show some aspects of being
information consumers, administrators, and acquirers. Then project applications in the
study are not mission-critical in terms of safety, but we would encounter a challenge with
respect to their Maslow need hierarchy.
This is that lower-level needs in the Maslow hierarchy such as safety and security of one’s
job or enterprise are high-priority until it meets a given level of satisfaction, but is not
high priority thereafter. Thus, we see that the clients had high priorities for availability
and backup/ recovery up to a basic level, but did not need more extreme levels such as
needed in such mission critical domains as passenger aircraft and nuclear power plants.
From a process standpoint, this implies that the use of Table 4 to baseline the attributes of
interest to stakeholder classes needs to reflect whether their value dependencies are
hygiene factors vs. mission-critical factors.
We can see from Table 13 and Table 16 that attributes such as correctness, availability/
backup-recovery, transition and support environment are generally high priorities from an
acquirer standpoint. As administrators, functional capabilities such as administration
support, notification, and information storage/ query / update are generally high
priorities. And those as representatives, of less-than-critical information consumers,
usability, performance, and interoperability are reasonably important, but not as high as
most of the others. However, the multi-dimensional nature of the clients makes it more
difficult to allocate priorities to stakeholder classes. As the other client classes were similar
in their stakeholder profiles, it is not surprising that their priority profiles were generally
similar. From a process standpoint, these conclusions imply that stakeholder negotiation
process agendas need to consider multi-role stakeholders.
97

Most of the community services projects share common functions such as administration
of community services, tracking of volunteers, donors, and grants, payment processing,
event tracking and management, and community communications involving third-party
advertising, e-newsletters, and social networking capabilities. Community services
organizations also need some information security to the extent of preventing disclosure
and modification of applicants’ personal information, such as social security number.
These indicate that requirements negotiation needs to address domain specific priorities
in addition to negotiating general dependencies.
5.4.5 Representativeness of Role-Playing Stakeholders
Even though the role-playing developers were initially inexperienced in their clients’
domains, they had gone through a sufficiently extensive set of mutual learning process
and experiences (site visits, goal-setting, prototypes, etc.), which make their priorities
comparable with those of the clients (correlations generally above 0.6 with some
exceptions, and as high as 0.92). Without the mutual learning process and experiences (as
with role-playing as users or maintainers), there was generally a positive correlation, but
below 0.6. However, there were occasional wide differences, indicating the process
implication for follow up on questionable role-player ratings.

98

5.5 Threats to Validity
Non-uniformity of projects
There were sources of variability among projects. Each project was different from one
another in terms of application, technical characteristics, client needs, and
communication skills among team members. For example, one team was not very good at
prototyping new features. In this case, even though the team was able to quickly reach an
agreement with the client’s win condition, their project package grade was not very good
because of the prototyping result. Also as mentioned in an earlier section, we had more
projects involve using COTS and web services in fall 2008 and fall 2009, which involved
different project guidelines and confounded results. In some cases, even though the initial
negotiation led to the right ball park with some candidate service products, if the
developers didn’t know how to evaluate a candidate vs. alternatives, it often limited them
in finding the most suitable solution. Such phenomena partially explained the lower
significance and some outliers had to be excluded from the results due to COTS
evaluation timing difficulties, leading to incomplete projects.
Non-equality of team experiences
Because students self-formed their teams, we didn’t control for equal team experience
levels. This led to possible influence on the project outcomes in that more experienced
teams may possess the skills to achieve higher grades, and they may also be more
conscious of adopting successful practices introduced in the course.

99

Other changes in the course
As mentioned in Section 5.4, changes in development process and guidelines also added
confounding effects to year-to-year project outcomes.
Attributes classification
Generalized classification of dependability attributes may bring some risks to the
stakeholders ratings of dependability strength. For example, end users may only be
concerned with the training provided to their work responsibilities, and training provided
to maintainers is less emphasized. When these two training requirements were classified
by the same attribute - training, the averaged priority rating tended to underestimate a
particular stakeholder’s priority on his concerned requirement.
The major threat to external validity is the generalization of our results
Our projects, to some extent, are representative of small development teams in industry
given that these projects stemmed from concrete real world problems, involved real client
representatives coming from variety of sources, developed by teams mixed with graduate-
level students (bachelor degree, some problems with English communication) and full-
time working professionals who were generally familiar with industry practices.
Considering that these teams also involved co-located and remote team members (some
located in mid west and east coast), our projects are also somewhat representative of
distributed development teams over different times and locations.
100

Chapter 6: Conclusions and Future Work
Poor requirements often lead to expensive project failures. Having multi-disciplinary
stakeholders collaboratively negotiate mutually satisfactory requirements contributes to
success, but remains a major challenge. Wiki is becoming more and more effective in
supporting collaborative activities, e.g. the case of Wikipedia, which suggests that wiki can
also be used to facilitate the collaborations during requirements negotiation.
The results to date from 46 software engineering projects over a three-year period at USC
who experimented with negotiating requirements using the WikiWinWin tool concluded
the following:
Better project outcomes were correlated with several critical factors of use, including use
early, use more, use often, and evolve of negotiation artifacts.
We adopted the shaper role from successful wiki uses in industry. The significant
correlation between shaper usage and project outcome indicated that having capable and
responsible shaper is another success factor for collaborative negotiation.
A number of complementary practices, such as front-end team-building, site-visits,
mutual learning, concurrent prototyping and COTS/services evaluation, helped project
teams achieve better outcomes.
User feedback indicated high consensus on the usefulness of WikiWinWin, but need to
further improve on it ease of use, particularly ease of use by clients, as many clients are
generally not familiar with editing in Wiki. Next generation of social networking
technologies, e.g. Facebook, Twitter, etc. provide new opportunities to experiment their
101

adoption in requirements negotiation, to continue improve on more user familiar
interface, as demonstrated by WinBook [Kukreja, 2012].
Our study provides an example of using wiki-based groupware in the domain of software
requirements negotiation. User experience revealed some useful factors to consider when
developing similar tool support for negotiation. These include:
 Provide effective and robust support in gathering, understanding, and
communicating needs
 Design flexible tool to accommodate individual use
 Facilitate knowledge sharing by allowing easy content update and preserving
history
 Reduce manual effort to improve efficiency

102

Bibliography
[Alexander and Maiden, 2004] I. Alexander and N.A.M. Maiden, Scenarios, Stories, Use
Cases: Through the Systems Development Life-Cycle. Chichester: Wiley, 2004.
[Arthi, 2009] B. Arthi, “Distributed Requirements Negotiations Using Mixed Media”. Int’l
Journal of Eng. and Technol., 2009.
[Boehm and Ross, 1989] B. Boehm and R. Ross, "Theory-W Software Project Management
Principles and Examples", IEEE Transactions on Software Engineering, Vol. 15, No. 7, July
1989.
[Boehm et al. 2000] B. Boehm, D. Port, and M. Al-Said, "Avoiding the software model-
clash spiderweb," Computer, vol. 33, 2000.
[Boehm et al. 2004] B. Boehm, et al., “The Nature of Information System Dependability: A
Stakeholder/Value Approach”, USC-CSSE Technical Report, 2004.
[Boehm and Kitapci, 2006] B. Boehm and H. Kitapci, "The WinWin approach: using a
requirements negotiation tool for rationale capture and use." Rationale management in
software engineering. Springer Berlin Heidelberg, 2006.
[Boehm and Jain, 2006] B. Boehm and A. Jain, “A Value-Based Theory of System
Engineering,” Proceedings, INCOSE 2006.
[Boehm et al. 2014] B. Boehm, J. Lane, S. Koolmanojwong, and R. Turner. The Incremental
Commitment Spiral Model: Principles and Practices for Successful Systems and Software.
Addison-Wesley Professional, 2014.
[Cheng et al. 2009] B. Cheng, P. Sawyer, N. Bencomo and J. Whittle, J., “A goal-Based
modeling approach to develop requirements of an adaptive system with environmental
uncertainty”, in Proceedings 12th ACM/IEEE International on Model Driven Engineering
Languages and Systems (MODELS-09). New York: ACM Press, 2009.
[Chi et al. 2010] E. Chi, S. Munson, G. Fischer, S. Vieweg and C. Parr, “Advancing the
design of technology-mediated social participation systems”. IEEE Computer, November
2010.
[Couglan and Macredie, 2002] J. Couglan and R.D. Macredie, “Effective communication in
requirements elicitation: a comparison of methodologies”. Requirements Engineering,
7(2), 2002.
[Damian, 2003] D. Damian, “A research methodology in the study of requirements
negotiations in geographically distributed software teams”. Proc. Workshop on
Comparative Evaluation in RE, 2003.
103

[Darimont et al. 1997] R. Darimont, E. Delor, P. Massonet and A. Van Lamsweerde,
“GRAIL/KAOS: an environment for goal-driven requirements engineering”, in Proceedings
ICSE-97. Los Alamitos CA: IEEE Computer Society Press, 1997.
[Davis, et al. 2006] Davis, A; Dieste, O.; Hickey, A; Juristo, N.; Moreno, AM., "Effectiveness
of Requirements Elicitation Techniques: Empirical Results Derived from a Systematic
Review," Requirements Engineering, 14th IEEE International Conference, 2006.
[Duan et al. 2009] C. Duan, P. Laurent, J. Cleland-Huang and C. Kwiatkowski, “Towards
automated requirements prioritization and triage”. Requirements Engineering, 14(2),
2009.
[Fernández et al. 2010] R. Fuentes-Fernández, J.J. Gómez-Sanz and J. Pavón,
“Understanding the human context in requirements elicitation”. Requirements
Engineering, 2010.
[Fisher and Ury, 1981] R. Fisher and W. Ury, “Getting to Yes”, Houghton Mifflin, 1981.
[Greenwood et al. 2012] P. Greenwood, A. Rashid and J. Walkerdine, “UDesignIt: towards
social media for community-driven design”, in Proceedings ICSE 2012. New York: ACM
Press, 2012.
[Giorgini et al. 2011] E.S.K. Yu, P. Giorgini, N.A.M. Maiden and J. Mylopoulos, “Social
modeling for requirements engineering: an introduction”, Social Modeling for
Requirements Engineering. Cambridge, MA: MIT Press, 2011.
[Hickey and Davis, 2004] A.M. Hickey and A.M. Davis, “A unified model of requirements
elicitation”. Journal of Management Information Systems, 2004.
[Horkoff and Yu, 2010] J. Horkoff, E.S.K. Yu, “Finding Solutions in Goal Models: An
Interactive Backward Reasoning Approach”, In Proceedings 29th Int. Conference on
Conceptual Modeling (ER 2010), Vancouver, Canada, 2010.
[Kaiya and Saeki, 2006] H. Kaiya and M. Saeki, “Using domain ontology as domain
knowledge for requirements elicitation”, in Proceedings 14
th
IEEE Int. Conference on
Requirements Engineering (RE’06), Minneapolis, USA, 2006.
[Kitapci, 2007] H. Kitapci, "Formalizing Informal Stakeholder Inputs Using Gap-bridging
Methods", Ph.D. thesis, Computer Science Department, University of Southern California,
Los Angeles, CA 90089, August 2007.
[Knauss et al. 2009] E. Knauss, O. Brill, I. Kitzmann, and T. Flohr, "SmartWiki: Support for
high-quality requirements engineering in a collaborative setting," in 2009 ICSE Workshop
on wikis for software engineering, 2009.
104

[Kukreja, 2012] Kukreja, N. "Winbook: A social networking based framework for
collaborative requirements elicitation and WinWin negotiations." Software Engineering
(ICSE), 2012 34th International Conference on. IEEE, 2012.
[Lamsweerde and Letier, 2000] A. Van Lamsweerde and E. Letier, “Handling obstacles in
goal-oriented requirements engineering”. IEEE Transactions on Software Engineering,
2000.
[Lee, 1996] M. J. Lee, "Foundations of the WinWin Requirements Negotiation System,"
Ph.D. thesis, Computer Science Department, University of Southern California, Los
Angeles, CA 90089, August 1996.
[Majchrzak et al. 2006] A. Majchrzak, C. Wagner, and D. Yates, “Corporate Wiki Users:
results of a Survey”, WikiSym '06: Proceedings of the 2006 international symposium on
Wikis, 2006.
[Marczak and Damian, 2009] S. Marczak and D. Damian, “How interaction between roles
shapes the communication structure in requirements-driven collaboration”, in
Proceedings 19th IEEE Int. Conference on Requirements Engineering (RE’01), Trento, Italy,
2009.
[Majchrzak et al. 2013] A. Majchrzak, E. Fife, Q. Min, and F. Pereira, “Activating the Tools
of Social Media for Innovative Collaboration in the Enterprise”, Springer Publishing
Company, Incorporated, 2013.
[Mallardo et al. 2007] T. Mallardo, F. Calefato, F. Lanubile, D. Damian, "The Effects of
Communication Mode on Distributed Requirements Negotiations", Proc. ICGSE
Workshop on Global Requirements Engineering, 2007.
[Mckinsey, 2012]
http://www.mckinsey.com/insights/business_technology/delivering_large-
scale_it_projects_on_time_on_budget_and_on_value
[Pew and Mavor, 2007] Mavor, Anne S., and Richard W. Pew, eds. Human-System
Integration in the System Development Process:: A New Look. National Academies Press,
2007.
[Ras, 2009] E. Ras, "Investigating Wikis for software engineering - Results of two case
studies," in 2009 ICSE Workshop on Wikis for Software Engineering, 2009.
[Riechert and Berger, 2009] T. Riechert and T. Berger, "Leveraging semantic data wikis for
distributed requirements elicitation," in 2009 ICSE Workshop on Wikis for Software
Engineering, 2009.
[Sateli et al. 2012] Bahar Sateli, Srinivasan Sembakkam Rajivelu, Elian Angius, and René
Witte. 2012. ReqWiki: a semantic system for collaborative software requirements
engineering. In Proceedings of the Eighth Annual International Symposium on Wikis and
Open Collaboration (WikiSym '12). ACM, New York, NY, USA.
105

[Snow, 1959] C. P. Snow, The two cultures and the scientific revolution, 1st American ed.
New York: Cambridge University Press, 1959.
[Solis and Ali, 2011] C. Solis and N. Ali, "An experience using a spatial hypertext Wiki," in
Proceedings of the 22nd ACM conference on Hypertext and hypermedia, 2011.
[Standish, 2013]
http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&v
ed=0CCAQFjAA&url=http%3A%2F%2Fwww.versionone.com%2Fassets%2Fimg%2Ffiles%2
FCHAOSManifesto2013.pdf&ei=cuQlVLrNJYTnoASU-
oKYAw&usg=AFQjCNGvPjCOnsnzMOwVXhV-c0ZjaIWZdQ&bvm=bv.76247554,d.cGU
[Sutcliffe, et al. 2011] A.G. Sutcliffe, S. Thew and P. Jarvis, “Experience with user-centered
requirements engineering. Requirements Engineering, 2011.
[Yang et al. 2012] H. Yang; A. De Roeck. V. Gervasi, A. Willis, B. Nuseibeh, “Speculative
requirements: automatic detection of uncertainty in natural language requirements”, in
Proceedings 20th IEEE Int. Conference on Requirements Engineering (RE’12), Chicago, Il,
USA, 2012.

Abstract (if available)

Abstract A major challenge in the development of software intensive systems is to define requirements to satisfy the different needs of its multiple inter-dependent stakeholders. Without win-win requirements, software projects often suffer unsatisfactory outcomes. As evidenced in multiple empirical and industry studies, developing the wrong functionalities and developing the wrong user interface are among the top software risk items

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Using social networking technology to improve collaborative requirements elicitation, negotiation, prioritization and evolution

PDF

Value-based, dependency-aware inspection and test prioritization

PDF

Domain-based effort distribution model for software cost estimation

PDF

Incremental development productivity decline

PDF

Using organized objectives to structure arguments for collaborative negotiation of group decisions in software design

PDF

Security functional requirements analysis for developing secure software

PDF

A value-based theory of software engineering

PDF

Formalizing informal stakeholder inputs using gap-bridging methods

PDF

The incremental commitment spiral model process patterns for rapid-fielding projects

PDF

A unified framework for studying architectural decay of software systems

PDF

A reference architecture for integrated self‐adaptive software environments

PDF

A framework for intelligent assessment and resolution of commercial-off-the-shelf product incompatibilities

PDF

Quantifying the impact of requirements volatility on systems engineering effort

PDF

Quantitative and qualitative analyses of requirements elaboration for early software size estimation

PDF

Software quality analysis: a value-based approach

PDF

A search-based approach for technical debt prioritization

PDF

Improved size and effort estimation models for software maintenance

PDF

Software connectors for highly distributed and voluminous data-intensive systems

PDF

Shrinking the cone of uncertainty with continuous assessment for software team dynamics in design and development

PDF

Policy based data placement in distributed systems

Asset Metadata

Creator Wu, Di (author)

Core Title WikiWinWin: a Wiki-based collaboration framework for rapid requirements negotiations

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Computer Science (Software Engineering)

Publication Date 12/09/2014

Defense Date 10/20/2014

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag collaboration,OAI-PMH Harvest,requirements negotiations,Wiki,WikiWinWin,win-win

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Boehm, Barry W. (committee chair), Mcleod, Dennis (committee member), Settles, F. Stan (committee member)

Creator Email diwu@usc.edu,dlite99@yahoo.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-520856

Unique identifier UC11297742

Identifier etd-WuDi-3102.pdf (filename),usctheses-c3-520856 (legacy record id)

Legacy Identifier etd-WuDi-3102.pdf

Dmrecord 520856

Document Type Dissertation

Format application/pdf (imt)

Rights Wu, Di

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA