Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Using organized objectives to structure arguments for collaborative negotiation of group decisions in software design
(USC Thesis Other)
Using organized objectives to structure arguments for collaborative negotiation of group decisions in software design
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
USING ORGANIZED OBJECTIVES TO STRUCTURE ARGUMENTS FOR COLLABORATIVE NEGOTIATION OF GROUP DECISIONS IN SOFTWARE DESIGN by Nan Jing A Dissertation Presented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHEN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (COMPUTER SCIENCE) August 2009 Copyright 2009 Nan Jing ii Dedication This dissertation is dedicated to my parents, Rui Jing and Shenying Wei, and my wife, He Wang. My parents have given me all their love and let me freely explore wherever my enthusiasm goes. They always have so much confidence and pride in me which makes me feel quite guilty whenever I tend to stop proceeding further. Their understanding on the value of education is truly beyond my comprehension. Without their support, there is no way I could possibly accomplish this. My wife, He Wang, has helped me in many ways and listened to my complaints about life. She was always there for me and gave me comfort and advice whenever I needed them. This work would not have been completed without her constant encouragement, tolerance and assistance. This dissertation is also dedicated to the memory of my motherly grandfather Mr. Jiacheng Wei and fatherly grandmother Ms. Liqun Liu. Both of them emphasized the importance of education to me and helped me with many lessons through their lives. They passed away in the years of my doctoral study. I miss them very much and hope all the best for them in another world. This dissertation is a result of collective efforts from my advisor Dr. Stephen Lu, my dissertation committee, and all colleagues at USC. They often give me valuable advice and insights about life, when I had a hard time of understanding it. In addition, they also give me room to try and make mistakes. Throughout my eight years at iii USC and IMPACT Lab, I have seen many people come and go; however, I am very grateful that many of them have taken me as their friend. They have taught me how to live, feel and love, which, in turn, become the catalyst to my desire to finish this dissertation. Last but certainly not the least, probably the most important, this dissertation is also dedicated to my son, Stephen Shang-En Jing. He was born on January 9, 2009, right in the middle of the process of my dissertation writing. Till the date when this dissertation is completed, he has grown into a wonderful 6-month old baby in spite of his father spending so much time away from him working on this dissertation. I love him so much and hope all the healthiness and happiness be with him. During these three months, my parents-in-law, Mr. Shutang Wang and Ms. Shurong Wang, have stayed with me, taken care of my family and myself, and helped me in every possible way. I love them and my appreciation for their generous help is beyond what can be described here. I truly thank all of them above from the bottom of my heart. iv Acknowledgements The writing of a dissertation is an exciting and exceptional journey, yet it is obviously not possible without the personal and practical support of numerous people. My first and most earnest acknowledgement goes to Dr. Stephen C.-Y. Lu, my Ph.D. advisor and chair of my committee. This dissertation is part of the research carried out through his vision in the last eight years. I am deeply indebted to him for his encouragement, advice, mentoring, and research support throughout my doctoral studies. He taught me how to ask questions and express my ideas. He showed me different ways to approach a research problem and the need to be persistent to accomplish any goal. I truly appreciate his patience and tolerance during my various puzzles. Besides my advisor, I would like to thank the rest of my dissertation committee: Dr. Paul Rosenbloom, who has given me invaluable feedback and insightful comments, and Dr. Yan Jin, who has asked me great questions and pointed me to correct directions. Both Dr. Rosenbloom and Dr. Jin have spent a significant amount of time for helping me during the course of my dissertation writing and revision, even in their valuable sabbatical time. Using this chance I hope to sincerely thank them for their great help, kind forgiveness and consistent support, without which the quality of this dissertation cannot be what it is like today. My thanks also go to Dr. Jonathan Gratch, Dr. Alexandre Francois, and Dr. David Traum for serving on my qualification committee and inspiring and advising me to pursue further in my research work. v I am fortunate to have the opportunity to work with a group of energetic people in the IMPACT laboratory. I have enjoyed every moment that we have worked together. All former and present IMPACT Lab members have taught me many things about work and life. I appreciate all their friendships and collective encouragement to finish this dissertation. I want especially to thank Jian Cai, Ping Ge, Kai-Lu Wang, Mohammad Danesh, Li Zhao, Pawat Chuslip, Wei Li and Yoichiro Suzuki, for sharing their technical wisdom and research ideas. During my work at IMPACT, I was supported in part by the grants of Construction Engineering Research Laboratory from U.S. Army Corp of Engineers. Specifically my thanks go out to Dr. Michael Case and Dr. Francois Grobler. They provided a lot of supports to the research projects I was working on. I am also grateful to the department of computer science and the graduate school of USC for all that they have taught me and encouraging me in various ways in my graduate study. There are far too many people to mention individually who have assisted in so many ways during my work of completing this dissertation. They all have my sincere gratitude. In particular, I would like to thank Andrew Wachter, who not only proof-read many chapters of this dissertation, but also provided stylistic suggestions to help me improve my writing. Without his help this work would be much less refined. I also wish to thank my colleagues from DivX Inc. for their help in completing the case study project in this dissertation. Particularly I am grateful to Jim Reesman who dedicated his precious time for reviewing my work and giving valuable input. vi Finally, the deepest thanks go to my wonderful parents, who cultivate and educate me with their endless love. It is my father who himself is a respectful professor and intrigues my interests in science since my childhood. And it is my mother who always believes I can complete the Ph.D. study and become a great scholar even at times when I myself feel lonely and desperate working on the dissertation. My final, and most heartfelt, acknowledgment must go to my lovely wife He Wang. Her support, encouragement, and companionship have turned my study and research journey into a pleasure. She is my source of strength and without her support any of my work would never have started or be much less finished. I have a great family and it is impossible to have my research career without their love. This dissertation is dedicated to them. To all of you, thank you. vii Table of Contents Dedication............................................................................................................... ii Acknowledgements................................................................................................ iv List of Tables............................................................................................................x List of Figures........................................................................................................ xi Abstract................................................................................................................. xii Chapter One: Introduction and Overview................................................................1 1.1 Background and Motivation ...........................................................................1 1.1.1 Challenges of Modern Software Design .................................................1 1.1.2 Group Decisions in Software Design......................................................2 1.1.3 Collaborative Negotiation of Group Decisions in Software Design.......5 1.1.4 Structured Arguments to Support Collaborative Negotiation of Group .7 Decisions in Software Design 1.1.5 Summary and Motivation .......................................................................8 1.2 Key Research Issues and Our Approach.......................................................11 1.3 Thesis Organization ......................................................................................14 Chapter Two: Literature Review............................................................................21 2.1 Introduction...................................................................................................21 2.2 Software Design............................................................................................22 2.3 Related Research in Group Decision............................................................26 2.3.1 Organizational Science .........................................................................26 2.3.2 Social Psychology.................................................................................29 2.3.3 Social Choice ........................................................................................31 2.3.4 Decision Science...................................................................................33 2.3.5 Group Decision in Software Engineering .............................................37 2.4 Collaborative Negotiation in Engineering ....................................................40 2.4.1 Challenges and Limitations...................................................................40 2.4.2 Engineering as Collaborative Negotiation Paradigm............................41 2.4.3 A Socio-Technical Framework .............................................................44 2.4.4 A Socio-Technical Co-construction Process ........................................43 2.5 Summary.......................................................................................................46 viii Chapter Three: Research Foundations, Approach, and Hypothesis.......................49 3.1 Introduction...................................................................................................49 3.2 Argument Structure: Toulmin’s Structure of Arguments.............................49 3.3 Objective Hierarchy: Value-focused Thinking.............................................54 3.4 Synthesis between the Objective Hierarchy and the Generic.......................57 Argument Structure 3.5 Research Hypothesis.....................................................................................60 3.6 Summary.......................................................................................................62 Chapter Four: Research Approach: Using Structured Arguments to Support ......64 Collaborative Negotiation 4.1 Introduction...................................................................................................64 4.2 Line of Argument..........................................................................................65 4.3 Overview of the Approach............................................................................68 4.3.1 Argument Generation and Exchange ....................................................72 4.3.2 Argument Analysis ...............................................................................77 4.3.3 Argument Evaluation ............................................................................83 4.4 A Collaborative Negotiation Process............................................................93 4.4.1 Overview...............................................................................................93 4.4.2 Pre-Negotiation: Task Proposal Modeling and.....................................96 Conflict Identification 4.4.3 Negotiation: Utilizing Structured Arguments Synthesized...................98 with Objective Hierarchy 4.4.4 Post-Negotiation: Consensual Agreement and Move to Next Phase..121 4.5 Summary.....................................................................................................121 Chapter Five: Case Studies and Hypothesis Validation.......................................123 5.1 Introduction.................................................................................................123 5.2 Case Study Background – Design Process and Data Collection ................125 5.2.1 A Real-life Software Design Process..................................................127 5.2.2 Data Collection Procedure ..................................................................131 5.3 Case 1..........................................................................................................133 5.3.1 Introduction.........................................................................................133 5.3.2 The Negotiation Process .....................................................................134 5.3.3 Lessons Learned..................................................................................137 5.4 Case 2..........................................................................................................139 5.4.1 Introduction.........................................................................................139 5.4.2 The Negotiation Process .....................................................................139 5.4.3 Lessons Learned..................................................................................165 5.5 Conclusion, Challenges and Limitations ....................................................167 Chapter Six: Impacts of Research........................................................................171 ix Chapter Seven: Summary, Conclusion, and Future Work....................................174 7.1 Summary.....................................................................................................174 7.2 Conclusion ..................................................................................................177 7.3 Future Work................................................................................................179 References............................................................................................................181 x List of Tables Table 1: Verbal Terms to Express Relative Individual Measurement of.......89 One Attribute Table 2: Verbal Terms to Express the Relative Importance of......................91 An Objective Table 3: An Example Objective Hierarchy for "Define ..............................103 Quality Attributes" Table 4: Importance of Objectives...............................................................104 Table 5: Stakeholders’ Perspectives (Attribute Values) for Product..........108 Manager’s Proposal Table 6: Stakeholders’ Perspectives: Importance of Objectives..................144 Table 7: Stakeholders’ Perspectives: Attribute Values per Proposal for.....150 ‘Build Communication Protocol' Table 8: Measurement Results and Preference Order..................................163 Table 9: Case Studies Results and Analysis ................................................164 xi List of Figures Figure 1: Structured Arguments, Collaborative Negotiation, Group...............9 Decisions, and Software Design Figure 2: Research Contexts, Foundations, and Approach............................14 Figure 3: Toulmin's Argument Structure.......................................................51 Figure 4: Synthesis Between Objective Hierarchy and Argument ................58 Structure Figure 5: Line of Argument ...........................................................................65 Figure 6: Argument-based Negotiation Approach in Relations to ................69 Design Tasks Figure 7: Interaction of Argument Generation, Argument Exchange, ..........71 Argument Analysis and Argument Evaluation Figure 8: Argument Generation and Exchange .............................................74 Figure 9: A Negotiation Approach for Collaborative Software Design ........94 Figure 10: An Example Argument (by the Engineer)..................................111 Figure 11: The Software Architecture Design Process................................128 Figure 12: Objectives and Attributes ...........................................................142 Figure 13: Argument Example.....................................................................155 Figure 14: Arguments Exchange .................................................................156 Figure 15: Argument Analysis (First Round) ..............................................157 Figure 16: Argument Analysis (Second Round)..........................................159 Figure 17: Argument Analysis (Third Round).............................................160 Figure 18: Argument Analysis (Fourth Round)...........................................161 xii Abstract Nowadays, driven by industry globalization and Internet revolution, modern software design is a group decision-making process which is often carried out by a team of stakeholders across geographical, disciplinary and temporal boundaries. These stakeholders have to negotiate collaboratively to achieve consensual agreements on their design decisions based on their competing objectives and different preferences. The identification, organization and negotiation of these objectives and preferences need to be well handled by the software design teams. To effectively support collaborative negotiation of group decisions, one of the critical requirements is to structure the negotiation arguments of multiple decision makers for multiple decision tasks, in order to make sure that all stakeholders have a common ground to effectively make rational group decisions for software design process. To satisfy this requirement, our approach is to develop a new framework of organizing objectives and preferences and structuring the arguments using this organized information. This framework is built based on a synthesis between an objective hierarchy and a generic argument structure. Enabled by this synthesis framework, our approach specifies how the arguments can be generated based on stakeholders’ proposals, objectives, criteria and preferences, and then exchanged amongst multiple stakeholders. This synthesis framework also helps us build the systematic models of analyzing the arguments relationship and evaluating the arguments based on the level of the objectives xiii achievement. Two case studies are presented for applying this new approach of structuring negotiation arguments in real-life software design processes. The results of these case studies have validated our hypothesis and justified our research approach. This research is an original attempt to develop a theoretical approach to use organized stakeholders’ objectives and preferences to structure collaborative negotiation arguments. This approach synthesizes applicable knowledge from decision science and social studies and leverages their advantages in the domain of software engineering. It overcomes the restriction and challenges of the existing approaches and defines a process with detailed guidance to improve the negotiation quality of software design team and the compatibility of stakeholders’ arguments. These contributions will eventually refine the ways of undertaking negotiation processes and resolve design conflicts in engineering domains. 1 Chapter One: Introduction and Overview 1.1 Background and Motivation This section explains the background and motivation of this thesis research. It begins with an overview of the challenges of modern software design, which is identified as a collaborative group decision-making process carried out by a team of stakeholders across geographical, disciplinary and temporal boundaries. Various studies of group decision are reviewed, and their shortcomings of meeting the challenges posed by modern software design are discussed. Based on these studies, we conclude that collaborative negotiation is a key issue in supporting group decision in modern software design. To carry out effective collaborative negotiations, a systematic framework for grounding the structured arguments with identified and organized stakeholders’ objectives is needed. 1.1.1 Challenges of Modern Software Design Software engineering research is concerned with how to improve the quality and efficiency of software design decisions in order to develop, test, implement and maintain functional and purposeful information systems that can satisfy demanding and evolving user requirements. Nowadays driven by industry globalization and Internet revolution, most software users are distributed and software design is carried out by multiple stakeholders, such as developers, architects, managers, executives, customers and sometimes outsourcing companies, each with varying needs, background and expertise. All stakeholders of software design projects must work together collaboratively as a team across geographical, disciplinary and temporal boundaries. Due to the highly distributed 2 and collaborative nature of the software design tasks, the design teams cannot rely on traditional means to conduct team meetings (e.g., face-to-face, blackboard, etc.). The software design process of such a distributed and collaborative design team is very dynamic and difficult to manage, as the objectives of the team members are not uniform and these members must make decisions collaboratively in a group based on their own objectives. At the same time, each member has different preferences for design alternatives. The elicitation of common objectives and integration of the multiple preferences need to be well handled in the collaborative software design teams. Therefore, one of the critical challenges for software design research is to investigate theories and develop methods that can support these distributed and collaborative group decision- making activities by organizing the competing objectives and integrating varying preferences for a team of decision makers, who are often multi-disciplinary, asynchronous, and geographically distributed. This challenge of better supporting group decisions for modern software design sets the overall direction of the research presented in this dissertation. 1.1.2 Group Decisions in Software Design The basic theories and methods of supporting group decision have been studied extensively by many different disciplines. These group decision studies can be generally organized into four interrelated categories: namely organizational science, social choice, social psychology, and decision science. Organizational science researchers has employed a satisficing paradigm (in contrast of the “optimizing” approach commonly 3 used in classical decision science) to find acceptable solutions under various constraints in the organization. To reach a consensus, and based on bounded rationality (in contrast to the “absolute rationality” idea in classical economics) [March and Simon, 1958; Newell and Simon 1972; Simon 1969], individuals engage with others in group decision- making processes to negotiate acceptable alternatives based on some shared goals. However, the research work to date has not fully resolved the challenges of software design group decisions, such as making group decisions bounded by cross-disciplinary organizations that are geographically distributed and operating in different timeframes. Meantime, social choice researchers in economics have long investigated various mathematical models of establishing social-welfare functions that can aggregate a profile of private preferences into a single shared preference for rational group decision makings [Goodman and Markowitz 1952]. The famous Arrow’s theorem in social choice research proves that groups in general cannot derive a fully rational social welfare function that can satisfy a set of minimal group rationality conditions. Modern social choice research has proven that only under a very restrictive set of conditions for special domains, which members can explicitly compare and negotiate their interpersonal preferences using ratings of continuous alternatives, to construct a group preference rationally [Harsanyi 1955]. Although this research provides solid theoretical ground for group decision making, in practice it does not specify how exactly the members in modern teamwork can identify, structure and negotiate these interpersonal preference in making collective choice. Social psychologist have developed empirical procedures and conducted many experimental studies to guide the dynamic evolution and construction of decision 4 perspectives during social interactions and group negotiation. Although this research does provide some general principles for decision makers to socially construct their perspectives, without solid theoretical grounds, it is very difficult to extend the previous studies in social psychology to the new challenges of group decision in modern software engineering. Decision science describes and prescribes decision-making and analyzing process in case of conflicting and cooperating issues to help decision makers jointly obtain rational decisions. Meanwhile, decision scientists who are interested in group decisions have extended decision science to deal with more than one decision maker. They have developed various negotiation models and decision analysis methods (such as game theoretic approaches) to help decision makers to arrive at agreements based upon the preferences and feedback of multiple individuals. However, these models and methods have not provided full support to decision makers who have to organize and integrate their multi-disciplinary objectives and preferences from distributed locations. Although the aforementioned works have developed some theoretical approaches and systematic methods to support group decision in general, none of them has resulted in solutions that can meet the key challenge of group decisions in modern software design, i.e., the identification, organization and integration of objectives and preferences for distributed, asynchronous, and cross-disciplinary group decision making. Neither the well-founded theoretical approach developed by the social choice research or the practical guidelines provided by the social psychology studies can fulfil the needs of group decision in modern software engineering. Organizational science studies are still 5 lack of guidelines of finding acceptable solutions within cross-disciplinary organizations that are geographically distributed and engage asynchronous communications. Most of existing models and approaches in decision science focus on the decision making activities between two decision makers on one single task (for example, games) and become conceptually intractable and computationally complex when applied for a group decision making process amongst multiple distributed decision makers who must work collaboratively with competing objectives and different preferences. In order to establish an adequate framework to support group decision in modern software design, it is necessary to thoroughly investigate the different schools of thought that contributed to this issue and obtain a deeper understanding of real-life group decision-making practices. 1.1.3 Collaborative Negotiation of Group Decisions in Software Design In modern software design processes, software designer always need to negotiate with each other in order to reach consensual agreements when they have conflicting opinions and competing demands in group decision making processes. The ability to negotiate with multiple stakeholders who have different technical expertise and diverse professional backgrounds is just as important as the ability to develop computation algorithms and build data structures. As in many situations, such an ability to effectively negotiate with others is the key to make a rational group decision in light of different objectives and preferences. However, in existing software engineering methodologies, little attention was placed on this type of group decision making activities, let alone the effectiveness of collaborative negotiation. Instead, most of such methodologies are more 6 geared towards analyzing the negotiation results of the decision making, i.e. solutions of the software design process. In one sense, these methodologies have neglected the impact of collaborative negotiation on the quality of software design. In fact, collaborative negotiation has already been well-researched in group decision research for the past several decades. Although they have been examined from several perspectives including the disciplines we have discussed in the last section (i.e. organizational science, social choice and decision science), it is only recently that approaches for managing negotiation in supporting group decisions have emerged, including heuristic-based negotiation approaches [Faratin, 2000; Kowalczyk and Bui, 2001; Fatima et al., 2002]; game- theoretic analysis [Rosenschein and Zlotkin, 1994; Kraus, 2001; Sandholm, 2002]; and argumentation-based negotiation approaches [Kraus et al., 1998; Parsons et al., 1998; Sierra et al., 1998]. However, when facing the challenges of group decision making processes in modern software design, none of them have either built theoretical foundations or provides practical guideline to identify and organize the objectives and preferences from decision makers and structure the negotiation arguments with these organized objectives and preferences in order to carry out an effective collaborative negotiation process in the virtual space. A new framework for grounding these structured arguments must be establish via a thorough study of the existing work and deep knowledge of real-life decision making activities. Based on this framework, we can develop systematic models to utilize these structure arguments to support effective collaborative negotiation. 7 1.1.4 Structured Arguments to Support Collaborative Negotiation of Group Decisions in Software Design Structured arguments have been found to be widely utilized in collaborative negotiation [Hart 1990; Jennings 1998; Marttunen 1992; Smith 1977]. For example, the work of Buckingham and his colleagues argue that standardizing an argument's structure facilitates the subsequent communication of the decision-making team since important meta-information of one decision maker can be more easily perceived and analyzed by others [Buckingham et al. 1997]. Stephen E. Toulmin’s 1958 work, Uses of Argument, has proposed a generic argument structure which becomes commonplace in structuring arguments. As well, Houp, Pearsall and Teheaux’s textbook, Reporting Technical Information, introduces Toulmin logic as providing “a way of checking your own arguments for those overlooked flaws and help you arrange your argument” [Houp et al. 1998]. Toulmin’s structure of argument provides the semantic symbols and data structure that support the argumentation process. It aims to clarify the reasoning process by encouraging parties to explicitly list important ground, backing, and relationship as they construct and rationalize ideas [Buckingham et al., 1997]. It has been proved in previous studies that by using this structure, it is generally easy for stakeholders to observe others’ objectives through examination of the ground and backing that the stakeholder expresses [Janssen and Sage 1996]. As such, Toulmin’s structure of argument has become a popular mechanism for structuring arguments in many previous studies of collaboration negotiation and proved 8 to facilitate more open and objective communication [Chang et al. 1995; Sillince et al. 1999; Sierra et al. 1998; Amgoud et al. 2000; Avery et al. 2001; Kraus, 2001; Rong et al. 2002]. However, this generic structure is still more intended for checking and arranging arguments for overlooked flaws [Houp, Pearsall, and Tebeaux 1998] instead of directly providing support for collaborative negotiation of group decision in software design, such as organizing and integrating stakeholders’ objectives and preferences in a well-defined structure for cross-disciplinary, geographically distributed and asynchronous decision makers. Also this generic structure and the existing negotiation approaches that were built based on this structure often informally define the utilization of structured arguments and loosely specify how arguments relate to other arguments with little guidance as how to evaluate them for the best [Zeleznikow 2002]. Therefore, one of the critical challenges to structure arguments to support collaborative negotiation of group decision in software design is to build a new framework to structure arguments with organized objectives and preferences and to develop models that utilize these structured arguments for carrying out effective collaborative negotiation. 1.1.5 Summary and Motivation Figure 1 shows the main concepts that we have discussed in this section: software design, group decision, collaborative negotiation, and the focus of this work: structured arguments. 9 Software Design Group Decisions Collaborative Negotiation Structured Arguments Practical Practical Theoretical Theoretical Cross-disciplinary, asynchronous and geographically distributed group decision process Identify, organize and integrate objectives and preferences Structure negotiation arguments of multiple stakeholders on multiple tasks Use organized objectives and preferences to structure negotiation arguments Figure 1: Structured Arguments, Collaborative Negotiation, Group Decisions, and Software Design As shown in Figure 1, our research is motivated by the practical challenges in modern software design and the needs for a better theoretical framework for its support. We first recognize the modern software design is a group decision-making process which is often carried out by a team of stakeholders that are across geography, disciplinary and temporal boundaries. Therefore, one of the practical challenges in supporting group decision in modern software design is to identify, organize and integrate the competing objectives and varying preferences from the distributed, cross-disciplinary and asynchronous stakeholders. Existing models and approaches of supporting group decision cannot satisfy 10 these challenges posed by modern software design. We also recognize that effective collaborative negotiation is a key issue in integrating stakeholders’ objectives and making rational group decisions, but it has not been well understood and supported either theoretically or practically in software design domain. To facilitate collaborative negotiation for group decision-making in a distributed and asynchronous virtual workspace such as modern software design, one of the critical requirements is to structure the negotiation arguments from multiple decision makers for multiple decision tasks, in order to make sure that all stakeholders have a common ground for reference and can effectively make rational group decisions for software design process. Previously, a generic argument structure has been defined by Toulmin [Toulmin 1958]. It has been widely utilized to build argument-based negotiation process models in many studies of collaborative negotiation and proved to facilitate more effective communication. However, these works do not effectively support collaborative negotiation of group decision in software design, such as organizing and integrating stakeholders’ objectives and preferences. Also there are remaining important issues that are unresolved in most of these works, such as a systematic guide to analyze the relationship amongst arguments and evaluate the arguments for the best. To resolve these issues and effectively support collaborative negotiation of group decision in software design, our approach is to develop a new framework of organizing objectives and preferences in the negotiation arguments and structuring these arguments for group decision-making activities. By utilizing these structured arguments, systematic models are built to generate, exchange, analyze and evaluate the negotiation arguments. These models can facilitate stakeholder to effectively 11 carry out a collaborative negotiation process in software design. As shown by the arrows between the different layers in Figure 1, this framework provides supports to collaborative negotiation that satisfies the need of group decision in resolving the cross- disciplinary, distributed and asynchronous challenges imposed by modern software design. To build such a new framework to structure arguments with organized objectives and preferences and to develop models that utilize these structured arguments for carrying out effective collaborative negotiation makes the motivation of this research work. 1.2 Key Research Issues and Our Approach As stated above, the key research issue of this thesis research is to develop a new framework that can organize stakeholders’ objectives and preferences during the collaborative negotiation process, and structure the negotiation arguments with these objectives and preference. In order to address this issue, we must first fully understand the two research foundations for this thesis work. Owing to the need for structuring the negotiation arguments with these stakeholders’ objectives and preferences, we introduce the first research foundation, which is the generic argument structure defined by Toulmin. This structure helps users place their negotiation information and meta-information into various argument components. The information and meta-information may include major claims, support data and additional persuasive perspectives such as justification, degree of desire and rebuttal condition. The process of preparing the argument components helps stakeholders objectively organize their understandings and opinions and the logic flow 12 between these components inherently specify the order of exchanging perspectives in the collaborative negotiation between the multiple stakeholders. However, although Toulmin defined the generic and well-adopted argument structure, this structure and the following practices (i.e., argument-based negotiation approaches) can only informally define the view of argument generation and loosely specify how arguments relate to other arguments with little guidance as how to evaluate the best proposals [Zeleznikow 2002]. It is believed that this generic structure is more intended as a way of checking and arranging arguments for overlooked flaws [Houp, Pearsall, and Tebeaux 1998] instead of providing direct supports such as sharing the understandings and providing operational guidance needed for collaborative negotiation. In order to overcome this shortage and better represent stakeholders’ understanding during negotiation, we introduce the second research foundation, an objective hierarchy, based on Value-focused Thinking (VFT) methodology. VFT helps stakeholders systematically identify their true values (i.e., what they really care about) and use them to derive the formation and organization of various fundamental and means objectives in decision-making problems [Keeney 1992]. The hierarchy will organize these objectives and their corresponding measurement criteria. It provides a common ground for multiple stakeholders to share their objectives clearly and declare their different preferences systematically towards multiple design task proposals regarding how well the respective objectives are achieved. Stakeholders can propose this hierarchy at the beginning of the design process and should always use and refine it during collaborative negotiation for various design tasks. 13 Based on the above two research foundations, this thesis work develops a new framework that is based on a strategic synthesis between the argument structure and the objective hierarchy, which clearly lays out the correspondence between the components in the argument structure and stakeholders’ objectives and preferences. Based on this synthesis framework, our approach specifies how the arguments can be generated based on stakeholders’ proposals, objectives, attributes (i.e., measurement criteria of objectives) and expected preferences. In other words, it explains how the required information, which is organized and shared based on the objective hierarchy, can be structured into the negotiation arguments. Enabled by this synthesis framework, our approach also specifies a practical way of analyzing the relationship of arguments and evaluating their performances in achieving the targeted objectives. Clustering analysis is applied to indicate the similarity and distance amongst these structured arguments and the analysis results are provided to the stakeholder to better understand and compare the arguments relationship and refine the arguments. Argument can also be evaluated to select the most preferred argument based on how well the arguments achieve the objectives. Figure 2, below, is a graphical illustration of our proposed approach described above. Based on the argument structure, the objective hierarchy and the synthesis between the two, we can develop a theoretical framework that organizes stakeholders’ objectives and preferences during the collaborative negotiation process and structures stakeholders’ negotiation arguments with these objectives and preferences. Based on these structured arguments, this work contributes to achieving effective collaborative negotiation by 14 building four interrelated research modules that utilize negotiation arguments to practically support group decisions in software design: argument generation from stakeholders’ objectives and preferences; argument exchange that helps stakeholders share their arguments based on the logic flow of argument components, argument analysis to understand the relationship between the arguments, and argument evaluation to aggregate individual preference to a collective choice. This dissertation explains the details of our approach of structuring the arguments and the four modules that utilize the structured arguments. Objective Hierarchy (Built using Value-focused Thinking Decision Framework) Generic Argument Structure (i.e. Toulmin’s Structure of Argument) Argument Generation Argument Exchange Argument Analysis Argument Evaluation Design Process (e.g., Common Software Design Procedures in Industries) Group Decision (e.g., Organizational Science; Social Choice; Decision Science; Sociology) Synthesis Synthesis Context Context Practical Practical Theoretical Theoretical Collaborative Negotiation Structured Arguments Figure 2: Research Contexts, Foundations, and Approach 1.3 Thesis Organization This thesis consists of seven chapters including the introduction and background presented here (Chapter 1). The rests are briefly summarized in the followings: 15 Chapter 2: Literature Review – This chapter begins with a brief review of the software development life cycle and identifies the focus of this research work in the phase of requirement analysis and software design. The challenge of providing a well-defined approach to support the group decision in software design is clearly presented. Then a variety of disciplines that have contributed to resolving this challenge are reviewed. Organizational science researchers propose a satisficing model to find an acceptable, good enough solution, reflecting "bounded rationality", in contrast to the concepts of “absolute rationality” and “optimizing”, and thus enable individual decision maker to engage in collaborative activities within organizations. Social psychologist has revealed that teamwork is a mutual, reciprocal, and dynamical social construction process, whereby each “constructs” others’ perspectives (e.g., objectives, preferences). Social choice researchers in economics have long investigated various social-welfare functions to aggregate private preferences into a single shared group preference for making collective choices. Decision scientists interested in group decisions have investigated various negotiation models and decision analysis functionalities that help to achieve group consensus among multiple interests and competing positions of stakeholders. Software engineering researchers develop group evaluation approaches and design negotiation models to help software designer team to obtain a consensus in case of conflicting design solutions. Most of the works we have reviewed reveal that collaborative negotiation is a widely accepted approach to support the group decision in software design. In an effort to developing such a collaborative negotiation approach, we 16 also introduce our previous work in this chapter: a Socio-Technical Co-construction Process (STCP) which is rooted in Engineering as Collaborative Negotiation paradigm and a Socio-Technical Framework and uses perspective models of stakeholders to manage a co-construction process taken by the engineering team. This chapter reviews all the above studies and discusses the contribution and limitation of each group. Based on the summary of all the reviewed work, in this chapter we also described the requirements of a new approach to support collaborative negotiation, which will be presented with details in the next chapter. Chapter 3: Research Foundations, Approach, and Hypothesis – This chapter introduces two theoretical foundations of this research work and discuss our research approach and the research hypothesis. The first foundation is the Toulmin’s structure of argument, which helps stakeholders more clearly and completely lists their perspectives, especially the ground and backing (of their arguments) which could be easily hidden or ignored otherwise. The second one is an objective hierarchy that is built based on value-focused thinking methodology that supports stakeholders to organize their objectives and criteria based on their understanding and expertise. After introduce the research foundations, we discuss our approach that synthesizes the generic argument structure and the objective hierarchy. This synthesis explains how multiple stakeholders can structure the negotiation arguments according to their objectives and preferences. It lays down the ground for our work to overcome the challenges in existing practices of argument-based negotiation approach by providing a feasible way of analyzing the arguments relationship and 17 evaluating arguments for the most preferred by the group. It also helps us advance the state of art in value-focused decision support framework by structuring stakeholders’ values into arguments for collaborative negotiation in which the design team co- constructs the values based on the structured arguments in an order defined by the logic flow betweens the argument components. Last in this chapter but certainly not the least, we discuss the research hypothesis that states our belief in that effective collaborative negotiation requires us to help stakeholders organize their objectives and preferences in the negotiation arguments and structure these arguments. Chapter 4: Using Structured Arguments to Support Collaborative Negotiation – The main result of our research is presented in this chapter. It starts with clarifying the line of argument, including background, goal, hypothesis, foundations, task and approach, based on findings and discussions in the previous chapters. Following the line of argument, this chapter describes how we utilize structured arguments to build the modules of argument generation, argument exchange, argument analysis, and argument evaluation. After then, we present the details about how a collaborative negotiation process can be developed based on the interaction of the aforementioned four modules. Finally, a summary of this chapter is given to emphasize the essence of this research work and outline the necessity of a case study as the empirical validation. Chapter 5: Case Studies and Hypothesis Validation – In order to demonstrate the application of our approach and validate its feasibility and effectiveness, this work has 18 carried out a series of case studies in real-life software projects. This chapter introduces two chosen case studies with adequate research data and different in nature. The first case study took place before our collaborative negotiation approach was fully developed. The stakeholders were only introduced to generic argument structure to organize and convey their thoughts. The second case study was a test of the approach that was fully adopted in a real-life software design process. This chapter discusses the backgrounds and details of these two case studies. In addition, the discussion of their results demonstrates the effectiveness of the support from our approach to group decision in real-life software design process. Chapter 6: Impacts of Research – In this chapter, we point out the impacts of this research to the domain of software design. This research is an original attempt to develop a theoretical approach to organize stakeholders’ objectives and preferences during the negotiation and structure them into negotiation arguments, in order to support collaborative negotiation of multiple stakeholders in group decision of software design. This approach synthesizes applicable knowledge from decision science and social studies and leverages their advantages in the domain of software engineering. Theoretically it overcomes the restriction and challenges of the existing approaches that focus on either decision optimization or negotiation analysis approach for the scenarios between two decisions makers on one task and are usually computational complex when applied for multiple decision makers on multiple tasks, while the method of structuring negotiation arguments proposed in this work can be easily scaled to a group decision making process 19 by multiple stakeholders. As such, this approach has built an objective hierarchy where stakeholders can jointly organize their objectives and preference in collaborative negotiation. It improves the generic argument structure by synthesizing it with the objective hierarchy in order to provide feasible and systematic methods to generate, exchange, analyze and evaluate arguments. It also advances the state of art in value- focused decision support framework by structuring the values of the entire design team by utilizing collaborative negotiation arguments. Practically this work has defined a process with detailed guidance to support stakeholders to generate exchange, analyze and evaluate their arguments for them to negotiate their decisions. With that support, software design team can still comply with their existing design process and within that process our approach will help them structure the stakeholders’ conflicting opinions and guide them through a well-managed negotiation process so that they can more focus on the negotiation content. Therefore, the interruption brought by our approach to the existing software design process is minimal while the negotiation quality of software design team and the compatibility of stakeholders’ arguments will be improved. Chapter 7: Conclusion and Future Work – This chapter concludes the research work presented in this dissertation and outlines future work. Overall, this work helps stakeholders structure their opinions, understandings and preferences into negotiation arguments and provide detailed guidance for how to generate, exchange, analyze and evaluate these structured arguments in collaborative negotiation to support the group decision in software design. This dissertation reviews the literatures from various schools 20 of studies which have contributed to supporting group decision and also discusses two research foundations of our work. Two case studies from real-life software design processes are presented to validate the feasibility and effectiveness of this approach. In future work we hope to provide more objective hierarchy templates having sample concepts and attributes, develop more comprehensive model to analyze stakeholders’ arguments, and extend the case study to more software design projects in various fields and with different scales in order to gather more research data and analysis results. In addition, we wish to transfer the lessons learned to other fields of engineering designs to broaden the research impacts and continuously improve this framework to eventually leading to the establishment of a scientific foundation for collaborative software engineering. 21 Chapter Two: Literature Review 2.1 Introduction This chapter begins with a brief review of the software design, which is identified as a group decision-making process often carried out by a team across geographical, disciplinary and temporal boundaries. The challenge of effectively carrying out collaborative negotiation in group decision for software design has been clearly presented in previous chapters. In order to resolve this challenge, we must establish a new framework for grounding the structured arguments with identified and organized objectives. There are many disciplines that have studied group decision. This chapter reviews these studies and discusses the contribution and limitation of each group in relation to collaborative negotiation and structured arguments. In addition, in this chapter we will discuss our previous efforts for developing a collaborative negotiation approach, in which we have designed a Socio-Technical Co-construction Process that is rooted in an Engineering Collaboration via Negotiation paradigm and a Socio-Technical Framework and uses perspective models of stakeholders to manage a co-construction process taken by the engineering team. This work lays down the key concepts for structuring the arguments in collaborative negotiation and the process for utilizing the structured argument in a group decision-making process. Based on the summary of all the reviewed research work, in this chapter we also described the requirements of our new approach, which will be presented with details in the next chapter. 22 2.2 Software Design Modern software engineering inevitably involves teams of stakeholders (e.g. engineers, engineering managers, product managers, and directors) collaborating on design of software artifacts and negotiating design requirements in order to reach common decisions. It is an intricate process that requires a lot of planning, designing, implementation and testing. One cannot develop perfect software with full functionality unless it goes through a full software development lifecycle and achieves every milestone in each phase by passing all the tests and quality analysis. The software development life cycle (SDLC) is a pre-set framework that is used by software industry for thorough understanding and developing effective information systems and software. Though software development lifecycle can have many versions and each version carries its own plus points and minus points, but all versions have same steps or activities, including feasibility check, in which the project is checked for feasibility, i.e. if it should get permission to proceed; requirement elicitation and specification, when the entire team will jointly decide a set of requirements for the system; software design and requirement analysis, where all stakeholders jointly considers the overall structure of the software and defines the strategy for its development; code generation and implementation that the code translation of the well-thought design from previous phase; testing and quality assurance, which decides the effectiveness and functionality of the product; and maintenance and support that provides answers to customers’ inquires and supply required changes with time and according to changing business needs. 23 Each stage of SDLC carries its own importance and plays a key role in success of any software development project. Our focus is in the software design and requirement analysis phase, which is one phase that is maximally involved with interaction and collaboration between engineers, managers, directors, and many others, however, with very little support of theories and/or approaches to facilitate collaboration and negotiation. Business analysts will perform a thorough study and analysis on specified requirements together with the business needs of organization and market trends. Architects and engineers investigate every technical challenge and have to make sure all the components and their workflow can be implemented on the selected platform. Managers and directors review the results from both business and technical sides and decide product development roadmap based on the organization budget. In summary, this phase considers the overall structure of the software and defines the strategy for its development. That is why software design and requirement analysis phase is somehow believed to be the most critical part of software development life cycle. Any flaw in this phase may prove very expensive for further stages of life cycle. On the other hand, most of the stakeholders of the software development life cycle show up in this phase in which they have to jointly analyze the design requirements and negotiate on technical architecture and development strategy. In real life software design, team negotiation is often an inevitable part of requirement analysis, architecture design and component analysis. During this process, the 24 stakeholders may disagree over what the critical business needs for organization and market trends are, how to map the functional requirements into system architecture, what components of the system are, and how they interact with each other. Therefore, Effective and efficient collaborative negotiation is undoubtedly the key for the whole team to succeed in this phase. Specific examples of collaborative negotiation in software design include: • In agile processes, such as eXtreme Programming, daily design meetings require very close collaboration focused on the same artifact. • Constant group meetings may carry out design activities such as application programming interfaces specification and system modules communications with different combinations of individuals. • When correcting defects, team members may negotiate with former team members, currently assigned to other projects, in order to determine the rationale for some design feature, which has subsequently been identified as problematic. • Refactoring the system design often involves relatively minor but widespread changes within, and sometimes outside, the development team. Team members who are kept informed of such activity are able to avoid potential conflicts. [Cook et al. 2005] Indeed, there is growing recognition that the research on how teams actually make design decisions and negotiate about design requirements can provide valuable insights for improving the quality and productivity of large-scale software system development efforts [Curtis 1986; Curtis et al. 1988; Soloway 1986]. Traditional group decision 25 models in various disciplines, such as software engineering, organization dynamics, and decision science, are not rich enough to thoroughly explain the real-world complexities faced by modern software design teams. Conventional group decision-making methodologies tend to suppress the presence of negotiation process with pure optimizing method, making any resolution untraceable and ignoring the issues in collaborative negotiation [Cook et al., 2003]. And most of the existing research was performed on tasks that were shorter, less complex and did not require the extensive integration of knowledge domains that characterizes software systems design [Curtis 1986; Curtis et al. 1988; Soloway 1986]. However, the software design process is essentially a knowledge sharing and integration process. Knowledge is the raw material which software design team works on and it is more than written scientific theories or the product documents [Yakemovic et al. 1990]. The design professional’s knowledge in particular is framed by a person’s objectives and preferences of multiple, ongoing group activities [Gersick 1988; Waiz 1988]. These objectives and perspectives are often very different and so the elicitation and integration of objectives and preferences need to be well handled in the software design teams. In addition, driven by industry globalization and Internet revolution, all these knowledge integration activities in modern software design may have to be done across the geographical, disciplinary and temporal boundaries. Therefore, one of the critical challenges in this distributed knowledge integration process is to identify and organize stakeholders’ objectives and perspectives and structure these objectives and perspectives into the negotiation arguments to assure these decision makers, who are 26 geographically distributed, cross-disciplinary and asynchronous, to have their arguments commonly structured in order to carry out a effective collaborative negotiation process. 2.3 Related Research in Group Decision As mentioned in Section 2.2, the software design phase is group decision-making process that is maximally involved with collaboration and negotiation between engineers, managers, architects, customers and many others, who are often geographically distributed, cross-disciplinary and asynchronous. Our challenge is to identify and organize these stakeholders’ objectives and perspectives and then structure these objectives and perspectives into the negotiation arguments. To put the discussion in perspective, this section reviews current research work of group decision in a variety of disciplines in relations to collaborative negotiation and structuring arguments. These disciplines include organizational science, social psychology, social choice, decision science and software engineering. 2.3.1 Organizational Science Organizational science uses the combined knowledge of economics, sociology, political science, anthropology, and psychology to investigate individual and group dynamics in an organizational setting, as well as the nature of organizations themselves [Ancona et al. 1999]. When people interact in organizations, different factors from those found in open groups in society come into play. Organizational studies attempt to understand and model 27 these factors in decision making process. Scientists in this domain have proposed the satisficing model [March and Simon 1958; Newell and Simon 1972; Simon 1969] which finds an acceptable, good enough solution, reflecting "bounded rationality", in contrast to the concepts of “absolute rationality” and “optimizing” from neoclassical economics [Elam et al. 1980]. This research recognizes there are always practical limits to human rationality and available resources, and these limits depend on the organizational policy, regulation and the environment in which individual interacts. It suggests individuals to engage in group decision-making process and structure their negotiation by the organizational limitations (e.g., policy, culture, etc.) to approach rationality as closely as possible in their joint actions. Organizational science holds that stakeholders always engage in collaborative endeavors as members of a purposeful “team.” Different from a group, a team consists of a distinguishable set of members who interact toward a set of common goals with specific roles or functions to perform [Olson 1965, 1971]. Members must be carefully selected so as to bring complementary expertise to the team, as well strategically assigned to different functions with clearly understood responsibilities [Straus and Layton 2002]. With such systematic organization, all team members will work collaboratively toward the team goal, rather than selfishly based on their separate interests. In such collaboration, optimizing can be viewed as the ideal way of jointly finding the solution that best fits the objectives. It involves constructing a model that incorporates all parameters and characteristics that have an effect on the optimal solution. Given all feasible values of 28 these characteristics, the model can then predict the optimal solution. However, in practice, such an approach is not possible. This is because such a model would be too time-consuming to construct, and in any case, it is not possible to model all values numerically. Also, the search for an optimal solution would be endless, because there are an infinite number of possible solutions. Therefore, the concept of satisficing is closer to what happens in practical life. Satisficing involves selecting a solution that is “good enough,” i.e., it falls within some specified criteria of acceptance. However, it is questionable whether this is an approach that is really good enough. While the optimizing approach is too ambitious, satisficing seems to be not ambitious enough. Of course, it depends on what criteria of acceptance that is specified; if the criteria are ambitious, the solution would be ambitious. However, it is hard to know at what level the limits should be set. The best approach seems to lie somewhere in between the two extremes: to search for a solution that satisfies a set of objectives in a best possible way, given the limited time and resources of the given project. In order to find this solution, organizational scientists actually suggest the decision makers to consider preference aggregation or value trade-off between the objectives in accordance with the relative importance of the criteria. The decision makers need to communicate and negotiate decision alternatives which have to be structured by the team goals and organizational limitations with consideration of bounded rationality and satisficing instead of optimizing the alternatives based on the attributes of each goal. However, this study to date has not considered challenges imposed by the group decision-making process in modern software design practices, such as achieving a consensual rationality bounded by multiple cross- 29 disciplinary organizations that are geographically distributed and operating in different timeframes. 2.3.2 Social Psychology Social psychology is the scientific study of how people's thoughts, feelings, and behaviors are influenced by the actual, imagined, or implied presence of others [Allport 1985]. It provides us with empirical studies and procedures of structuring and constructing human perspectives in the context of social interaction. Social psychologists typically explain human behavior as a result of the interaction of mental states and immediate social situations. In Kurt Lewin's famous heuristic formula, behavior (B) can be viewed as a function of the person (P) and the environment (E), B=f(P,E) [Lewin 1951]. In general, social psychologists have a preference for laboratory based, empirical method of investigation. Within this field of study, our particular interest is in social cognition, which is part of psychological research and study “self-conceptions” and what the individual perceives to be within her horizon of actions [Aronson et. al. 1951]. Specifically, it investigates how individuals perceive, influence, and relate to others by studying the relationship between minds and behaviors in groups. Unlike organizational science studies, which focus on the overall structure and organization of groups without focusing on individuals’ minds, social cognition looks at the “inside” to understand how individual minds change in the context of social interactions. As early as 1978, Vygotsky suggested that social interaction between individuals provides inputs to most 30 epistemological processes [Vygotsky 1978]. Social interaction with others and participation in the life of one’s social community are fundamental underpinnings of human cognition. Human perspective and its social or cultural environment cannot be separated from, but rather are integral to human cognition. Additionally, the relationship between the individual and her social environment are parts of a mutually constructed whole [Bredo 1994]. Based on the premise that “minds can be shaped by others,” social cognition research has long proven that individuals’ perspectives change their preferences, which guide their decisions. In social cognition, a perspective is defined as a set of objectives that shape the way an individual perceives reality and responds to it. These studies also confirm that individuals’ perspectives are not static, but are always dynamically evolving when they interact with others in social settings. In teamwork, specifically, research has revealed that individuals often participate in a mutual, reciprocal, and dynamical social construction process, whereby each “constructs” (i.e., influences) others’ perspectives (e.g., objectives, preferences). Based on this finding, social psychologists have developed empirical procedures and conducted many experimental studies to guide the dynamic evolution and construction of human perspectives during social interactions and group negotiation. Their studies provide us with practical principles for how to structure human perspectives within the context of the social interaction and how to guide the dynamic evolution and perspectives construction during this social interaction. However, without solid theoretical grounds, it is still very difficult to extend the previous studies in social psychology to the new 31 challenges of guiding the structuring and co-construction of stakeholders’ perspectives in social interaction to support group decision in modern software engineering. 2.3.3 Social Choice As a branch of social science, social choice studies provide us with solid theoretical background regarding how many individual preferences can be logically aggregated to form a consistent collective preference for group decision making [Fishburn 1973]. One of the most well-known studies in this field is Arrow’s possibility theorem [Arrow 1959]. It states that “no social choice procedure can be derived from simple rules of democratic decision making in groups”. In details, this theorem proves that no social preference function can satisfy the four criteria (i.e., universal domain, Pareto principle, independence from irrelevant alternatives, and non-dictatorship) of democratic decision- making in groups. Because this theorem suggests that there is no procedure for combining individual rankings that can lead to a consistent group ranking, some have insisted on a rigid position that “group decision-making by all is not rational, never consistent, and inherently chaotic” [Hazelrigg 1998]. If this theorem is universally true under all conditions, then group decision problems, which depend on finding a good group preference, become theoretically impossible and all group decision practices are then unavoidably ad hoc and eventually chaotic [Hazelrigg 1996]. However, quite often in engineering domains, the four criteria in the impossibility have been already weakened. For example, engineering decisions are never fully democratic (in contrast of non- 32 dictatorship) and always need collaboration and co-construction. Engineering decisions are always restricted (in contrast of universal domain) by customer needs and corporate policies. Also engineers do not need to explore the complete alternative set (in contrast of independence of irrelevant alternatives) at the beginning and only have to focus on the promising ones. Modern social choice research has proven that only under a very restrictive set of conditions for special domains, which members can explicitly compare and negotiate their interpersonal preferences using ratings of continuous alternatives, to construct a group preference rationally [Harsanyi 1955]. The same has been proven analytically by decision scientists, who have used preference ratings, not rankings, to derive a group preference that can satisfy Arrow’s four criteria of democratic decision- making [Keeney 1976]. Nevertheless, social choice research to date has been largely based on the assumption that individuals’ preferences remain static, regardless of their interactions in terms of collaborative negotiation. Although this research provides solid theoretical ground for how to logically aggregate individual preferences to form a consistent collective preference in group decision making, in practice it does not specify how exactly the members in modern teamwork can identify, structure and negotiate these interpersonal preference based on alternative ratings in making collective choice, which remains as the challenge of our work. 33 2.3.4 Decision Science Decision scientists interested in group decisions have investigated various negotiation models and decision analysis functionalities that help to achieve group consensus among multiple interests and competing positions of stakeholders. However, these models and functionalities have not provided full support to decision makers who have to identify, organize and integrate their multi-disciplinary objectives and perspectives from distributed locations and asynchronous communications. The schools of study in this field include game-theoretic analysis [Rosenschein and Zlotkin, 1994; Kraus, 2001; Sandholm, 2002]; heuristic-based approaches [Faratin, 2000; Kowalczyk and Bui, 2001; Fatima et al., 2002]; and argumentation-based approaches [Kraus et al., 1998; Parsons et al., 1998; Sierra et al., 1998]. The details of each study in relations to our work are explained below. Rooted in economics, game theory studies interactions between self-interested agents. The objective of game theory is to determine the best (i.e., most rational) decision a player can make, using mathematical modeling. In order to do so, the player must take into account the decisions that other agents can make and must assume that they will act rationally as well. A solution in game theory is generally found when players’ strategies are in equilibrium: a player’s strategy is the best response to the other’s strategies. Tools from game theory can help decision makers understand and predict the outcome of a negotiation and then help them make strategic decisions in group decision making 34 process [Nagarajan and Sosic 2008]. Game theory can be divided into two main approaches. Non-cooperative game theory is strategy oriented, meaning it studies what players will do in a specific context in order to win over their opponent. In contrast, cooperative game theory studies how players can cooperate to reach a win-win situation when the global gains are higher with cooperation than without. A frequently mentioned drawback of game theoretic approaches is the perfect rationality assumption. In order to select the best strategy, the player must know the entire environment as well as the opponent’s knowledge. Otherwise, it is not possible for the player to estimate the most rational choice. Unfortunately, in real world situations, players have private information hidden from their partners in the decision making process. So an effective and systematic method to elicit and organize the information from each team member is still in need to make rational group decision using game theoretical approaches. A way to overcome the game theory limitations described previously is to use heuristic approaches. Heuristic-based negotiation is based on search strategies where the objective, instead of finding the optimal solution, is to find a good solution in a reasonable time. Multiple approaches can be used, depending on the search strategy deployed. Stakeholders do not need to be perfectly rational and information can be kept private. Basically, the space of possible agreements is represented by contracts having different values for each issue. Using its own utility function, a stakeholder must compute the value of each contract. Proposals and counter-proposals are exchanged over the different contracts and search terminates either when the time limit has been reached or when a 35 mutually acceptable solution has been found. Kraus presented a review of applications of heuristics to negotiations and pointed out where it represents an advantage over other approaches [Kraus 1997]. Klein worked on a simulated annealing based approach for negotiation of multi-interdependent issues in contracts [Klein 2003]. Rahwan have worked on defining a method for designing heuristics-based negotiation strategies for negotiation agents, by analyzing the environment and the agent capabilities [Rahwan 2007]. While heuristic methods do indeed overcome some of the shortcomings of game- theoretic approaches, they also have a number of disadvantages [Jennings et al. 2001]. First, the models often lead to outcomes that are sub-optimal, because the information and space that the negotiation team can explore is always limited by the design of the heuristics method, which is usually an ad-hoc. Second, because of the ad-hoc design of the heuristic method, it is very difficult to predict precisely how the team and stakeholders will behave and there is usually no guaranteed solution at the end of the execution of the heuristics. Consequently, these approaches usually need extensive evaluation through simulations and empirical analysis, which is not often available due to the resource limitation in engineering processes. Argument-based negotiation approaches follow a generic structure of arguments defined in [Toulmin 1958] which helps stakeholders lay out their negotiation information and meta-information into various components and this information includes major claims, support data and additional persuasive perspectives such as justification, degree of desire and rebuttal condition. In the negotiation approaches presented previously, stakeholders 36 cannot justify to their partner why they refuse an offer or what part of the offer was problematic. Proposals do not include the explanations of the positions and considerably limit the potential of negotiation. The idea behind argumentation-based negotiations is precisely to give this additional information (e.g. a justification about why the partner should accept a proposal) to stakeholders, helping the negotiation process by identifying part of the support data and background information that does not get explored otherwise. Different authors have presented applications of argumentation-based negotiation models [Buttner, 2006; Atkinson et al. 2005; Capobianco et al. 2005; Jennings et al. 2001]. These approaches can increase the efficiency of negotiation process by adding information that was not used before. By revealing new information, the partner can be persuaded that a certain proposal is better than it thought. Based on the advantage of these approaches in systematically and effectively organizing and conveying stakeholders’ perspective, argument-based negotiation approaches have been our major concern in this work. One of the main limitations of these approach falls in that the stakeholder must be able to analyze the arguments and calculate their value in order to better understand the relationship between the arguments or choose one best argument. This is because, although Toulmin defined the generic and well-adopted argument structure, he proposed his views on argumentation informally in loosely specifying how arguments relate to other arguments and providing little guidance as how to evaluate the best [Zeleznikow 2002]. It is still more intended as a way of checking and arranging arguments for overlooked flaws [Houp, Pearsall, and Tebeaux 1998] instead of directly supporting group decision-making such as specifying the relationship between the structured 37 argument and the governing factors in decision making (e.g., stakeholders’ objectives and perspectives). 2.3.5 Group Decision in Software Engineering Most of the existing research for group decision in software engineering falls in two categories: group evaluation approaches and design negotiation models. We review both categories in this section with regard to collaborative negotiation and structuring arguments. An example of the first category is ATAM, which distributes the architectural documents and business requirements to the stakeholders, elaborates and prioritizes scenarios, conducts design evaluation and finally develops a complete technical report [Kazman et. al. 1998; Lee 2005]. It helps the engineers understand the consequence of software design with respect to the system’s quality attributed requirements and business goals. In addition, it helps the developers to determine where the risks and tradeoffs exist in various software design strategies. However, as an evaluation method for software design, ATAM does not explicitly take any human objectives or perspectives into its evaluation rationale, such as, individual goals and personal interests from the stakeholders themselves. As a result, it is not always certain that the stakeholders will accept the solutions provided by ATAM. Another software design evaluation method, developed from ATAM by the same group of researchers, is CBAM (Cost Benefit Analysis Method) [Moore 2003; Kazman 2005]. CBAM collates high-priority scenarios from ATAM, and refines and prioritizes scenarios to formulate business goals. It then develops architectural strategies for scenarios, calculates the total economic benefit for 38 each strategy, and chooses architectural strategies based on business values. CBAM explores, analyzes, and makes technical decisions regarding software architecture design alternatives with consideration of economic factors [Hoh et. al. 2002]. However, it is still not clear how the explored alternatives relate to stakeholders’ objectives and preferences and how well the solutions satisfy these objectives, especially in modern software design process where these objectives and preferences may be in geographically distributed locations and asynchronous communications. In addition, this approach only evaluates different design solutions but does not specify any operational guidance for stakeholders to negotiate conflicting design decisions in the group decision-making process for software design. A good example in the second category is the Win-Win negotiation model developed by the Centre of Software Engineering at USC [Boehm et. al. 1999]. It provides a generic framework for software requirement negotiation. In the Win-Win model, stakeholders begin by first eliciting their own desired “win conditions”, identifying issues (e.g., conflicts), generating options to resolve these issues, negotiating options and finally reaching agreements. However, without a well-defined group decision making process, in the Win-Win model stakeholders still need to generate and negotiate the architecture alternatives manually by a rather ad-hoc process [Hoh et al. 2002]. Furthermore, the Win- Win negotiation model is based on a software engineering approach called MBASE (Model-Based Architecting and Software Engineering), which detects the conflict in software development by identifying model clashes (e.g. success models) [Boehm et al. 39 2002]. Most of the identified clashes in this approach, however, come from past success models representing previous win conditions, which are only a subset of stakeholders’ objectives and preferences. Therefore, some of the potential conflicts caused by different objectives and preferences may not be accounted for because these differences are not explicitly modeled in this approach. In the second category, the most relevant work to our present study is the MPARN model, which guides stakeholders from design options to agreements by using multi-criteria preference analysis techniques [Hoh et al. 2002]. The MPARN process begins to identify the conflicts in the stakeholders’ needs following the Win-Win process and then explores resolution options. After this, it supplements the Win-Win process by eliciting stakeholder preferences. It also assesses how well each of the generated options performs on stakeholder criteria. As a negotiation approach with the goal of supplementing the Win-Win process, MPARN provides some conflict analysis and resolution strategies. However, since it is based on the Win-Win process, it inherits some of the same limitations of the latter. For example, stakeholders’ negotiation arguments are not well structured since their arguments include knowledge from different disciplines. Nor are their objectives and preferences explicitly modeled and organized, especially in a group decision-making process that is across the geographical, disciplinary and temporal boundaries, although these objectives and preferences are important factors in determining the outcomes of group decision-making tasks. 40 2.4 Collaborative Negotiation in Engineering 2.4.1 Challenges and Limitations As mentioned earlier, though extensive research has been done in the field of negotiation, various limitations exists. Most of existing practices only provides negotiation analysis capabilities for the scenarios involving two decision makers on one task. They become conceptually intractable and computationally complex when applied to the scenarios that involve multiple distributed decision makers with competing objectives and different preferences. And they have not specified a practical means to identify and organize objectives and preferences amongst multiple stakeholders, nor an operational approach to guide the stakeholder through a negotiation process for multiple tasks. In order to resolve the challenge of our work in structuring negotiation arguments with identified and organized objectives and preferences, in this section we will review our previous work in a well-designed Socio-Technical Co-construction Process (STCP) which is rooted in an Engineering Collaboration via Negotiation (ECN) paradigm and a Socio-Technical Framework (STF). This work helps us specify operational guidance for the software design team to carry out a negotiation process utilizing structured arguments for negotiation tasks. This section will explain these concepts (i.e., ECN, STF and STCP) in details. 41 2.4.2 Engineering as Collaborative Negotiation Paradigm Real-world negotiation tasks undertaken by engineering teams are always driven by many conflicting social, economical and technical (SET) factors. Traditional engineering research has mainly focused on technical factors, with some recent efforts being extended to consider the economic factors. While recognizing the importance of both technical and economical considerations, our past research has been focusing on social factors, and more specifically, on their interactions with technical factors. We view an engineering team activity such as developing software, as a technical activity with a human purpose. Therefore, when a team of engineers (i.e., multiple stakeholders) with differing life cycle concerns come together to develop new software, it can lead to a complex socio-technical campaign. To resolve this challenge, an Engineering Collaboration via Negotiation (ECN) paradigm is developed in our past work and can best support this type of socio- technical campaign. In this paradigm, we have defined the meanings of several key concepts that lay down the foundation for the research framework and process. Now we first give the definition of Engineering as Collaborative Negotiation (ECN). ECN is a socio-technical negotiation activity, where a team of stakeholders with different expertise and mixed objectives to co- construct consensual agreements of some engineering matter. 42 More details regarding the concepts used in this definition are explained as follows: Socio-technical Compared with traditional approaches, ECN treats engineering decision making as a technical activity within dynamic social contexts instead of a purely technical activity. Negotiation With a general definition, “to confer with another so as to arrive at the settlement of some matter,” given by Webster’s dictionary, we limit ourselves in the domain of engineering domain and negotiation is viewed as an on-going activity of social decision-making, where two or more interdependent parties with some common or conflicting interests engages in back-and-forth communication aimed at reaching a settlement Stakeholders Stakeholders are parties with direct or indirect interests in the outcomes of a common undertaking. Regardless of whether stakeholders are directly involved in a decision at a particular moment, they are always given the opportunity to express their perspectives toward that decision in the ECN approach. The stakeholder perspectives evolve dynamically through the co-construction process in ECN. Different Expertise In ECN, engineering expertise must be expanded from purely natural sciences to include not only all social, economical, and technical (S.E.T.) knowledge but also the ability to negotiate effectively with other stakeholders. This expansion requires techniques that go beyond the traditional engineering approaches. The social-technical framework 43 developed in our research is an example of these new techniques and will be discussed in the next section. Mixed Objectives ECN focuses on the knowledge of artificial which is based on human agreements. Human agreements are driven by their objectives. Objectives can be in concert or in conflict, which in turn determines stakeholders’ various cooperative or adversarial decision behaviors and leads to the final agreement or disagreement. Co-construct/Co-construction Co-construction is a key activity in ECN. In ECN, the stakeholders perspectives are influenced by each other and mutually modified through negotiation, and finally arrive at a shared reality (e.g., new knowledge in the form of consensual agreements) of a particular matter. This makes ECN different from the traditional engineering, in which the perspectives of stakeholders are assumed to be static. Consensual Agreements Unlike traditional engineering approaches that search for precise answers optimized for pre-defined objectives, the final outcomes of ECN are consensual agreements that suffice to meet the interests of all involved stakeholders. When stakeholders reach consensual agreements, it indicates that, for all parties, the payoffs from the consequences of joint decisions are better than if each party had decided to go-it-alone. In ECN, consensual agreements are the consequences of joint decisions. They are negotiation process dependent, and change with the stakeholders’ objectives and/or social settings. 44 2.4.3 A Socio-Technical Framework Based on this ECN paradigm, we have developed a Socio-Technical Framework (STF) as a foundation to pursue basic research in collaborative negotiation. The STF has its roots in the Social Construction Theory proposed by Peter Berger and Thomas Luckmann in 1966 [Berger and Luckmann 1996], which states that meaning and institutions (e.g., a collaborative decision during software development) are a jointly negotiated and agreed construction between those participating in an endeavor. More specifically, our STF uses perspective models of stakeholders (e.g., software designers) to integrate social interactions with technical decisions, and then uses these models to manage decision conflicts during a co-construction process taken by the engineering team. The three core components of our STF research framework are: (1) technical decision processes taken by the engineering team, (2) perspective models of stakeholders, and (3) conflict management strategies for collaborative negotiation [Lu and Cai 1999]. The technical decision process refers to a series of tasks and states that must be performed by stakeholders according to some pre-established steps adapted from the specific domain practices or mandated by corporate policies. Perspective models describe stakeholders’ viewpoints towards the concepts in a particular design campaign. Conflict management provides guidance to reconcile decision conflicts by managing decision-making process and helping stakeholders co-construct stakeholders’ perspectives. 45 2.4.4 A Socio-Technical Co-construction Process Based on the foundations of ECN diagram and the STF framework, we have taken the next step to develop a specific process through which this conceptual framework can be further detailed and structured for a collaborative negotiation process. This process, which we called Socio-Technical Co-construction process (STCP), specifies eight steps with sufficient operational details as follows: I. Define a starting “baseline process” for the chosen application domains, as the basis to be co-constructed (i.e., changed) later, upon the agreement by all involved stakeholders. II. Identify a group of “stakeholders” who have an interest in the outcomes of, and will directly or indirectly participate in, the co-construction process of a particular collaborative campaign (e.g. software design). III. Propose an initial “concept structure” for a particular engineering process to organize the concepts provided by the team. IV. Establish the initial “perspective model” for all participating stakeholders to express opinions for each concept in the concept structure. V. Build the “perspective model state diagram” (PMSD) for each concept in the concept structure. VI. Perform the “perspective analysis” on the current PMSD to understand the closeness or distance of different stakeholders’ perspectives at that particular moment. VII. Conduct the “conflict management” tasks according to the results perspective analysis. 46 VIII. Obtain a “shared reality” as a result of the co-construction process. This final product of the STC process is a shared reality, which is a broader concept than traditional approaches (e.g., a finished design in terms of a product model). The socio-technical co-construction process (STCP) provides us with research context and grounds of building a new negotiation process, as STCP specifies eight steps with sufficient operational details to guide the stakeholders through the process of co- constructing group decisions. However, stakeholders who work in STCP are required to fully share their perspectives (e.g., proposals, objectives, preferences, and justifications) yet they have not been guided about how to organize their perspectives and structure the negotiation arguments with these perspectives during the process. 2.5 Summary To summarize the above, in this section we have discussed the contributions and limitations of previous works on group decision in relation to collaborative negotiation and structured arguments. Existing research approaches focus on different aspects of group decision and some of them provide considerable contributions for understanding the basic characteristics of collaborative negotiation. However, most of these approaches focus on analyzing the negotiation scenario between two stakeholders on one task. When these analytical approaches are applied for multiple stakeholders over a sequence of tasks, either the computation is too complex to handle or there is a lack of theoretical support to extend these approaches to such scenarios. In addition, existing approaches usually 47 simplify the impact of meta-information in the negotiation such as stakeholders’ objectives and perspectives. Although it is impossible to fully share this meta-information among stakeholders in group decision of software design, this meta-information still need to be explicitly elicited, organized and negotiated to reach an optimal degree of consensus upon the software design issues and resolve conflicts to deliver a desirable design result [Clancey 1993]. Furthermore, even in the approaches which have specified the structure or steps to elicit and model this meta-information, such as argument-based approaches and social-technical co-construction process, they are still in need of feasible and effective approaches of centralizing and organizing this meta-information in a group decision-making process that is across geography, discipline and temporary boundaries, such as modern software design. Therefore, this research focuses on the specific task of organizing the objectives and perspectives from decision makers and structuring these objectives and perspectives into the negotiation arguments to assure these decision makers, who are geographically distributed, cross-disciplinary and asynchronously communicating, to have their arguments commonly structured in order to carry out a effective collaborative negotiation process in group decision of software design. We have followed a generic structure of arguments defined by Toulmin and utilized that as a template to help stakeholders to lay out negotiation information and meta-information in a systematic process defined based on the socio-technical co-construction process. The novelty of our approach is, by synthesizing this generic argument structure with an objective hierarchy that organizes 48 stakeholders’ objectives and perspectives, in ensuring stakeholders have commonly structured arguments and providing operational and feasible guidance for the stakeholders to generate, exchange, analyze and evaluate their structured argument during the collaborative negotiation process. Our approach explains to the stakeholders how they can organize their objectives and perspectives, use this information to generate their arguments and then exchange the arguments in necessary conditions. In addition, it applies clustering analysis based on the concrete meanings of the arguments (given by the synthesis) for better understanding the relationship between the arguments and developing a feasible way of evaluating the arguments to choose the most preferred one. The remainder of this dissertation describes this approach in details. 49 Chapter Three: Research Foundations, Approach and Hypothesis 3.1 Introduction Based on the findings and discussions in the previous chapter, this chapter introduces the two theoretical foundations of this research work. The first foundation is the Toulmin’s structure of argument, which is utilized to structure negotiation arguments and define a systematic process for eliciting and exchanging structured arguments. The second one is an objective hierarchy based on value-focused thinking methodology to organize stakeholders’ objectives and preferences. After that, we discuss our main approach, which is a synthesis between structured arguments and objective hierarchy to overcome the limitations in the existing practices of argument-based negotiation approaches. Last but certainly not the least, we present our main hypothesis that results from inductive reasoning; from the practical challenge of supporting group decisions in software design through how to effectively carry out collaborative negotiation by utilizing structured arguments. 3.2 Argument Structure: Toulmin’s Structure of Arguments Practicing collaborative design and negotiation dialogue have been found to be positively linked with argument development and critical thinking skills [Hart 1990; Jennings 1998; Jin, Geslin and Lu 2005; Marttunen 1992; Smith 1977]. Furthermore, the work of Buckingham and his colleagues argue that standardizing an argument's structure facilitates its subsequent communication since important meta-information and 50 relationships can be more easily perceived and analyzed by others [Buckingham Shum et al. 1997]. Stephen E. Toulmin’s 1958 work, Uses of Argument, has become commonplace in structuring arguments - Toulmin acknowledges as much in the preface to his 1984 text, An Introduction to Reasoning [Toulmin 1984]. For example, Houp, Pearsall and Teheaux’s textbook, Reporting Technical Information, introduces Toulmin logic as providing “a way of checking your own arguments for those overlooked flaws. It can also help you arrange your argument” [Houp et al. 1998]. The goal of developing arguments in negotiation is to persuade or convince others that one's reasoning is more valid or appropriate. Toulmin's structure of argument provides the language symbols and data structures that support the argumentation process (see Figure 3). Toulmin’s structure is procedural and the layout of this structure focuses on the movement of accepted data to the claim through a warrant. Toulmin also recognizes three secondary elements that may be present in an argument: backing, qualifier, and rebuttal. Backing is the authority for a warrant, provides credibility for the warrant, and may be introduced when the audience is unwilling to accept the warrant. A qualifier indicates the degree of force or certainty that a claim possesses. Finally, rebuttal represents certain condition or exception under which the claim will fail and hence anticipates objections that might be advanced against the argument to refute the claim [Toulmin 1958]. As such, Toulmin’s argument structure becomes a popular mechanism for structuring arguments between negotiating stakeholders. It aims to clarify the reasoning process by encouraging parties to make explicit important assumptions, distinctions, and relationships as they construct and rationalize ideas [Buckingham Shum et al., 1997]. 51 Figure 3: Toulmin's Argument Structure We selected Toulmin argument structures to structure the negotiation arguments in our approach after considering a number of possible approaches and structures applied or developed for negotiation [Janssen and Sage 1996]. Negotiation is a process that involves both qualitative and quantitative concepts. Many of the formal approaches such as game theory and decision analysis have focused traditionally on the quantitative aspects of negotiation including goals, alternatives, and consequences. The qualitative aspects of negotiation such as argumentation and negotiator perspectives are typically given low priority and are subsumed in other quantitative concepts, although they are important in their own respect from a practical viewpoint because in many cases negotiation approaches and stakeholders’ interaction directly affect the negotiation outcome. The negotiation approach described in this work will capture these qualitative aspects inside the structured arguments. 52 Other types of structures than Toulmin’s argument include: influence diagrams and Bayesian networks, which are oriented toward quantitative analysis and are not easy to build, structured and conceptual modeling, which may be used to construct a Toulmin argument structure, and Wigmore structured evidence, which comes the closest to Toulmin and is an argument structure based on an elaborate set of symbols [Janssen and Sage 1996]. The Wigmore approach, although closest to our Toulmin based approach, is much more complicated than Toulmin. The Toulmin argument structure is easy to use and understand and is widely applicable [Janssen and Sage 1996]. Our studies suggest that, when the argument structure is framed appropriately, it is easy for software designer to understand and to use that in negotiating design solutions. Many of these approaches are described in the contemporary literature [Sage 1992; Schum 1994]. There have also been a number of other studies concerning the implementation of Toulmin based logic [Lagomasino and Sage 1985; Sage 1991; Laskey et al. 1989]. In addition, Jassen and Sage’s study shows there are several advantages to a graphical or structured depiction (i.e., boxes and lines as in the Toulmin’s argument structure) than natural language description [Janssen and Sage 1996]. They have stated the reasons like follows: first, visualization eases comprehension. The components of the argument are explicitly represented, meaning that it is easier to identify the particular elements of an argument, and the structure comprising of these elements facilitates the elicitation of the elements. Second, it is easy for the person filing the boxes to see what is missing as well as the reasoning that has been put forth. In this regard, it is easier to compare arguments 53 between multiple experts and between claim and counterclaims than between statements in generally unstructured discourse [Janssen and Sage 1996]. It is true that using the Toulmin’s argument structure, which is generally more objective than implicit arguments, it is hard for stakeholders to hide bias because the grounds and backing of an argument are clearly listed and described to support the claims. Therefore, all stakeholders’ perspectives are generally relatively easy to be fully observed by others through examination of the ground and warrants that the stakeholder expresses [Janssen and Sage 1996]. The process of preparing the argument components has been proven to be an objective way to organize and convey stakeholders’ perspectives. In addition, the logic flow between these components inherently helps us specify the order of exchanging perspectives in the group negotiation between the multiple stakeholders. As mentioned earlier in this section, argument structure has been used to build an argument-based negotiation process model in many previous studies including collaborative design and proved to facilitate more objective and fair communication [Chang et al. 1995; Sillince et al. 1999; Sierra et al. 1998; Amgoud et al. 2000; Avery et al. 2001; Kraus, 2001; Rong et al. 2002]. However, there are remaining unresolved issues in most of the above work, such as a systematic guide to analyze the relationship amongst arguments and evaluate the arguments for the best in an operational negotiation process. Our research pushes this forward by developing a collaborative negotiation approach that fully exploits the structured arguments and synthesizes it with an objective hierarchy in the context of the Socio-Technical Co-construction Process for group decision in software design. More 54 details about the objective hierarchy are discussed in Section 3.3 and the synthesis is introduced in Section 3.4. 3.3 Objective Hierarchy: Value-focused Thinking Based on the belief that conflicts in software design has roots in stakeholders who have different perspectives (e.g., understandings and expectations) of the design task, our approach uses an objective hierarchy as an organization of the objectives on which the stakeholders can declare their perspectives Furthermore, these objectives should also help stakeholders reveal the real “values” the stakeholders want to get out of the design task. These values are the expectations for why the stakeholders join in a negotiation process to reach an agreement in case of a conflicting design task. They are the things about which stakeholders (or their organizations) really should care. It is assumably always subjective and socially constructible. Our approach focuses on only those professional values that are mandated by the organization for the software team to follow. This being said, the concept of “value-focused thinking” is introduced in this work to help stakeholders formulate and propose their objectives. The theory of value-focused thinking for decision-making was built by Ralph Keeney. “Values are fundamental to all that we do and thus values should be the driving force for our decision-making” [Keeney 1992]. Every stakeholder has his own value when he makes any decision. For example, when selecting a platform for developing web service, an individual contractor who favors open source may always choose Java based 55 frameworks since most of those are open-source to the public and everything is free to use, however, a manager who has more concern with platform and development support may choose .Net as the main platform since using that it is more convenient for him to directly contact Microsoft for customer service instead of asking a question in the public community and desperately waiting for some one to kindly give an answer.” Keeney emphasizes that each decision maker pursues his own values in a group decision-making process and the goal they participate in this process is to maximize his value. Value-focused thinking helps stakeholders identify their values in terms of fundamental objectives [Keeney 1992]. This work assumes a hierarchical structure of stakeholders’ understanding for the design process, from abstract to detail, in that one’s “values” drives “objectives” and “perspectives” that in turn determine the “preferences” for various “alternatives” (i.e., proposals) for the design task. In this work, values represent the highest (i.e., the most abstract) level that is always subjective and formed by one’s education, professional, cultural background and experience. Values help stakeholders identify objectives and then drive the “perspective” (i.e., the less abstract) indicating their preferences for the proposals (alternatives in value-focused thinking) of the assigned design task. Value-focused thinking has provided practices for deriving objectives from identified values in decision-making problems [Keeney 1992], based on which the proposed procedure to use value-focused thinking in our approach to propose stakeholder’s objectives is described as below: 56 a. List the values for individual design tasks (on which the team are negotiating) b. Identify the set of objectives based on the values c. Organize these objectives to clarify relationships among them To build our objective hierarchy, following Keeney’s work [1992], we classify the values into fundamental objectives and means objectives. Fundamental objectives concern the ends in a specific decision context and the means objectives are the ways to achieve those ends. As for the previous example in this section, the fundamental objective of both the contractor and the manager is to build a web service application successfully. The means their objectives are different: the engineer’s is to employ a Java related web service platform which is free and open source while the manager goes for .Net with assurance of development support. In this objective hierarchy, we also define attributes for evaluating how well these objectives are achieved. These attributes describe the objectives and have common interpretation to every stakeholder. For example, “the number of interdependent application programming interfaces between modules” is an attribute of the objective “reduce modules coupling”; “the percentage of functions/methods which contribute to a single, well-defined task” is an attribute of the objective “increase module cohesion.” “Network latency” and “data throughput” are two attributes of the objective “improve network performance.” These attributes can be used as the common scales to evaluate the negotiation arguments for the degree to which they achieve the objective (that the attributes belong to) and thus yield independent measurement values for each argument. Then the arguments can be ranked for how well they achieve the objectives. The 57 evaluation results can be either a quantitative value within a pre-defined range, or a qualitative value, e.g., support, neutrality (indifferent or uninterested), opposition, controversy (not support but match the interest for the group). Section 4.1.3 explains the argument evaluation in details. 3.4 Synthesis between the Objective Hierarchy and the Generic Argument Structure The “argument” in our approach is first structured based on the generic argument structure defined by Toulmin [1958]. Note that this generic structure, by itself, stays as a general guide for collaborative negotiation and does not specify how to analyze the relationship amongst the arguments and evaluate all the arguments for the best. Our research resolves this challenge by synthesizing the generic argument structure (as discussed in Section 3.2) with the objective hierarchy (as discussed in Section 3.3) and then provides a feasible way of analyzing the relationship of arguments and evaluating arguments based on the meanings given to the argument components from the objective hierarchy. The synthesis is shown in Figure 4 and its details are explained as follows: 58 Data Warrant Claim Backing Qualifier Rebuttal Argument Structure Attributes State of Agreement Objectives Warrant Task Proposals Measurement (all objectives) Measurement (own objectives) Design Process Concept Structure Task Proposals State of Agreement Objectives Attributes Stakeholder Preferences Objective Hierarchy Design Process Figure 4: Synthesis Between Objective Hierarchy and Argument Structure As shown in Figure 4, the claim is proposed by the stakeholder and consists of a sequence of actions/objects to implement the task. The data specifies the state of team agreement about the design process (e.g., tasks, objectives) together with any applicable information to support the claim. The state of team agreement can be accomplished by previous design tasks and arguments, such as the results of the previous tasks, the common understanding of the team for these tasks, and the objectives jointly proposed by the team. The applicable information include the stakeholder’s understanding and expectation for the current task and any facts that can be used to justify there proposal. This data describes the initial state of the current task, presents the applicable information, and therefore provides background support for the claim. Stakeholders’ objectives are in 59 the place of warrant, identifying the value that stakeholders want to achieve along with clarification of the relationship between the value and the current state of agreement. These objectives, as warrants, justify that the proposal can achieve the value based on the state of agreement (i.e. data). The attributes of each objective, corresponding to the ‘backing’ component, further explain the objectives by describing their measurement criteria and then validate the relationship amongst the objectives, the proposal and the current state of agreement. Based on these objectives and attributes, the measurement result regarding the achievement of the aforementioned objectives by stakeholder’s own proposal work as a ‘qualifier’ to indicate the degree of desire of the stakeholders for the proposal. Similar results regarding the achievement of the objectives by others’ proposals work as ‘rebuttal’ and describe possible conditions that could fail the claim or suspend the warrant. Based on the concrete meanings (proposals, objectives, attributes, measurement results) given to the argument components from the objective hierarchy, clustering analysis based on the composition of the arguments (objectives and attributes) can applied to indicate the similarity and distance amongst these arguments in their objectives/attributes and the analysis results are provided to the stakeholder for better understanding of the arguments relationship and recommendations to refine the arguments. Similar analysis based on the attributes value of the arguments is applied to indicate the distance amongst these arguments in their measure results and provide the recommendations for arguments refinement as well. When the stakeholders cannot agree upon on one task proposal, 60 argument evaluation is taken based on the level of the objectives achievement to get the best choice. More details about argument analysis and evaluation will be discussed in the next chapter. 3.5 Research Hypothesis This section presents the hypothesis of this work. In real-life software design processes, we have observed that without a well-defined approach of structuring stakeholders’ arguments, it was often very difficult and time-consuming for multiple stakeholders with various backgrounds and expertise to understand how others arrived at their judgments and what to choose when two stakeholders give contradictory answers. Moreover, without a solid base to identify and organize stakeholders’ objective and preferences, it is difficult for stakeholders to track whether their objectives and preferences become more mutually agreed or even more diverse in the negotiation process. As such, based on the need to effectively support the collaborative negotiation for group decision in software, our hypothesis is that, using organized objectives and preferences to structure stakeholders’ negotiation arguments can help the design team improve their efficiency, by decreasing the time spent for negotiating on design tasks before reach consensual agreements, and enhance the quality of their collaborative negotiation of the design team by increasing the number of common objectives of the team during the negotiation process. 61 In order to test this hypothesis, this work has first devised a synthesis between the two research foundations (as discussed in Sections 3.3 and 3.4), i.e., the objective hierarchy which organizes the objectives and preferences of the stakeholder, and the generic argument structure defined by Toulmin [1958] that lays out the component including major claim and additional persuasive justifications. Based on this synthesis framework, this work develops four interrelated modules that can further test the hypothesis: argument generation which exactly specifies how to generate the structured arguments based on the integration between generic argument structure and stakeholders’ objective hierarchy; argument exchange that helps stakeholder to share their arguments based on the a pre-defined order in align with the logic follow of argument components; argument analysis that provides methods to analyze the commonality of objectives/attributes in stakeholders’ negotiation arguments to test our hypothesis; and argument evaluation that measures all the arguments based on how well they accomplish the objectives and lead the completion of the negotiation process (the completion time of which will be used as another testing metric for the hypothesis). The details of these four modules will be discussed in Chapter 4. Chapter 5 will present the case studies in which our research approach and modules have been applied for real-life software design processes in order to validate the hypothesis discussed in this section. 62 3.6 Summary The discussion in this chapter gives the following conclusions: First, structured arguments, e.g., Toulmin’s argument structure, help stakeholders more clearly and completely list their perspectives, especially the ground and backing (of their arguments) which could be easily hidden or ignored otherwise. The process of laying out the argument components provides our approach with a way of eliciting and organizing stakeholders’ perspective. In addition, the logic follow between the argument components defines the order of exchanging these perspectives amongst multiple stakeholders. The second research foundation of our work is an objective hierarchy that supports stakeholders in organizing their objectives and criteria based on their backgrounds and expertise. It is built based on ‘value-focused thinking’ methodology that helps stakeholder formulate and propose their objectives. This hierarchy represents stakeholders’ real ‘values’ and provides evaluation measure for the achievement of the objectives. Based on the two foundations, we have presented our main approach, which is a synthesis between the structured arguments and the objective hierarchy. This synthesis gives a solid ground for multiple stakeholders to share their objectives and criteria and generate the negotiation arguments accordingly. It overcomes the challenges in existing practices 63 of argument-based negotiation approach by providing a feasible way of analyzing the arguments relationship and evaluating arguments for the most preferred by the group. It also helps advances the state of art in value-focused decision support framework by defining guidance of structuring and co-constructing the values in the entire group based on the structured arguments and the logic flow betweens the argument components. Last but certainly not the least, we have summarized the main hypothesis of this dissertation, which actually has led to the establishment of our research foundations and formation of our approach for this work. Our hypothesis is that by identifying and organizing the stakeholders’ objectives and preferences, and then structuring the negotiation arguments with the organized objective and preferences, the software design team can improve their efficiency by decreasing the negotiation time for design tasks and enhance the quality of their collaborative negotiation by increasing the number of the common objectives during the negotiation process. Based on these conclusions, Chapter 4 presents four interrelated modules that are built based on the synthesis between the generic argument structure and the objective hierarchy. These modules specify the details about how to generate, exchange, analyze and evaluate the structured arguments. Then, a case study, as the test of our approach, will be reported in Chapter 5. 64 Chapter Four: Research Approach: Using Structured Arguments to Support Collaborative Negotiation 4.1 Introduction The first part of this chapter clarifies the line of argument based on findings and discussions in the previous chapters. This line of argument explains the details and relationship between the research background, the goal of our research, the hypothesis, the research foundations, and our research approaches. Then in this chapter we give a detailed discussion about our research approach that utilizes structured argument to build four research modules: argument generation from stakeholders’ objectives and preferences; argument exchange that helps stakeholders share their arguments based on the logic flow of argument components, argument analysis to understand the relationship between the arguments, and argument evaluation to aggregate individual preference to a collective choice. A presentation of a collaborative negotiation process based on the interaction of these modules follows. This process helps stakeholders generate structured arguments based on organized objectives/perspectives and then guides the stakeholders through negotiation activities where their arguments can be exchanged, analysed, refined and finally evaluated based on the level of objectives achievement for a best choice. Finally, a summary of this chapter is given to emphasize the essence of this research work and outline the necessity of a case study as empirical validation. 65 4.2 Line of Argument The preceding chapters have produced a line of argument as illustrated in Figure 5. Structured arguments for Collaborative Negotiation of Group Decisions in Software Design Group decisions: need to organize and integrate objectives/preferences Improve the effectiveness of collaborative negotiation for group decision in modern software design Collaborative negotiation: need to structure arguments on the semantic level Background Goal Structuring stakeholders’ negotiation arguments on the semantic level based on organized objectives and preferences will reduce the time of collaborative negotiation and increase the commonality of objectives during the collaborative negotiation process Software design: group decision activities across disciplinary, geographical, and temporal boundaries Hypothesis Approach Synthesize argument structure with a value-focused objective hierarchy to overcome the challenges in existing practices Argument generation – based on objectives/perspectives Argument exchange – information flow between stakeholders Argument analysis – to understand arguments relationship Argument evaluation – to select the most preferred by team A negotiation process – interaction of above four modules Line of Argument Approach Structured arguments: need to structure arguments with organized objectives/preferences and develop models that utilize these structured arguments Figure 5: Line of Argument The background for the work was based on the fact that the need of structuring argument to support collaborative negotiation of support group decision in software design. We first recognize that modern software design is a group decision making process that is often carried by a team of stakeholders that are across geography, disciplinary and temporal boundaries. One of the practical challenges to support making such group decision to identify, organize and integrate the competing objectives and varying 66 preferences from the distributed, cross-disciplinary and asynchronous stakeholders. Existing approaches of supporting group decision cannot satisfy these challenges posed by modern software design. In this work we also recognize that effective collaborative negotiation is a key issue in integrating stakeholders’ objectives and making rational group decisions, but it has not been well understood and supported either theoretically or practically in software design domain. To satisfy the need of supporting collaborative negotiation for group decision-making in a distributed and asynchronous workspace like modern software design, one of the critical requirements is to structure the negotiation arguments from multiple decision makers for multiple decision tasks, in order to make sure that all stakeholders have a common ground for reference and can effectively make rational group decisions for software design process. By utilizing these structured arguments, systematic models are built to generate, exchange, analyze and evaluate the negotiation arguments. These models can facilitate stakeholder to effectively carry out a collaborative negotiation process in software design. The goal of this work was therefore to improve the effectiveness of collaborative negotiation of group decision in software design by developing a new framework that can organize stakeholders’ objectives and preferences during the collaborative negotiation process, and structure the negotiation arguments with these objectives and preference. Our hypothesis was that by structuring stakeholders’ negotiation arguments based on organized objectives and preferences, the design team will improve their efficiency by decreasing the time spent for negotiating on design tasks before reach consensual 67 agreements, and enhance the quality of their collaborative negotiation by increasing the number of their common objectives during the negotiation process. In order to achieve this goal, we have discussed two research foundations on which our research approach will be developed. The first foundation is structured arguments defined by Toulmin’s argument structure. The process of preparing the argument components is an objective way to organize and convey stakeholders’ opinions and understandings. And the logic flow between these components inherently helps us specify the order of exchanging perspectives in the collaborative negotiation between the multiple stakeholders. However, existing research work in argument-based negotiation processes is lack of systematic guide to analyze the relationship amongst arguments and evaluate the arguments for the best. In order to resolve this challenge and better represent stakeholders’ understanding, we introduce the second foundation, which is the objective hierarchy built based on value-focused thinking methodology. This hierarchy is proposed by the stakeholders and organizes fundamental and means objectives. It provides a common ground for multiple stakeholders to share their understanding and declare their preferences towards design task proposals regarding how well the objectives are achieved. Based on the two research foundations, we have built our main approach based on a synthesis between the objective hierarchy and the generic argument structure. By synthesizing this objective hierarchy with the structured arguments, this approach specifies how the arguments can be generated based on stakeholders’ proposals, objectives, attributes (i.e., measurement criteria of objectives) and preferences, and then 68 exchanged amongst multiple stakeholders. This synthesis also provides a feasible way of analyzing the relationship of the arguments and evaluating the arguments based on the level of the objectives achievement. Based on these foundations, our approach also utilizes structured arguments synthesized with the objective hierarchy to devise a collaborative negotiation process to support group decision in software design. The next section discusses the details of the four modules in our research approach including argument generation, argument exchange, argument analysis and argument evaluation, followed by a section that specifies the details of the negotiation process based on the interaction of the aforementioned four modules. 4.3 Overview of the Approach The main focus of this work is to build a collaborative negotiation approach based on structured argument to support collaborative software design. A ‘big picture’ of this research approach in relation to the design process (a set of design task proposals) is illustrated in Figure 6. 69 Design Task 1 Design Task 2 Design Task 3 Proposal Development Argument Generation and Exchange Argument Analysis and Evaluation Requirements Specification/Goals Objective Hierarchy Objective Hierarchy Refinement and Reformulation Objective Hierarchy Refinement and Reformulation Proposal Development Argument Generation and Exchange Argument Analysis and Evaluation Proposal Development Argument Generation and Exchange Argument Analysis and Evaluation Figure 6: Argument-based Negotiation Approach in Relations to Design Tasks The software design phase starts with the requirements specification and the goals given by the organization. The stakeholders begin to develop design task proposal for the first task based on current state of team agreement (e.g., common understanding of organization goals) and also built the objective hierarchy by applying the value-focused thinking methodology. The objective hierarchy includes fundamental objectives for which the task is undertaken, the mean objectives which helps achieve the fundamental, and the attributes measuring the achievement of the objectives. Based on the objective hierarchy, stakeholders can declare their perspectives regarding how well the objective is achieved by the proposals in terms of the attribute value. If there are conflicts, the 70 arguments are generated based on the task proposals, objectives and perspectives. Generated arguments are exchanged and that may lead to a revision of the proposals (action, object and resource) and a re-generation of the arguments (including changes in proposals, objectives, attributes and measurement results). After exchanging the argument, if there is no single argument commonly accepted by the team, an argument analysis will take place to help the team better understand the relationship and evolution (i.e., the change after each iteration of negotiation processes) of the arguments. The argument analysis may lead to a revision of the proposals and a re-generation of the arguments as well. At the end, if there is still none of arguments commonly accepted, all the arguments will be evaluated by aggregating stakeholders’ preferences for an optimal choice or a well-informed team leader will make the choice. Within each design task, there is an iterative loop between proposal development, argument generation/exchange and argument analysis/evaluation. The loop goes on when the design team opts to continue negotiating conflicting proposals for this design task and improving the common understanding of that. The loop ends when an argument is either commonly accepted or chosen by an evaluation function (i.e. group preference aggregation) or a team leader. The objective hierarchy built in one design task is always carried on to the next task where it is more refined and reformulated. The loop is the main part of the negotiation process in our approach, which focuses on the generation, exchange, analysis and evaluation of structured arguments. Figure 7 is a sequence diagram that shows the interaction amongst argument generation, argument exchange, 71 argument analysis and argument evaluation, as shown in steps (1) to (14). It is from another point of view looking at the major modules of the negotiation process. Argument Generation (1) Task Proposal on Current Agreement Argument Exchange (2) Exchange Claim and Data (3) Identify Fundamental/ Means Objectives (4) Exchange Warrant (5) Declare Attributes For Each Objective (6) Exchange Backing (7) Argument Refinement (11) Measurement Using Own/Group Value Model Argument Analysis Objective Composition Cluster Attribute Value Cluster (12) Exchange Qualifier & Rebuttal (13) Argument Refinement Argument Evaluation (8) Assign Objectives Weights (9) Score Argument (10) Preferences Aggregation (14) Argument Ranking If not accepted If not accepted If not accepted If not accepted If accepted If no refinement (e) (a) (b) (d) (c) Figure 7: Interaction of Argument Generation, Argument Exchange, Argument Analysis and Argument Evaluation As shown in Figure 7, task proposals on current state of agreement will be exchanged as claim and data (Steps 1 and 2). Fundamental and means objectives will identified and exchanged as warrant (Steps 3 and 4). Attributes for these objectives will be declared and exchanged as backings (Steps 5 and 6). When all the objectives and attributes are proposed, stakeholders can also choose to refine their arguments (Step 7) based on the new mutual understandings of each other’s expectations and measurement criteria and/or the results of the argument analysis in the composition of objectives/attributes as shown 72 in Link (a). Measurement results of the arguments against stakeholder’s own and group objectives will be developed by aggregating stakeholders’ preferences with objectives’ weights (Steps 8, 9, and 10) and then exchanged as qualifiers and rebuttals (Steps 11 and 12). All the exchanges will take place if the previous argument component is not accepted, otherwise the team will proceed to the end of the whole process as shown in Link (b). Based on the results of argument analysis in attribute value (i.e. either natural attributes value or stakeholders’ preferences) as shown in Link (c), stakeholders can refine their argument (Step 13). All the argument refinement can be changes in the design task proposals and/or stakeholders’ objectives, as shown in Links (d) and (e). If the group cannot commonly accept one argument, all argument will be ranked based on their measurement results and the one with highest score will be recommended to the group leader (Step 14). The following sections will explain the details of these four modules. 4.3.1 Argument Generation and Exchange In generating stakeholders’ negotiation arguments, claims and data are collected from the baseline process including design tasks proposals and states of agreement. Warrant, backing, qualifiers and rebuttals are obtained from the objective hierarchy and stakeholders’ perspective models. Based on the Toulmin definition of structure, the claim is the proposal of the argument. In our approach, it is how a stakeholder proposes to implement the design task in terms of the sequence of the actions/objects. The data consists of the initial state of the task – the joint agreement achieved by the design team 73 before they work on this task. The warrant is the set of the objectives that the team wants to achieve from this task based on the initial state. Therefore, the data actually validates the feasibility and applicability of the claim and the warrant justifies this validation between the data and the claim. Backing is the attributes of each objective that further explain the objectives by describing their measurement criteria and then validate the relationship amongst the objectives, the proposal and the current state of agreement. Qualifier and rebuttal are actually the measurement results regarding the accomplishment of the objectives by one’s own and other’s proposals respectively, so that one’s own measurement result (qualifier) can indicate the degree of desire of the stakeholder for the proposal while measurement results for other’s proposals (rebuttal) describe the chances that could fail the proposal (claim). To build a negotiation argument in this way, stakeholders will have a better understanding of each other because they share not only their claims but also their underlining reasons and desires (e.g., perspectives). Figure 8 shows a detailed process of generating and exchanging the arguments. 74 Task Proposal on Current Agreement (Data & Claim) Identify Fundamental/Means Objectives (Warrant) Declare Attributes for Each Objective (Backing) Argument Analysis Proposal Measurement Using Own/Group Value Model (Qualifier and Rebuttal) Architecture Component Data structure Interface Constraints Software Design Features If not accepted Exchange Claim and Data Exchange Warrant Exchange Backing Argument Refinement Exchange Qualifier and Rebuttal Cost Argument Evaluation for Ranking If not accepted If not accepted If not accepted Attributes for Architecture, Component, Data Structure, Interface, Cost Attributes Subjective to Knowledge and Opinions: Support vs. Opposition Argument Generation And Exchange Argument Refinement Objective Composition Clusters Attributes Value Clusters (a) (d) (c) (b) refine Figure 8: Argument Generation and Exchange If the claim and the data are not accepted by other stakeholders who jointly work with her on this task, then the stakeholder begins to identify the objectives which the task proposal will accomplish and present them as the warrant to others. The objectives may come from general software design features including architecture, data structure, programming interface, application constraints and cost, as shown in Link (a). If the claim, data and warrant are not accepted by others, then the attributes of those objectives (in warrant) will be declared and presented as backings to describe the measurement criteria of the objectives and confirm the proposal can accomplish the 75 objectives by achieving good performance on each attribute. As explained by Link (b), the attributes can involve either quantitative measure (e.g., general performance predicts for the software design) or qualitative measure (e.g., support or opposition to the proposal based on the how well it accomplishes the objectives). Up to this point, after all the stakeholders present their claims, data, warrants and backings, the initial objectives hierarchy (including all fundamental/means objectives and attributes) for the entire group has been set up. If the claim, data, warrant and backing are not accepted by others, the stakeholder has a chance to refine her proposal (claim) based on the group objectives hierarchy and follow the aforementioned procedure from the beginning. In this step, as shown in Link (c), argument analysis based on the composition of the arguments (objectives and attributes) will be applied to indicate the similarity and distance amongst these arguments in their objectives/attributes and the analysis results will be provided to the stakeholder for better understanding of the arguments relationship and recommendations to refine the arguments. If the stakeholder chooses not to refine her claim or the refined version (claim, data, warrant and backing) is still not accepted, the stakeholder will give the qualifier that is, instead of solely subjective opinion, the measurement result regarding how well her proposal accomplishes the objectives given in her warrant, indicating the degree of her desire for the proposal. The rebuttal should be also given in this step, as the measurement result regarding how well her proposal accomplishes the objectives proposed by team, 76 indicating the probability of this proposal can be rejected if this result is rather unsatisfactory. To get the measurement result based on the measurement criteria (attributes), value functions are developed to transform the unit score for each measurement criteria into “value unit” on a scale of 0 to 1 [Kirkwood 1997]. Different weights will also be given to the objectives, where these measurement criteria (attributes) belong, to indicate the varying degree of their importance. A sum of the unit score for measurement criteria for one alternative, with consideration of the unique weights of these criteria, will be assigned to the alternative as the total measurement result. When all the involved stakeholders give qualifier and rebuttal for their proposals, they should be offered one more chance to refine the proposal based on the measurement results and then follow the steps. Here argument analysis based on the attribute value of the arguments will be applied again, as shown in Link (d), to indicate the similarity and distance amongst these arguments in the regard of their measurement results and the results will be provided to the stakeholder for recommendations to refine the arguments. If the stakeholder chooses not to refine her claim or the refined version is still not accepted, all the arguments will be evaluated and ranked. The argument with best evaluation result will be recommended to the design team. 77 4.3.2 Argument Analysis Toulmin proposed his argument structure informally yet provided no guidance in our knowledge as to understand the relationship between arguments or evaluate the best argument to identify implausible ones. Part of the focus of this dissertation is to better understand the relationship of arguments and how the evolution of this relationship indicates the quality of the negotiation by our approach. This task becomes feasible in our approach based on more concrete meanings given to each argument component based on its synthesis with the objective hierarchy. For this task, we can cluster arguments based on either their attribute value or the composition of objectives and attributes in the argument and then understand the relationship between arguments using clustering analysis. The composition of the objectives and attributes shows what value the argument tries to accomplish (what the stakeholder ‘cares’). Attribute value shows how well the argument accomplishes the objectives. They help understand why the argument is generated and how it works. In both cases, each attribute acts as a dimension in the clustering analysis functions. The value in these dimensions is either determined by whether the corresponding attribute has been declared by the stakeholder in the argument, or assigned by the stakeholders (i.e. the attribute value). Then this analysis result can be provided to the stakeholders for better understanding of arguments’ relationship and recommendation to refine and arguments. First, we define the similarity of two arguments (e.g., A i and A j ) as the “distance” d ij . In the case of analyzing the arguments based on attributes values, if d ij equals 0, it means 78 two arguments are compatible. In the case of analyzing the arguments based on their compositions in objectives and attributes, if d ij equals 0, it means two arguments are structurally equivalent in ‘value.’ Definition 1: Arguments A i and A j , in an analysis of the compositions in objectives/attributes, are structurally equivalent if o ik = o jk and a ikm = a jkm , for g k ,..., 2 , 1 = (where g is the total number of objectives of A i ), m= 1,2,…,h (where h represents the total number of attributes of objective o ik ) where o ik is k th objective in the warrant of A i , and o jk is k th objective in the warrant of A j , a ikm is the m th attribute that belongs to o ik and is in backing of A i, and a jkm is the m th attribute that belongs to o jk and is in backing of A j Definition 2: Arguments A i and A j , in an analysis of attribute value, are compatible if o ik = o jk and a ikm = a jkm and V(a ikm ) = V(a jkm ), where o ik is k th objective in the warrant of A i , and o jk is k th objective in the warrant of A j , a ikm is the m th attribute that belongs to o ik and is in backing of A i , and o jk is k th objective in the warrant of A j , V(a ikm ) is the value of m th attribute in backing of A i and belongs to o ik , and V(a jkm ) is the value of m th attribute in backing of A j and belongs to o jk Since structural equivalence is a mathematical property, which is seldom actually realized in collected arguments, we use power distance and single linkage (i.e., nearest neighbor), assuming the stakeholders in the design team tend to cluster together, to calculate the 79 distance between arguments and string ‘similar’ arguments into clusters to build a cluster tree. Single linkage (nearest neighbor) in our work takes the distance between two clusters as the distance of the two closest arguments (nearest neighbors) in the different clusters and then string closest clusters to form new clusters. Power distance is usually used to calculate the distance between respective arguments based on dimensions with various weights. Definition 3: For arguments A i and A j , for each attribute a m (m = 1,2,…, h where h is the total number of attributes of A i ) in A i, the distance between the two arguments is defined as: distance(A i ,A j ) = sum ( |x im – y jm | p ) where m = 1, 2, …h p is the weight of individual dimensions, i.e. the importance of each attribute. in the case of analysis based on attribute value, x im is the value of a m in A i , y jm is the value of a m in A j ; if a m does not exist in A j , both x im and y jm are 0, in the case of analysis based on objectives/attributes composition x im is either 1 if attribute a m exists in A i , or 0 if otherwise y jm is either 1 if attribute a m exists in A j , or 0 if otherwise Next, we will apply the hierarchical clustering that is a data analysis technique and suited for partitioning the arguments into sub-classes. It groups entities into subsets, so that entities within a subset are relatively similar to each other. The hierarchical clustering 80 procedures can be divided into two different approaches: agglomerative and divisive. The agglomerative approach to cluster analysis, used by the nearest and farthest neighbor algorithms, is a bottom-up clumping approach where we begin with n singleton clusters and successively merge clusters to produce the other ones. The minimal spanning tree, on the other hand, uses the divisive approach, which is a top-down approach to successively split clusters to produce others. In the group decision of software design, the bottom-up clumping approach is more suitable to analyze the relationship amongst various arguments from the team members. There are three algorithms using the bottom-up clumping approach: nearest neighbor, furthest neighbor, and average linkage. The difference lies what to base on to group the objects: the smallest distance (nearest neighbor), the largest distance (furthest neighbor) or the average distance (average linkage). Since stakeholders tend to group with the partners who have ‘identical’ or ‘compatible’ arguments, we use nearest neighbor to string ‘similar’ arguments into clusters to build a cluster tree. Nearest neighbor algorithm, also called single linkage, uses the smallest distance between objects in the two teams. The distance between two clusters r and s is the similarity between two arguments. It is defined as: ) ,..., 1 ( ), ,..., 1 ( ), min( ) , ( , s r j i n j n i d s r d ∈ ∈ = 81 The computational method of nearest-neighbor clustering algorithm is: Initialize C: C’=n; D(i)={x(i)}; I = 1,…,n while(C’<>C) { C’=C’-1; find nearest clusters D(i) and D(j); D’={D(i), D(j)}; } return C clusters Nearest neighbor in our work takes the distance between two clusters as the distance of the two closest arguments (as ‘nearest neighbors’) in the different clusters and then string closest clusters to form new clusters. In general, a cluster tree with simple structure and fewer levels of clusters means all of the arguments are either more compatible or more structurally identical. Considering the simplest case, a cluster tree with only one level indicates all arguments have equal distance from each other. Then, stakeholders can have a better understanding when looking at the clustering results of this argument analysis. The height of the cluster indicates the distances (similarity) among the arguments. If most of the arguments are within a cluster having short height, they are quite compatible with each other. In this case, the short branch illustrates a group of ‘similar’ arguments which may be an indicator of ‘merging the arguments’ to the design team. On the other hand, if the clustering levels are high, the arguments are quite distant from each other. It denotes that 82 the stakeholders’ objectives and perspectives are not well shared or it is vastly different in how well the arguments achieve the objectives. And thus it may be an indicator of ‘choosing one argument that is most preferred’ instead of ‘merging the arguments.’ Based on this discussion, we define the similarity of the arguments in a cluster tree as its weighted average height. Definition 4: For a given clustering tree for a group of arguments, the weighted average of the height of the cluster levels is defined as: ∑ ∑ = = = l i i l i i i n h n h ˆ 1 1 (1) h i is the height of cluster level i n i is the number of all arguments in cluster level The average height indicates the similarity/distance of arguments in a clustering tree and thus a decrease (or increase) of the average height of the clustering tree indicates the better (or worse) quality of a negotiation process and less (more) incompatibility of arguments. In more details, a composition-based cluster tree with simple structure and few levels means all of the stakeholders have similar objectives and attributes in their arguments – they want similar value when working on the same task for that they probably just have different proposals to achieve the value. If composition-based cluster tree tends to have 83 simpler structure and fewer levels after each iteration of negotiation processes, it indicates the less incompatibility of stakeholders’ objectives and better quality of the negotiation process. Similarly, an attribute-value-based cluster tree with simple structure and few levels means the arguments have similar achievement of objectives. If this cluster tree tends to have simpler structure and fewer levels after each iteration of negotiation processes, it also indicates the less incompatibility of arguments and better quality of the negotiation process. In addition, we can recommend the design team to refine similar arguments in the same cluster, including the proposals, objectives and perspectives, or rearrange the design task or design team to control the similarity of the arguments. More details of these recommendations will be given in Section 4.4. 4.3.3 Argument Evaluation The last central module of this approach is the argument evaluation. At the end of negotiation, if all the stakeholders can jointly agree on a particular implementation, they can take that as the ultimate solution of the negotiation. Otherwise, all the arguments must be carefully evaluated for resolutions. This evaluation is comparison of the performance of a potential solution to the design objectives in order to see how well it fulfills these objectives. The evaluation determines the preference order of the arguments and forms a basis for recommendation to stakeholders’ decisions about what argument to choose. 84 Considering various stakeholders objectives and the difference in these objectives, a model for measuring performances on a common scale, weighting the criteria, and aggregating it all into a common measure of ‘good’ arguments seems necessary. However, the question of whether or not such an aggregation is fruitful is often raised. Some argue that the worth of an option is not a simple additive function of the worth of its various components. This may be true, but addition still seems to be the most suitable way to determine the overall worth, mainly because it is easy to understand and implement. Others argue that the aggregation hides value judgments and conceals details. To this last argument we have to argue that the contrary is true. Since the aggregation requires explicit formulation of measurements and value judgments, it serves to better document the negotiation and decisions makings, and to promote discussion and common understanding about the design problems. In this way it may help define the general nature and context of negotiations and thus promote their qualities. The process of assigning weights and scoring arguments may also help to identify weak points where the design needs to be improved. Software design involves evaluation with respect to both quantitative (e.g., general performance predict) and qualitative criteria (e.g., stakeholder preferences). Both kinds are important factors in determining the evaluation results. The quantitative attributes of a system may be easily measured numerically therefore in existing literature it has been given much attention. Instead, the qualitative criteria can only be measured on a relatively coarse scale because humans have a limited vocabulary to describe qualities. 85 Examples of such qualitative scales were given, e.g. excellent, good, fair unsatisfactory, poor. Also, in the early design phase there is much uncertainty and there is not enough data available to justify very detailed performance assessments. Therefore, it seems most worthwhile to base the evaluation on a fairly coarse scale. To build this method, we have investigated the field of multiple objective decision evaluation (also known as Multi-Criteria Decision Making, MCDM). The focus of MCDM is usually on a single decision-maker who unilaterally chooses between alternatives whose outcomes differ on two or more “objectives” [Yoon and Hwang 1995]. In MCDM there has been a variety of strategies developed to calculate the best choice for a decision-making problem. Optimization selects the first alternative with the best value for single criterion or aggregate measure. It also establishes a set of constraints, which are the requirements on the alternatives. Depending on the decision-making problem, different optimization techniques can be applied: linear programming, nonlinear programming, discrete optimization, etc. [Nemhauser et al. 1989]. Multiple criteria optimization applies for the case of having a finite number of criteria and an infinite number of the feasible alternatives (meeting the requirements) or finite number of feasible alternatives but only in implicit form [Steuer 1986]. Satisficing strategy eliminates all alternatives that fail on one constraint (involving one or more criteria) and then consider another constraint. Mixed scanning strategy decomposes the overall decision objective into a hierarchic structure of criteria and alternatives and builds a pair- 86 wise comparison matrix to calculate relative weights of the criteria, which are then used to arrive at a score for each alternative [Etzioni 1967]. Another strategy, Analytical Hierarchy Process (AHP) a pairwise comparison method developed by Saaty [1980] to assign quantitative weights to criteria and scores to alternatives. Statistical technique, also known as simple additive weighting (deterministic value function), ranks the alternatives from best to worst and assume one can characterize the consequences of each alternative with certainty or that only their expected value matters. Utility function methods are similar to deterministic value function methods, except that they consider uncertainty in the criteria values for each alternative [Anderson et al. 2002]. Because it is important that our approach can be used in the early software design phase, the evaluation approach should be quick and simple to use. Our review in multi-criteria evaluation approaches suggested that some sort of Simple Additive Weighting (SAW) approach would be most appropriate. Various applications of SAW approaches have been found in group decision making approaches and support tools. The SAW model appeared to be the most simple and intuitive of the evaluation approaches. Also most of the evaluation approaches that were found to be applied within the software design domain, used some variations of the SAW model. The other models that were found, were either mathematically complicated, very time-consuming to use, or required input data that would be almost impossible to find. The main advantage of the SAW approach is that it makes value judgments explicit, thereby acting to increase mutual understanding among the design team participants about what is important to focus on. It was also important 87 that both qualitative and quantitative values could be incorporated in the model for we set up the link between the verbal and numeric scales for qualitative values and normalize the quantitative value into the common numeric scales with qualitative ones. Based on a well-designed evaluation process including scale defining, score collecting and weight trade-off, the performance score and criteria weight will be aggregated into an overall evaluation result for a specific argument. As such, the evaluation process is separated into three steps: 1. Define measurement scales The value units of each attribute are transformed into a common scale. 2. Assign objective weights The objectives are weighted based on the chosen scales. 3. Score argument The arguments are scored based on the attribute values calculated for natural attributes or declared as stakeholders’ perspectives. 4. Preference aggregation The weights and the scores are aggregated to obtain an overall evaluation of each alternative. The details of each step in this process will be explained below: Step 1: Define measurement scales 88 In order to sum the individual measurements for a total score, the value derived from all the evaluation measures (i.e., attributes defined in the objective hierarchy) need to be on same measurement scale (e.g., 1-10). If the attributes are quantitative, we can normalize the attribute value to be within a pre-defined scale. If the attributes are qualitative, they are usually in different units and measured on different scales, it is impossible to sum the individual measurements to obtain a total score. To solve this problem, we have to provide detailed guidance to transform the units of each evaluation measure into “value units” on a common scale which is discussed below. Imagine an evaluation measure that has its worst possible score at x and its best possible score at y and the values associated with these extremes are 1 and 10, respectively. To determine intermediate value units of the attributes for alternatives that score between the extremes, various methods have been suggested in previous literature, in which, arguably, the easiest and most feasible method is direct assessment, where the decision makers uses her judgment and experience to provide value units associated with each alternative’s evaluation measure. The common measurement scale need not have a large number of intervals. This is because fine gradations do not make sense for the qualitative criteria; these criteria can only be described verbally, and stakeholders have a limited vocabulary to express qualitative gradations. In addition, it does not make sense to have a very fine scale because of the large uncertainties of the performance predictions in the early design 89 phase. A 9- or 10-point scale seems to be the maximum usable gradation. When measuring the attribute value, we have 5 main terms to express the relative individual measurement of one attribute in terms of support or opposition, see Table 1. The qualitative criteria are rated directly on this scale. The categories in between can be used when the decision-maker hesitates between the neighboring qualifications. Relative judgment Direct judgment 10 Strong support Excellent 9 8 Support Good 7 6 Neutrality/Controversy Fair or unknown 5 4 Opposition Unsatisfactory 3 2 Strong opposition Poor 1 Table 1: Verbal Terms to Express Relative Individual Measurement of One Attribute Step 2: Assign objective weights Weights are indicators of the influence of the individual objective on the decision. The objective weights will be populated to the weights of its attributes. In fact, the weight assigned to an attribute is essentially a scaling factor which relates the value on that attribute on all other attributes. For example, if an attribute A has a weight which is twice that of attribute B, this should be interpreted that the decision-maker values 10 value points on attribute A the same as 20 value points on attribute B and would be willing to trade one for the other. Or put another way, the decision-maker should be indifferent to a trade between 1 unit of A and 2 units of B. Thus, consistency checks may have to be 90 made using such trade-off questions or indifference questions after the weights are assigned to all the attributes. The most simple and intuitive of the weighting approaches, i.e., the ranking (i.e., listing all the objectives in order of importance) or direct rating (rate all the objectives independently using a pre-defined importance scale), all rely on the notion of attribute importance. This concept has been criticized because it assumes that decision-makers are able to appropriately adjust the importance judgments in relation to the scales of the attribute value, such as the attribute ‘a reduce cost of network storage’ is certainly more important if it can be over the range of 0 to millions of dollars than that of 0 to hundreds of dollars. Von Winterfeldt and Edwards states that the evidence on whether or not people can appropriately adjust the importance judgments, is mixed [1986]. However, they also suggest extra caution can be used to appropriately apply intuitive weighting approaches, i.e. elicit the natural weights that the decision-makers have in mind in the early stage and make sure the decision-makers know what it is and understand the relation between value ranges and weights. In our approach, we will provide a pre-defined importance scale (as shown in Table 2) for stakeholders to independently rate all objectives. The subtle idea, that a weight of an 91 attribute is actually a scaling factor that relates scores on that attribute to scores on all other attributes, should be well explained to the stakeholders in the negotiation process. Relative judgment Direct judgment 10 Very important Excellent 9 8 Somewhat more important Good 7 6 Important Fair or unknown 5 4 Somewhat less important Unsatisfactory 3 2 Very less important Poor 1 Table 2: Verbal Terms to Express the Relative Importance of an Objective Step 3: Argument scoring Once the arguments to be evaluated are known, data needs to be collected in order to score each argument in relation to the evaluation measures, i.e. each attribute in the objectives hierarchy. In case of a natural attribute, the data may be a quantitative value. In case of a ‘support v.s. opposition’ attribute, the data may be a qualitative value derived from the stakeholder’s perspectives. In fact, to determine score of the qualitative attributes for alternatives, various methods have been suggested in previous literature, in which, arguably, the easiest and most feasible method is direct assessment, where the decision makers uses her judgment and experience to provide value units associated with each alternative’s evaluation measure. Step 4: Aggregation 92 In this step, the data collected from Steps 2 (objectives weights) and 3 (argument score) are used to form an overall aggregation function to rank evaluation results in a manner consistent with the decision-maker’s preferences for the arguments. As discussed earlier in this section, the simple additive weighting (SAW) model is proposed to aggregate the performance grades and the objective weights. The calculation of final score for an alternative is defined as follows: Definition 5: for a given alternative A j f j = ∑ = m i ij ig c 1 , j = 1 …n (2) where f j is the final score for alternative A j , n is the number of alternatives, m is the number of criteria, c i is the normalized weight of the attribute c i g ij is the performance grade (score) for alternative A j with respect to attribute c i . This approach uses simple additive weighting function to obtain an overall performance for each arguments against the objectives proposed by the design team. The calculations needed are so simple that the design meetings in software team can easily produce the result without too much interruption. With the result of these calculations, stakeholders can have a better understanding of the preference order of the arguments, which get the 93 design team or the manager better informed when making a choice amongst the negotiating arguments. 4.4 A Collaborative Negotiation Process 4.4.1 Overview A collaborative negotiation process based on the four modules discussed in Section 4.3 is the main strength of this approach. It adapts the theoretical foundation and practical steps of the socio-technical co-construction process and utilize the structured arguments that are synthesized with the objective hierarchy. This process describes how the stakeholders prepare structured arguments based on the objective hierarchy, and then carry out the negotiation process in which these arguments are exchanged, analyzed and evaluated. Figure 9 below illustrates this integrated approach, which has three inter-related phases. 94 Post-Negotiation Analysis Phase Step 1: Identify Stakeholders Step 2: Design Baseline Process Step 3: Check Differences Step 4: Propose Concept Structure Step 5: Declare Perspective Step 7: Argument Evaluation Step 6.a: Argument Generation & Exchange Step 8:Proceed to Next Phase Check other tasks No conflict Pre-Negotiation Modeling Phase Negotiation Phase conflict Step 6.b: Argument Analysis Figure 9: A Negotiation Approach for Collaborative Software Design First, the Pre-negotiation phase starts with identifying a set of “stakeholders” who have an interest in the outcome of the co-construction process, in which they will directly or indirectly participate in the baseline process of software design. This approach also needs to employ a baseline process that captures the required technical task-work in a predetermined order. For example, it can be a commonly accepted software design “workflow” suggested by the domain experts or standard operating procedures instituted by the company. The team starts the design process, asks each stakeholder to propose an implementation for specific tasks in the design process, and checks the differences (i.e. conflict) between implementation proposals. Then it determines whether to initiate a negotiation by identifying conflicting design proposals. 95 Second, the Negotiation phase helps the stakeholders prepare for their arguments, and guides them into a negotiation process using the prepared arguments. In this phase the stakeholders jointly propose an objective hierarchy and declare their perspectives (e.g., preferences) based on the objectives. Then based on the design tasks, objectives and perspectives, the stakeholders are guided to systematically generate and exchange their negotiation arguments. If no argument is commonly accepted at the end of exchanging the arguments, the argument analysis will take place in two folds: based on the composition of objectives/attributes in the arguments and based on the attributes value of the arguments. In addition, the argument analysis may lead to a revision of the objectives, perspectives and proposals, or a re-generation of the arguments. Finally, if there is still none of arguments commonly accepted, all the arguments will be evaluated by aggregating stakeholders’ preferences and ranked to recommend an optimal choice or a well-informed team leader will make the choice based on the evaluation results. Lastly, the Post-negotiation phase with only one step assures that the stakeholders obtain a commonly accepted software design implementation and proceed to next phase of the software development life cycle. The following subsections explain these three phases in more details. 96 4.4.2 Pre-Negotiation: Task Proposal Modeling and Conflict Identification The goal of the pre-negotiation phase is to identify all potential conflicts by checking the differences between task implementation proposals, and help the stakeholders decide whether to negotiate on the identified conflicts. For example, in a common software design task “define quality attributes,” team members often have different views. Product managers may suggest performance and usability as the most important attributes, while engineers may argue that security and maintainability is most important for the long run. Meanwhile, engineering managers may believe that portability is critical, due to possible future options to migrate the software to a variety of the operating platforms. We will use this example to explain how to capture and organize relevant information in this section. There are five specific steps in this pre-negotiation phase of our collaborative negotiation approach. Step 1 – Identify the “stakeholders” who participate in the software design team. Stakeholders are those software design team members who have an interest in the process and/or outcomes of the software design decisions (i.e., implementations) and may directly or indirectly participate in the negotiation process. Step 2 – Prescribe a “baseline software design process” A baseline software design process is defined as a series of necessary technical task-work that must be undertaken by the team to develop a software design solution. Our approach takes this design process as the baseline to begin the software design process. This 97 process and its associated standard design task-works are generally pre-defined based on the domain practices or chosen for the stakeholders by management, e.g., Objective Oriented Design Process or Layer based Design Process, which comes with a set of standard procedures. Also in this step, the software corporation must set up common goals for the software design team to achieve and define a set of shared values that all team members follow during the design activities. Goals and values set the direction for the design team to identify and define their decision objectives in the negotiation phase. Step 3 – Ask stakeholders to implement the above design tasks and check the difference in their implementation details. Although stakeholders jointly work on the design tasks according to the baseline process prescribed above, due to their divergent background, interest, experience, and expertise, they will undoubtedly come up with different technical decisions when proposing implementations for these tasks. In our approach, the implementation proposal of a software design task is defined as a logical sequence of actions/objects, combined with necessary resources including time and staff. For example, regarding the example software design task “define quality attributes,” a possible implementation proposal can be specified as: {Objects: performance, security and usability; actions: define performance, security and usability as quality attributes according to the functional requirements; resources: the design team work for one day.} 98 Therefore, if there are different decisions, objects or resources in the stakeholders’ proposals for a specific design task, the team will declare a decision conflict. A typical conflict could be, for example, that the stakeholders are choosing different objects. Just like the example mentioned at the beginning of this section, the objects (i.e., quality attributes) for the task are different amongst all the stakeholders. In case of a conflict, the process will continue to the Negotiation phase next to generate, exchange, analyze and evaluate the arguments for conflict resolution. Otherwise (i.e., no conflicting decision), the process will move forward directly to the Post-Negotiation phase (see Section 4.4.4) with mutual agreements on how to implement all design tasks. 4.4.3 Negotiation: Utilizing Structured Arguments Synthesized with Objective Hierarchy In this phase, the participating stakeholders are guided to negotiate with each other based on structured arguments until a mutual agreement is reached. In most of the existing practices, an argument-based negotiation process is generally undertaken according to the following two stages: • Stakeholders generate argument claims (or counter claims) for concerned issues and provide supporting data, and • Stakeholders exchange and respond to others’ claims (or counter claims) and their associated supporting data [Sierra et al. 1998] However, these practices have not resolved the challenges of analyzing how arguments relate to other arguments for refining conflicting arguments or how to evaluate the 99 arguments for the best [Zeleznikow 2002]. In addition, most of them provide little guidance in how to generate the arguments based on the decision making process. As mentioned, what is in need is an operational negotiation process built based on the arguments to support group decision making. The novelty of our approach is to resolve this challenge by utilizing the structured arguments to design a collaborative negotiation process referencing the socio-technical co-construction process, where these arguments can be generated, exchanged, analyzed and evaluated based on the synthesis between the structured arguments and the objective hierarchy. Based on the detailed discussion in Section 4.3 regarding argument generation, exchange, analysis and evaluation, this section discusses the negotiation phase that is based on the interaction of these modules. This phase consists of four steps (4, 5, 6, 7), where Steps 4 and 5 guides the stakeholders to building the objective hierarchy and declare stakeholders’ perspectives. Steps 6 (including 6.a and 6.b) and 7 discuss how the arguments are utilized based on the objectives and perspectives collected in Steps 4 and 5. Step 4 - Propose an “objective hierarchy” for the identified conflicting design task. Negotiating conflicting implementation proposals of a design task in the baseline process indicates some differences in the stakeholders’ objectives and perspectives (i.e., preferences for arguments and objectives). These objectives includes the fundamental objectives (as the values) for which the task is undertaken, the means objectives that helps achieve the fundamental, and the objective’s attributes for measuring the proposal about the degree to which the objective are achieved if the proposal is accepted. These 100 attributes describe the objectives and should have a common interpretation to every stakeholder. If an objective does not have any means objectives or any attributes that are naturally used to interpret the objective (e.g., ‘size of network communication data’ is a so-called natural attribute of the objective ‘limit network bandwidth’), an attribute ‘support vs. opposition’ will be added, based on which stakeholder in later steps can declare their perspective as either support or opposition. Based on the goals defined at the beginning of the design process, the software team should be able to identify objectives and attributes in this step. As discussed earlier, our approach uses an objective hierarchy to organize the objectives and capture their differences. This hierarchy is jointly built by the stakeholders based on their understanding and expectations (“values”) of the design tasks. And the objectives in this hierarchy will be dynamically changed by the social interactions among the stakeholders. In reference to the information in a concept structure, the stakeholders can declare their perspectives regarding how important the concepts (i.e. the weights of the objectives) are and how much each proposal is supported or opposed. The latter will be obtained in the next step based on the values assigned by the stakeholders for the objectives’ attributes. The objective weights are collected in this step after these concepts are declared in the structure. In details, the relative importance of each objective in 1-to- 10 scale according to Table 2 defined in Section 4.3.3: 101 10 = Very important 8 = Somewhat more important 6 = Important 4 = Somewhat less important 2 = Very less important 1 = Lowest importance To get more accurate results, the objective weights were collected for each fundamental and means objective. After the perspectives were collected, the weight of each means objective was adjusted as the average value of its weight and the weight of its corresponding fundamental objective. The importance of an attribute was same as that of its objective. To explain the concept objective hierarchy further, we continue to use the software design task “define quality attributes” as an example. Table 3 below describes this example objective hierarchy, including information about stakeholders, objectives, and attributes. There are three stakeholders in this example: product manager, engineer, and engineering manager. Product manager’s fundamental objectives are to guarantee software performance and usability and the means objective is to beat the quality of video rendered by competitor’s applications (for performance) and build a friendly user interface to handle the video discovery, selection and rendering on mobile device (for usability). An attribute ‘support vs. opposition’ is added for each aforementioned means 102 objective respectively. The engineer, on the other hand, believes that maintainability and security are most important. His means objective is to build a prototype system for estimating the efforts taken to add or remove a module (for maintainability) and for checking the best level of the data encryption that can be achieved (for security). Meanwhile, the fundamental objectives of the third stakeholder, the engineering manager, are portability and development cost. The means objective for portability is to estimate the efforts of migrating the software to a new operating system with the attribute of ‘support vs. opposition’. For development cost, the means objective is to build a project roadmap having the workload estimation and the attribute is the ‘support vs. opposition’. Based on this objective hierarchy, the stakeholders will also declare the objective importance that is shown in Table 4. 103 Stakeholder Fundamental Objectives Means Objectives Attributes Performance Beat the quality of video rendered by competitor’s applications Support vs. opposition Product Manager Usability Build a friendly user interface to handle the video discovery, selection and rendering on mobile device Support vs. opposition Maintainability Build a prototype system for estimating the efforts taken to add or remove a module Support vs. opposition Engineer Security Build a prototype system for checking the best level of the data encryption which can be achieved The maximum number of bits in the encryption algorithm which can be applied Portability Estimate the efforts of migrating the software to a new operation system Support vs. opposition Engineering Manager Development cost Build a project roadmap Support vs. opposition Table 3: An Example Objective Hierarchy for "Define Quality Attributes" 104 Table 4: Importance of Objectives Objectives Product Manager Engineering Manager Engineers Importance Adjusted Importance (for means objective ) Performance 10 8 10 9.3 n/a Performance- beat the quality of video rendering by competitor’s applications 10 8 8 8.6 8.9 Usability 10 8 8 8.6 n/a Usability-build a friendly user interface to handle the video discovery, selection and rendering on mobile device 10 8 6 8 8.3 Maintainability 6 8 10 8 n/a Maintainability- build a prototype system for estimating the efforts taken to add or remove a module 6 6 8 6.6 7.3 Security 6 6 8 6.6 n/a Security-build a prototype system for checking the best level of the encryption 4 6 8 6 6.3 105 which can be achieved Table 4, Continued Portability 8 10 8 8.6 n/a Portability- estimate the efforts of migrating the software to a new operation system 6 8 8 7.2 7.8 Development cost 6 8 6 6.6 n/a Development cost-build a project roadmap 6 10 6 7.2 6.9 Step 5 – Each stakeholder declares ‘perspective’ based on the above objective hierarchy. Once an objective hierarchy is established by the team, each attribute of each objective should be assigned a value before the arguments are evaluated. In case of a natural attribute, the value was generally numeric and calculated by a commonly accepted method based on the individual case, such as average number of the clicks per use case or the total count of supported handset platform. In case of the added attribute ‘support vs. opposition’, each stakeholder can express their own perspectives via social interaction based on their expertise and understanding, and this opinion should describe their position of either supporting or opposing the argument according to how much the objective (to which the attribute belongs) can be accomplished if the argument is accepted. 106 Negotiation, where the stakeholders express and exchange their own opinions about the objectives and attributes, is generally a very complex human phenomenon in teamwork that consists of many inter-related psychological and organizational factors. There is no practical way that a complete analytical modeling of negotiation can be fully developed and incorporated for a group of decision makers. As a result, our approach takes a rather simplified view by focusing on modeling the dynamic impacts of negotiation, i.e. on the evolving “perspectives” of the stakeholders, as they express their opinions toward the objective hierarchy. These dynamically evolving perspectives are declared for the proposed objectives of which the stakeholders have common interests or some expertise. In other words, the perspective dynamically depicts a stakeholder’s perceptions of his or others’ objectives. These perceptions could include the stakeholders’ desire for their ideas to succeed and their support for or disagreement with how well their own or others’ argument can achieve the objectives, either proposed by themselves or others. Therefore, the perspectives indicate the difference in the stakeholders’ perceptions that cause the conflict in the technical proposals of the tasks and put the negotiation into necessity. Moreover, these perspectives will be further analyzed in order to systematically evaluate the arguments in our negotiation approach. Although stakeholder perspectives are often highly subjective in nature, some quantitative methods are needed in order to define the measurement scales of the perspectives and further analyze these perspectives for argument evaluation. In our research, referencing our discussion in Section 4.3.3 regarding defining measurement 107 scale (Table 1), we define a 1-10 measurement scale to quantify the stakeholder’s perspectives of either supporting or opposing the arguments. In other words, when the task proposals were being evaluated, for the “support vs. opposition” attribute, stakeholders declared their perspectives about the proposal’s value based on their expertise and understanding. The perspective will be one of the following: 10 = Strong support, i.e., the proposal will most likely help achieve the objective 8 = Support, i.e., the proposal will likely help achieve the objective 6 = Neutrality (fair, unknown or uninterested), i.e., the proposal may not either contribute to or harm the achievement of the objective, or Controversy, i.e., the proposal may have some effect in achieving the objective, but the decision maker is not clear of what kind of effect it may have; 4 = Opposition i.e., the proposal will likely bring negative effects in achieving the objective. 2 = Strong opposition i.e., the proposal will most likely bring detrimental effects in achieving the objective. 1 = Strongest opposition i.e., the proposal will definitely bring detrimental effects in achieving the objective. For example, in the above “define quality attributes” design task, the proposal from the product manager is to define performance and usability. His proposal will be measured by each stakeholder against every objective attribute. Table 5 describes the measurement 108 results, which, as discussed earlier, are either calculated numeric value in case of a natural attribute or stakeholders’ perspectives in case of a ‘support vs. opposition’ attribute. Product Manager’s Proposal: Define performance and usability for the software product Objectives/Attributes Product Manager Engineer Engineering Manager Average Value Performance-beat the quality of video rendering by competitor’s applications- support vs. opposition Strong support (10) Support (8) Neutrality (6) 0.67 Usability-build a friendly user interface to handle the video discovery, selection and rendering on mobile device - support vs. opposition Support (8) Strong support (10) Neutrality (6) 0.67 Maintainability-build a prototype system for estimating the efforts taken to add or remove a module-support vs. opposition Neutrality (6) Support (8) Controversy (6) 0 Security-build a prototype system for checking the best level of the data encryption which can be achieved-the maximum number of bits in the encryption algorithm which can be applied-support vs. opposition 256 256 256 256 Portability-estimate the efforts of migrating the software to a new operating system- support vs. Opposition Support (8) Neutrality (6) Support (8) 0.67 Development cost-build a project roadmap having the workload estimation-support vs. opposition Controversy (6) Neutrality (6) Support (1) 0 Table 5: Stakeholders’ Perspectives (Attribute Values) for Product Manager’s Proposal 109 Step 6.a – Argument Generation and Exchange In generating the negotiation arguments, claims and data are collected from the baseline process representing technical decisions. Warrant, backing, qualifiers and rebuttals are obtained from the objective hierarchy and stakeholders’ perspective models. Based on the Toulmin definition of structure, the claim is the proposal of the argument. In our approach, it is how a stakeholder proposes to implement the design task in terms of the sequence of the actions/objects. The data consists of the initial state of the task – the joint agreement achieved by the design team before they work on this task. The warrant is the set of the objectives that the team wants to achieve from this task based on the initial state. Therefore, the data actually validates the feasibility and applicability of the claim and the warrant justifies this validation between the data and the claim. Backing is the attributes of each objective that further explain the objectives by describing their measurement criteria and then validate the relationship amongst the objectives, the proposal and the current state of agreement. Qualifier and rebuttal are actually the measurement results regarding how well the proposal achieves her own objectives and the objectives proposed by the team, so that the measurement result for own objectives (qualifier) can indicate the degree of desire of the stakeholder for the proposal while the measurement results for team objectives (rebuttal) describe the chances that could fail the proposal (claim). To build a negotiation argument in this way, stakeholders will have a better understanding of each other because they share not only their claims but also their underlining reasons and desires (e.g., perspectives). Figure 10 below describes an 110 argument example from an engineer’s perspective. As shown in the figure, the claim for the task “define quality attributes” is to define the attributes of maintainability and security for the software product. The data describes the initial state (of this task), which include design requirements, application constraints and architecture style, and all applicable facts to support the claim. To justify the use of the data, the warrant has his fundamental and means objectives that state why the claim is proposed based on the data. The backing of this argument is the attributes of his objectives that further explain the warrant by providing its measurement scales. The measurement result given by the engineer for her own objectives is included in qualifier while the measurement result for the team’s objectives is the rebuttal that describes his perspective regarding the performance regarding how well his argument may achieve the objectives proposed by the entire team. 111 Backing (Attributes): Fundamental Objective: Performance - Means objective: build a prototype system to estimate the efforts taken to add/remove a module - Attribute: support v.s. opposition Fundamental Objective: Usability Fundamental objective: build a prototype system to check the best level of data encryption - Attribute: the maximum number of bits in the applicable encryption algorithm Rebuttal (Evaluation for the Objectives of the Team): How well the proposal achieve the objectives proposed by the team Claim (Task Proposal): Define maintainability and safety for the software product Qualifier (Evaluation for Own Objectives): How well the proposal achieve the objectives in the warrant Warrant (Objectives): Fundamental Objective: Portabi - Means objective: build a prototype system to estimate the efforts taken to add/remove a module Fundamental Objective: Security - Means objective: build a prototype system to check the best level of data encryption Data (State of Agreement): Specified Design Requirements Defined Quality Attributes Identified Functional Modules Figure 10: An Example Argument (by the Engineer) In exchanging the arguments, different argument components can be exchanged in an order, instead of at once, so that this exchange will continue only if the stakeholders have not reached a consensual agreement. Such order of exchanging argument components is actually defined by the logic flow between the argument components. The claim should always be proposed first to clearly state the stakeholders’ position. The data can be exchanged with the claim as it shows the facts behind the claim. In fact we suggest the claim and data should be exchanged together since assumably the claims are actually conflicting (otherwise the negotiation should not take place) and thus the data is necessary to present in this step. After the data it should be the warrant that justifies the 112 use of data for proving the claim. Backing comes next as further facts to support the warrant. The last two components that should be exchanged are qualifier indicating degree of desire for the proposal and rebuttal indicating the condition that may suspend the warrant. In details, the order of exchanging the arguments is defined as follows: First, the stakeholder begins to work on design task and develop her claim based on the data, i.e. her task implementation proposal based on the state of agreement that can validate the proposal is a further effort to achieve the goals. If the claim and the data are not accepted by other stakeholders who jointly work with her on this task, then the stakeholder begins to identify the objectives which the task proposal will accomplish and present them as the warrant to others. If the claim, data and warrant are not accepted by others, then the attributes of those objectives (in warrant) will be declared and presented as backings to describe the measurement criteria of the objectives and confirm the proposal can accomplish the objectives by achieving good performance on each attribute. Up to this point, after all the stakeholders present their claims, data, warrants and backings, the initial objectives hierarchy (including all fundamental/means objectives and 113 attributes) of entire team has been set up. If the claim, data, warrant and backing are not accepted by others, the stakeholder has a chance to refine her proposal (claim) based on the group objectives hierarchy and follow the aforementioned procedure from the beginning. In this step, argument analysis based on the composition of the arguments (objectives and attributes) will be applied to indicate the similarity and distance amongst these arguments in their objectives/attributes and the analysis results will be provided to the stakeholder for better understanding of the arguments relationship and recommendations to refine the arguments. Next, if the stakeholder chooses not to refine her claim or the refined version (claim, data, warrant and backing) is still not accepted, the stakeholder will give the qualifier that is, instead of solely subjective opinion, the measurement result regarding how well her proposal accomplishes the objectives given in her warrant, indicating the degree of her desire for the proposal. The rebuttal should be also given in this step, as the measurement result regarding how well her proposal accomplishes the objectives proposed by team, indicating the probability of this proposal can be rejected if this result is rather unsatisfactory. In this step, when all the involved stakeholders give qualifier and rebuttal for their proposals, they should be offered one more chance to refine the proposal based on the measurement results and then follow the steps. Here argument analysis based on the attribute value of the arguments will be applied again to indicate the similarity and distance amongst these arguments in the regard of their measurement results and the results will be provided to the stakeholder for recommendations to refine the arguments. 114 If the stakeholder chooses not to refine her claim or the refined version is still not accepted, all the arguments will be evaluated and ranked based on deterministic analysis. The argument with best evaluation result will be recommended to the group leader. Step 6.b – Argument Analysis In our approach, clustering analysis is applied to indicate the similarity and distance amongst these arguments based on either the composition of objectives/attributes or the attributes values of the arguments, and string ‘similar’ arguments into clusters to build a cluster tree. These analysis results will be provided to the stakeholder for better understanding of the arguments relationship and recommendations to refine the arguments. Argument analysis will take place within the flow of generating and exchanging argument components (that is why the number of this step is 6.b since argument generation and exchange is in 6.a). There are two points in time when the argument analysis will be applied. First point is when the claim (task proposal), data (state of agreement), warrant (objectives), and backing (attributes) are exchanged yet accepted by the others, the argument analysis based on the composition of the objectives/attributes can be applied to indicate the similarity amongst the arguments and build clusters tree of arguments based on their similarity in the objectives/attributes composition. In reading 115 the analysis results, a composition-based cluster tree with simple structure and few levels means all of the stakeholders have similar objectives and attributes in their arguments – they want similar value when working on the same task for that they probably just have different proposals to achieve the value. On the other hand, if the clustering levels are high, the arguments are quite distant from each other. In one sense, it denotes that the stakeholders’ objectives and perspectives are not well shared. And it may be an indicator of ‘choosing one argument that is most preferred’ instead of ‘merging the arguments’. If composition-based cluster tree tends to have simpler structure and fewer levels after each iteration of negotiation processes, it indicates the less incompatibility of stakeholders’ objectives and better quality of the negotiation process The second point is when stakeholders have generated and exchanged their qualifier (i.e. the measurement results regarding how well the proposal accomplishes the objectives given in the warrant) and rebuttal (i.e. the measurement results regarding how well the proposal accomplishes the objectives proposed by the team) and yet they still have not make a joint agreement on one argument. Here argument analysis based on the attribute value of the arguments will be applied again to indicate the similarity and distance amongst these arguments in the regard of their measurement results and then build the clustering tree by stringing similar clusters. Similarly as the previous analysis, cluster tree with simple structure and few levels means the arguments have similar achievement of objectives. If this cluster tree tends to have simpler structure and fewer levels after each 116 iteration of negotiation processes, it also indicates the less incompatibility of arguments and better quality of the negotiation process. In addition, the analysis results derived from this step will provide the design team or the team leader with recommendations to refine the arguments. The following describes some of the typical argument refinement mechanism: Refine Design Proposals/ Process In case of similar arguments (in the clustering tree) in the composition of objectives and perspectives but different task proposal, stakeholders can refine the task proposals and/or the design process to increase the similarity between the arguments to work towards a consensual agreement. a. Identify further means objectives to achieve the fundamental objectives and add more actions/objects accordingly b. Add or remove stakeholders from a given task. It is even possible to add/remove stakeholders associated with a task to avoid the conflict situation. c. By evaluating the feasibility of planned design tasks, we can prevent some scenario of distant arguments by noticing their potential existing earlier to the stakeholder. d. Identify the arguments with low similarities and reveal the differences earlier. 117 Refine Objectives and Perspectives In case of distant arguments (in the clustering tree) in the composition of objectives and perspectives, or similar arguments (in the clustering tree) in attributes value, it is possible to directly refine stakeholders’ objective and perspectives to increase the similarity between the arguments to work towards a consensual agreement. a. Focus on the stakeholders who have ‘separated’ argument (from other clusters) and ask them to consider refining or adding objectives. b. Ask the stakeholder with different distant models to talk to each other on certain issues. Build communication channels to increase their interaction chances. c. Clarify the meaning/definition of fundamental objectives so that people have shared understanding. d. Help stakeholder generate more objectives (e.g. separate objectives, generate more means objectives) to isolate their perspectives. Rearrange Design Team The argument analysis developed above provides feasibility to analyze the relationship between the arguments of the stakeholders, understand their interaction, and better control the design teamwork. a. The clustering tree shows the grouping features of the arguments. If two stakeholders have very distant arguments, the team can apply certain methods of promoting their interaction, such as suitable cross-trainings based on their expertise and backgrounds. 118 b. Ask the stakeholder with similar arguments to communicate more and explore the possibility of combining their arguments c. Suggest the stakeholders to review the relevant product information during certain tasks. d. Provide the stakeholders with the information of the negotiation and solutions for similar design tasks in the past Step 7 – Argument Evaluation As the stakeholders’ arguments are generated, exchanged and analyzed during negotiation, their objectives and perspective models may evolve due to deepened understanding of each other. If all the stakeholders can jointly agree on a particular argument claim, they can take that claim as the final resolution. Otherwise, all the arguments must be carefully evaluated for resolutions. The evaluation method analyzes the stakeholder perspectives of the objectives within the arguments and compares the argument claims based on the result. In this work, a simple additive weighting function (a.k.a. weighted average) is used to build the evaluation method which ranks the arguments from most desired to least desired assuming stakeholders can characterize the consequences of each argument with certainty. Furthermore, “weighted average” is also applied when evaluating the arguments based on their value for the objective attributes with varying importance. Weighted average, by its definition, means an average that takes into account the proportional relevance and strength of each component, rather than treating each component equally. 119 As discussed in Section 4.3.3, the argument evaluation in our work includes four steps, i.e. define measurement scale, assign objective weights, score the arguments and aggregate the preferences. In these four steps, the measurement scale has been defined in Section 4.3.3, the objective weights have been defined in Step 4 and the arguments are scored in attribute values (either natural attributes’ values or stakeholders’ preferences) in Step 5, both as stakeholders’ perspectives. In this step, the preferences and weights are aggregated to derive final argument evaluation results which are used to rank the arguments and select the one that is most preferred by the team. Before the arguments were measured, all the attributes values should also be first normalized into range of [0, 1] in order to decrease the effects of differences in the numeric range of attribute values and thus allows easier and better cross-attribute comparisons. For normalizing the values of natural attributes, since the value of the attributes are usually either zero or positive, but we did not know the maximum value of an attribute. So the normalized value is: (3) Where d i is the attribute value and n is the number of the proposals 120 For normalizing the values of ‘support vs. opposition’, since we already knew the maximum value (‘10’ as ‘strongest support’) and minimum value (‘1’ as ‘strongest opposition’) of this attribute, then the value can be normalized as: (4) Where d i is the attribute value, d max =10 and d min =1 Here formula (2) defined in Section 4.3.3 is used to derive final evaluation result, based on the attribute values and their normalization results. For example, the evaluation of product manager’s proposal is: Evaluation = = 6 ) t_cost developmen migration encryption modularity I friendly_U quality rendering_ ( + + + + + = 6 ) 9 . 6 * 66 . 0 9 . 7 * 89 . 0 3 . 6 * 75 . 0 3 . 7 * 66 . 0 3 . 8 * 89 . 0 9 . 8 * 89 . 0 ( + + + + + = 5.95 Based on the evaluation results, a most preferred argument (i.e., the one with highest evaluation score) will be recommended as the final agreement of the negotiation for the design task. They will move back to Step 3 above to check for further decision conflicts with other tasks. These iterations continue until no more conflict is found (i.e., no more negotiation is necessary), and the team moves to the Post-Negotiation phase as described below. 121 4.4.4 Post-Negotiation: Consensual Agreement and Move to Next Phase In the Post-negotiation phase, the stakeholders have achieved consensual agreement for the design process: completed all negotiation activities and are committed to accept one jointly agreed software design that consists of agreements for each design task. The step in this last phase as described below. Step 8 – Obtain a commonly accepted software design. At the end of the collaborative negotiation process for each design task, the design team should either agree on a commonly appraised argument, or take the evaluation results and accept the argument with top evaluation score as the agreement. After this process runs for all the design tasks, the team should be ready to continue to the next phase of the software development lifecycle with the implementation proposals in the agreed arguments for all the tasks. In addition, the result of the collaborative negotiation processes also includes the objectives and perspectives which have been collected and constructed in the negotiation phase and can be very useful for future collaboration within the same group of stakeholders on similar software design tasks. 4.5 Summary In this chapter, we first lay down the line of argument for our research work based on a summary of the findings in the previous chapters. Second, we provide an overview of our research approach and discuss its four main modules (i.e., argument generation, exchange, 122 analysis and evaluation). Then we present the details of our collaborative negotiation process based on the interaction of aforementioned four modules. This well-structured process helps stakeholders organize and share objectives/perspectives and then generate structured arguments including claims, support and further persuasive meta-information. It also guides the stakeholders through negotiation activities where their arguments can be exchanged, analysed, refined and finally evaluated based on the level of objectives achievement for a best choice. In order to prove the feasibility and effectiveness of our approach, a case study in a real-life collaborative software design process will be presented and discussed in the next chapter. 123 Chapter Five: Case Studies and Hypothesis Validation 5.1 Introduction This work has developed an approach of structuring negotiation arguments and utilizing these structured arguments to support the group decision in software design practices. In this section, we will discuss validating this approach. General methods of validating a framework and methodology include mathematical proofs, system simulations, case studies and experiments. Within these methods, mathematical proofs and system simulations are more suitable to models that are purely quantitatively defined. As this research has developed a negotiation approach that has quantitative (e.g. stakeholders’ preferences) and qualitative (e.g. value of natural attributes) factors, case studies and the experimental approach fit better with the purpose of validating our research. Therefore, case studies were undertaken in real-life software design processes to prove the effectiveness of our approach. As mentioned in Section 3.5, our hypothesis is that that, by structuring stakeholders’ negotiation arguments based on organized objectives and preferences, the design team can improve their efficiency by decreasing the time spent for negotiating on design tasks before reach consensual agreements, and enhance the quality of their collaborative negotiation, by increasing the number of the common objectives during their negotiation process. As such, the following two criteria will be measured to validate our hypothesis and justify our research approach: 124 a. Reduce the total time taken for completing software design phase b. Improve the commonality of the stakeholders’ objectives in the negotiation process In order to validate this hypothesis, the following strategies were taken to select the case studies: a. Compare the results of the cases with the support of this research approach and the results of the cases without the support of this research approach. b. Conduct the case studies, of which the results are analyzed and compared together, within one team to rule out the variance that may be caused by different teams. This team should consist of stakeholders with various expertises. It should be a relatively mature team, which have gone through multiple software design processes and have obtained stable understanding and opinions for design tasks before we take the case studies. This can help us separate the effects caused by repetitive practices of the case studies from the effects by using this research approach. c. The design team should work on different tasks (i.e. design tasks for different software functionalities) to assure the team has to collaboratively negotiate on and jointly build common understandings for each task, instead of working on same task multiple times and using the common understanding built in previous cases. In addition, these tasks should have similar levels of complexity so that the levels of difficulty in the challenges that the design team has in making jointly agreed decisions are similar as well. This can help us draw accurate conclusions when 125 comparing the case study results to validate whether the efficiency and quality have been improved. With these strategies, this work has chosen and carried out a series of case studies in real- life software projects and this chapter will introduce two case studies in this series. These two studies were undertaken in the same design team. In the studies this team works on different tasks with similar levels of complexity. The first case study took place before our collaborative negotiation approach was fully developed. The stakeholders were introduced to generic argument structure and they learnt how to use this structure to organize and convey their thoughts. The second case study was a test of our approach of structuring arguments, which was fully implemented and applied in a real-life software design process. The first study was more close to the existing design process and mainly included preparing design task proposals, generating, exchanging and discussing their arguments, and making a final decision. The second study, with support of our approach, included preparing design task proposals, building objective hierarchy, capturing stakeholders’ perspectives, generating, exchanging, analyzing and evaluating negotiation arguments, and making a final decision. The comparison between the two case studies shows the validation of the research hypothesis and justification of our approach to support the collaboration negotiation for group decision in software design. 5.2 Case Study Background – Design Process and Data Collection The case studies were taken in one real-life software development group. The following criteria were used to select a collaborative software design process for these studies: 126 1. The software design team is composed of multiple stakeholders, preferably with various backgrounds and domain knowledge, e.g. at least three distinctive disciplinary areas. 2. The design solutions call for systematic collaborative negotiations among stakeholders. 3. Each stakeholder is able to explain the required domain knowledge and his or her perspectives in a more or less structured manner. [Lu et al. 2005] According to the above criteria, for the case studies, DivX Inc. was chosen. It is a leading digital media company and provides software for customers to create, share and view high-quality videos on a variety of devices. The specific process that was studied is the collaborative design process to create a software application on mobile platforms that plays video files in mp4 format (.avi, .divx, etc) for the DivX mobile player. The design team composition was four software engineers, one engineering manager, one engineering director and one product managers. In the software design processes for DivX mobile players, case studies were chosen based on the strategies that were discussed in Section 5.1. There are two case studies that will be presented here. They were carried out in the design processes for two versions of DivX mobile players (versions 0.92 and v0.93). Both of them were undertaken by the same software design team and these two versions had different functionality with similar level of design complexity, which was measured by reviewing and analyzing the product 127 specification documents. The first case study (in DivX mobile player version 0.92) was undertaken without the support of this research approach. The second case study (in DivX mobile player version 0.93) was a test of a full implementation of this approach in structuring negotiation argument. The rest of this section introduce the details of this software design process and outlines the data collection procedure. 5.2.1 A Real-life Software Design Process Figure 11 shows a general software design and development process undertaken by the DivX Mobile Player project. As with most of the well-established processes for designing software architecture, this process focuses on working with requirements, functionality and quality attributes. At the top of the figure are the stakeholders that define and/or collect the requirements for the application. They also collect application constraints (e.g., company policy and design guideline) and determine the architecture style (e.g., layer- based system). Then a project requirement specification document is composed including all of the requirements, constraints and the architecture style. Based on the requirement specifications, the stakeholders define the quality attributes (e.g., usability, scalability, maintainability, etc.) against which the functional architecture is evaluated. These quality attributes will be evaluated after the team completes a functional architecture design that includes functional modules and communication protocol. In case the quality attributes are not met, the architecture will be modified and the process goes back to estimate the quality attributes again. When the quality attributes are met, the process continues if there are no more requirements (otherwise, it will go back to define new requirements). 128 Engineers Marketing Technical Lead Management Define Quality Attributes Modify Architecture Estimate Quality Attributes More Requirements Design Functional Architecture Continue to Development Yes Not OK OK No Specify Player Design Requirements Gather Design Requirements Collect Application Constraints Determine Architecture Style Identify Functional Modules Build Communication Protocol Figure 11: The Software Architecture Design Process During the software design process, different stakeholders played various roles based on their respectively professional expertise and they had to work jointly on common tasks. The product manager aimed to specify every requirement of the software product in order to satisfy the customers’ needs. The engineering manager designed the overall system architecture, divided it into each component module and speculated the interaction of these modules. The software engineers needed to provide detailed technical solutions for each module and communicate functional requirements defined by the manager and customers. The director’s role was to supervise the whole design process and ensure it goes in the correct direction in terms of resources and time. These stakeholders with 129 different backgrounds and expertise had to cooperate with each other throughout the whole life cycle of the software design process. The steps in this process are explained in detail below: a. Specifying design requirements This step was mainly to dig out the application’s design requirements, such as the product manager typically turned in a set of requirements based on customer surveys and competitor’s offers, whereas the engineering director's major requirement was that development costs stay low. The engineering manager needed to get these requirements, considers which requirements were essential to the application, and negotiated with engineers to determine what requirements were technically doable. The next thing in this step was to gather all the possible application constraints from engineering director and probably engineering/product managers. A constraint could be anything from a company policy to a set of architectural guidelines for an existing product line. Like requirements, application constraints could be heavily contested territory, although in most cases constraints were less easily removed or set aside than requirements. The next thing that needed to be determined was the architectural style of the software, in which there were many candidates from which to choose. These included layered architecture (most common for enterprise applications), pipes and filters, database- 130 centered, client-server architectures, virtual machines, and more. The architectural style that the team chose provided the conceptual framework around which the player application was built. b. Defining quality attributes In this step, the whole team (based on their individual background and professional expertise) proposed the most critical quality attributes including performance, reliability, security, usability, modifiability, portability, and interoperability. The entire design team must understand and agree on the application's quality attributes. c. Designing functional architecture With the functional requirements set, constraints accepted, the architectural style chosen and quality attributes defined, the next step was to set down the functional architecture. This mainly included identifying each functional module and building communication protocols between these modules. The ways the functional architecture in DivX Mobile Player was designed involved going through a set of use cases (i.e. a detailed description of the action flow in each function requirement) one by one and sorting out the modules, classes and the interaction between them for each use case. After defining the modules and classes, the design team began to define the methods of each class and application programming interfaces between the classes. 131 d. Estimating quality attributes After the functional architecture was specified, the next step was to determine whether the architecture could meet the quality attributes. If the quality attributes were not met, the team should change the architecture so that all of them were met. After that, the team must go through the quality attributes estimation again to see if and how the changes have affected them, and whether the quality attributes were met. If the quality attributes were met, the design team might go through iteration if they needed to add new requirements. If no new requirements were added, they would go on to the development phase. 5.2.2 Data Collection Procedure Using our approach, we interviewed four software engineers (for brevity, in the case studies presented here, we only discuss the data from the lead software engineer), one engineering manager, two product managers, and one engineering department director. We helped them structuring their opinions into arguments and guided them through the negotiation process of utilizing these structured arguments. The general guidance for carrying out the negotiation process is provided as follows: For new versions of the DivX Mobile Player, the software design process started when the requirements were gathered and so did the negotiation process when multiple stakeholders worked on one task. In the pre-negotiation phase, all the stakeholders in our 132 experiment initiated a software design process for the DivX Mobile Player. When there were different proposals from the stakeholders on same task, the difference was captured and the team led to the negotiation phase. During the negotiation phase, the objective hierarchy was jointly proposed by the stakeholders and the perspectives of each stakeholder were captured indicating their opinions regarding the importance of the objectives/attributes and the value of objectives’ attributes based on the achievement of the objectives by each proposal. Then all the information was used to generate arguments which would be exchanged between the stakeholders. The results of argument analysis were provided to the design team and certain guidelines of refining the argument or rearranging the design tasks/process were also suggested. At the end of this process, if necessary, we engaged the argument evaluation approach to recommend a most desired proposal as an outcome of the negotiation. The design team (or the team leader) had to make a choice based on this recommendation. Upon this agreement, the process flowed back to the pre-negotiation process to check the difference in proposals for other tasks until no further differences were found for all the tasks. When there were no more proposals differences detected for all the negotiation tasks, the design team moved to the post-negotiation phase and reviewed the software design solution, which consists of the commonly agreed proposals for all the design tasks. When there was no more proposal refinement necessary, the team proceeded to the next step of the software development lifecycle. 133 5.3 Case 1 5.3.1 Introduction This case was a test taken in the software design process for DivX mobile player 0.92. It was before our approach was fully developed and introduced to the DivX software design team. In a training session, the generic argument structure was introduced to the design team who then used it in their negotiation about conflicting design proposals. The reason we are undertaking this case study in such settings is in two folds: first, it would make more sense if we are comparing the results using our approach with that using some sort of proven support for structuring arguments instead of the pure ad-hoc process; second, this gives the design team an initial understanding of the argument structure and lays down the grounds of fully utilizing our approach. The design team consisted of the following: z Product manager: specify the software product requirements based on the customers’ needs z Engineering manager: design the overall system architecture, divide it into each component module and speculate the interaction of these modules. z Software engineers: provide detailed technical solutions for each module and communicate functional requirements defined by the manager and customers. z Director: supervise the whole design process and ensure it goes in the correct direction in terms of resources and time. 134 In the next Section 5.3.2, we discuss the details of the negotiation process within one design task, presented by examples, and Section 5.3.3 discusses the lessons learned from these negotiation processes. 5.3.2 The Negotiation Process For each design task, the engineering director, as the design team leader, suggested a 4- step procedure, which was the existing practice combined with the use of argument structures. a. Engineering director explained the design task b. Stakeholders developed the implementation proposals c. Stakeholders generated, exchanged, and discussed arguments d. The design team made a joint agreement or the team leader made the choice The details of each step in this process are presented as follows: a. Engineering director explained the design task The engineering director explained the details of each task (e.g. initial state and needs) in the design process. b. Stakeholders developed the implementation proposals A quick brainstorming session was carried out for stakeholder to generate their implementation proposals. For the task ‘Determine Architecture Style’, the proposals are listed below: 135 Director: model-view-control layered architecture that is used most commonly for enterprise applications Engineering manager: database-centered architecture Engineer: client-server architecture with Java messaging services Product manager: no proposal at this time c. Stakeholders generated, exchanged, and discussed arguments Based on the generic argument structure that was introduced at the beginning of the process, in this step, stakeholders formed their arguments around the proposals they had and argued about the validity of their own proposals (i.e. the decisions about how to perform the design task). They also presented the underlying reasoning that made the decisions. The following dialogs are examples that we extracted from the task ‘Determine Architecture Style’: Engineering manager: We need a database-centered architecture, and here is why: because the database can help all the modules communicate with integral data Engineer: I see. But I still prefer a client-server application with Java Messaging Services. The messaging service can also enable all the modules to send or receive data. Engineering manager: With database-centered design, we do not have to use Java Messaging Services. Engineer: Sure, but why we choose the database-centered if the same advantage can be achieved by the other way? Engineering manager: Let me think about it … 136 …… Engineering manager: We need a database-centered architecture, and here is why: because the database can help all the modules communicate with integral data Product manager: That’s fine. I am OK with the choice although I am not so clear of the reason. d. The design team made a joint agreement or the team leader made the choice In the example we used previously ‘Determine Architecture Style’, stakeholders spent around 4 hours for completing the negotiation for the solutions and there were 7 rounds of generating/exchanging the arguments amongst the stakeholders before a joint agreement was made. The total planned time for the software design phase was 5 days, 40 hours. The design team completed the entire design process using more than four and a half days, accurately speaking, 37 hours. According to the team, half of the time had been spent for explaining the underlying reasons behind the decisions, especially for every task there was a fair amount of explanation that came from similar sources. There were nine tasks for which the stakeholders carried out the negotiation activities and generated/exchanged arguments. The average number in the rounds of generating and exchanging the arguments was 9 before the stakeholders reached an agreement for each design task or the team leader made a choice. Within the nine tasks, there were two tasks in which the design team had jointly agreed on one argument. In the other seven tasks, 137 the design team failed to achieve a consensual agreement, and the engineering director (the team leader) had to choose one alternative as the solution, which was not much favored by the team but still kept the teamwork going on. 5.3.3 Lessons Learned The design team found that using the generic structure to prepare their arguments was useful to organize and convey their thoughts. This way helped them understand how others arrived at their judgments and brought up interesting discussions about issues that might otherwise have been overlooked. However, it took a fair amount of time for the stakeholders to prepare a full structured argument (including the claim, support data, warrant, etc.) in the negotiation mainly because of the lack of experience and insufficient guideline of building the argument structure. Even though they had given claims, count-claims and reasoning course, they were not very satisfied with what they had since this information is rarely complete or accurate (as they so stated afterwards). Some kind of guideline about how to effectively and accurately generate arguments is needed. Stakeholders also found that most the arguments from same person but for different tasks were very much like, especially the meta-information (e.g., supporting data and warrant). The information in these arguments came from their position, expectation, experience 138 and even ‘gut feelings’. However, this information they have collected for one task was usually poorly recorded and then either lost or hard to find (or relate to) in the next tasks. So it was quite time-consuming and required redundant efforts to prepare this same or similar information for each task. In summary, the team felt that the argument structure was useful to organize their opinions and present them in the team meetings. It provided a common reference for a structured discussion of software design criteria and for the team evaluation of all the arguments. It also spurred the search for additional information and meta-information, which might be overlooked otherwise. However, some problems were encountered with respect to constructing the arguments and persisting the meta-information in the arguments for multiple tasks. Apart from this, the arguments seemed to be helpful for the team members to understand each other and for the team lead to review all the structured arguments and make a final decision. However, even though the team had some a priori feeling about what solution would be more feasible, they would feel much more comfortable that the final solution (chosen by the team lead) was also justified by using some sort of evaluation method. 139 5.4 Case 2 5.4.1 Introduction The second case study was a test in the software design process for DivX mobile player 0.93. It was after our approach was fully developed and introduced to this real-life software design process. All the actual members in the real design team participated in this study. In a training session given in the process, the collaborative negotiation approach was introduced to the design team with examples and detailed guidelines to make sure the team has sufficient understanding and abilities to use our approach in the process. As the first case study is more close to the existing design process, compared with that, this case study serves to illustrate how well our collaborative negotiation approach achieves our research goals and supports the group decision in software design. The design team is the same as the first case study. Section 5.4.2 discusses the details of the negotiation process, presented by example in one design task, and Section 5.4.3 discusses the lessons learned from these negotiation processes. 5.4.2 The Negotiation Process For each design task in this case study, an 8-step procedure was suggested, which was a combination of our collaborative negotiation process and the common practice previously taken by the design team. 140 a. The engineering director explained the design task b. Our approach of structuring arguments was introduced c. Stakeholders developed the implementation proposals d. Stakeholders built the objective hierarchy e. Stakeholders declared their perspectives f. Stakeholders generated and exchanged arguments g. Arguments were analyzed and the results are provided h. Arguments were evaluated and the one with highest ranking was recommended This procedure was accepted and the details of each step in this process are presented as follows: a. The engineering director explained the design task The engineering director explained the details of each task (e.g. initial state and needs) in the design process. b. Our approach of structuring arguments was introduced Before the stakeholders worked on the task, we held a technical training in which the collaborative negotiation approach developed in this work was introduced to the design team. The details of each step in the structured negotiation process were presented using examples and specific guidelines. After the training, the design team understood how to generate and exchange the arguments. They also realized the argument analysis results would be calculated and provided as recommendation for them to refine the arguments 141 and at the end there might be an evaluation applied for all the arguments if there is no argument commonly accepted. c. Stakeholders developed the implementation proposals Using the task ‘Build Communication Protocol’ as an example, the proposals given by the stakeholders are listed below: Product manager: use XML format specified by the company partners as the communication protocol. Engineering manager: use a CSV format, i.e., comma separated values. Engineers: use open and well-adopted XML format that is used for general device and software specification. Engineering director: build a proprietary XML format. d. Stakeholders built the objective hierarchy The objective hierarchy that stakeholders built in this case study was a flowchart including all the objectives and attributes as well as the relationship between the objectives and attributes. For example, if a subject stated that ‘We should maximize the software usability on the mobile handsets by keeping UI (user interface) simple, by which I mean it should take the user as little navigation as possible to locate the desired content’, this was actually interpreted as suggesting one fundamental objective ‘maximize usability on handsets’, two means objectives ‘keep UI simple on handset browsers’ and ‘require little navigation to locate content’. One attribute, ‘average number of clicks per use case,’ 142 was also added by us for the objective ‘require little navigation to locate content’ to measure this. Figure 12 shows the Objectives and Attributes flowchart including all the objectives and attributes proposed by the stakeholders. Maximize Usability on Handsets Maximize Application Reliability Maximize Software Efficiency Maximize Code Portability Keep Development Feasibility and Maintainability Product Manager Engineering Manager Engineering Director Engineers Minimize Development Cost Keep UI Simple on Handset Browsers Reduce Data Entry Allow Offline Uses Limit Graphical Rendering Limit Bandwidth Use of the Network Module Limit Memory Allocation Simple to Develop Port Code to A Brand Spectrum of Devices Save Network Requests if Offline More Choice than Input Less Navigation to Locate Content Easy to Build Categories Use Less Connections Compress Network Data Build Application for Various Platforms Test Application on Various Carriers Keep Good Modularity Less Coupling More Functional Cohesion Reduce Interaction with Other Phone Modules Provide Error Reporting Flat Learning Curve Coupling Cohesion Number of Supported Platforms Number of Supported Platforms Compression Rate Number of Connection Per Use Case Frequency of Choice over Input Average Number of Clicks Per Use Case Support vs Opposition Support vs Opposition Support vs Opposition Support vs Opposition Attributes Means Objectives Means Objectives Fundamental Objectives Stakeholders Figure 12: Objectives and Attributes As mentioned in Step 4 in Section 4.4.3 (Propose an objective hierarchy), if an objective does not have any means objectives or any attributes that are naturally used to interpret the objective (also called natural attributes below), as shown in Figure 12, an attribute ‘support vs. opposition’ was added by the researcher. 143 In this work, as specified in the Section 4.3.3 (Argument evaluation), additive weighting function (a.k.a. weighted average) was used to measure the arguments, i.e. the proposals were measured based on the attribute values and the ‘importance’ of the attributes. Therefore, the stakeholders had to give the relative importance of each objective in a 1- to-10 scale as follows: 10 = Very important 8 = Somewhat more important 6 = Important 4 = Somewhat less important 2 = Very less important To get more accurate results, this ‘importance’ value were collected for each fundamental and means objectives. After then the importance of each means objective was calculated as the average value of its importance and the importance of its corresponding fundamental objective. The importance of an attribute was same as that of its objective. Table 6 shows the relative importance of the objectives and attributes in the task ‘Build Communication Protocol.’ 144 Table 6: Stakeholders’ Perspectives: Importance of Objectives Objectives Product Manager Engineering Manager Engineering Director Engineers Importance Maximize Usability 8 8 8 8 8 Maximize Usability- Keep UI Simple on Handheld Browser 8 8 8 8 9 Maximize Usability- Keep UI Simple on Handheld Browser-Use Less Navigation to locate Content 10 8 8 8 8.5 Maximize Usability- Reduce Data Entry-Easy to Build Data Category 8 6 6 6 6.5 Maximize Usability- Reduce Data Entry-More Choice than Input 8 8 6 6 7 Maximize Usability- Allow Offline Uses- Save Network Requests 8 8 8 4 4.5 Maximize Usability- Limit Graphical 6 6 4 4 5 145 Rendering Table 6, Continued Maximize Application Reliability 8 8 10 8 8.5 Maximize Application Reliability- Provide Error Reporting 8 8 6 6 7 Maximize Application Reliability- Reduce Interaction with Other Phone Modules 6 8 6 4 6 Maximize Software Efficiency 8 8 8 6 7.5 Maximize Software Efficiency- Limit Bandwidth 6 8 8 6 7 Maximize Software Efficiency- Limit Bandwidth Use-Use Less Connection 8 10 8 8 8.5 Maximize Software Efficiency- Limit Bandwidth Use- Compress Data 6 6 6 8 6.5 146 Maximize Software Efficiency- Limit Memory Allocation 6 8 6 6 Table 6, Continued 6.5 Maximize Code Portability 6 8 8 8 7.5 Maximize Code Portability- Port Code to A Brand Spectrum of Devices 4 6 8 8 6.5 Maximize Code Portability- Port Code to A Brand Spectrum of Devices- Build Application on Various Platforms 6 8 8 8 7.5 Maximize Code Portability- Port Code to A Brand Spectrum of Devices-Test Application against Various Carriers 6 8 8 8 7.5 Minimize Development Cost 6 6 10 6 7 Minimize Development 4 6 8 8 6.5 147 Cost-Simple to Develop Table 6, Continued Minimize Development Cost-Easy to Scale Up 6 6 10 6 7 Minimize Development Cost-Flat Learning Curve 4 4 8 8 6 Maximize Software Maintainabili ty 6 8 10 8 8 Maximize Software Maintainabili ty-Keep Good Modularity 4 6 8 8 6.5 Maximize Software Maintainabili ty-Keep Good Modularity - Less Coupling 2 6 10 8 6.5 Maximize Software Maintainabili ty-Keep Good Modularity – More Functional Cohesion 2 6 10 8 6.5 148 e. Stakeholders declared their perspectives When the task proposals were being evaluated, each attribute of each objective was assigned a value. In case of a natural attribute, the value was generally numeric and calculated by a commonly accepted method based on the individual case, such as average number of the clicks per use case or the total count of supported handset platforms. In the case of the added attribute ‘support vs. opposition’, each stakeholder declared their perspectives for its value based on their expertise and understanding and the perspective would be one of the following: 10 = Strong support, i.e. the proposal will most likely help achieve the objective 8 = Support, i.e. the proposal will likely help achieve the objective 6 = Neutrality (fair, unknown or uninterested), i.e. the proposal may not either contribute to or harm the achievement of the objective, or Controversy, i.e. the proposal may have some effect in achieving the objective, but the decision maker is not clear of what kind of effect it may have; 4 = Opposition i.e. the proposal will likely bring negative effects in achieving the objective. 2 = Strong opposition i.e. the proposal will most likely bring detrimental effects in achieving the objective. All the perspectives for one ‘support vs. opposition’ attribute were integrated together into a single value for this attribute, and then the values of all attributes were used to 149 evaluate each argument regarding how its proposal achieves the objectives. The evaluation results will determine the preference order of these arguments. Table 7 shows stakeholders’ perspectives for each argument regarding how well it achieves each objective. These perspectives are represented by values for attributes of the objective. 150 Table 7: Stakeholders’ Perspectives: Attribute Values per Proposal for ‘Build Communication Protocol’ Objective-Attribute Product Manager Proposal (Partner XML) Engineer Proposal (Adopt open XML format) Engineer Manager Proposal (Comma Separated Value) Engineer Director Proposal (Build own XML format) Maximize Usability-Keep UI Simple-Take Little Navigation-Average Number of Clicks per Page 3 clicks 3 clicks 3 clicks 3 clicks Maximize Usability-Keep UI Simple-Build Content Categories-Support vs. Opposition Product Manager: Support Engineer: Neutrality Engineering Manager: Opposition Engineering Director: Opposition Product Manager: Support Engineer: Support Engineering Manager: Opposition Engineering Director: Neutrality Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Support Engineering Director: Opposition Product Manager: Neutrality Engineer: Support Engineering Manager: Controversy Engineering Director: Support Maximize Usability- Reduce Data Entry-Have More Choice than Input-Frequency of Choice over Input 22:2 22:2 22:2 22:2 Maximize Usability-Allow Offline Uses-Save Network Requests if Offline-Support vs. Opposition Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Opposition Engineering Director: Neutrality Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Opposition Engineering Director: Neutrality Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Neutrality Engineering Director: Neutrality Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Neutrality Engineering Director: Support Maximize Application Reliability-Provide Error Reporting-Support vs. Opposition Product Manager: Support Engineer: Product Manager: Support Engineer: Product Manager: Support Engineer: Product Manager: Support Engineer: 151 Neutrality Engineering Manager: Neutrality Engineering Director: Neutrality Support Engineering Manager: Controversy Engineering Director: Neutrality Controversy Engineering Manager: Support Engineering Director: Neutrality Table 7, Continued Support Engineering Manager: Controversy Engineering Director: Support Maximize Application Reliability-Reduce Interaction with Other Phone Modules – Support vs. Opposition Product Manager: Neutrality Engineer: Controversy Engineering Manager: Opposition Engineering Director: Controversy Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Neutrality Engineering Director: Controversy Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Neutrality Engineering Director: Controversy Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Neutrality Engineering Director: Support Maximize Software Efficiency-Limit Bandwidth Use-Use Fewer Connections-Support vs. Opposition Product Manager: Neutrality Engineer: Controversy Engineering Manager: Opposition Engineering Director: Controversy Product Manager: Neutrality Engineer: Support Engineering Manager: Neutrality Engineering Director: Neutrality Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Support Engineering Director: Neutrality Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Neutrality Engineering Director: Support Maximize Software Efficiency-Limit Bandwidth Use-Easy to Compress Network Data-Support vs. Opposition Product Manager: Neutrality Engineer: Controversy Engineering Manager: Opposition Engineering Director: Controversy Product Manager: Neutrality Engineer: Controversy Engineering Manager: Opposition Engineering Director: Controversy Product Manager: Neutrality Engineer: Controversy Engineering Manager: Support Engineering Director: Controversy Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Neutrality Engineering Director: Support Maximize Software Product Product Product Product 152 Efficiency-Limit Memory Allocation-Support vs. Opposition Manager: Neutrality Engineer: Opposition Engineering Manager: Opposition Engineering Director: Controversy Manager: Neutrality Engineer: Neutrality Engineering Manager: Opposition Engineering Director: Controversy Manager: Neutrality Engineer: Support Engineering Manager: Support Engineering Director: Controversy Table 7, Continued Manager: Neutrality Engineer: Neutrality Engineering Manager: Neutrality Engineering Director: Support Maximize Code Portability-Port Code to A Brand Spectrum of Devices-Build Application for Various Platforms-Number of Supported Platforms 3 Windows Mobile, Symbian, Palm 3 Windows Mobile, Symbian, Palm 3 Windows Mobile, Symbian, Palm 3 Windows Mobile, Symbian, Palm Maximize Code Portability-Port Code to A Brand Spectrum of Devices-Test Application against Various Carriers-Number of Supported Carriers 4 Verizon, AT&T, T-Mobile, Sprint 4 Verizon, AT&T, T-Mobile, Sprint 4 Verizon, AT&T, T-Mobile, Sprint 4 Verizon, AT&T, T-Mobile, Sprint Minimize Development Cost-Feasible and Simple to Develop-Support vs. Opposition Product Manager: Neutrality Engineer: Controversy Engineering Manager: Neutrality Engineering Director: Controversy Product Manager: Neutrality Engineer: Support Engineering Manager: Neutrality Engineering Director: Controversy Product Manager: Neutrality Engineer: Controversy Engineering Manager: Neutrality Engineering Director: Opposition Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Neutrality Engineering Director: Support Minimize Development Cost-Easy to Scale Up-Support vs. Opposition Product Manager: Neutrality Engineer: Neutrality Product Manager: Neutrality Engineer: Support Product Manager: Neutrality Engineer: Opposition Product Manager: Neutrality Engineer: Support 153 Engineering Manager: Controversy Engineering Director: Controversy Engineering Manager: Controversy Engineering Director: Support Engineering Manager: Controversy Engineering Director: Opposition Table 7, Continued Engineering Manager: Controversy Engineering Director: Support Minimize Development Cost-Flat Learning Curve-Support vs. Opposition Product Manager: Support Engineer: Controversy Engineering Manager: Neutrality Engineering Director: Neutrality Product Manager: Neutrality Engineer: Support Engineering Manager: Neutrality Engineering Director: Neutrality Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Support Engineering Director: Neutrality Product Manager: Neutrality Engineer: Support Engineering Manager: Neutrality Engineering Director: Support Maximize Maintainability-Keep Good Modularity-Less Coupling-Support vs. Opposition Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Controversy Engineering Director: Neutrality Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Controversy Engineering Director: Neutrality Product Manager: Neutrality Engineer: Opposition Engineering Manager: Controversy Engineering Director: Opposition Product Manager: Neutrality Engineer: Support Engineering Manager: Controversy Engineering Director: Support Maximize Maintainability-Keep Good Modularity-More Functional Cohesion-Support vs. Opposition Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Controversy Engineering Director: Neutrality Product Manager: Neutrality Engineer: Neutrality Engineering Manager: Controversy Engineering Director: Neutrality Product Manager: Neutrality Engineer: Opposition Engineering Manager: Controversy Engineering Director: Opposition Product Manager: Neutrality Engineer: Support Engineering Manager: Controversy Engineering Director: Support 154 f. Stakeholders generated and exchanged arguments Up to this step, stakeholders had prepared task proposals, objectives, and perspectives. They used this information to generate the arguments. Based on the Toulmin definition of structure, the claim was the design task proposal. The data consist of the initial state of the task – the joint agreement achieved by the design team before they work on this task. The warrant was the set of the objectives that the team wanted to achieve from this task based on the initial state. Backing was the attributes of each objective that further explained the objectives by describing their measurement criteria and then validated the relationship amongst the objectives, the proposal and the current state of agreement. Qualifier and rebuttal were actually the measurement results regarding how well the proposal achieved her own objectives and the objectives proposed by the team, so that the measurement result for own objectives (qualifier) could indicate the degree of desire of the stakeholder for the proposal while the measurement results for team objectives (rebuttal) described the chances that could fail the proposal (claim). An argument example is given in Figure 13 below. It was generated by the engineering director. As shown in the figure, the claim for the task “Build Communication Protocol” is to build a proprietary XML format. The data described the initial state (of this task) which included design requirements, quality attributes and functional modules. To justify the use of the data, the warrant had his fundamental and means objectives that stated why the claim was proposed based on the data. The backing of this argument was the attributes of his objectives that further explained the warrant by providing its 155 measurement criteria. The measurement result given by the engineering director for his own objectives was included in qualifier while the measurement result for the team’s objectives was the rebuttal that described his perspective regarding the chances that other arguments might win. Backing (Attributes): Fundamental Objective: Reliability Means objective: Error reporting; Reduce interaction Attributes: Support vs. Opposition Fundamental Objective: Efficiency Means objective: Limit bandwidth use; Limit memory allocation Attributes: # of connections; compression rate, Support vs. opposition Fundamental Objective: Portability Means objective: porting code to device/platforms Attributes: # of supported platforms; # of supported carriers Rebuttal (Evaluation for Others’ Objectives): How well the proposal achieves the objectives proposed by the team Claim (Task Proposal): Build a proprietary XML format Qualifier (Evaluation for Own Objectives): How well the proposal achieves the objectives in the warrant Warrant (Objectives): Fundamental Objective: Reliability - Means objective: Error reporting; Reduce interaction Fundamental Objective: Efficiency - Means objective: Limit bandwidth use; Limit memory allocation Fundamental Objective: Portability -Means objective: porting code; testing app. on various platforms Data (State of Agreement): Specified Design Requirements Defined Quality Attributes Identified Functional Modules Figure 13: Argument Example In the process of generating the arguments, the stakeholders were also guided to exchange each component of their arguments. The order of exchanging these components was defined based on the logic flow between the components: the component was exchanged only if the components, which stayed ahead of it, were not accepted. Figure 14 shows this order in details. For example, the claim and the data in one argument were 156 exchanged first before all other components were exchanged or even generated. If they were not accepted, the warrant was provided. If still not accepted, the backing was exchanged. If the combination of claim, data, warrant and backing were not accepted, the qualifier and rebuttal were the final batch to be exchanged before all the arguments were evaluated to find out the one which were most preferred. Data Claim Data Claim Warrant Data Claim Warrant Backing Data Claim Warrant Backing Qualifier Rebuttal State of Agreement Task Proposal Objectives Proposal Evaluation and Stakeholder Perspectives Attributes Figure 14: Arguments Exchange g. Arguments were analyzed and the results are provided One thing to note is, in the process of generating and exchanging arguments, argument analysis was applied when the claim, data, warrant and backing were exchanged and when the all the components were exchanged, respectively. In the first case, the analysis was done based on the composition of objectives and attributes in the arguments to 157 indicate the distance amongst these arguments in their objectives/attributes. In the second case, the analysis was done based on the attributes value of the arguments to indicate the distance amongst these arguments in the regard of the evaluation results about how well they had achieved the objectives. In both cases, the analysis results were provided to the stakeholder for better understanding of the arguments relationship and recommendations to refine the arguments. Please refer to the Step 6.b in Section 4.4.3 for more details. As an example, Figures 15 - 18 shows a series of clustering trees built in the argument analysis based on the composition of objectives/attributes, using single linkage and power distance, on the task ‘build communication protocol’. All the results have been normalized in the 0-5 scale for a clear presentation. These clustering trees helped us track the commonality of stakeholders’ objectives, based on which we determine the compability of their arguments. Figure 15: Argument Analysis (First Round) Dendrogram(Single linkage) 3 4 2 1 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 1 2 3 4 5 Distance 0 4000 158 In the diagram in Figure 15 and the followings, the X axis represents the stakeholder: 1, 2, 3, 4 are product manager, engineering manager, engineering director and engineer, respectively. This diagram shows in the first round of the arguments generation/exchange, the engineering manager and engineer had ‘close’ arguments as the diagram indicated the arguments contained (i.e. the stakeholders had) similar objective/attributes. The commonality of these two stakeholders’ objectives is high, i.e., they have more compatible arguments. The engineering director’s argument is relatively distant from those of the first two stakeholders and the product manager had the most ‘distant’ argument in the group. According to Definition 4 in Section 4.3.2, the average height of the cluster tree is 4.19. Based on the analysis result, the following actions were recommended: 1. Asked the product manager and the engineering director (who have ‘separated’ argument from others) to consider refining or adding objectives. 2. Asked the product manager and the engineering director to talk to the engineering manager and the engineer on this task or have some cross-trainings on their backgrounds and expertise. 3. Suggested the engineering director, engineering manager, and the engineer to review the relevant product information on the design task. 4. Asked the engineering manager and the engineer to explore the possibility of combining their arguments. 159 Figure 16 shows the clustering tree built based on the analysis result in the second round. De n d r ogram ( Si ngle li nkage) 3 4 2 1 0 0. 5 1 1. 5 2 2. 5 3 3. 5 4 4. 5 5 0 1 2 3 4 5 Di st a n ce 0 4000 Figure 16: Argument Analysis (Second Round) As shown in Figure 16, in the second round, the stakeholder did slightly better in the compatibility of the arguments (especially the product manager), i.e. they had more common objectives than last round. The average height of the tree is 4.15. The design team got the same recommendations and in particular the engineering director was asked to spend more time talking to other team members about his thoughts and expectations, or consider revising his objectives and attributes. Figure 17 shows the clustering tree built based on the analysis result in the third round. 160 De n d r ogram ( Si ngle l inkage) 4 2 3 1 0 0. 5 1 1. 5 2 2. 5 3 3. 5 4 4. 5 5 0 1 2 3 4 5 Di st a n ce 0 4000 Figure 17: Argument Analysis (Third Round) The average height of the cluster tree is 4 as shown in Figure 17, indicating the compatibility in stakeholders’ arguments had been improved. The engineering director and the product manager had got relatively compatible arguments, probably because of the recommended communication and cross-training which were particularly focused on them. Figure 18 shows the clustering tree built based on the analysis result in the fourth round. The compatibility in the stakeholders’ arguments was much further improved in the fourth round, i.e. the stakeholders have more commonly agreed objectives in this round. The average height is 3.65, representing a 12.8% decrease from the initial average height of 4.19, which indicated the stakeholders had much more common objectives/attributes than in the first round. The tendency of achieving more compatibility in stakeholders’ 161 arguments also indicated the positive impact of our recommendations to the design team and the fine quality of the structured negotiation process. D endr ogram ( Si ngle l i nkage) 4 2 3 1 0 0. 5 1 1. 5 2 2. 5 3 3. 5 4 4. 5 5 0 1 2 3 4 5 D i st ance 0 4000 Figure 18: Argument Analysis (Fourth Round) After these four rounds of argument generation, exchange and refinement, the design team determined that even they obtained better understanding of each other, but they did not jointly agreed on any argument. Therefore, in order to find an argument as the outcome of the negotiation process, due to the time constraints, they decided to proceed to the argument evaluation phase with refined arguments, objectives and perspectives. h. Arguments were evaluated and the one with highest ranking was recommended After the generation, exchange and analysis of the arguments, even though the stakeholders obtained more mutual understandings and developed more compatible 162 arguments (as indicated by the argument analysis results), they might not commonly accept one argument as the outcome of the negotiation process. This is when the argument evaluation should take place by aggregating the stakeholders’ perspectives (objective importance and attribute value) to derive the one that is most preferred by the team and recommend it to the team leader. As discussed in the Step 7 of Section 4.4.3, before the arguments were evaluated by how their proposals achieved the objectives, the attributes values were first normalized into range of [0, 1] in order to decrease the effects of differences in the numeric range of attribute values and thus allows better cross- attribute comparisons. Based on the attribute values and their normalization results, the arguments were evaluated using the weighted average of all attribute values achieved by the argument. Please refer to Section 4.4.3 (step 7) for more details of normalization and argument evaluation. Based on this evaluation method and the collected data, still using the example “build communication protocol,” Table 8 shows the evaluation results and preference order of the proposals. 163 Alternative/Claim Proposed by Measurement Results Preferred Order Use XML format Specified by partners, e.g. Partner <A> device Specification Product Manager 0.68 2 Use open and well-adopted XML format for device and software specification Engineer 0.59 3 Use CSV format – Comma Separated values (Name- Value Pairs) Engineer Manager 0.56 4 Build and use XML format for device specification and registration Engineer Director 0.77 1 Table 8: Measurement Results and Preference Order Based on the measurements results and calculated preference order, the argument from the engineering director was most preferred and recommended as the resolution of the negotiation. In this example ‘Build Communication Protocol’, stakeholders spent around 2.5 hours for completing the negotiation for the solutions and there were 4 rounds of generating/exchanging the arguments amongst the stakeholders before they decided to proceed to evaluate the arguments. Table 9 shows the main results of the two case studies and the analysis of these results in order to validate the research hypothesis and justify our research approach. 164 Hypothesis Validation/Cases Second Case Study (with new approach) First Case Study (with existing practice) Analysis Time for Completing the Negotiation 21 hours 37 hours Negotiation time is decreased by 43% Average Rounds of Arguments Generation and Exchange 6 rounds 9 rounds Average number of the rounds of argument generation and exchange is decreased by 33% Average Height of Arguments’ Clustering Trees (Commonality of Stakeholders’ Objectives) First round: 4.19 Second round: 4.15. Third round: 4 Fourth round: 3.65 N/A The average height is decreased by 12.8%, i.e. the commonality of stakeholders’ objectives (the compatibilities of stakeholders’ arguments) is improved Table 9: Case Studies Results and Analysis for Hypothesis Validation The total planned time for the software design phase was the same as the first case, 5 days, 40 hours. As shown in Table 9, in fact it took the design team around 2 and half days (around 21 hours) to complete the entire design process. The average number in the rounds of generating and exchanging the arguments was 6 before the stakeholders reached an agreement for each design task. Compared with the 37 hours for the entire design process and 9 rounds in average for the arguments generation and exchange in the first case study, it showed both the total time taken for the software design phase and the average number in the rounds of generating and exchanging the arguments (before an 165 agreement is reached) had been decreased, by 43% in time and 33% in average number of the negotiation rounds respectively. Also based on the argument analysis results in the second case study, the commonality of stakeholders’ objectives (i.e., the compability of their arguments) has been improved after each round of arguments generation and exchange. Overall, the decrease in the negotiation time and the improvement in the commonality of stakeholders’ objectives have validated our research hypothesis and justified our research approach. 5.4.3 Lessons Learned Even though this case study was still limited to draw final conclusions about whether such an approach would improve the overall design process, this study still provided information that can be used to draw some conclusions. The software design team found that a well-defined approach of structuring arguments and a negotiation process of utilizing structured arguments was a feasible and effective way for multiple stakeholders to share understandings and co-construct group decisions. They also found that the hierarchical way of structuring the objectives was a useful method to organize the expectations of the entire team and find the right level of the detail in which to do the evaluation of these objectives. The objective hierarchy provided a solid base for them to construct the arguments and later to evaluate the arguments. When working on multiple tasks, it also enables them always to refer back to the common understandings built in the previous tasks. 166 In the negotiation process, it was confirmed that the commonality of stakeholders’ objectives was improved after several rounds of generating, exchanging and refining the arguments. The stakeholders felt that the argument analysis and the refinement guidelines helped them obtain better understandings of each other’s expectations, preferences and opinions which they did not pay enough attention to previously. The team had some trouble determining what recommendations they should take to refine arguments, objectives and design tasks, and how exactly they should carry out these generic recommendations. Eventually they had adapted these recommendations into their own contexts and figured out how to combine suitable arguments and revise the objectives. The argument evaluation was quite straightforward as it was possible to calculate all the stakeholders’ preferences in numbers and compare these numbers to derive the most preferred one, all happening in a timely manner. However, the predetermined scales did not turn out to be well suited for all the criteria. Part of the scales was much often used than others, so the differences between the alternatives were small. To be able to better see the differences between the alternatives, the scales should have been smaller. Another concern is the team felt that they did not have enough information or accurate data to predict all performances with certainty. Nevertheless, all the team members felt quite comfortable with using this kind of methodology, and the evaluation went smoothly and without major difficulties. Especially they felt more convinced when the most preferred one was found by preferences aggregation and recommended to the team leader who then 167 selected it, even though the team already had priori feeling that the final solution was to be chosen by the leader anyways. 5.5 Conclusion, Challenges and Limitations In this chapter, we have presented two case studies of the collaborative negotiation approach in real-life software design processes. During the software design phase, our approach supported the design team to structure the arguments with their objectives/preferences and well utilize the structured arguments, including arguments generation, exchange, analysis and evaluation. The objective hierarchy jointly built by the software team provided a sound base for the team to understand others’ expectations and declare their perspectives. It helps sharing and persisting this common understanding of multiple stakeholders for multiple tasks. Based on the objective hierarchy, stakeholders in the design team were able to organize their arguments in an effective and objective way with grounds and backing explicitly listed and followed a pre-defined order of exchanging their argument components under necessary conditions. The analysis results had helped stakeholders understand the relationship between the arguments and provided them with recommendation of refining arguments and/or objectives. The argument evaluation aggregated stakeholders’ preferences and recommended the argument with best evaluation result to the design team. In some sense, it also gave better justification to the team for choosing an argument as the outcome of the negotiation, than just having the team lead choose one. 168 From the negotiation results, it has been confirmed that by following our approach, both the total time taken for completing software design phase and the average number in the rounds of generating/exchanging arguments before having a joint agreement have been reduced. The commonality of stakeholders’ objectives (i.e. compatibilities of stakeholders’ arguments) has been increased, indicating they have obtained a better mutual understanding of each other’s expectation, preference and reasoning process. The measurement results for both of these criteria have validated our hypothesis and justified our research approach. There are two main challenges in this practice that we have handled: first, due to the nature of software industry, i.e. high development cost and urgent marketing needs, the software design phase was very short in order to fit to the overall schedule for the business requirements. In this case study, we have managed to catch up with the progress of the existing software design process, arranged different steps of the negotiation process within the design task, and kept the impact of the negotiation activities to the design process to a minimum level. The second challenge we were facing was all of the stakeholders needed to be inspired and guided to express their own understanding, and understand others’ opinions accurately, so the meaning of each objective and perspective could be well understood during the interaction. Our attempt to resolve this challenge was to explain our approach in the training session with detailed guidelines and examples, and make appropriate recommendations for argument refinement (e.g., build communication 169 channels, promote cross-training, etc.) to facilitate the communication between the stakeholders. A couple of limitations within this study should be taken into consideration when interpreting these discussions and analysis. The strategies we have used to choose case studies and validate the research hypothesis have inevitably limited the scope of applying this approach as we were making choices, instead of taking every possible case in the software design processes. The small sample size of the results, the data that was collected in a snapshot of time and the restricted exposure we had to real life software projects, due to the intensively competitive nature in software industry, also limited the generalization of these discussions. To minimize the impact of these limitations in validating our methodologies, we were interviewing a group of stakeholders both horizontally across different disciplines (product, management and engineering) and vertically spread in the corporate hierarchy (engineers, managers and directors). In the future work, a more rigorous case study could be taken against more comprehensive corporate settings such as multi-national companies, and across different industries instead of only being in the software business. Second limitation we want to discuss here is that since this validation approach was more of a descriptive method instead of an explanatory one. We could not provide the controlled conditions of mathematical models or any definite logic about cause-and-effect relationships. In addition, our case study, as all others, relied on descriptive information 170 provided by different people. This gave room for important details to be left out. Furthermore, some of the information collected was recollections of past events and therefore subject to the accuracy of the project records, existing documents and stakeholder’s memory. 171 Chapter Six: Impacts of Research This research is an original attempt to structure negotiation arguments with organized objectives and preferences and to develop models that utilized these structured arguments for carrying out effective collaborative negotiation. In this research we have built theoretical foundations and provided practical guidelines to identify and organize the objectives and preferences from decision makers and structure the negotiation arguments with these organized objectives and preferences in order to support collaborative negotiation of group decision in the virtual workspace. In more details, this research provides contributions to software engineering both in theory and practice, which are discussed as follows: In theory, this work contributes to improving the existing practices of collaborative negotiation, which mainly focuses on two stakeholders on one decision task. It develops a new approach of structuring the arguments of the design team with their objective and preferences that are organized and persisted in a value-focused objective hierarchy built for multiple decision tasks. This hierarchy provides the software design team with a platform to identify their objectives, declare their perspectives and persist these information for the design process in which they can often refer back to the common understanding that they have built in the previous decision tasks. By synthesizing this objective hierarchy with a generic argument structure, the fundamental contribution of this new approach of structuring arguments is to provide a feasible and systematic way 172 for stakeholders to generate, exchange, analyze and evaluate their negotiation arguments effectively. This synthesis improves the generic argument structure by providing feasible methods to track the relationship between the arguments and evaluate them for the most preferred. It also advanced the practices of group decision making by improving the argument-based negotiation approaches with enhanced argument structure and advanced analyzing and evaluation functionalities. In one sense, it builds a linkage between collaborative negotiation, which is a qualitative group decision methodology concerned with building and sharing arguments, and quantitative decision support tools such as simple additive weighting functions. In practice, without trying to replace any existing software design process, this work describes a negotiation process that utilizes structured arguments and works within a generic software design process. The negotiation process only takes place if there is difference detected between the design task proposals, and continues if no argument is commonly accepted. By this way, the interruption brought to the existing process is kept to a minimum while the effectiveness of establishing common understanding amongst stakeholders is much improved. Based on this process, this research has conducted case studies of applying this approach for a real-life software design process in order to illustrate how negotiation arguments can be generated, exchanged, analyzed and evaluated. These studies present how this approach can improve the effectiveness of the software design team in making group decisions by reducing the time of completing software design phase with a final design solution and improving the compability of their 173 negotiation arguments within certain number of rounds of generating and exchanging these arguments. With further theoretical improvements and more experimental results, these impacts will eventually refine the ways of managing negotiation processes and resolving design conflicts in engineering domains. 174 Chapter Seven: Summary, Conclusion, and Future Work 7.1 Summary This dissertation describes a research framework to structure arguments with organized objectives and preferences of multiple stakeholders to support their collaborative negotiation of group decision in software design. Based on this framework, our approach has developed four interrelated and systematic models to utilize these structured arguments including argument generation from stakeholders’ objectives and preferences; argument exchange that helps stakeholders share their arguments based on the logic flow of argument components, argument analysis to understand the relationship between the arguments, and argument evaluation to choose a most preferred argument based on how well the objectives have been achieved. In this section we will review the entire dissertation and emphasize the essence of each chapter. We first review the software development life cycle and identify the focus of our work in the phase of software design. The challenge of developing a new approach of structuring arguments to support collaborative negotiation of group decision in software design is clearly presented. Then we reviewed a variety of disciplines that have contributed to resolving this challenge. And we also discussed the contributions and limitations of the approaches in each discipline. Based on the summary of all the above studies plus the contribution and limitation of each, we have also described the requirements of a new 175 approach to support collaborative negotiation by structuring arguments with organized objectives and preferences of multiple stakeholders. Based on the requirements of a new approach, we first introduce two theoretical foundations for this work and discuss our research approach and the research hypothesis. The first foundation is the generic argument structure defined by Toulmin, which helps stakeholders explicitly list their perspectives (e.g., the ground and backing of their arguments) that can be easily hidden or ignored otherwise. The second foundation is an objective hierarchy that we built based on the value-focused thinking methodology to support multiple stakeholders to identify and organize their objectives and criteria. After the introduction of these two research foundations, we discuss our approach that synthesizes the generic argument structure and the objective hierarchy. This synthesis framework explains how stakeholders can generate the structured arguments according to their objectives and preferences. With this synthesis framework it helps us to overcome the challenges in existing practices of generic argument structure by developing feasible ways to analyze the arguments relationship and evaluating the arguments for the most preferred by the team. Lastly we state our research hypothesis that by structuring stakeholders’ negotiation arguments based on organized objectives and preferences, the design team will spend less time for negotiating on design tasks before reach consensual agreements, and they should have more common objectives during the collaborative negotiation process 176 Next, we present the main work of our research. It starts with clarifying the line of argument, including background, goal, hypothesis and approach, based on findings and discussions in the previous chapters. Following the line of argument, we describe how these structured arguments can be utilized to build research modules for generating, exchanging, analyzing and evaluating negotiation arguments. After then, we present the details about how a collaborative negotiation process can be developed based on these four interrelated modules. With a summary of this approach, we also outline the necessity of case studies as the empirical validation. This dissertation also presents the case studies that we have conducted in real-life software design processes in order to demonstrate the application of our approach and validate our research hypothesis. Hereby we have introduced two case studies with adequate data and different in nature. The first case study was undertaken without applying our approach. The second case study was a test of our application in order to compare with the results of the first case. We discuss the details and results of these two case studies and demonstrate the validation of our hypothesis by quantitative data analysis. Last but certainly not least, we have pointed out the impact of this research work in supporting collaborative negotiation of group decision in software design. This work is an original attempt to build a theoretical framework to support collaborative negotiation by organizing stakeholders’ objectives and preferences and structuring them into the 177 negotiation arguments. Theoretically this work improves the practices of the generic argument structure by providing feasible and systematic approaches to generate, exchange, analyze and evaluate structured arguments. It also advances the state of art in value-focused decision support framework by structuring the values of the entire design team by utilizing structured arguments. Practically the framework and approaches developed in this work have devised a collaborative negotiation process with detailed guidance for multiple stakeholders to generate exchange, analyze and evaluate their arguments. With that support, software design team can still comply with their existing design process and within that process our approach will help them structure their conflicting opinions and guide them through the phases of generating, exchanging, analyzing and evaluating their structured arguments. Therefore, the interruption brought by our approach to the existing software design process is minimal while the effectiveness of collaborative negotiation in the software design team and the compatibility of stakeholders’ arguments will be improved. 7.2 Conclusion This framework is built based on a strategic synthesis between the generic argument structure and an objective hierarchy. This synthesis framework lays out the correspondence between the components in the argument structure and stakeholders’ objectives and preferences. Based on the synthesis framework, our approach specifies how the arguments can be generated based on stakeholders’ proposals, objectives, attributes (i.e., measurement criteria of objectives) and expected preferences. In other 178 words, it explains how the required information and meta-information in collaborative negotiation can be structured into negotiation arguments. Enabled by this synthesis framework, this work also specifies a practical way of analyzing the relationship of arguments and evaluating their performances in achieving the targeted objectives. Clustering analysis is applied to indicate the similarity and distance amongst these structured arguments and the analysis results are provided to the stakeholder to better understand and compare the arguments relationship and refine the arguments. Arguments can also be evaluated to select the most preferred argument based on how well the arguments achieve the objectives. Overall, this work has synthesized applicable knowledge from relevant studies and then carefully evaluated this knowledge in light of existing practices in collaborative software design. With our approaches and the case studies that we conducted in real-life software design processes, we have demonstrated that structuring stakeholders’ negotiation arguments based on organized objectives and preferences will support the design team to spend less time for negotiating on design tasks before reach consensual agreements, and have more commonly agreed objectives during the collaborative negotiation process. Certainly, this approach has some limitations and two significant ones are listed as follows. First is the complexity for gathering the research data. For instance, in order to generate arguments in the negotiation phase, the stakeholders need extra work to identify objectives and declare perspectives based on their expertise and understandings. Due to 179 the high-cost and tight-schedule nature of software industry, this limitation may become a roadblock to implement this approach in real life as it may not fit into the overall schedule of the software development life cycle. Second, in our research we assume that every objective has attributes that can be used as common scales to measure all the arguments and yield independent measurement value for each alternative. Therefore, we can use cardinal preferences in evaluating arguments and find out the mostly desired one. This assumption has been proven feasible in our case study but may need much more verification when the approach is applied in more projects and fields. 7.3 Future Work Although the provided approach and analyses constitute a set of valuable and feasible methods to support collaborative negotiation of group decision in software design, there are some improvements that can still be done. In this section we will briefly describe several interesting research topics that are worth investigating further. First, when building the objective hierarchy, more templates having sample objectives and attributes should be devised and provided to the stakeholders to start with when proposing their own objectives and attributes. These templates should be proposed based on software engineering or other related domain and covers the most common objectives of that domain. 180 Second, it is necessary to develop more accurate and comprehensive model to quantify stakeholders’ perspective and also take into account the evolutions of stakeholders’ perspectives under different circumstances. The argument evaluation results will be much improved when the perspectives and their evolution are well collected. Third, we need to extend the case study to more software design projects, preferably in various fields and with different scales, and then gather more research data and argument analysis results. We may also need to employ more advanced analysis methods and metrics especially regarding how effectively this approach has helped software team carry out the collaborative negotiation activities. Overall, this work gains us an enhanced option to structure arguments in order to effectively support collaborative negotiation in group decision of modern software design processes. In future work we wish to transfer the lessons learned to other fields of engineering designs, such as new product developments, to broaden the research impacts. Also with more intensive theory refinement and data analysis results gathered, the framework and approaches will be continuously improved to eventually leading to the establishment of a scientific foundation for collaborative software engineering. 181 References Adolphs, R (1999), "Social Cognition and the Human Brain", Trends in Cognitive Sciences 3: 469-79. Aldrich, J., Garlan, D., Schmerl, B., Shaw, M., and Wing J. (2006), “Software Engineering Research in the Computer Science Department of at Carnegie Mellon”, a web tutorial, http://www.csd.cs.cmu.edu/research/areas/softeng/. Alexander, C. (1979), “The Timeless Way of Building”, Oxford University Press. Amgoud, L., Maudet, N., and Parsons, S. (2000), “Modeling dialogues using argumentation, MultiAgent Systems”, Proceedings. Fourth International Conference on 10-12 July 2000 Pages: 31 – 38. Anderson, R.M., Hobbs, B. F., and Bell, M.L. (2002), “Multiobjective decision making in negotiation and conflict resolution”, Chapter 6 of Topic 1.40.4, “Formal Models for Conflict Resolution” edited by K.W. Hipel, prepared for The Encyclopedia of Life Support Systems (EOLSS). Apache Software Foundation (2006), “Apache Projects”, www.apache.org. Arrow, K. J. (1950), "A Difficulty in the Concept of Social Welfare", Journal of Political Economy 58(4), August, 1950, pp. 328–346. Arrow, K. J. (1951), “Social choice and individual values”, Wiley, New York. Arrow, K. J. (1977), “Current developments in theory of social choice”, Social Research 44(4): 607–622. Arrow, K. J. (1987), “Reflections on the essays. In Arrow and the foundations of the theory of economic policy”, ed. G. Feiwel, U.K.: Macmillan, pp. 727–734. Avery, J., Yearwood, J., and Stranieri, A. (2001), “An Argumentation based Multi-agent System for eTourism Dialogue”, Proceedings of International Workshop on Hybrid Intelligent Systems (HIS'01), December 2001, Adelaide, Australia, pp.194-210. 12. Axelrod, R. (1984), “The Evolution of Co-operation”, Basic Books Inc., NY. Argote, L., McEvily, B., and Reagams, R. (2003), "Managing knowledge in organizations: An integrative framework and review emerging themes”, Management Science, vol. 49, no. 4, pp. 571-582, 2003. Baker, K., Greenberg, S., and Gutwin, C. (2001), “Heuristic Evaluation of Groupware Based on the Mechanics of Collaboration”, Proceedings of the 8th IFIP International Conference on Engineering for Human-Computer Interaction, p.123-140, May 11-13, 2001. 182 Baker, K., Greenberg, S., and Gutwin, C. (2002), “Empirical development of a heuristic evaluation methodology for shared workspace groupware”, Proceedings of the 2002 ACM conference on Computer supported cooperative work, November 16-20, 2002, New Orleans, Louisiana, USA. Barker, R., Holloway, L.P., and Meehan, A. (2001), "Supporting Negotiation in Concurrent Design Teams", Proceedings of the Sixth International Conference on CSCW in Design, 2001, pp. 243 - 248. Begole, J., Tang, J. C., and Hill, R. (2003), “Rhythm modeling, visualizations and applications”, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.11-20, November 02-05, 2003, Vancouver, Canada. Berger, P.L., and Luckmann, T. (1966), “The Social Construction of Reality - a Treatise in the Sociology of Knowledge”, Anchor Books, Garden City, NY. Boehm, B., Egyed, A., Port, D., Shah, A., Kwan, J., and Madachy, R. (1999) “A Stakeholder Win-Win Approach to Software Engineering Education”, Annals of Software Engineering. Boehm, B., Port, D., Huang, L. G., and Brown, W. (2002), “Using The Spiral Model and MBASE to Generate New Acquisition Process Models: SAIV, CAIV, and SCQAIV”, CrossTalk, pp. 20-25 January. Briggs, R.O., de Vreede, G.J., and Nunamaker, J.F. (2003), "Collaboration Engineering with ThinkLets to Pursue Sustained Success with Group Support Systems", Journal of Management Information Systems, 19 (4), 2003, 31-64. Brockriede, W. and Ehninger, D. (1960), “Toulmin on Argument: An Introduction and Application”, Quarterly Journal of Speech, Vol. 46, 1960, pp 44-53. Buckingham, S., MacLean, A., Bellotti, V., and Hammond, N. (1997), “Graphical Argumentation and Design Cognition”, Human-Computer Interaction, 12(3), pp. 267- 300. Bui, T. (1987), “Co-op: A Group Decision Support System for Cooperative Multiple Criteria Group Decision-Making”, Springer-Verlag. Bui, T. (1993), “Designing Multiple Criteria Negotiation Support Systems: Framework, Issues and Implementation,” in MCDM: Expand and Enrich the Domains of Thinking and Application, eds. Tzeng et. al., Springer Verlag. Bui, T. (1994), “Evaluating Negotiation Support Systems: A Conceptualization”, in Proceedings of the 27th Annual Hawaii International Conference on Systems Sciences, Vol. 4, 316-324. Burnett, R. (1993), “Conflict in Collaborative Decision-making”, in Professional Communication: The Social Perspective, N. Roundy Blyler and C. Thralls, Eds., Newbury Park, CA: Sage Pub., 1993, pp. 144-162. 183 Campbell, M.I., et. al. (1999), “A-Design: An Agent-based Approach to Conceptual Design in a Dynamic Environment”, Research in Engineering Design, Vol. 11, 172-192. Chang, A.M. and Han, T.D. (1995), “Design of an Argumentation-based negotiation support system”, System Sciences, 1995, Vol. IV, Proceedings of the Twenty-Eighth Hawaii International Conference on Volume: 4, 3-6 Jan. 1995 Pages: 242 - 251 vol.4. Chen, J., Anane, R., Chao, K., and Godwin, N. (2002), “Architecture of an Agent-Based Negotiation Mechanism”, in Proceedings of the 22nd international Conference on Distributed Computing Systems (July 02 - 05, 2002), ICDCSW, IEEE Computer Society, Washington, DC, 379-384. Choi H., Panchal J.H., Allen K. J., Rosen W.D., and Mistree F. (2003), “Towards a standardized engineering framework for distributed collaborative product realization”, presented at the Design Eng. Tech. Conf. Comput. Inform. Eng. Conf., Chicago, IL, 2003. Clancey, W. J. (1993), “Guidon-Manage Revisited: A Socio-Technical Systems Approach”, Journal of Artificial Intelligence in Education Vol. 4(1):5-34. Clancey, W. J (1997), “The Conceptual Nature of Knowledge, Situations, and Activity”, in Feltovich, P., Hoffman, R. and Ford, K., Eds. Human and Machine Expertise in Context, 247-291. AAAI Press, CA. Cook, C. and Churcher, N. (2003), “An Extensible Framework for Collaborative Software Engineering”, Proceedings of the Tenth Asia-Pacific Software Engineering Conference Software Engineering Conference, p.290, December 10-12, 2003. Cook, C. and Churcher, N. (2005), “Modeling and measuring Collaborative Software Engineering”, In Proceedings of the Twenty-Eighth Australasian Conference on Computer Science - Volume 38 (Newcastle, Australia). V. Estivill-Castro, Ed. ACM International Conference Proceeding Series, vol. 102. Australian Computer Society, Darlinghurst, Australia, 267-276. Cooper, S. and Taleb-Bendiab, A. (1998), "CONCENSUS: Multiparty Negotiation Support for Conflict Resolution in Concurrent Engineering Design", Journal of Intelligent Manufacturing, 9: 155-159, 1998. Cross, R., Borgani, S.P., and Parker, A. (2001), "Beyond Answers: Dimensions of the Advice Networks”, Social Networks, vol. 23, pp. 215-235.2001. Cross, R., Rice, R. E., and Parker, A. (2001), "Information Seeking in Social Context: Structural Influences and Receipt of Information Benefits", IEEE Trans. Systems, Man, and Cybernetics-Part C: Application and Reviews, vol. 31, no.4, pp. 438 – 448, 2001. Curtis, B. (1986), “By the way, did anyone study any real programmers?” Papers presented at the first workshop on empirical studies of programmers on Empirical studies of programmers, p.256-262, June 1986, Washington, D.C., United States. 184 Curtis, B., Krasner, H., and Iscoe, N. (1988), “A Field Study of the Software Design Process for Large Systems”, Communications of the ACM, 31 (11). Curtis, B., Krasner, H., and Iscoe, N. (1998), “A Field Study of the Software Design Process for Large Systems”, Communications of the ACM, v.31 n.11, p.1268-1287, Nov. 1988. Dahl, R.A. (1956), “A Preface to Democratic Theory”, Charles R. Walgreen Foundation lectures, Chicago, University of Chicago Press. Davis, R. (1979), “Interactive Transfer of Expertise: Acquisition of New Inference Rules”, Artificial Intelligence 12, p121-157. Davis, R. and Smith, R.G. (1983), “Negotiation as a Metaphor for Distributed Problem Solving,” Artificial Intelligence, Vol. 20, 63-109. Deutsch, M. (1973), “The Resolution of Conflict”, Yale University Press, New Haven. De Vreede, G.J. and Briggs, R.O. (2005), "Collaboration Engineering: Designing Repeatable Processes for High-Value Collaborative Tasks", Proceedings of the 38th Hawaii International Conference on Systems Sciences, Hawaii, IEEE Computer Society Press, 2005. De Vreede, G.J., Vogel, D.R., Kolfschoten, G.L., and Wien, J.S. (2003), "Fifteen Years of in-situ GSS Use: A Comparison across Time and National Boundaries", Proceedings of the Thirty-Sixth Hawaii International Conference on System Sciences, Hawaii, IEEE Computer Society Press, 2003. Dommel, H.P. and Garcia-Luna-Aceves, J. J. (1997), “Floor Control for Multimedia Conferencing and Collaboration”, Multimedia Systems, v.5 n.1, p.23-38, Jan. 1997. Dunskus, B. V. (1995), “Using Single Function Agents to investigate conflict”, AIEDAM, Vol. 9, No. 4 299-313. Durfee, E.H. and Lesser, V.R. (1989), “Negotiating Task Decomposition and Allocation using Partial Global Planning,” in Distributed Artificial Intelligence, Vol.2, eds. M. Huhns and L. Gasser, San Mateo, CA: Morgan Kaufmann. Easterbrook, S. (1994), “Resolving requirements conflicts with computer-supported negotiation”, In Requirements Engineering: Social and Technical Issues, M. Jirotka and J. A. Goguen, Eds. Academic Press Professional, San Diego, CA, 41-65. Etzioni, A. (1967), “Mixed-Scanning: A "Third" Approach to Decision-Making”, Public Administration Review, Vol. 27, No. 5. (Dec., 1967), pp. 385-392. Faratin, P. (2000), Automated Service Negotiation Between Autonomous Computational Agents, Ph.D. thesis, Department of Electronic Engineering, Queen Mary and Westfield College, University of London. 185 Fatima, S., Wooldridge, M. and Jennings, N.R. (2002), “Multi-issue Negotiation under Time Constraints”, in Castelfranchi, C. & Johnson, L. (eds.), Proceedings of the 1st International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS-2002), New York, USA. ACM Press, pp. 143–150. Feather, M.S. (1989), "Detecting Interference when Merging Specification Evolutions," Unpublished report, Information Sciences Institute, University of Southern California, Marina del Rey, CA (1989). Fielding, R.T. and Kaiser, G. (2006), “The Apache HTTP Server Project”, Internet Computing, IEEE, Volume 1, Issue 4, July-Aug. 1997 Page(s):88 – 90. Fishburn, P.C. (1973), The Theory of Social Choice, Princeton University Press, Princeton, N.J. Fogel, K.F. (2001), Moshe Bar, Open Source Development with Cvs, Coriolis Group Books, Scottsdale, AZ, 2001. Fulkerson, R. (1996), “The Toulmin model of argument and the teaching of composition”, in B. Emmel, P. Rescb, & D. Tenney (Eds.), “Argument revisited, argument redefined: Negotiating meaning in the composition classroom”, Pp. 45-72, Sage: Thousand Oaks, CA, 1996. Gearhart, K.A. (1992), “A Collaborative Writing Project in a Technical Communication Course”, Technical Communication, 39(3), 360-366. Gersick, C.J. (1988), “Time and Transition in Work Teams: Toward a New Model of Group Development”, the Academy of Management Journal, 31, 1 (1988), 9~4i. Greenberg, S. (1989), “A Survey of Computer-Supported Co-operative Work”, Draft Report, Alberta Research Council, Calgary, Canada. Grohowski, R., McGoff, C., Vogel, D.R., Martz, B., and Nunamaker, J.F. (1990), "Implementing Electronic Meeting Systems at IBM: Lessons Learned and Success Factors", Management Information Systems Quarterly, 14 (4), 1990, 327-345. Hamilton-Weiler, S. (1990), “Collaborative Classrooms: Building a Community of Writers,” Writing on the Edge, vol. 2, no. 2, pp. 19-30, 1990. Hample, D. (1977), “The Toulmin Model and the Syllogism,” Journal of the American Forensic Association, Vol. 14, summer 1977. Harsanyi, J.C. (1955), “Cardinal Welfare, Individualistic Ethics, and Interpersonal Comparisons of Utility”, Journal of Political Economy, 63/, 309-321. Hart, K.A. (1990), “Teaching thinking in college. Accent on improving college teaching and learning” (ERIC Document Reproduction service No. ED 332 613). Hauser, J.R. and Clausing, D. (1988), The House of Quality. Harvard Business Review, May-June, 63-73. 186 Hazelrigg, G.A. (1996), “The Implications of Arrow's Impossibility Theorem on Approaches to Optimal Design”, ASME Journal of Mechanical Design, 118/2, 161-164. Hazelrigg, G.A. (1998), “Framework for Decision-Based Engineering Design”, Journal of Mechanical Design, Transaction of the ASME, 120/4, 653-658. Hernandez G. and Seepersad C.C., “Design for maintenance: A game theoretic approach,” Eng. Optim., vol. 34, no. 6, pp. 561–577, 2002. Hillinger, C. (2005), "The Case for Utilitarian Voting," Discussion Papers in Economics, University of Munich, Department of Economics. Hocker J.L. and Wilmot W.W. (1985), Interpersonal Conflict, Second Edition, William C. Brown, Dubuque, IA, 1985. Houp, K.W., Pearsall T.E., and Tebeaux E. (1998), “Reporting Technical Information”, 9th Edition. Oxford UP New York. 1998. Huhns, M.N. (ed) (1987), “Distributed Artificial Intelligence”, Morgan Kaufmann Publishers Inc, Los Altos CA. Huhns, M. and Gasser L. (1989) (eds), Distributed Artificial Intelligence, Vol. 2, San Mateo, CA: Morgan Kaufmann. In, H., Olson D., and Rodgers T. (2002) “Multi-Criteria Preference Analysis for Systematic Requirements Negotiation”, IEEE International Computer Software and Applications Conference (COMPSAC 2002), pp. 887-892, Oxford, UK. In, H., Olson, D., and Rodgers, T. (2001), “A Requirements Negotiation Model based on Multi-criteria Analysis”, Requirements Engineering, 2001, Proceedings of Fifth IEEE International Symposium on 27-31 Aug. 2001 Page(s):312 – 313. Irish, R.K. (2003), “The Uses of Toulmin: the Value of an Explicit Model of Argument for Academic and Working engineers”, Proceedings of Professional Communication Conference, IPCC 2003, IEEE International Volume/Issue 21-24 Sept. 2003 Page(s): 8. Janssen, T. (1996), Toulmin Argument Structures for Science Assessment, Doctoral Dissertation, George Mason University, Fairfax, VA, 1996. Janssen, T. and Sage, A.P. (1996), “Group Decision Support using Toulmin Argument Structures”, IEEE International Conference on Systems, Man, and Cybernetics, Volume: 4, on page(s): 2704-2709 Vol.4, Oct. 1996. Jelassi, M. T. and Foroughi A. (1989), “Negotiation Support Systems: An Overview of Design Issues and Existing Software,” Decision Support Systems, Vol. 5, 32-49. Jin, Y., Geslin, M., and Lu, S. C.-Y. (2005), “Impact of Argumentative Negotiation on Collaborative Engineering”, IMPACT Laboratory, Department of Aerospace and Mechanical Engineering, University of Southern California, Los Angeles, USA 2005. Kahneman, D. (2003), “Maps of Bounded Rationality: Psychology for Behavioral Economics”, The American Economic Review. 93(5). pp. 1449-1475. 187 Kautz, H., Selman, B., and Shah, M. (1997), “Referralweb: Combining Social Networks and Collaborative Filtering”, Communications of the ACM, 40(3):63–65, 1997. Kazman, R. (2005), “The Essential Components of Software Architecture Design and Analysis”, Software Engineering Conference, 2005, APSEC '05 12th Asia-Pacific 15-17 Dec. 2005 Page(s):1 pp. Digital Object Identifier 10.1109/APSEC.2005.103. Kazman, R., Klein, M., Barbacci, M., Longstaff, T., Lipson, H., and Carriere, J. (1998), “The Architecture Tradeoff Analysis Method”, Engineering of Complex Computer Systems, 1998. ICECCS '98. Proceedings of Fourth IEEE International Conference on 10-14 Aug. 1998 Page(s):68 – 78. Keeney, R. L. and Raiffa, H. (1976) Decisions with Multiple Objectives: Preferences and Value Tradeoffs, J. Wiley & Sons, NY. Keeney, R.L. (1992) “Value-focused Thinking”, Harvard University Press, Cambridge, 1992. Kersten, G. E. (1985), “NEGO - Group Decision Support System”, Information and Management, Vol. 8, No. 5, 237-246. Kersten, G.E., Michalowski, W., Szpakowicz, S., and Koperczak, Z. (1991), “Restructurable Representations of Negotiation,” Management Science, Vol. 37, No. 10, 1269-1290. Khan, F., Fisher, T., Shuler, L., Wu, T., and Pottenger, W. (2002), “Mining Chat-room Conversations for Social and Semantic Interactions”, 2002. Kilker, J. (1999), “Conflict on Collaborative Design Teams: Understanding the Role of Social Identities”, IEEE Technology and Society Magazine. Fall, 12-21. Kirkwood, C. W. (1997), “Strategic Decision Making: Multi-objective Decision Analysis with Spreadsheet:, Belmont, CA, Wadsworth Publishing Company, 1997. Klein, M. (1994), "Integrated Support for Cooperative Design Coordination: Managing Processes, Conflicts and Memories", in: S.Y.Nof (eds), Information and collaboration models of integration, Kluwer Academic Publishers, Netherlands, 1994, pp. 435-459. Klein, M. (1995), "Conflict Management as Part of an Integrated Exception Handling Approach", Artificial Intelligence for Engineering Design, Analysis, and Manufacturing, 9: 259-267, 1995. Kontio, M. (2004), “Architectural Manifesto - Design Software Architectures”, http://www-128.ibm.com/developerworks/wireless/library/wi-arch7/#, October. Kowalczyk, R. and Bui, V., (2001), “On Constraint-based Reasoning in E-negotiation Agents”, In Dighum, F. & Cortes U. (eds.), Agent-Mediated Electronic Commerce III (Lecture Notes in Computer Science, Vol. 2003), Berlin: Springer-Verlag, pp. 31-46. Kraus, S. (2001), “Automated Negotiation and Decision Making in Multiagent Environments”, Lecture Notes in Artificial Intelligence 2086, pp 150. 188 Kraus, S. (2001), Strategic Negotiation in Multi-Agent Environments, Cambridge, MA, MIT Press. Kraus, S., Sycara, K. and Evenchik, A. (1998), “Reaching Agreements through Argumentation: a Logical Model and Implementation”, Artificial Intelligence 104 (1-2), 1-69. Krogstie, J., Lindland, O.I., and Sindre, T., (1995), “Towards a Deeper Understanding of Quality in Requirements Engineering,” Proceedings of the 7th International Conference on Advanced Information Systems Engineering (CAiSE’95) (Springer-Verlag, Jyvaskyla, Finland, 1995), pp 82-95. Lagomasino, A. and Sage, A.P. (1985), “An Interactive Inquiry System”, Large Scale Systems, Vol. 9, No. 3, 1985, pp. 231-244. Lagomasino, A. and Sage, A. P. (1985), “Representation and Interpretation of Information for Decision Support with Imperfect Knowledge”, Large Scale Systems, Vol. 9, No. 2, 1985, pp. 169-181. Lara M.A. and Nof S.Y. (2003), "Computer-supported Conflict Resolution for Collaborative Facility Designers", International Journal of Production Research, 41(2): 207-234, 2003. Laskey, K. B., Chen, M. S., and Martin, A. W. (1989), “Representing and Eliciting Knowledge about Uncertain Evidence and its Implications”, IEEE Transactions on Systems, Man, and Cybernetics, Vol. 19, No. 3, May 1989, pp. 536-545. Lawrence, A. B. (2001), “Towards a Useful Methodology Discipline”, Journal of Economic Methodology, 8, 2001, 3-10. Lee Y. and Choi H.-J. (2005), “Experience of Combing Qualitative and Quantitative Analysis Methods for Evaluating Software Architecture”, Proceedings of the Fourth Annual ACIS International Conference on Computer and Information Science (ICIS 2005) 152-157. Lim, L.-H. and Benbasat, I. (1991), “From Negotiation to Negotiation Support Systems: A Theoretical Perspective,” Working Paper, Faculty of Commerce and Business Administration, University of British Columbia, Vancouver, B. C., Canada. Lu, S. C.-Y. (2001), “Engineering as Collaborative Negotiation: A New Foundation for Collaborative Engineering Research”, The ECN Working Group of the International Institution of Production Engineering Research (CIRP), see http://wisdom.usc.edu/ecn. Lu, S. C.-Y. (2003), “Engineering as Collaborative Negotiation: A New Paradigm for Collaborative Engineering Research”, http://wisdom.usc.edu/ecn/about_ECN_what_is_ECN.htm. Lu, S. C.-Y. (2007), “A Scientific Foundation of Collaborative Engineering”, 2007 International Academy of Production Engineering (CIRP) Keynote Paper (STC-Dn: Design), Annals of the CIRP Vol. 56/2/2007. 189 Lu, S. C-Y. and Cai, J. (1999), “Modeling Collaborative Design Process with a Socio- Technical Framework”, Proc. Sixth ISPE Int’l Conf. Concurrent Engineering, Bath, UK. Lu, S. C-Y., Cai, J., Burkett, W., and Udwadia, F. (2000), “A Methodology for Collaborative Design Process and Conflict Analysis”, CIRP Annals, 49(1), 69-73. Lu, S. C-Y. and Cai, J. (2000), “STARS: A Socio-Technical Framework for Integrating Design Knowledge over the Internet”, A Special Issue in IEEE Internet Computing, Vol. 4, No. 5, pp. 54-62. Lu, Stephen C.-Y. and Cai, J. (2001), “A Collaborative Design Process Model in the Socio-technical Engineering Design Framework,” Artificial Intelligence Engineering Design, Anal. Manufacturing, vol. 15, pp. 3–20, 2001. Lu, S. C-Y., Elmaraghy, W., Schuh, G. and Wilhelm, R. (2007), “A Scientific Foundation of Collaborative Engineering”, CIRP Annals - Manufacturing Technology, Volume 56, Issue 2, 2007, Pages 605-634. Lu, S. C.-Y., Zhang, J.-Y., Wang, C-T., and Grobler, F. (2005), “Modeling Design Processes and Stakeholder Perspectives to Support Collaborative Engineering Negotiation: a Case Study of Designing Individualized Prostheses over the Internet”, IJCAT 23(1): 2-12 (2005). Martson, C.M. (2000), “Game based Design: A Game Theory based Approach to Engineering Design,” Ph.D. dissertation, Georgia Inst. Technol., Atlanta, GA, 2000. Marttunen, M. (1992). “Commenting on Written Arguments as a Part of Argumentation Skills – Comparison between Students Engaged in Traditional vs. On-line Study”, Scandinavian Journal of Educational Research, 36(4), 289-302. Moore, M., Kazman, R., Klein, M., and Asundi, J. (2003), "Quantifying the Value of Architecture Design Decisions: Lessons from the Field", Proceedings of the 25th International Conference on Software Engineering (ICSE 25), Portland, Oregon, May 2003. Nash, J (1950), The Bargaining Problem, Econometrica, Vol. 18, 155-162. Neiger, D. and Churilov, L. (2006), “Intelligent Decision Support through Synchronized Decomposition of Process and Objectives Structures”, System Sciences, HICSS, Proceedings of the 39th Annual Hawaii International Conference, Volume 2, Issue, 04-07 Jan. 2006 Page: 31a - 31a. Nemhauser, G.L., Rinnoy Kan, A.H.G. and Todd, M.J. (1989) Handbooks in Operations Research and Management Science: Volume 1 Optimization, North-Holland, Amsterdam. Newman, M.E.J., Watts, D., and Strogatz, S. (2002), “Random Graph Models of Social Networks”, in Proceedings of the National Academy of Sciences of the United States of America, volume 99, pages 2566–2572, February 2002. 190 Newman, M.E.J. (2001), “The Structure of Scientific Collaboration Networks”, in Proceedings of the National Academy of Science, volume 98, pages 404–409, January 2001. Nikolaidis, E. (2007), “Decision-based Approach for Reliability Design,” ASME Journal of Mechanical Design, vol. 129, no. 5, pp. 466–475, 2007. Nunamaker, J.F., Dennis, A., Valacich, J., Vogel, D., and George, J.F. (1991), "Electronic Meeting Systems to Support Group Work", Communications of the ACM, 34 (7), 1991, 40-61. Parsons, S., Sierra, C. and Jennings, N. (1998), “Agents that Reason and Negotiate by Arguing”, Journal of Logic and Computation 8 (3), 261-292, Post, B.Q. (1993), "A Business Case Framework for Group Support Technology", Journal of Management Information Systems, 9 (3), 1993, 7-26. Qualley, D. and Chiseri-Strater, E. (1994), “Collaboration as ‘Reflective Dialogue: A Knowing ‘Deeper than Reason,’ ” J. Advanced Computer vol. 14, no. 1, pp 111-130, 1994. Raiffa, H (1982), “The Art and Science of Negotiation”, Cambridge, MA: Harvard University Press. Rapoport, A., (ed) (1974), “Game Theory as a Theory of Conflict Resolution”, D. Reidel Publ. Co., Dordrecht, Holland. Reagans, R., and MeEvily, B. (2003), “Newark structure and knowledge transfer: The transfer problem revisited”, Working Paper, Columbia University, New York, 2003. Robbins, S.P. (1974), Managing Organizational Conflict: A Non-traditional Approach, Prentice Hall, NJ. Robbins, S.P. (1989), Organizational Behavior: Concepts, Controversies, and Applications, (fourth edition) Prentice Hall, NJ. Rong, J., Geng, S. J., Valasek, J., and Ioerger, T.R. (2002), “Air Traffic Conflict Negotiation and Resolution using an Onboard Multi-agent System”, Digital Avionics Systems Conference, 2002, in proceedings. The 21st, Volume: 2, 2002, Pages: 7B2-1 - 7B2-12 vol.2. Rosenschein, J. S. (1985), “Rational Interaction: Co-operation among Intelligent Agents”, Ph.D. Thesis, Report No STAN-CS-85-1081, Dept of Computer Science, Stanford University, Stanford, CA. Rosenschein, J. S., and Genesereth, M. R. (1985), “Deals among Rational Agents”, in Proceedings of Ninth International Joint Conference on Artificial Intelligence, p91-99. Rosenschein, J. and Zlotkin, G., (1994), Rules of Encounter: Designing Conventions for Automated Negotiation among Computers, Cambridge, MA, MIT Press 191 Rymer, J. (1993), “Collaboration and conversation in learning communities: The discipline and the classroom,” in Professional Communication: The Social Perspective, N. Roundy Blyler and C. Thralls, Eds. Newbury Park, CA: Sage Pub., 1993, pp. 179-195. Saaty, T.L. (1980), The Analytic Hierarchy Process. New York: McGraw-Hill. Sage, A.P. (1992), Systems Engineering, John Wiley and Sons, New York, 1992. Sage, A.P. (1991), “On the Processing of Imperfect Information Using Structured Frameworks,” Chapter 7 in Kandel, A. (Ed.), Fuzzy Expert Systems, CRC Press, New York, 1991, pp. 99-1 12. Sandholm, T., (2002a), “Algorithm for Optimal Winner Determination in Combinatorial Auctions”, Artificial Intelligence 135 (1-2), 1-54. Sandholm, T., (2002b), “eMediator: a Next Generation Electronic Commerce Server”, Computational Intelligence, Special issue on Agent Technology for Electronic Commerce 18 (4), 656-676. Schum, D. (1994), Evidential Foundations of Probabilistic Reasoning, John Wiley and Sons, New York, 1994. Schwartz, M. and Wood, D. (1993), “Discovering shared interests using graph analysis”, Communications of the ACM, 36(8):78–89, Aug 1993. Scott, J. (1991). Social Network Analysis: A Handbook. SAGE Publications, 1991. Selfa, D.M., Carrillo, M., and Del Rocio Boone, M. (2006), “A Database and Web Application Based on MVC Architecture”, Electronics, Communications and Computers, 2006. CONIELECOMP 2006. 16th International Conference on 27-01 Feb. 2006 Page(s):48 – 48. Sengupta, K., Te'eni, D. (1991), “Reducing Cognitive Conflict through Feedback in GDSS: an Experimenting the Formulation of Group Preferences”, System Sciences, 1991. Proceedings of the Twenty-Fourth Annual Hawaii International Conference on Volume iii, Issue, 8-11 Jan 1991 Page(s):631 - 640 vol.3 Shakun, M.F. (1992), “Negotiation”, Group Decision and Negotiation, Vol. 2. Shaw, M.L.G., and Gaines, B.R. (1988), “A Methodology for Recognizing Consensus, Correspondence, Conflict, and Contrast in a Knowledge Acquisition System”, Proceedings, Third Knowledge Acquisition For Knowledge-Based Systems Workshop, Banff, November 1988. Shaw, M.L.G., and Woodward, J.B. (1989), “Mental Models in the Knowledge Acquisition Process”, Proceedings, Fourth Knowledge Acquisition for Knowledge-Based Systems Workshop, Banff, October 1989. Shen, W., Norrie, D.H. and Barthes, J.P (2001), “Multi-Agent Systems for Concurrent Intelligent Design and Manufacturing”, Taylor and Francis, London, UK, 2001. 192 Sierra, C., Jennings, N.R., Noriega, P., and Parsons, S. (1998), “A Framework for Argumentation-based Negotiation”, in Singh, Rao, A. & Wooldridge, M. (eds), Intelligent Agent IV: 4th, International Workshop on Agent Theories, Architectures and Languages (ATAL – 1997) (Lecture Notes in Artificial Intelligence, Vol 1365), Berlin, Springer-Verlag, pp 177-192. Sillince, J.A.A. and Saeedi, M.H. (1999), “Computer-mediated Communication: Problems and Potentials of Argumentation Support Systems”, Decision Support Systems, 26, 287-306. Simon, H. (1956), “Rational Choice and the Structure of the Environment,” Psychol. Rev., vol. 63, no. 2, pp. 129–138, 1956. Simon, H. (1957). “A Behavioral Model of Rational Choice", in Models of Man, Social and Rational: Mathematical Essays on Rational Human Behavior in a Social Setting. New York: Wiley. Simpson, T.W., Seepersad, C.C., and Mistree, F. (2001), “Balancing Commonality and Performance within the Concurrent Design of Multiple Products in a Product Family,” Concurrent Eng. Res. Appl., vol. 9, no. 3, pp. 177–190, 2001. Smith, D.G. (1977), “College Classroom Interactions and Critical Thinking”, Journal of Educational Psychology, 69(2), 180-190. Soloway, E. (1986), “What to Do Next: Meeting the Challenge of Programming-in-the- large”, Papers presented at the first workshop on empirical studies of programmers on Empirical studies of programmers, p.263-268, June 1986, Washington, D.C., United States. Song, J., Almeida, P., and Wu, G. (2003), "Learning by Hiring: When is Mobility More Likely to Facilitate Interfirm Knowledge Transfer?" Management Science, vol. 49, no. 4, pp. 351-365, 2003. Stary, C., (1991), “Modeling Decision Support for Rational Agents”, Proceedings of the European Simulation Multiconference, ed. E. Mosekilde, pp. 351-356. Stefik, M., Foster, G., Bobrow, D. G., Kahn, K., Lanning, S., and Suchman, L. (1987), “Beyond the Chalkboard: Computer Support for Collaboration and Problem Solving in Meetings”, Communications of the ACM 30 (1). Steuer, R. E. (1986), Multiple Criteria Optimization: Theory, Computation and Application, Wiley Series in Probability and Mathematical Statistics, John Wiley, New York, 546 pp. Strauss, A. (1978), Negotiations: Varieties, Contexts, Processes and Social Order, Jossey- Bass Publishers, San Francisco, CA. Suh, N.P., (1990), The Principle of Design. Oxford University Press, Oxford. 193 Sun Microsystems (2002), “Designing Enterprise Applications with the Java 2 EE Platform”, Second Edition, http://java.sun.com/blueprints/guidelines/designing_enterprise_applications_2e/titlepage. html Sycara, K. (1989), “Multiagent Compromise via Negotiation,” in Distributed Artificial Intelligence, Vol. 2, eds. M. Huhns and L. Gasser, San Mateo, CA: Morgan Kaufmann. Sycara, K. (1991), “Problem Restructuring in Negotiation”, Management Science, Vol. 37, No. 10, ‘1248-1268. Tiwari, S. and Gupta, A. (1995), “Constraint Management on Distributed Design Configurations”, Engineering with Computers, 199-210. Toulmin, S. (1958), The uses of argument, Cambridge University Press, London. Toulmin, S., Rieke, R., and Janik, A. (1984), An Introduction to Reasoning, Macmillan Publishing, New York, 1984. Vincent, T.L. (1983), “Game Theory as a Design Tool”, Journal of Mechanisms, Transmissions, and Automation in Design, Vol.105. Wall, J.A. and Calister, R.R. (1995), “Conflict and Its Management”, Journal of Management, Vol. 21(2), 515-558. Waiz, D.B. (1988), “Longitudinal Study of Group Design of Computer Systems”, Ph.D. dissertation, University of Texas, Dec, 1988. Wassenaar, F. and Chen, W. (2003), “An Approach to Decision-based Design with Discrete Choice Analysis for Demand Modeling,” ASME J. Mech. Design, vol. 125, no. 3, pp. 480–497, 2003. Wassenaar, H., Chen, W., Cheng, J., and Sudjianto, A. (2005), “Enhancing Discrete Choice Demand Modeling for Decision-based Design,” ASME J. Mech. Design, vol. 127, no. 4, pp. 514–523, 2005. Wong, S.T.C. (1997), “Coping with Conflict in Cooperative Knowledge-based Systems”, IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans. Vol. 27, No. 1, 57-71. Wu, F., Huberman, B., Adamic, L., and Tyler, J. (2004), “Information Flow in Social Groups”, Physical, 337:327–335, 2004. Yakemovic Burgess, K.C. and Conklin, E.J. (1990), “Report on a Development Project Use of an Issue-based Information System”, Proceedings of the 1990 ACM conference on Computer-supported cooperative work, p.105-118, October 07-10, 1990, Los Angeles, California, United States. Yoon, K.P. and Hwang, C.-L. (1995), "Multiple Attribute Decision Making: An Introduction", Thousand Oaks, Sage Publications. 194 Zeleznikow, J. (2002), “Risk, Negotiation and Argumentation - A Decision Support System Based Approach”, Law, Probability and Risk, Vol. 1, pp. 37-48, July 2002. Zhao, G. and Deng, J. (2001), "Cooperative Product Design Process Modeling", Proceedings of the Sixth International Conference on CSCW in Design, 2001, pp. 236- 242. Zhuang, R. (1999), “Conflict Detection in Web Based Concurrent Engineering Design”, Master Thesis Proposal, University of Florida, 1999.
Abstract (if available)
Abstract
Nowadays, driven by industry globalization and Internet revolution, modern software design is a group decision-making process which is often carried out by a team of stakeholders across geographical, disciplinary and temporal boundaries. These stakeholders have to negotiate collaboratively to achieve consensual agreements on their design decisions based on their competing objectives and different preferences. The identification, organization and negotiation of these objectives and preferences need to be well handled by the software design teams. To effectively support collaborative negotiation of group decisions, one of the critical requirements is to structure the negotiation arguments of multiple decision makers for multiple decision tasks, in order to make sure that all stakeholders have a common ground to effectively make rational group decisions for software design process.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
A synthesis approach to manage complexity in software systems design
PDF
Extraction of preferential probabilities from early stage engineering design team discussion
PDF
Techniques for methodically exploring software development alternatives
PDF
A transparent framework of trust-based collaborative decision-making
PDF
Proactive detection of higher-order software design conflicts
PDF
Using social networking technology to improve collaborative requirements elicitation, negotiation, prioritization and evolution
PDF
Design-time software quality modeling and analysis of distributed software-intensive systems
PDF
Information sharing, deliberation, and collective decision-making: A computational model of collaborative governance
PDF
A synthesis reasoning framework for early-stage engineering design
PDF
Collaborative stimulation in team design thinking
PDF
Software architecture recovery using text classification -- recover and RELAX
PDF
Decoding information about human-agent negotiations from brain patterns
PDF
Decision support systems for adaptive experimental design of autonomous, off-road ground vehicles
PDF
The interpersonal effect of emotion in decision-making and social dilemmas
PDF
Managing functional coupling sequences to reduce complexity and increase modularity in conceptual design
PDF
Designing‐in performance: energy simulation feedback for early stage design decision making
PDF
Coordinating social communication in human-robot task collaborations
PDF
Computationally efficient design of optimal strategies for passive and semiactive damping devices in smart structures
PDF
Computer aided visual analogy support (CAVAS) for engineering design
PDF
A social-cognitive approach to modeling design thinking styles
Asset Metadata
Creator
Jing, Nan
(author)
Core Title
Using organized objectives to structure arguments for collaborative negotiation of group decisions in software design
School
Viterbi School of Engineering
Degree
Doctor of Philosophy
Degree Program
Computer Science
Publication Date
07/30/2009
Defense Date
04/16/2009
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
argument structure,collaborative engineering,collaborative negotiation,computer supported cooperative work,group decision,OAI-PMH Harvest,software design
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Lu, Stephen C.-Y. (
committee chair
), Jin, Yan (
committee member
), Rosenbloom, Paul S. (
committee member
)
Creator Email
jing@usc.edu,jingnan@gmail.com
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-m2434
Unique identifier
UC1499195
Identifier
etd-Jing-2900 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-564867 (legacy record id),usctheses-m2434 (legacy record id)
Legacy Identifier
etd-Jing-2900.pdf
Dmrecord
564867
Document Type
Dissertation
Rights
Jing, Nan
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Repository Name
Libraries, University of Southern California
Repository Location
Los Angeles, California
Repository Email
cisadmin@lib.usc.edu
Tags
argument structure
collaborative engineering
collaborative negotiation
computer supported cooperative work
group decision
software design