Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
A synthesis approach to manage complexity in software systems design
(USC Thesis Other)
A synthesis approach to manage complexity in software systems design
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
A SYNTHESIS APPROACH TO MANAGE COMPLEXITY IN SOFTWARE SYSTEMS DESIGN by Hung-Fu Chang A Dissertation Presented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (COMPUTER SCIENCE) August 2013 Copyright 2013 Hung-Fu Chang ii Dedication To the memory of my grandmother, Wu-Chu Chang. To my parents, De-Yi Chang and Mei-Fong Chen, and my wife, Wenhui Zhu. iii Acknowledgements My Ph.D. study is a long journey. During these years, many things happened expectedly or unexpectedly so I have received assistances and care from all of my teachers, family, and friends. I hope this acknowledgement can articulate my sincere gratitude to them. Firstly, I would like to express my deepest appreciation to my advisor, Dr. Stephen Lu, who patiently guided me throughout my research and studies. Dr. Lu always encouraged me to work for more fundamental research ideas that can bring greater impacts to the academia. What he taught me really made me a more mature scientist. Secondly, I want to thank my committee members: Dr. Barry Boehm, Dr. Robert Neches, and Dr. Yan Jin. Their advices enriched every aspect of my research and their words encouraged me to overcome many difficulties that I encountered. I particularly want to thank for the supports of Dr. Neches. He was working at Washington D.C. during the last two years of my study; however, he still provided me many suggestions, which taught me a lot in my experiment. Thirdly, I would like to express my gratitude to my family, especially my parents and my wife, for their unconditional love, support, and understanding. My wife, Wenhui, has been with me through many life changes and challenges. I know they always stand iv beside me, no matter what happens. Their supports are always the source of my strength. Without them, there would be no me today. Then, I also would like to acknowledge my friends, Dr. Chang Chen, Dr. Supannika Koolmanojwong, Dr. George Zouein, Dr. Ang Liu, Yeh ―Eric‖ Lin, Hui ―Daisy‖ Wang, and Qianyu ―Cherry‖ Liu, who worked and laughed with me during my Ph.D. life. I always remember those interesting talks among us and the helps from all of you. I am very grateful for your accompanying in these years. I also want to thank for the opportunities from the Department of Computer Science because I worked as a teaching assistant most of time in my study. I especially want to thank for Lizsl De Leon for her kindly helps, particularly while I was away from school. Finally, to all my teachers, family, and friends, I want to say ―thank you‖ again for what you have done for me. v Table of Contents Dedication ...................................................................................................................... ii Acknowledgements ...................................................................................................... iii List of Figures ............................................................................................................ viii List of Tables ................................................................................................................ xi Abstract ....................................................................................................................... xii Chapter 1: Introduction ............................................................................................... 13 1.1 Introduction ...................................................................................................... 13 1.2 Motivation......................................................................................................... 14 1.3 Key Research Challenges in Design to Manage Complexity in Software Systems ............................................................................................................. 16 1.4 Proposal Organization ..................................................................................... 17 Chapter 2: Literature Reviews .................................................................................... 19 2.1 Introduction ...................................................................................................... 19 2.2 Complexity Research ....................................................................................... 20 2.2.1 Complexity Research in Analysis Perspective ......................................... 20 2.2.2 Complexity Research in Design Perspective ............................................ 25 2.3 Complexity Research in Software Engineering ............................................ 26 2.3.1 Software Complexity Measure ................................................................. 27 2.3.2 Complex Software Design Principle ........................................................ 29 2.4 Design Theory and Methodology Research in Engineering......................... 29 2.5 Design Theory and Methodology Research in Software Engineering ........ 33 2.5.1 Level Approach ........................................................................................ 34 2.5.2 Structured Approach ................................................................................. 34 2.5.3 Data Structure Approach .......................................................................... 36 2.5.4 Object-oriented Design Method ............................................................... 37 2.5.5 Component-based Design Method ........................................................... 42 2.5.6 Architecture-based Design Method .......................................................... 44 2.5.7 Product Line Based Design Method ......................................................... 44 2.6 Summary of Past Research ............................................................................. 47 vi Chapter 3: Managing Complex Software Systems - The Need for Synthesis in Software Design ........................................................................................................... 49 3.1 Introduction ...................................................................................................... 49 3.2 Synthesis VS Analysis Approaches in Design Thinking ............................... 49 3.2.1 Characteristics of Analysis – Structural Decomposition .......................... 51 3.2.2 Characteristics of Synthesis – Function vs Expected Behavior ................ 53 3.3 Complexity in Software System Designs ........................................................ 62 3.3.1 Traditional System Complexity of Software ............................................ 62 3.3.2 Design-centric Complexity ....................................................................... 63 3.4 The Need for a New Design Approach to Manage the Complexity of Software System .............................................................................................. 64 Chapter 4: Hypothesis of the Proposed Research ...................................................... 67 4.1 Problem Statement .......................................................................................... 67 4.2 Hypothesis......................................................................................................... 67 Chapter 5: A Synthesis Approach in Design .............................................................. 69 5.1 Overview of the Synthesis Approach in Design ............................................ 69 5.1.1 Integrating Function and Expected Behaviors .......................................... 69 5.1.2 Forward Synthesis Model ......................................................................... 72 5.2 Realization of Function – Abduction ............................................................. 75 5.2.1 Abduction ................................................................................................. 75 5.2.2 Functional Coupling ................................................................................. 76 5.3 Synthesis-based Software Design Framework .............................................. 78 5.4 The Process of Synthesis-based Software Design Framework .................... 81 5.5 An Illustration Example .................................................................................. 87 Chapter 6: Experimental Validation and Results ...................................................... 94 6.1 Validation Procedure ....................................................................................... 94 6.2 Measurement Methods .................................................................................... 95 6.2.1 Graph Transformation .............................................................................. 97 6.2.2 Graph-based Similarity ........................................................................... 100 6.2.3 Graph-based Measurement ..................................................................... 102 6.3 Preliminary Data Analysis and Case Study ................................................ 106 6.3.1 Data Analysis .......................................................................................... 106 6.3.1 Case Study .............................................................................................. 112 6.3.1.1 Case Study Plan ........................................................................... 112 6.3.1.2 SICND method ............................................................................ 115 6.3.1.3 Design Tasks ............................................................................... 118 vii 6.3.1.4 Results and Analysis ................................................................... 120 6.4 Experiment Design ......................................................................................... 127 6.4.1 Group Division Method .......................................................................... 129 6.4.2 Threats to Validity .................................................................................. 131 6.5 Experiment Result and Discussion ............................................................... 135 6.5.1 Experiment A .......................................................................................... 135 6.5.2 Experiment B .......................................................................................... 138 6.5.3 Experiment C .......................................................................................... 144 6.5.4 Survey and Framework Feedbacks ......................................................... 146 Chapter 7: Contributions and Future Direction ...................................................... 148 7.1 Contributions ................................................................................................. 148 7.2 Fundamental Weakness and Limitation ...................................................... 149 7.2.1 Fundamental Weakness .......................................................................... 149 7.2.2 Framework Limitation ............................................................................ 150 7.2.3 Validation Limitation .............................................................................. 151 7.3 Future Work ................................................................................................... 152 Bibliography .............................................................................................................. 154 Appendix ................................................................................................................... 165 A. Software Design – Login System (OO) ....................................................... 165 B. Software Design – Login System (SSDF) .................................................... 169 C. Software Design – Product System ............................................................. 173 D. Customer's Needs of the Visualization of Attack Surfaces for Targeting (VAST) ........................................................................................................... 179 E. SSDF Design Procedure of the Visualization of Attack Surfaces for Targeting (VAST) ......................................................................................... 181 F. OO Design Procedure of the Visualization of Attack Surfaces for Targeting (VAST) ......................................................................................... 183 G. Survey Form .................................................................................................. 185 viii List of Figures Figure 2.1 Related Fields......................................................................................... 19 Figure 2.2 Design in the Ideal Knowledge ............................................................. 30 Figure 2.3 Four Domains of Axiomatic Design ..................................................... 33 Figure 2.4 Data Flow Diagram ............................................................................... 35 Figure 2.5 Activity Diagram ................................................................................... 36 Figure 2.6 Jackson Structure Diagram Describes a Data Structure .................. 37 Figure 2.7 Use Case Diagram ................................................................................. 39 Figure 2.8 Three Special Fusion Diagrammatical Forms .................................... 41 Figure 3.1 Synthesis and Analysis in Gero’s FBS Model ..................................... 50 Figure 3.2 Design as Decision Making ................................................................... 52 Figure 3.3 Design from Analysis Perspective ........................................................ 53 Figure 3.4 Distinction between Function and Expected Behaviors..................... 56 Figure 3.5 Two Different Views of the World ....................................................... 58 Figure 3.6 An Example of Taxonomy Space ......................................................... 59 Figure 3.7 Complexity from Design Perspective (Notional Expression) ............ 63 Figure 5.1 Traditional Method – One Dimensional Decomposition ................... 70 Figure 5.2 Integrating Function and Behavior Views – Two Dimensional Decomposition ........................................................................................ 70 Figure 5.3 Real World Design – Diagonal Specialization .................................... 71 Figure 5.4 Software Synthesis Model ..................................................................... 72 Figure 5.5 Three by Three Uncoupled and Decoupled Design Matrixes ........... 77 Figure 5.6 Software Design Synthesis Framework ............................................... 80 Figure 5.7 Six Sub-steps in Step2 ........................................................................... 82 ix Figure 5.8 Class Diagram for Showing the Possible Entity Sources in the Object ................................................................................................................. 84 Figure 5.9 The First Two Level Zigzagging .......................................................... 88 Figure 5.10 The Third Level Zigzagging Under DataIO ....................................... 89 Figure 5.11 Two Possible GUI Entities .................................................................... 90 Figure 5.12 Class Diagrams Mappings of Two Possible GUI Entities .................. 91 Figure 5.13 The Third Level Zigzagging Under ScriptGenerator ........................ 92 Figure 5.14 Two Possible Structures XMLHandler ............................................... 93 Figure 6.1 Process of Producing Complexity and Iteration................................. 96 Figure 6.2 Transform classes into nodes in the graph ......................................... 98 Figure 6.3 Transform associations into bi-direction edges in the graph ............ 98 Figure 6.4 Inheritance transformations ................................................................ 99 Figure 6.5 Inheritance transformations for associations ..................................... 99 Figure 6.6 Process of Preliminary Data Analysis ............................................... 107 Figure 6.7 Number of SLOC at Each Version .................................................... 110 Figure 6.8 Number of Classes at Each Version................................................... 111 Figure 6.9 Different Development Phases ........................................................... 111 Figure 6.10 IDEF0 Diagram of Creating Structural Identical Customer Need’s Description ........................................................................................... 115 Figure 6.11 Syntax Analysis on the Sentence ........................................................ 116 Figure 6.12 The Mapping Sentence........................................................................ 117 Figure 6.13 OOAD Experience VS Complexity Reduction Rate ........................ 121 Figure 6.14 Programming Experience VS Complexity Reduction Rate ............ 122 Figure 6.15 UML Experience VS Complexity Reduction Rate ........................... 123 Figure 6.16 Average Complexity Each Iteration in Case Study ......................... 124 x Figure 6.17 Number of Iterations Used to Finish the Design in Case Study ...... 124 Figure 6.18 Normalized Average Training and Performance between Non-Drop and Drop Groups ................................................................................. 126 Figure 6.19 Design Experience vs Complexity Reduction of OO Design Result 136 Figure 6.20 Number of Iterations Used to Finish the Design .............................. 140 Figure 6.21 Average Complexity at Each Iteration (B1 and B2) ........................ 141 Figure 6.22 Average Complexity at Each Iteration (VAST and B1) .................. 141 Figure 6.23 Average Complexity at Each Iteration (VAST and B2) .................. 142 Figure 6.24 Maximum Complexity Variation at Each Iteration ......................... 142 Figure 6.25 Average Complexity at Each Iteration (B2 and C) .......................... 145 xi List of Tables Table 2.1 Three Axioms in GDT ........................................................................... 31 Table 3.1 Comparison between Function and Expected Behavior Views ........ 60 Table 6.1 Validation Procedure ............................................................................ 95 Table 6.2 Metric Definition ................................................................................. 103 Table 6.3 Two Stage Case Study Plan ................................................................ 114 Table 6.4 Customer Need’s Description of Login System ................................ 118 Table 6.5 Customer Need’s Description of Product System ............................ 119 Table 6.6 Case Study Participant’s Background Statistics .............................. 120 Table 6.7 Number of Iterations Between Baseline and Participants ............... 125 Table 6.8 Background Statistics of Drop and Non-drop Group ...................... 126 Table 6.9 Experiment A Setting .......................................................................... 128 Table 6.10 Experiment B Setting .......................................................................... 128 Table 6.11 Rationale of Experiment B ................................................................. 129 Table 6.12 Experiment C Setting .......................................................................... 129 Table 6.13 Significance F Value of Linear Regression ....................................... 136 Table 6.14 Background Statistics of A1 and A2 Groups .................................... 137 Table 6.15 Performance Difference between Blind and Non-Blind Groups .... 137 Table 6.16 Complexity Improvement ................................................................... 138 Table 6.17 Background Statistics of B1 and B2 Groups .................................... 139 Table 6.18 Statistics of Number of Iterations in Group B1 and B2................... 139 Table 6.19 Background Statistics of Group B2 and C ........................................ 144 Table 6.20 Statistics of Number of Iterations in Group B2 and C .................... 145 Table 6.21 Statistics of Subject’s Survey ............................................................. 146 xii Abstract Modern software systems have become increasingly complex due to heightened customer demands on more functions and reliability. When software engineers face the requirements of lower costs, shorter time, and easier maintenance, the challenges of designing complex software rise sharply. Complexity of a software system in traditional software design approaches can only be analyzed after the design is done. As a result, complexity is treated as an inherited property of engineered software systems which can only be reduced and managed by expensive and time-consuming iterative methods. In this research, we propose a new software design approach which treats complexity as an evolving property of the systems being designed. In this way, the software design will have the control of systems complexity during design, rather than only given the chance of analyzing it afterwards. By treating complexity as a design variable and suggesting a new synthesis design method, our research can systematically reduce and manage complexity when designing modern software systems to meet high customer needs. This fundamentally changes the notion of systems complexity and offers a better method for designing complex software. This research proposes a new software design framework, which is based on a synthesis theory, to complement the existing object-oriented design method. The theoretical backgrounds, the new synthesis design framework, and the process will be developed and then demonstrated with a series of comparative tests to validate our research results. 13 Chapter 1: Introduction 1.1 Introduction To date, due to the increasing demand for more function, software systems become more and more complex. Plus the needs for lower cost, higher quality, shorter delivery time and easier maintenance, the difficulty of the software rises. From the lessons learned from the analysis of the complex nature system, using a single hierarchical decomposition, which divides the structure of the software system into smaller and manageable blocks from abstract level to specific level, is the common technique to overcome the challenges of the complex system design. Even though this decomposition practice has been widely applied for decades, its deficiencies have clearly emerged. Because increasing user demands lead to complicated software requirements, this brings challenges for software design methods to reduce complexity in software design. Designing a technical artifact must be purposeful and functional. The purpose is to produce the design through the designer‘s decisions to fulfill customers‘ needs. The function of the system derived from design decisions, according to the needs, may cause additional complexity to the system if poor decisions are made during the design. One must avoid bad decisions in order to reduce the overall complication of software systems. 14 The backward analysis perspective embedded in conventional software design methods is one of the reasons for this deficiency. Unlike analysis that focuses on the ―as-is‖ software structure, forward synthesis treats complexity as a ―to-be‖ variable. It emphasizes the process from the system‘s function, which is chosen by the designer, to the structure construction. To remedy this deficiency, a new Synthesis-based Design Framework is suggested in this research. This framework supplements the traditional object-oriented (OO) software design method by offering additional supports in the synthesis part of design. To compare the new design framework and the OO design method, a set of design experiments that contains multiple software projects will be performed. Since complexity of software causes the problem of increasing overall inter-dependencies among modules and less design efficiency due to the poorly management of these inter-dependencies, a group of measurements based on dependencies is used in the comparison to evaluate the complexity. With these design experiment results, we expected to show that our Synthesis-based Design Framework can certainly manage the complexity in software design to avoid the unnecessary complexity. 1.2 Motivation Research about complex system and complexity has been a key topic for decades. Many previous empirical studies showed that high software complexity resulted in poor understandability, modification, and testability of modules. Because one module 15 highly depends on other modules, a local change could have wide and unpredictable repercussions. In addition, more dependencies between modules cause less flexibility to divide the labor of coding. Those difficulties due to high complexity become harder to overcome in a large scale and highly functional demanding software system. To deal with complex software system designs, modularization with the information hiding principle, which is suggested by Parnas, is a solution. Through modularization, a hierarchical structure is formed in a software system. However, even though modular structure with information hiding has been shown effective to remove the influence of global data that used by many subroutines, how the structure is created and what the module is for are missing in this approach. Especially, when those decisions are made in the early stage, they are more critical than later stages. However, one single structural hierarchy lacks of the ability to distinguish intertwined requirements and decisions. A more abstract entity is the result of the decision made according to the requirement, but the entity itself is also a ―what‖ for the more specific decisions to fulfill. Hence, this method can only manage the abstraction; it poorly separate ―what‖ and ―how‖ in designs. Complexity in past software research is treated as a result of the design so that most approaches focus on the metrics of the source codes that is already a consequence of the design. Therefore, if the complexity of the software system is still high, the 16 designer uses iteratively refinement to reduce it. However, complexity should be considered as an evolving property of the system rather than regarding it as a design result. Existing one structural hierarchy approach that overlooks the relations between requirements (what) and decisions (how) are still insufficient to handle the complex software system design. As a result, the need of a new software design approach emerges. The new design approach must manage complexity in software design in order to reduce the overall level of complexity of the system. 1.3 Key Research Challenges in Design to Manage Complexity in Software Systems Previous research indeed only deals with complex system but our work treats complexity as a variable in design of the software system. The term, complex system, is the result of the design, which means it has been already complex. Rather than thinking in this way, we believe complexity is a result of the design. It is also an attribute of the system, that is, bad design approach creates unnecessary complexity. Once we have a good design approach, the complexity of the resulting system can be reduces. Therefore, our major challenge is to come up with a new proposed software design approach by addressing old method‘s overlooking portion of the design. This new approach can aid the current OO software design method to manage the complexity and it can also consistently be performed from the abstract to detailed design stage. 17 Hence, the approach should also cooperate with the existing libraries or commercial- off-shelf (COTS) components to better allocate the component to avoid mismatches. 1.4 Proposal Organization Chapter II: Literature Review This chapter reviews the literature relevant to this research work and summarizes the different aspects of viewing complexity and design approaches. It also discusses requirements of developing a new design approach to support the management of complexity in software system design. Chapter III: Managing Complex Software System – the Need for Synthesis in Software Design This chapter discusses the analysis and synthesis perspective in the software design. According to the characteristics of the modern software system, we explain the need for the Synthesis-based design approach. Chapter IV: Hypothesis of the Proposed Research The hypothesis of the research problem of this dissertation is described in this chapter. Chapter V: A Synthesis Approach in Design In this chapter, the theoretical basics of synthesis part of the design, such as the functional view and reasoning method of the decision making, are explored in order to explain the foundation of Synthesis-based Design Framework. A step by step process is also shown to explain how to apply the design framework. 18 Chapter VII: Experimental Validation This chapter outlines our two stage validation plans and the results of the validations. In the validation, a template-based method is used to compare traditional Object- oriented design approach and our design framework. A graph-based complexity measurement is used to investigate the result. Chapter VIII: Contributions and Future Direction The significance of this Synthesis-based Software Design Framework and its impact to software design are discussed in this chapter. In addition, the discussion of further research is also included in this chapter. 19 Chapter 2: Literature Reviews 2.1 Introduction Our research involves the fields of complexity as well as engineering design theory and methodology. As it develops a new software design approach, it is certainly related to the software design research. Besides, this new approach treats the complexity of the system as variable that we can handle during the design. It also strong connects to the complexity research. Finally, since our work is also inspired by physical engineering design methods, it has a relation to field of engineering design theory and methodology. We divide this review in the following four sections. Figure 2.1 Related Fields 20 2.2 Complexity Research Research regarding ―complexity‖ is a field of interdisciplinary research from the interplay of physics, mathematics, biology, economy, and engineering and computer science. As Weaver (1948) pointed out the problem of simplicity in science, one key task of this field is to develop a body of scientific theory of complexity in order to overcome the simplifications and idealizations that have lead to unrealistic models in these sciences. Complexity theories can also be treated as mean to prescribe the rules for complex systems, such as formulating general principles to guide the behavior of complex systems. Since complexity has been widely studied among various fields, we summarized them into analysis and design perspectives. 2.2.1 Complexity Research in Analysis Perspective In analysis perspective, the complexity theory has three major categories. First, algorithmic complexity, which includes the mathematical complexity theory and information theory, claims the complexity of a system lies in the difficulty faced in describing system characteristics. Second, deterministic complexity, which deals with chaos theory and catastrophe theory, posits the interaction of certain variables causing systems prone to sudden discontinuities. The last is aggregate complexity. This one concerns how to create a system with complex behaviors by considering individual element work‘s contribution. 21 Algorithmic complexity In this category, two kinds of contributions are indentified. The first kind primarily focuses on calculating the effort required to solve a mathematical problem. Some problems, like spatial statistics and geographic information science, are unsolvable due to the high complexity. Some problems, such as finding the shortest path through a network, are very hard to solve in non-trivial cases. The research of above problems is useful because it guides the practitioners to choose the right technique. The work in the other kind mostly lies in information theory and indentifies complexity as the simplest computation algorithm that can reproduce system behaviors (Chaitin 1992). By using information theory, tons of interactions between elements of a system can be condensed into a simple measure. Norton and Ulanowicz (1992) applied information theory to study the ecological community structure on biodiversity. Barnes and Ducan (1991) pointed out that incorrectly equating data with knowledge causing the difficulty of applying algorithm complexity concept in social or environmental phenomena. Vast realms of human endeavor like lived experience lie beyond algorithmic expression. Deterministic complexity One of the key characteristics of deterministic complexity is to use mathematics and mathematical attractors. The term deterministic complexity refers to the premise that some key variables related through a set of known equations that can describe the 22 behavior of a complex system when simple mathematic terms are employed to potentially understand chaotic or catastrophic systems. May (1976) found that the standard logistic model of population growth settles on a single value while key parameter‘s values of the growth equation lies in certain ranges. Some researchers paid attention to complex system that is sensitive to initial condition to understand situations where small changes in the initial system configuration may lead to large or non-linear effects. Several of them (Nijkamp and Reggiani 1990; Byrne 1997) examined a variety of chaotic systems for ―bifurcation‖ that is the term to the potential for system variables to jump suddenly from one attractor to another. Others (Renfrew and Cooke 1979; Lowe 1985; Brown 1995) applied catastrophe attractors to describe changes in natural phenomena with the exception of a few attempts to portray changes in social system. Another research area in deterministic complexity is about strange attractor and fractal. The strange attractor means some variable‘s value that is not in a normal range could cause the chaos. The fractal refers self-referential pattern that is remaining unchanged regardless of the scale of the patterns. Many researchers have applied the concept of strange attractor (Mainzer 1996) and fractal (White and Engelen 1993; Pecknold et al. 1997; Mandelbrot 1997) to understand nature systems. 23 Deterministic complexity also introduces new thinking in certain fields, especially in an analogical manner. For example, postmodernists (Hayles 1991; Warf 1993; Jeanne 1997) have embraced deterministic complexity in this way. Using the concept to understand how local individual interactions affect large-scale systems since the postmodern concern for local interactions influences geographic concepts of scale. Research on foreign trade and international development also uses the concept of strange attractors and deterministic complexity (Savit 1993; Keen 1997), as does work attempting to explain the decline of political systems in catastrophic terms (Renfrew and Cooke 1997; Lowe 1985). However, despite some advances, deterministic complexity is still not easy to apply to social phenomena even though there are provoking exceptions (Wong and Fotheringham 1990; Krider and Weinberg 1997). Aggregate complexity Algorithmic and deterministic complexity depends on simple mathematical equations and assumptions of the way complex system works. Instead, aggregate complexity attempts to access the holism and synergy resulting from the interaction of system components. Four interrelated attributes that define a complex system are important in aggregate complexity. They are relationship between entities, internal structure and surrounding environment, learning and emergent behaviors, and the different means by which complex systems change and grow. The relationship between components is the most crucial concept in aggregate complexity (Manson et al. 2001). For example, in the economical system, components 24 could be consumers or firms so they exchange or redistribution of information is relationship. Relationships of different strengths between components define the internal structure of the system. The environment focuses relationships between the system and anything outside of it. In terms of learning and emergent behaviors, Holland (1992) thought the learning of a system as it remembers through the persistence of internal structure. For example, the memory of an economic system may exist in the experience of individuals. Emergence is function of synergism, whereby system level characteristics do not result from superposition but instead from interactions among components even through the system capacity is larger than the sum of its parts (Lansing and Kremer 1993). However, emergent phenomena are unable to predict or control (Lansing and Kremer 1993; Youssefmir and Huberman 1997). Changes and evolutions happen in a complex system majorly through three different types of transition. First, a key feature of a complex system is self-organization, that is, the property allows it to change its internal structure to better interact with its environment. Self-organization also let a system learn from piecemeal changes in its internal structure. Second, a system becomes dissipative when outside forces or internal disturbance drive it to a highly unorganized state (Schieve and Allen 1982). Holling‘s work (1978, 1995) illustrated how small disturbances can trigger large scale redistribution of resources and connectivity within the internal structure of a system. 25 The last is self-organized criticality. It refers to the ability of complex systems to balance between randomness and stasis. Bak and Chen (1991) explained it as that a system can reach a critical point where its internal structure is at the edge of collapsing without actually happening. Self-organized criticality can be treated as a form of self- organization where the rate of internal restructuring is almost too rapid for the system to accommodate (Scheinkman and Woodford 1994). Most self-organized criticality research is restricted to ecological and bio-geophysical systems (e.g. Andrade Jr. et al 1995; Correig et al. 1997). The value of aggregate complexity is its challenge to conventional notions of stability and change because science in general sees systems as stable entities. However, it is sometimes problematic to use the insight due to the ambiguous boundary between the system and its components. 2.2.2 Complexity Research in Design Perspective Unlike analysis perspective complexity, complexity in designing a system does not look at the physical domain, such as examining the structure of machines, biological cells or individuals in economies, the view of complexity in design perspective focuses on the artificial system‘s function that the designer assigns to fulfill the customer needs. Suh (2005) defined this new complexity as a measure of uncertainty in achieving functional requirement of a system within their specified design range. This complexity is also based on Suh‘s Axiomatic Design (AD) theory. 26 Suh categorized the complexity into four types: (1) time-independent real complexity; (2) time-independent imaginary complexity; (3) time-dependent combinatorial complexity; (4) time-dependent periodic complexity. Time-independent real complexity results from the poor design decisions; that is, the functional requirement cannot always be achieved with the available implementation. Time-independent imaginary complexity is not real. It exists due to lacking of understanding about system design or system architecture. This usually happens when there are many functional requirements in the design and they must be satisfied in a special sequence. Both time-dependent complexities consider the system ranges vary with time. Time- dependent combinatorial complexity happens because some elements in the system could gradually change as time pass even though their initial design is less complex. If this type of complexity is left unmanaged, the system will eventually collapse. Time- dependent periodic complexity is the complexity that only exists with a finite time period. That is, at every time interval, system range move away due to element‘s gradually changes but the system resets to initial state before it reaches the chaotic state to fail. One example of such a technical system is the airline scheduling system. 2.3 Complexity Research in Software Engineering Software complexity research began in 1970s due to the need of quality control of the large systems. It can be categorized into two fields to measure the complexity of software or a single program and to study guidelines to design a complex software system. 27 2.3.1 Software Complexity Measure The best known early approach of measuring software complexity was proposed by McCabe (1976). McCabe developed the concept of ―cyclomatic number‖ that represents the number of linearly independent execution paths through the program. To calculate that, he considers the program as a directed graph in which the edges are lines of control flow and the node are line segments of code. Another well-known metric should Halstead‘s metric (1987). He basically views a program as an expression of language; therefore, his method seems to measure the complexity of a language. Although Halstead‘s method was influenced by the cyclomatic number, he also considered number of variables, programming effort and statement count. Much successive research took McCabe‘s and Halstead‘s approach as a starting point. For example, Myer (1977) extended McCabe‘s metric to suggest an interval complexity measure, which considered the influence of number of decision statements and the number of conditions in predicates. Hansen (1978) combine logical complexity and expression complexity by combining McCabe‘s and Halstead‘s methods into a two- tuple complexity measure, in which program complexity is calculated according to number of branches and the number of operators in the program. Henry and Kafura (1981) presented a measure that is sensitive to the structural decomposition of the program into procedures and functions. Their method depends on procedure size and the information flow that is in and out of the procedures. 28 Woodward et al. proposed a control flow complexity measure based on the number of ―knots‖ in a program. A knot is unavoidable control path crossing when a program is transformed into a directed graph. One of the benefits in this measure is knot count over the cyclomatic number; thus, the method gives a better indication of being structured and readability. Kitchenham and Walker (1986) thought software structural or product complexity is measured by module metrics and inter-module metrics. Module metrics that focuses at individual module level (subprogram or class) measure the internals of the module such as size, data structure, or control flow complexity. Inter-module metrics that measure the connections between modules are regarded as system level complexity. Since complexity measures often based on program structures could not truly reflect a developer‘s familiarity of a piece of program that he/she works on, alternative perspective that learns from the Basili‘s (1980) and Curtis‘s (1979) complexity views is based on cognition, regards as software cognitive complexity. Such a field has been endorsed by Curtis (1979), Davis (1984), and Kearney et al. (1986). Later, Cant et al. presented the Cognitive Complexity Model based on an analysis of the programming process. This model consists of a graphical representation and a mathematical representation to show the comprehension process during the understanding, modifying and debugging source codes. 29 2.3.2 Complex Software Design Principle The way to handle complex software system design can trace back to the general problem solving strategy that is to ―divide and conquer‖ the problem. In this way, software system is decomposed through its hierarchical layers; as a result, the design becomes simpler to address and realize at the lower levels. However, from the pioneer research of complex software system design (Parnas 1972), a critical principle should be included. Parnas suggested ―information hiding‖ as the decomposition criterion to divide a software system into a hierarchical (or modular) structure. Parnas indicated its purpose is to make inaccessible certain details that should not affect other parts of the system. In other words, information hiding means that a module that is characterized by information hides from other modules in a hierarchical software structure. 2.4 Design Theory and Methodology Research in Engineering Design theories or methodologies always play a key role in engineering design research. In this field, scientists tried to figure out human design activities to present a method in system development. We intentionally avoid discussing Systematic Design (Pahl and Beitz 1996) and TRIZ method (Altshuller and Shulyak 1998) because they mostly are related to process of the practices and principle of resolving requirement conflicts respectively. Therefore, except these two, we introduce other related design methods as follows General Design Theory 30 General Design Theory (GDT) proposed by Yoshikawa (1981) forms a mathematical formulation of the design process and a justification of design knowledge. It is a descriptive model that explains how design is conceptually performed in terms of knowledge manipulation. In GDT, a design process is regarded as a mapping from the ―function space‖ to ―attribute space‖ and both of them should be defined over the ―entity concept set‖ (see Figure 1). An entity is a concrete existing object and an entity concept is its abstract mental impression conceived by human. For example, an entity concept might be color or size, which is the property of the entity. These properties are called ―abstract concepts‖ in GDT and include attributes. Figure 2.2 Design in the Ideal Knowledge Because Yoshikawa thought the ultimate goal of GDT aims at a scientific manner to design, based on its three axioms (see Table 1), it uses mathematical set theory and topology combining with its ―Ideal Knowledge‖ concept to describe (or to prove) designs in the ideal world. Ideal Knowledge assumes the one with it knows all the elements of the entity set (i.e. all the entities) and can describe each element (i.e. 31 entity) by abstract concept with ambiguity. According to Ideal Knowledge, designs can be thought as immediately direct mappings because we know everything perfectly. However, this situation never exists in a real world. Table 2.1 Three Axioms in GDT Axioms in GDT Axiom 1 : Axiom of recognition Any entity can be recognized or described by its attributes and/or other abstract concepts. Axiom 2 : Axiom of correspondence The entity set and its representation have one-to-one correspondence. Axiom 3 : Axiom of operation The set of all abstract concepts is a topology of the entity set. To deal with real cases, three factors should be considered in the design. First, design is not a simple mapping. The designer is in the process of seeking a solution that satisfies the constraints or requirements. Second, it is hard to formalize the concept of function objectively due to subjectively sense that can vary from person to person. Third, one must take physical constraints into account because they are mentioned in Ideal Knowledge. As a result, GDT introduce Real Knowledge that considers above three factors. By Real Knowledge, the design is regarded as a process in which the designer sets the goal and tries to satisfy the specifications without violating any physical constraints. Similar to designing with Ideal Knowledge, mathematical explanation can also be applied to Real Knowledge designs because the concept of physical law is used to associate with the attributes in GDT. From the mathematical description of designing with Real Knowledge, a design process can be regarded as a convergence process, but 32 solutions are not guaranteed existed as a single one. It is possible to have no solution or multiple solutions. Axiomatic Design Axiomatic Design was developed by N.P. Suh at MIT in the late 1970s. Its hypothesis is that there exist certain fundamental principles that govern good designs. Four key elements are included in this design framework; thus, design domains, axioms, hierarchies and zigzagging. The whole process interplays between four key domains that are: the customer domain, the functional domain, the physical domain, and the process domain. The design process begins in the customer domain with the features that the customers are looking for in a technical system, called the customer needs (CNs). Mapping from customer to functional domain is used to find out functional requirements (FRs) of a system. Design parameters (DPs) are the set of properties that satisfy the FRs and describe the design objects in the physical domain. Process variables (PVs) finally relate the design parameters to the manufacturing process, which outlines how the product is to be made. Hierarchical decomposition is conducted in the four domains and zig-zag mapping is carried out from left to right between these domains. 33 FR FR 1 FR 2 FR 1.1 FR 1.3 FR 1.2 Conceptual Design CN FR DP PV Customer Functional Physical Process Detail Design Function Assignment ZigZagging DP DP 1 DP 2 DP 1.1 DP 1.3 DP 1.2 PV PV 1 PV 2 PV 1.1 PV 1.3 PV 1.2 Figure 2.3 Four Domains of Axiomatic Design The framework also consists of two axioms called the independence and information axioms. The independence axiom states that one should ―maintain the independence of functional requirements.‖ This means that mapping between the FRs and the DPs should fulfill in a way such that each FR can be satisfied without affecting the other FRs. The information axiom states that one should ―minimize the information content of a design,‖ which implies that the ―simplest‖ design should be the best. 2.5 Design Theory and Methodology Research in Software Engineering Software systems are virtual systems that are thought to be very different from physical systems. Research in software design methods is often developed separately from engineering design research. According to the characteristic of the approach, past software design methods can be classified into five kinds. They are level, structured, data structure, object oriented, component based, architecture based, and product line 34 based approaches. The evolution and details of the design method in each category are listed below. 2.5.1 Level Approach Niklaus Wirth (1971) addressed the top-down (usually terms stepwise refinement) approach to solve the classical ―eight-queens problem‖ in his pioneer paper. The top- down considers a program with various levels and the process starts at the top (system) level, according to functional decomposition, breaks down the system into smaller functional modules. It leads to the decomposition of tasks into sub-tasks and of data into data structures. Through this step-by-step process, a solution is produced. However, since the method is algorithm-like and relatively closed to programming level, it almost cannot be employed in large software systems. 2.5.2 Structured Approach In the structured approach, data flow characteristic is used to derive program structure and operations performed on the data. This continuous node to node data flow is transformed by the functions. In 1970‘s, a family of structured method were created such as Structured Analysis and Design Technique (SADT), Software Requirements Engineering Methodology (SREM) and Problem Statement Language/Problem Statement Analyzer (PSL/PSA). Structure Analysis and Design Technique (SADT) that was proposed is the typical example of structured methods. PSL/PSA focused on producing design documents automatically from computer-aided systems and SREM emphasized the requirement capture skills. In SADT, Structured Analysis (SA) primarily used two main the requirement analysis techniques that capture the system 35 requirements, namely, data flows analysis and activity analysis. Data flow analysis produces an artifact, data flow diagram (DFD), which shows the data flow between the system‘s functions. Activity analysis uses activity diagrams to represent relations between inputs, controls and activities. DFD is a directed graph, where the nodes represent the interfaces from or to external environments and the edges represent data flows between nodes. The whole data flow transformation process can be further divided into sub process. Structured Design is a systematically top-down process for mapping processes of DFD to program modules. Various extensions were invented to support the basic SADT method in order to deal with certain unique software system designs such as real-time systems. In these varied versions, some diagram like state transitions diagrams or some extra notations of DFD are placed into the analysis. Data Store Process Data Flow Figure 2.4 Data Flow Diagram Activities diagrams use boxes and arrows to illustrate a set of top-down decomposition structure of activities. A box shows the activity that consumes input data (arrows enters from left) and produces outputs (arrows exit from right). Controls that are not 36 consumed by the activity represent commands which influence the execution of the activity. Activity Activity Activity Input Output Control Figure 2.5 Activity Diagram 2.5.3 Data Structure Approach Jackson Structured Programming (JSP) and Jackson System Development (JSD) methods that were both invented by Jackson in 1980‘s are data structure oriented approaches. JSP primarily focus on the coding and program analysis skills. However, JSD supports all the system development steps during the software life cycle and consists of three main phases – modeling, network and implementation phases. In the modeling phase, designer model the problem domain of the real world. The network phase is for designers to add functions to the model. The implementation phase covers physical design parameters such as physical data design and results in the final system. In JSD, JSP can be used to produce the final code. In JSP, engineers analyze and model the structure of the data with three data constructors – concatenation, iteration and selection, and then describe those data in the form of a data structure diagram that 37 is called Jackson tree. Once the input and output data structure trees have been done, the relations among nodes between two trees are explored. Program is written according to the diagram. Data #1 Data #2 Data #5 Data #4 * Data #3 Figure 2.6 Jackson Structure Diagram Describes a Data Structure 2.5.4 Object-oriented Design Method Object-oriented design (OOD) method is currently widely used. It was originated by Russ Abbott and then many design methods such as Grady Booch‘s approach have been proposed since the late 1980‘s. Until today, many approaches have been suggested. The idea behind OOD was to create software systems by modeling them in terms of the real-world ―objects‖ and the operations performed on those objects. Even though researchers have various descriptions of the characteristics of this method, class concept, polymorphism, inheritance, abstraction, and encapsulation are often described as the features. A class represents a type that classifies a set of objects sharing with common behaviors and structures. Hence, an object is an instance of a class, and it encapsulates attributes that can be data structures or basic data attributes. Operations are procedures and contain methods that operate the attributes. Despite 38 having these OO common concepts, the way that different approaches suggested to create a software system needs to be explored. Their details are described below. Shlaer and Mellor in 1992 suggested the behaviors of the object should be modeled by the conventional state-transition diagrams, which laid the basis of a genuinely OO even though some argued that it was still data-driven. Coad and Yourdon at the same time incorporated behavioral ideas into a simpler OO method that comprehensively supports for the features of the C++ programming language. As a result, Coad‘s method immediately became popular. Object-modeling Technique (OMT) (Rumbaugh et al., 1991) followed Coad‘s approach. It captured the idea of adding operation to entity-type description to make class models from Coad‘s approach; however, it used a different notation from all previous methods. Some evolutions can also be found in OMT. Although it is also data-driven, it separated processes from data by using data flow diagram separately from the class diagrams. It used state-transition diagram to describe the life-cycle of instances. Besides, it made useful advises on how to connect OO programming to relational databases. Because these suggestions on precisely modelling the language and database, it became a primary method for the C++ and relational database development. 39 Different from those data-driven OO approaches, Wirfs-Brock and her colleagues developed the responsibility-driven design (RDD) technique. The most notable contributions of the RDD were the idea of using CRC cards and stereotype for design. CRC showed the responsibilities of the class and its collaborations with other objects as a starting point of the design. Using such technique can help the designer to identify the objects in the beginning of the software design. Rather than directly doing OO analysis, Jacobson et al. in 1992 suggested the Objectory (Object Factory) method that uses the use case to begin the analysis (see Fig. 2.6). The classes in the software were then derived from the use cases. Goldberg and Rubin in their Object Behaviour Analysis (OBA) used stereotypical scripts in use cases. Use case technique marked a significant step forward in Object-oriented Software Engineering (OOSE). Actor Use Case 1 Use Case 2 Use Case 3 Figure 2.7 Use Case Diagram Methodology for Object-Oriented Software Engineering of Systems (MOSES) was proposed by Henderson-Sellers and Edward in 1994. It was the first OO method that includes a full development process, a metric suite and a way to manage reuses. 40 Almost contemporaneously, Semantic Object Modelling Approach (SOMA) that was proposed by Graham was influenced by MOSES. SOMA especially focused on the areas of the requirement engineering, rigor principles in processes and the agent-based system. Hierarchical Object-oriented Design (HOOD) method that targeted Ada programming language was developed for European Space Agency (ESA 1989 and Robinson 1992). Even though HOOD is based heavily on the form used in Booch‘s earlier work, it leans on the use of written text to describe the abstract design model and uses s a diagrammatical representation. Unlike other methods, there are no specific forms used for capturing behaviors or data modeling aspects of the object model. Instead, two key concepts were introduced. First, HOOD identifies the existence of both ―passive‖ and ―active‖ objects in a system. A passive object essentially provides a service; which means, the operations of a passive object should be only activated by the invocation of the method. An active object can provide a parallel thread of execution, and it may interact with active or passive objects. The second leading concept is the hierarchy described in HOOD diagram. It uses a ―use‖ and a ―functional composition‖ hierarchy. However, HOOD limits itself to the object characteristics of modularity but does not attempt to bring in the concept of the class hierarchy. Fusion was developed at Hewlett-Packard (HP) laboratory in the UK (Coleman et al. 1994) and tried to combine good techniques from other published methods. It 41 concerns the description of identifying the different viewpoints with a mix of text and diagrams. Three principal differences in the notation of the Fusion modeling lays in its object model, object interaction graph and object visibility graph (see Fig. 2.7). Object model in Fusion is strongly related to Entity-relation diagram (ERD) notation. Functionalities regarding collaboration between objects are described in object interaction graph and the graph was constructed for each system operation. Object visibility graph should be considered as most special to Fusion. It is used to define the reference structure of classes in the system, in which, identifying the objects that class instances need to access and the forms that these references should take. In addition to the representation of the model, the process of the Fusion method is strictly defined as eight distinct steps. The first four steps that belong to black-box modeling are counted as the analysis of the object model and the remaining four steps are white-box design steps. Therefore, in the analysis stage, starting from the requirement document, the object model, system interfaces, operation and behaviors are sequentially added into the analysis model. This result setups the base for the design of the system. Attribute Class Name Aggregate Class Class 1 Class 1 Class 1 Name Role 1 Role 2 C C represents the cardinality of the aggregation Object Model Object Name: Class Object Interaction Graph Object 1: Class 1 Object 2: Class 2 Object 3: Class 3 System_operation() Message 1 (parameter list) Message 2 (parameter list) (Controller Object) (Collaborator Object) Class A Object 1: Class B Object Visibility Graph Lifetime of the object 1 is not bound to the lifetime of the class that is accessing it Figure 2.8 Three Special Fusion Diagrammatical Forms 42 The state of art OO approach can be thought to be the Unified Process (UP) which was developed primarily by Grady Booch, James Rumbaugh and Ivar Jacobson. Particularly, the Unified Process is much influenced by the concept of the Objectory. Like Fusion, it also merges many ideas from various sources but its process is much less sequential than Fusion. There are four phases in the development process and they are Inception, Elaboration, Construction and Transition. More importantly, UP considers various elements associate OO modeling to be supported by a universal graphical notation; as a result, the Unified Modeling Language (UML) was created for this purpose. Recently, UML becomes very popular in many engineering fields. UP is also extended to various commercial forms and the most famous one should be the Rational Unified Process (RUP). 2.5.5 Component-based Design Method Component-based design (CBD) showed up due to the trend of emerging large-scale and complex applications and business platform standards. It mimics the physical system design, where components have well-defined roles and interfaces, in this way, software components can be reusable easier. In addition, it is also thought as a higher level of abstraction than objects. The interfaces of a component are typically defined by using an interface definition language (IDL). In early CBD research, to distinguish it from other methods by introducing the component concept, many studies suggested the definition of component (Brown and Short 1997; Szyperski 1998; Heineman and Coucil 2001). According to those 43 definitions, the core concept of the component is independent, by which a component should not have any context-specific dependencies. One extreme case is generally regarded as Commercial Off-The-Shelf (COTS). It refers to ready-made, self- contained and available-for-sale merchandise. Therefore, a COTS component is a quite expressly back box in its nature, so engineers have no control over its functionality or performance (Boehm and Abts 1999). According to the component concepts, three principal requirements for designing a component are processing well- defined functionality, providing a set of well-defined interfaces, and clearly specifying any explicit dependencies. To use component in software design, seeking components and fitting them together are two crucial tasks. Pohthong and Budgen (2000; 2001) suggested two strategies, element first and framework first. Element first begins with identifying the general needs of the design problem. Then, engineers search for a set of components that satisfy functionality and finally compose them to form a system. Framework first is to decompose the design problem into fairly well-defined sub-problems and then to find out a set of components that fit individual sub-problems. In term of fitting components together, Garlan et al. (1995) firstly discussed the mismatches between the expectations that each component makes about its context. Yakimovitch et al. (1999) provided examples of how mismatches can be created due to the inconsistencies of component packaging, type of control, information flow, binding time and synchronization between components. 44 However, there is no specific design process in this field (Budgen 2003). Research about CBD mostly focuses on the component itself, such as component interoperation. This could be the design process is strongly driven by the availability of existing components. (Budgen 2003). 2.5.6 Architecture-based Design Method Architecture Based Design (ABD) method was proposed by Bachmann et al. in 2000. Its goal is to design the high level software architecture for a product line or long-lived system. There are three foundations for the ABD method. The first is to decompose the function recursively based on the techniques of coupling and cohesion. The decomposition should be examined from logical, concurrency and deployment views. The second is to satisfy the quality and business requirements through choosing the architecture style. The last is the use of software templates. 2.5.7 Product Line Based Design Method The concept of the software product line (SPL) is a set of software systems that share a set of common assets that are developed using a common means of production. Design approaches in field embrace the concept of SPL to create a software system, and they often consider design at the architectural level. A Component-Oriented Platform Architecting Method for Families of Software Intensive Electronic Products (COPA) that is developed by the Philip Research Labs is one of the results of its Gaudi project. The COPA covers business, architecture, 45 process and organizational aspects to find out the best fit. It starts from analyzing customer needs. In its architecting phase, the inputs are facts, stakeholder expectations, (existing) architectures and architect‘s intuition. The output of this phase is a lightweight architecture. Moreover, the method balances the bottom-up component-based and top-down architecture-centric approaches to design the architecture. Weiss and Lai (1999) introduced the Family-Oriented Abstraction Specification and Translation (FAST) process that has been successfully applied at Lucent Technologies in the domain of the telecommunication infrastructure. The FAST defines a full product line engineering process with activities and artifacts. It also divides the process of product line into the sub-processes of domain qualification, domain engineering, and application engineering. Domain qualification begins with receiving the general needs of business according two cases, which are paying little or no attention to domain engineering or engineering the domain to make production of family members more efficient. The output of the domain qualification is an economic model. Domain engineering generates a language for specifying family members, an environment for generating family members based on their specifications, and a process of producing family members using that environment. In the beginning of application engineering, engineers get the requirements and application engineering finally output family members through generation to response the requirements. 46 Feature-oriented Reuse Method (FORM) was suggested by Kang et al. (1998) as an extension of Feature-oriented Domain Analysis (FODA) method. The FORM prescribes how feature model is used to develop domain architecture and reuse component. It starts with feature modeling to discover, understand and capture the variability and commonalities of a product line. Domain engineering in FORM is to perform context analysis to create the feature model, reference architecture, and reuse components. Then, the application is implemented after features have been chosen from the feature model, the application architecture has been selected from reference architecture, and reuse components have been decided from those reuse component candidates. KorbA stands for ―Komponentenbasierte Anwendungsentwicklung‖ that is German for ―component-based application development‖. It was developed by Atkinson et al. (2000) in Fraunhfer IESE to model software architecture. The most important parts of KorbA‘s product line engineering are framework engineering and application engineering. It also defines implementation, release, inspection and testing aspects of the process. The process of this method is that, firstly, realizing the context to perform framework engineering, elicitation of requirements within the scope of the framework from customer‘s contacts with software development organization, and then designs of software components. 47 QADA (Quality-driven Architecture Design and quality Analysis) is developed at VTT, the Technical Research Centre of Finland, to emphasize the quality attributes in software design. Initially, the QADA uses requirement engineering to collect the driving ideas of the system and technical properties of the system. In fact, its requirement engineering part actually includes the factors leading to the software architecture design. The output of QADA covers not only conceptual and concrete level software architecture design, but also the analysis of the information that concerns the quality of the design. 2.6 Summary of Past Research From the review of complexity research, most past research were based on examining existing systems, in which all their elements have already been interacting and structure has already been formed. Thus, they analyzed the system as the way it is. Especially in software complexity research, the emphasis is on measuring the source codes of the software system. This ―as-is‖ complexity view also brings the inspiration to complex system designs. Through observing the characteristic of complex systems, they are formed hierarchically. Therefore, the use of hierarchical structural decomposition is the common strategy to deal with a design problem. Among all the past software design methods, this decomposition technique is widely applied in various approaches at different levels (e.g. program or system level) to divide the structure of the system. 48 As a result, we want to argue that complexity can no longer be treated as the analysis perspective research in software designs if we want to have a simpler system in the end. The new software design approach has to embrace the concept of new design perspective complexity as a mean to enhance the current software design method. Except this, it also needs to combine explicitly expressing the design procedure from functional requirement to the structure of the software to become workable. 49 Chapter 3: Managing Complex Software Systems - The Need for Synthesis in Software Design 3.1 Introduction From past research, complex software system design cannot be handled well so that the system usually ends up with high complexity. In this chapter, we argue that focusing on the synthesis in designs in order to reduce complexity in software system. Therefore, we first define synthesis in designs. Then, we explain that traditional software complexity is not appropriate for building a simpler software system. Another kind of definition, which regards complexity from design perspective, should be applied to reduce the complexity. Since complexity is caused by bad design decisions in this new view, synthesis that focuses on the decisions of making software architecture should be emphasized in complex software designs. Finally, we summarize why a new design approach that focus on synthesis for reducing complexity is needed. 3.2 Synthesis VS Analysis Approaches in Design Thinking The terms ―synthesis‖ and ―analysis‖ can be traced back from Greek; they mean literally ―to put together‖ and ―to loosen up‖, respectively. In general, synthesis is the procedure by which a whole is formed by combining separated elements or components. Analysis is the opposite procedure: breaking a substantial whole into parts or components (Ritchey 1996). 50 Figure 3.1 Synthesis and Analysis in Gero’s FBS Model In design, the definition of ―synthesis‖ can be found in Gero‘s Function-Behavior- Structure (FBS) model (see Fig. 3.1). In the FBS model, there are eight elementary design activities that are linked by five elements. Before synthesis and analysis are specifically explained, we first describe four related elements: function (F), expected behaviors (Be), actual or structural behavior (Bs), and structure (S). Function is the intention or purpose of the designer. It is represented by using ―to do what (to + verb)‖. For example, the function of a car brake is ―to stop‖. Expected behavior is a set of effects that the designer expects to fulfill the function when a device or component is taken into consideration. Structure (or solution structure) is generated from the expected behaviors and it represents how a system is formed. Actual (or structural) behavior is the effect that actually happens due to the structure of the system. 51 The link from a set of expected behaviors forward to a solution structure is called ―synthesis‖ and the link from a solution structure backward to a set of structural behaviors is called ―analysis‖. Synthesis is the transformation of the expected behavior into a structure that is proposed to exhibit the expected behavior. Analysis is the derivation of the actual structure. FBS also indicates that a completed design should include both synthesis and analysis. In addition, according to Ritchey‘s point of view (1996), as scientific methods, synthesis and analysis should always go hand in hand and complement one another: to regard one as being inherently better than the other is in fact meaningless. As a result, a good design should be well-balanced between synthesis and analysis. Synthesis should also be performed before analysis and the result of analysis is used to evaluate the function. Because the structure of the system is created through synthesis, analysis is built upon synthesis. 3.2.1 Characteristics of Analysis – Structural Decomposition Briefly, design is a series of decision makings that maps a set of elements in the requirement domain to a set of elements in the solution structure domain. The mapping between these two domains means that the solution structure satisfies a specific set of requirements. There are multiple ways of mapping, and how they map each other depends on the design decisions (see Fig. 3.2). Elements in the requirement domain are functions of the system (i.e. the designer‘s intentions or what designers want to achieve). Elements in the solution structure are built to form the system according to a certain formation. In software system design, elements in the 52 requirement domain refer to requirements, and elements in the solution structure domain represent a set of components, programs, or objects that satisfy the requirements. Figure 3.2 Design as Decision Making Traditional design approaches are from an analysis perspective. To shift the design from abstract to specific, the designer only decomposes the solution structure into hierarchical modules. Elements in structure are expected to be independent to each other. To evaluate the design, the designer has to examine if a set of elements in the solution structure domain can satisfy every requirement (see Fig. 3.3). More importantly, due to the focus on the structural decomposition, traditional design approaches emphasize evaluating if the behaviors of its structure can fulfill the requirements. This often leads to the result of neglecting the way to synthesize the solution structure – the synthesis part is just ad hoc passed through in designs. 53 Figure 3.3 Design from Analysis Perspective The gap of the traceability between requirements and their design decisions explicitly exists while a system is designed from analysis perspective (Turban et al. 2009). Paech et al. (2002) pointed out that traceability is typically seen as a set of bidirectional relationships between requirements and their fulfilling design entities (Gotel and Finkelstein 1994). Because a system‘s solution structure is decomposed before looking back to see if requirements are satisfied in analysis perspective, traceability links are retrieved or established after decisions are made on the structure. Therefore, this causes very complicated traceability relations – fewer simple or linear bidirectional links. The shortcoming of such complex traceability is to endanger a project‘s consistency and common understandings of its stakeholders (Lindvall 1994; Knethen 2001; Ebert 2005). 3.2.2 Characteristics of Synthesis – Function vs Expected Behavior The terms ―function‖ and ―behavior‖ are commonly used in design research and practices. However, due to the careless treatment of their semantic and syntactic 54 distinctions, they are often misunderstood conceptually and/or mis-used interchangeably when describing design requirements and/or specifications. This has caused some confusion in the design research community when communicating and comparing different design approaches. To make a clear distinction between function and behavior in design, according to the FBS, Gero defines function as the designer‘s intentions or purposes; behavior is how the structure of artifact (real, existing, or imaginary) achieves its intended functions. He further defines two different types of behaviors: expected behaviors (Be) and structure (or actual) behaviors (Bs). Expected behaviors (Be) are what the designer wants the system (to be or being designed) to behave. Actual behaviors (Bs in Fig. 3.2) are those effects that the real (designed) system produces within the environment based on its embodied structure (either physically, like an airplane, or virtually, like software). Even though Gero‘s FBS model has distinguished functions (F), expected behaviors (Be), and actually behaviors (Bs), it does not clarify the difference and respective roles of function and expected behaviors in design practice. Consequently, they are still blurred in most design research and many design methodologies. Fundamentally and epistemologically, we can argue that function (F) and expected behavior (Be) are very different in their natures and contexts of expression; they 55 represent two different ways of design thinking. In the design of a system, unless the notion of some sort of ―embodiment‖ (e.g., ―a thing‖ - no matter whether it is physical or virtual, actual or imaginary) comes into the designer‘s mind, behaviors of the system could never be described. In other words, the design cannot express the behaviors of nothing (i.e., he/she must have something in mind when he/she expresses its behaviors). Function, on the other hands, represents the designer‘s metal intention without regard to any imaginary or real thing that has existed or is to be made in the world). Therefore, the function should be considered as abstract (i.e., ―thing-free‖ or ―solution-neutral‖) requirements (see Fig. 3.4) – i.e., functional requirement (FR). In short, when using the term ―function‖ in describing design specifications, the designer‘s thinking should not be guided or constrained by anything nor be guided by any ideas of physical ―embodiment‖. For example, when designing a system, a possible functional requirement is ―to move‖. If the designer thinks of (or has it in the back of his/her mind) something like a car or an airplane, he/she can describe the expected behaviors (Be) as ―wheels rotate to move‖ in his/her design specifications. In comparison, if the designer uses the term ―functional requirement‖ to describe ―to move‖, there is no notion of any ―embodiment‖ (or nothing imaginary or tangible exists in the back of his mind). Hence, neither properties nor constraints (from cars or airplanes) are implicated or unknowingly added from the upcoming ―embodiment‖. 56 Figure 3.4 Distinction between Function and Expected Behaviors Many design approaches have neglected the important uses of ―function‖ (or functional requirement – FR) in their method or process. One famous example is the object-oriented methods in software design, such as object-oriented analysis and design (OOAD) or object-oriented software engineering (OOSE) methods. In object- oriented design, the notion of FR is neglected; the designer uses expected behaviors (Be) to describe the required effects of the object that will be created in the software system. Even in some design reasoning research, function is sometimes considered as abstract behavior without the careful distinction as described above (Bracewell and Sharpe 1996). Therefore, we can classify existing design approaches into two different categories based on the designer‘s view toward the real world. On one hand, a designer can describe his/her design intentions as functional requirements (FR) entirely with no regard to a thing (or anything) that may have already existed (or will exist) in the real world. Using the ―thing-free‖ design thinking to describe design intentions is extremely hard because we, especially as an educated being, have all experienced 57 many ―things‖ as part of our lives and education experiences. Therefore, it seems a bit ―un-natural‖ to ask the design to think his/her intentions in a perfectly ―solution- neutral‖ manner in design practice. However, as un-natural as it may be, this free- thinking approach will provide the designer a limitless amount of innovation possibilities to come up with an embodiment later that is completely new and previously unseen. So, if the designer views the world as a blank sheet of paper (i.e., without any previously existing things), then he/she will have the possibility of being totally innovative in his/her design creation. For example, using the Axiomatic Design method, the designer uses the term functional requirement (FR) because he views the world as ―nothing has ever existed and anything is possible‖ (see Fig. 3.5). Therefore, in the design of a thing, the only notation is purely what he/she wants or what he/she targets ideally and abstractly – that is function. In expected behaviors, a designer views the world as ―a collection of things‖. The system that he/she wants to build is just one of ―them‖ even though it is yet to be determined during the design. Once he/she starts a system design, a prior thing (or something) exists in his/her mind. It could be abstract and vague in the beginning; at the end, it must become concrete and detailed. As a result, the designer‘s job is to specify the behaviors of the thing being created so that he/she can build the structure later accordingly. Since there is an embodiment embedded in the designer‘s mind during the design, this style of design is not ―thing-free‖. It can be influenced by things that already exist in the real world since those embodiments limit the thinking of the designer. 58 Figure 3.5 Two Different Views of the World The usage of expected behaviors (Be) deeply influences the OO methods in software design. In OO thinking, objects that are considered as independent entities can be composed into a system. How they interact with each other or the way they operate to compute or save data is specified by their class. In other words, the class is the specification of its instantiated object (Meyer 1997). Classes represent the types of object. Expressing their specification and their inter-relationships, by which forms a taxonomy space (see Fig. 3.6), is the key to inheriting properities and attributes from similar (or common) objects. Since the design of an object-based software system always starts with certain objects (or entities) that the designer must describe and already have in mind, for the above reasons, using expected behavior (Be) is the only appropriate way. For example, the use case approach in OOSE is applied to elicit the behaviors of system, which exists in the designer‘s mind, from the interaction between actors and their objects; therefore, the designer can later specify them in classes and construct the taxonomy hierarchies. This approach is practical because it can take 59 advantage of the existing thing; that is, reusing through taxonomy space. However, it constrains the designer‘s creativity. More seriously, due to the complexity inherited from the existing things, it also leads to more complex software systems. Figure 3.6 An Example of Taxonomy Space In addition, comparing physical and software system designs, the system‘s expected behaviors (Be) and actual behaviors (Bs) distinguish these two kinds of designs. Physical systems within the environment are often constrained by the law of physics. Due to the complications of physics, the expected behavior (Be) and actual behavior (Bs) are often explicitly different. Hence, function is needed to express the ideal design specification. On the other hand, in software design, expected behavior (Be) and actual behavior (Bs) are often implicitly different. Only the distinction can be discovered because of two main reasons. First, performance of certain behaviors cannot meet the expectation (i.e. Bs is worse or better than Be), due to the hard-to- predict influence of the system‘s runtime environment (e.g. hardware or virtual machine). Second, the system is implemented with human‘s logical errors and bugs 60 are not detected, which results in the defective actual behaviors that the designer does not want. However, the above two reasons cannot be considered during the design time because the runtime status is changing and unpredictable, the detailed specification of the runtime environment is often decided in the later design stage, and the implementation errors can never be told in designs. Hence, software designers often use expected behaviors (Be) as software specifications and are unfamiliar with the notion of the function. Table 3.1 Comparison between Function and Expected Behavior Views Required Function Expected Behavior World View Without a thing in mind With a thing in mind Advantages More creative More feasible Disadvantages Difficult, bad for reuse, property inheritance More constraints Approaches with the notation of function are solution-neutral and more creative, but it also suffers challenges of un-nature thinking style. Designing systems by using expected behaviors is considered being more feasible in practices and generally helpful to reuse (i.e. taxonomy space), but it also has disadvantages of less freedom due to the prior embodiment constraints (see Table 3.1). Therefore, if designing a system from a synthesis perspective, the designer must create a structure from solution-neutral functions, and the process is also hugely different from analysis perspective design approaches. The designer does not only decompose the structure, he/she also decides the function - selecting each element in the requirement domain. Making such a decision of choosing function is subjective. In 61 other words, this process is based on the designer‘s preference; thus, the function is selected by the designer to satisfy customer needs. For example, the customer may need ―the data to be reusable‖. The function determined by the designer could be ―to save the data into storage‖ and ―to take out the data from that storage‖. In addition, to reason the function, dividing an upper level function into more specific sub-functions should be a way to specialize the designer‘s decision when more information is collected due to the design decisions that have been made or the structural elements that have been built. Furthermore, to realize entities in solution structure domains from functions, the decision cannot be random or no-limit because designs always have some constraints from other existing entities, environment, boundary, budget, and etc. Therefore, this realization must follow a human‘s rationality. Rational design thinking can also be found in many past design studies. (Simon 1990; Suh 2001; Franssen and Bucciarelli 2004). In summary, function and expected behaviors have their own advantages. Particularly, using function is more difficult but thinking of expected behaviors is more practical. Therefore, while the designer maps the specified functions to the structural elements in synthesis, the ideal approach is to integrate both views of the world – this is also the justification and foundation of our dissertation research. 62 3.3 Complexity in Software System Designs 3.3.1 Traditional System Complexity of Software No matter whether a system is natural or artificial, it usually consists of many elements with interconnected or dependent relationships. When a number of elements have complicated relationships or dependencies among them in a system, ―complexity‖ occurs. Unlike traditional program complexity that is based on control graph determined by a program‘s logic, many previous studies (Kitchenham and Walker 1986; Cant 1995) suggested that dependencies between modules should be used as one complexity measure for the overall software system since software with more inter-dependent modules is usually more difficult to understand from a cognitive perspective. More importantly, due to the numerous relations or dependencies among its elements, the behaviors of the system that caused by a local module‘s change is hard to predict. However, both program complexity and overall software system complexity metrics are based on the analysis of the structure of the software system by investigating the source code. These complexity definitions, which come from much empirical research, treat the software complexity as an ―as-is‖ property without considering the complexity in the design before the structure is determined. 63 3.3.2 Design-centric Complexity Rather than thinking about complexity as existing property or a result of the design to be analyzed, another perspective of complexity, design-centric complexity (DC), was proposed based on functional view (Suh 2003; Lu and Suh 2009). The DC is defined as ―the uncertainty of achieving function in design‖. In the DC, complexity is treated as a ―to-be‖ property of the system. Thus, in designing a system, the complexity should be caused by the dependencies that are created by designer‘s decisions. The DC indicates design decisions as a major factor that causes the complexity of the resulting system. Therefore, a highly complex software system could be made due to poor design decisions. Figure 3.7 Complexity from Design Perspective (Notional Expression) Technical systems are designed to satisfy functional requirement (or function) in order to meet customer needs. Considering both aspects at the same time is the designer‘s challenge. Therefore, these two components contribute to the overall system difficulty 64 (i.e., the level of complexity of the system). The first is the complication due to customer needs, other resources, and physical constraints. The second is complexity due to poor design decisions that incur a higher uncertainty of satisfying functional requirements (or function). According to DC, we can define the first type of system difficulty (complication) as ―inborn complication‖ and the second type as ―acquired complexity‖. In order to reduce the level of complexity of the system, the designer needs to avoid unnecessary complexity. In Fig. 3.7, more demanding customer needs cause a higher level of complexity of the system because of increasing inborn complication. However, the designer cannot moderate customer needs since this is not a suitable strategy in such highly competitive markets. Customers‘ demands set the baseline of the level of complexity of the system (point a 1 in Fig. 3.7). The designer has to avoid bad design decisions in order to reduce the acquired complexity (move a to a 1 in Fig. 3.7). Because traditional software design approaches only consider the program structure, that is, complexity includes acquired complexity and inborn complication, ―synthesis‖ is the way to reduce acquired complexity. 3.4 The Need for a New Design Approach to Manage the Complexity of Software System Traditional software design methods learn from practices and the observation of the end results or existing systems (i.e. ―as-is‖ property of the system or analysis perspective), and they focus on the structural decomposition; therefore, how the designer synthesizes the software architecture from the function is often neglected. 65 More specifically, any previous design approach carelessly or quickly passes the synthesis part of the design. Even though one may argue that the OO design approach proposes using use case to synthesize the software architecture (or the objects), unfortunately, the synthesis part is still ad-hoc without systematic steps or it often targets at too detailed level or smaller scale. An approach that can consider both system and module levels as well as is operated from abstract stage to specific stage is lacking. However, we must argue that those old ways poorly reduce ―acquired complexity‖. Thus, once the main system structure or architecture has been at hoc decided, the acquired complexity is set. This means that the level of complexity is fixed. It is also the baseline for the iterative refinement to reduce if they would like to refine the result. Therefore, focusing on synthesis is necessary since it directly reduces the acquired complexity. In this way, the overall system complexity should decrease. The traceability relation between functional requirements and their decisions of the software structure in analysis perspective is very complicated. This can result in incomprehensible and inconsistent design decisions. Since synthesis emphasizes the reasoning of how the software structure is decided, the tracing links from functional requirements to their realizations are directly created along with the synthesis reasoning process. Furthermore, this should bring in simpler (i.e., one requirement to one decision) traceability dependencies. 66 To improve these deficiencies, we must pay attention to the synthesis part. However, this does not mean the analysis part should be excluded. Instead, both parts should co- exist and balance. The synthesis part should be processed before the analysis. The forward synthesis also contributes the early stage design that is regarded to be more influential than the later stage. As a result, the following research issues should be motivated in the new approach: (1) How the program structure is created according to the functions that assigned by the designer – to better support the early stage design. (2) How to consistently manage complexity before the whole system is created – without analyzing the whole source codes. (3) To suggest a common model can describe system designs. (4) In current object-oriented software system designs, how to better allocate objects and components so that the design can be more efficient. (5) How the proposed approach copes with the existing software libraries or components (e.g., COTS or programming language API). 67 Chapter 4: Hypothesis of the Proposed Research 4.1 Problem Statement Traditional research suggested that complex software system design results in high structural dependency among its internal modules and poor efficiency of creating software system. The highly dependent modules in the complex software system cause difficulties of its implementation, maintenance, and testing. The poor design efficiency leads to use more iterations to produce the software system. These two deficiencies both come from a poor design approach. Our research problem is to provide a new approach to improve this problem. 4.2 Hypothesis By developing the systematic forward synthesis in the design to treat the complexity as a ―to-be‖ variable resulting from the design, the complexity of the software system can be reduced and better managed. In addition, the whole design becomes more efficient. Hypotheses to be tested in the validations are presented as follows: H1: Systematic forward synthesis design approach can reduce Ns, where Ns is the number of structural dependencies per module in a software system. 68 Systematic forward synthesis helps designs to decrease software system‘s overall structural dependencies. The structural dependency of the software system can be measured by counting the number of dependencies per module. H2: Systematic forward synthesis design approach can increase T, where T is the efficiency of designing a software system. Systematic forward synthesis helps the designer to result in a software system with less time spent or fewer iterations used in a design. 69 Chapter 5: A Synthesis Approach in Design 5.1 Overview of the Synthesis Approach in Design 5.1.1 Integrating Function and Expected Behaviors Our goal is to synthesize the advantages of function and expected behaviors in order to suggest an ideal design approach. However, bridging these two perspectives in one approach is not easy. Traditional design methods that use one hierarchy to deal with abstraction end up with using function for the higher node of the hierarchy and describing behaviors for the lower (more specific) nodes (see Fig. 5.1). Obviously, it is caused by poor separation of function and expected behavior. Our suggested method, which can combine these two views and also distinguish them at the same time, is to add one more hierarchy for function, beside the structural hierarchy, to expand one more dimension of thinking (i.e., the horizontal mapping from left to right). To lay out these two different hierarchies, zigzagging operation is used; thus, the horizontal mapping (i.e., zig operation) between two hierarchies creates elements of the structure from the function, and the zag operation from upper level structural hierarchy to lower level function hierarchy specifies the sub-function (see Fig. 5.2). In other words, the designer uses the functional view (or thinking) to map horizontally and think of the expected behavior in the zag operation because expected behaviors can be considered only when the embodiment comes out. 70 Figure 5.1 Traditional Method – One Dimensional Decomposition Figure 5.2 Integrating Function and Behavior Views – Two Dimensional Decomposition The idea of introducing an additional hierarchy can be extended for the whole system development. Since the real world development includes extracting customer needs, deciding function, creating the system structure, and manufacturing the system, we can expand one domain that only has structural hierarchy to four different domains to describe the whole development. The other three domains are customer, functional, and process. Each element in the customer domain is a description of the customer‘s need, the function is in the hierarchy of the functional domain, elements in the structure domain represent system‘s components, and the node of the hierarchy in the process domain shows how to manufacture the component or entity of the system (see 71 Fig. 5.3). Due to the two dimensional operation (i.e., mapping and decomposition), the real world is a diagonal specification process – not just a top-down decomposition. Therefore, in real world designs, the low level function (in the functional hierarchy) and detailed customer needs do not cause the designer‘s caution because it is hard for customers to tell specific needs, and the detailed function is also hard to decomposed in reality. On the other hand, the designer also cannot describe the high level structure and process because they are too abstract to be described. Figure 5.3 Real World Design – Diagonal Specialization Most importantly, the transition from function to expected behavior in real designs happens between the functional and structure spaces. After the embodiments (or entities) have been constructed in the structure space, the designer can refer to the embodiment and then create the expected behaviors. This transition is also the most difficult part in the design for the designer. 72 5.1.2 Forward Synthesis Model The core and transition part of the design, where the designer produces the entities from functions, is described in the forward synthesis model (see Fig. 5.4). The model includes three different spaces, functional, structure, and taxonomy, and five kinds of operations with their directions, ―has-a‖ specialization, ―is-a‖ specialization, realization, classification, and instantiation. Figure 5.4 Software Synthesis Model Functional space Function (F) is the element in the functional space. It presents the designer‘s abstract wish – what the designer wants from the system. Function is chosen by the designer in order to satisfy the customer‘s needs. To represent function, the designer must use ―to 73 + verb‖ in his/her description. For example, the customer wants to keep and show their photos. To fulfill this, function can be ―to save the photo files‖. While the designer decides the function, top-down specialization is utilized to decompose the upper level function into sub level function. This top-down specialization maintains ―abstraction‖ of the function and all function forms a hierarchy. This hierarchy maintains a ―has-a‖ or ―part-of‖ relationship in the space. Structure space Each entity should be a physical component or a software object in the system. For example, a wheel is one of the entities in a car system or a button in the user interface is the entity in the software system. Each of them inside the structure space is formed to realize the corresponding function. All entities can also be specified to result in a ―part-of (has-a)‖ hierarchy. Realization between functional and structure spaces is not just a simple horizontal mapping. The designer has to use ―zigzagging‖ maneuvers between the functional and structure spaces. The ―zig‖ operation that horizontally maps the function to the same level of the entity by thinking of the function creates the ―mean-of‖ dependency between the functional and structure spaces. The ―zag‖ operation that gives the additional constraints from the upper entity to the down level function by following the expected behavior of the entity guides the sub level vertical functional 74 decomposition. For example, if the designer chooses the brake as the entity for realizing the function ―stop the wheels‖, the additional constraints for the sub level function could be ―something attached on the axis‖. Therefore, the functional decomposition in the functional space is often limited to the upper level entity that the designer creates. This additional constraint is different from the system‘s overall constraints, such as development time, costs, resources; it originates from the entity the designer decides. Because realization is the most crucial part of the synthesis model, we will discuss the details in later sections (section 5.2). Taxonomy space Taxonomy space is a collection of some entities from the known world. Each element in the taxonomy space is a class, which extracts the common structure of the known entity in the structure space. A class is created through the classifying entities in the structure space for reuse purposes. It represents the type of some entities with common attributes or properties. Unlike the other two spaces, the hierarchies in this space are ―is-a‖ structures; that is, the sub level class ―is a‖ node of the upper level class. For example, ―a tree is a plant‖ or ―a sedan is a car‖. All the classes inside the taxonomy space are used as background knowledge for reuse in designs; that is, each entity in the structure space is always an instance of its corresponding class. For example, a steel car wheel with a 15 inch diameter is an instance of the wheel and this diameter is its attribute that has the value 15 inches. 75 Therefore, through the instantiation, the designer can make the entity in the structure space. Furthermore, from the above synthesis model, the AD is just an application of the model of forward design synthesis. However, AD lacks the taxonomy space because reuse and knowledge of the entity are usually implicit in the physical system design. 5.2 Realization of Function – Abduction 5.2.1 Abduction The designer must decide the components of system rationally because he/she has to make decisions based on the related design information, such as constraints, goals, and intentions. To realize the entities in the structure space, abduction is the method that applies to the horizontal mapping between functional and structure spaces. Abduction was proposed by Peirce (1960). In the logical system, it is one of the three reasoning modes (i.e. the other two are deduction and induction). Generally, abduction is a technique that can generate ―explanations‖ for given ―observations‖ or ―goals‖. For example, while ―shoes-are-wet‖ is observed, ―rained-last-night‖ should be one plausible explanation among the generated hypothesis set (Poole 1994). Unlike deduction and induction, abduction is a type of critical thinking rather than symbolic logic. Deduction is a form of analytic inference based on logical demonstration. Peirce suggested that it is drawing the logical consequence from 76 premises. The conclusion is true given that the premises are also true. The most noteworthy feature of abduction is the capability of introducing new knowledge, and it is also the only logical reasoning that offers this ability. For seeking plausible explanations, Pierce‘s framework is usually formulated in the following mode: The surprising phenomenon, X, is observed. Among hypotheses A, B, and C, A is capable of explaining X. Hence, there is reason to pursue A. Analogizing above mode, function in the functional space can be treated as the phenomenon that is observed. Therefore, the element in the software architecture should be selected among the generated hypothesis set as long as it can satisfy (explain) the function. 5.2.2 Functional Coupling In order to manage the complexity of the software system design, during the realization of the function, the designer has to be very careful about functional coupling. The definition of functional coupling is that two functions share the same entity; that is, two functions‘ realizations associate with the same entity. Since the thinking direction of the forward synthesis is from functional space to structure space, the designer has to evaluate the functional coupling after the entities are created through abduction. 77 To detect the functional coupling, using a design matrix [A] is necessary because the design matrix [A] denotes a dependency between functions and entities. Therefore, at each level of the ―has-a‖ specialization, the mapping between the functional and structure spaces can form a design equation {F} = [A]{E}, where {F} represents function, and {E} represents entity. According to the characteristics of the [A], three kinds of designs, uncoupled, decoupled and coupled, can be found. If all the non-zero elements in [A] are only in the diagonal position, it is an uncoupled design. The decoupled design is identified while all the elements in the upper triangular of [A] are equal to zero. Hence, if [A] is not the uncoupled or decoupled case, it must be a coupled design. The uncoupled design means that each function is only satisfied by one corresponding entity; for example, in Fig. 5.5, F 1 is only related to E 1 (i.e., F 1 = A 11 E 1 ). The function (F) in the decoupled design is not wholly independent; thus, the independence of the function occurs when entities can be determined in a particular sequence. Figure 5.5 Three by Three Uncoupled and Decoupled Design Matrixes 78 According to the definition of design-centric complexity (see section 3.2), the designer must avoid (functional) coupled designs because it causes increasing acquired complexity. Functional coupling is a new concept to complexity research. It is majorly different from physical coupling, which is often well-known by research communities. The design matrix has to be maintained during the whole design. In other words, once the coupled design is detected, the designer must choose a different entity from his/her abduction. 5.3 Synthesis-based Software Design Framework Due to a lack of a good way to create objects from functions in traditional OO software system design, the model of forward synthesis is used to supplement the design method. This new Synthesis-based Design Framework is expected to improve the old OO design by introducing the systematic way to synthesize the solution structure of the software system; therefore, the rationale of the design decision can be explicitly shown and the solution structure should be more consistent with the functional requirements. Fig. 5.6 shows the details of the Synthesis-based Software Design Framework. The designer first takes the customer needs as input and then decides the functional requirement (or functions) of the system to satisfy the customer needs. After the functional requirement is chosen, the designer performs the platform- independent design so the entities of the system (i.e. the objects) are built to realize the functional requirement. During the platform-independent design, the class could also be specified to represent the common structure of the objects. Finally, according to the 79 platform-independent design, the designer chooses the specific technologies, such as certain libraries or components, which will be used to implement the system to do the platform-specific design. Hence, inside the functional, structure and platform domains, the top-down operation in the domain is specialization, which uses decomposition to maintain the abstraction in a hierarchy. For example, the upper level functional requirement is decomposed into sub level functional requirements in the function domain and all functional requirements form a ―part-of‖ hierarchy. The top-down specialization in the taxonomy and platform domains are different; all the elements should form multiple ―has-a‖ hierarchies that represents the classification relationship instead of forming a single ―part-of‖ hierarchy. The horizontal zigzagging operation is across two adjacent domains. The zig operation (i.e., mapping from left to right domain at the same level of the hierarchy) is realization, which means that the element in the right domain is ―mean-of‖ the element in the left domain (e.g., entity in the structure domain is mean of the corresponding functional requirement in the functional domain). Since the design synthesis model suggests using abduction to realize the functional requirement to create the entity (as mentioned in section 5.2), this mapping involves the designer‘s abduction in this operation. The zag operation maps from the node of the hierarchy of the right domain (e.g., E) to the sub level nodes of the hierarchy of the left domain (e.g. FR 1 and FR 2 ), which often brings the additional constraints to the sub level nodes in the left domain from the upper level node in the right domain. It is clear that the 80 designer can clarify various types of constraints, understand the design rationale, and avoid the dependencies (coupled) during the zigzagging. These benefits could seldom be seen in the traditional OO design and are particularly useful during the whole development, even in the system maintenance phase. Figure 5.6 Software Design Synthesis Framework Similar to the role of taxonomy space in the model of forward design synthesis, two operations, classification and instantiation, happen between taxonomy and structure domains as well as between library and platform domains. For classification, both 81 taxonomy and library domains extract the common form of the elements from structure and platform domains; respectively, for reuse purposes. Hence, in the structure domain, the entity (object) should be specified by the class in the taxonomy domain and common entities (objects) can also be described by the same class. For example, a ―red steel ball with 6 inch radius‖ and a ―blue wood ball with 7 inch radius‖ both belong to the class ―ball‖ in the taxonomy domain. The designer can define all the properties of the ball class by assigning radius, material, and color. The ball class can be treated as a knowledge or dictionary for the designer to decide the entity. Similar to the taxonomy domain, hierarchies in the library domain mean the categories of the platform-dependent entities (PEs). Since the designer thinks the specification of the class in both taxonomy and library domains as the knowledge or dictionary, the intention of instantiation operation is to let the designer reuse the knowledge to make a specific instance from the hierarchies (knowledge). 5.4 The Process of Synthesis-based Software Design Framework There are three main steps for using the Synthesis-based Software Design Framework. We explain the detail of each step below. Step 1: Elicit customer ’s needs Customer‘s needs are collected during interviews or meetings with clients. To ascertain the client‘s actual needs rather than what the designer thinks, an elicitation 82 technique is often applied. Then, those real customer‘s needs are organized and listed in the documents for the designer to start the step 2. Step 2: Specify functional requirement, entity, structural constraints, platform- specific entity and platform-specific constraints through double zigzagging operations After all customer‘s needs have been extracted, the designer starts to specify the functional requirements, the structure of system and the platform-specific realizations. This step is the most complicated one in these three principal steps. It can be divided into six sub-steps. They are shown as Fig. 5.7. Except step 2.1 to 2.3, the whole process of the rest of the sub-steps in step 2 (step 2.4 to step 2.6) is a loop. This loop ends while the designer thinks that all the design details are done. Figure 5.7 Six Sub-steps in Step2 Step 2.1 - Choose an initial FR The beginning operation is for the developer to choose the highest FR. This root FR represents the most abstract system function (i.e. the overall expected behavior of the system). Due to lacking of more concrete details, this highest FR only shows the intention of the designer. 83 Step 2.2 – Create Es by realizing the respective FRs The designer lays out the FRs and the Es through the ―zigzagging‖ operation. All the FRs are based the customer‘s needs. Step 2,2 is similar to step 2.5; however, it is a special case of step 2.5. It is the highest node in the entity hierarchy, which represents the whole system. This system contains many components and objects inside itself. Step 2.3 – Create PEs by realizing from the respective Es Platform-specific entities are based on the technologies that the designer chooses to use for implementing the system. It could be related to the libraries, the Commercial off-the-shelf (COTS) products, the programming language, the operation system and so on. However, since the designer does not have enough information about the whole software system structure at the root of the hierarchy, it should be impossible to specify which platform dependent component that will realize the highest entity in the structure domain. The designer could only decide which technology will be used in the platform domain. Step 2.4 – Specify FRs and associated Cs according to the upper level FR and E The designer specifies more detailed FR by decomposing the upper level FR into sub- level FRs. However, this specialization operation is not just like traditional one dimensional decomposition. It is also limited to the upper level entity; that is, the designer‘s choices of the sub-level FRs are not at-hoc or random. The designer must 84 consider the upper level entity. Therefore, additional constraints are often identified through the ―zag‖ operation. For example, if the upper level DP decides to use three- tier structure and the third tier is a database structure, the sub-level FR could be limited to choose the FR – to communicate with the storage through a protocol. Besides, the FR decomposition is very different from the traditional functional decomposition in the structured software design approaches. The FRs do not concern any sequence of the operation or any data flow. The FRs are the functionalities (or the expected behaviors) that the system or the component has to perform. As a result, if the designer needs, they can even apply some technique, such as the use case analysis, to help themselves to specify the functionalities of the system from the customer needs. Step 2.5 – Specify sub-level Es and associated Cp according to the upper level E and PE Figure 5.8 Class Diagram for Showing the Possible Entity Sources in the Object 85 At this step, the designer has to create the program structure to realize the FR. This operation in OO design is to identify all the objects inside the system; thus, the composition of the objects and the instances (of the class) passed between objects, both need to be clearly allocated. Since the class is the specification of the object (its instance), it describes the structure of the object. The designer creates the ―part-of‖ hierarchy (rather than inheritance relationship) and can use the graphical notation like the UML class diagram to show the static relationship between objects. More specifically, attributes or parameters made of user-defined data types (i.e. classes) also belong to the entity that the designer creates (see the class diagram representation in Fig. 5.8). In addition, objects in the class diagram that contains aggregation and composition relationships are also considered as entities. Between the root and lowest levels, components or higher level objects (which compose lower level objects) are chosen (through the specialization operations) to compose the overall architecture. During this specialization, the designer must always consider the associated interfaces and names. However, this operation is not ad-hoc or random, either. Those level lower entities should be consistent with the decision that is made by the designer at the higher level. Two primary contributions show the way the designer creates the program structure (object). First, similar to step 2.4, the program structure that the designer creates also is limited to the upper level PE. The additional platform-specific constraint Cp could be added along with the zag operation. Second, the realization from FR to E should 86 also be an abduction process. Based on the information that the designer has, he/she abducts the possible entities and then evaluate the most appropriate one. If the designer only abducts one entity and the entity cannot pass the evaluation, he/she has to do this operation again at this level. Since we propose to reduce the complexity of the software system, in order to perform the evaluation, we use the design matrix [A] to detect the functional coupling between FR and E; that means, the designer can only use the entity that is functional uncoupling or decoupling. One must notice that the designer does not design from the class hierarchy - that is to think from the top-down class hierarchy. In order words, the abstract class or the inheritance is considered while the designer realizes the FR and reusing some component is possible. In other words, because the designer reuses objects or considers the common interface of objects, the class hierarchy is formed. For example, when the designer think object X and Y could share some interfaces during the realization, a parent class, as the common specification for both object X and Y, could exist in the taxonomy domain. Step 2.6 – Mapping the PEs from Es To realize the Es is more intuitive. The designer just need to consider the platform- specific technological choices and then maps them to the Es respectively. The designer can find what component the PE is instantiated from the library, which is provided by programming language, COTS, and so on. 87 Step 3: Supplement the design by modeling from other viewpoints The step 2 does not limit the designer to build a static structure (like UML class diagram) only. The designer may create other representations to help them to understand better some relations like functional coupling between the entities. However, if those representations that are created in step 2 are not enough for the designer to express the whole design result, they can plot the internal relationship between modules at different viewpoints. For example, the sequence diagram is one of the UML diagrams in OO design to show the dynamic message between objects and it may not be needed at step 2. Therefore, it can be drawn at this step. 5.5 An Illustration Example To demonstrate how the synthesis model works, the following software system is used. The customer needs of the application are ―constructing a specific LoadRunner script by combining multiple XML files and a special formatting template file‖, ―the user can select the input files from a graphical user interface (GUI)‖, and ―the user can determine the output path from the GUI‖. According to the customer‘s needs, we can decide the functions and their corresponding entities through our proposed model. As pointed out in section 5.1.1, the upper levels of the nodes of the structural hierarchy should be highly abstract; they only present the embodiment that the designer imagines. The root of the functional hierarchy is ―to generate LoadRunner‖. It is the most abstract wish of the designer; 88 that is, no detailed functions are specified. After the designer decides a system to process by using a block (see Fig. 5.9) to represent it, he/she maps the most abstract function to the system. When the designer does the zag operation, an additional constraint ―the component must be in the system‖ comes out. Following the constraint, he/she decomposes the root function into two sub-functions: ―to process the script‖ and ―to take and display information‖. To map these two functions or to explain them by using embodiments, two components (objects in OO programming) inside the system (see the block diagram in Fig. 5.9) that can enable these two functions are defined. Therefore, at the second level of the structural hierarchy, two components (or objects), ScriptGenerator and DataIO, are created. Figure 5.9 The First Two Level Zigzagging While decomposing the function – ―to take and display information‖, the designer decides two sub-functions can be divided and both of them constrained by the DataIO module. The DataIO in the designer‘s mind is a component that is for users to operate 89 the input and output data. Therefore, the two sub level functions under the ―to take and display information‖ are ―to operate input and output data‖ , ―to save output data‖, and ―to transform input data‖. They are mapped to the ―GUI‖, ―OutputHandler‖ and ―InputHandler‖ components, respectively (see Fig. 5.10). DataIO: for showing data To take and display information GUI: for manipulating the input and output InputHandler: for processing the input data To operate input and output data To transform input data DataIO ScriptGenerator System GUI InputHandler To save output result OutputHandler: for writing the output data OutputHandler Figure 5.10 The Third Level Zigzagging Under DataIO The entity in the designer‘s mind influences the decomposition of the function. In Fig. 5.11, two different GUIs can satisfy the functional requirements that the designer decides. These two GUIs will give different constraints and behaviors when the designer specifies the sub level functions. If the designer selects A, the behaviors of the entity can be ―text fields for keying in the input, template, and output paths‖ and ―an operation for passing those data‖. If the designer determines B, the behaviors of the entity should be ―a text field for keying in the file path‖, ―options for selecting the type of the file‖, ―an operation for passing data‖, ―an operation for adding the chosen path‖, and ―a text field for showing the chosen file paths‖. Since the embodiment at 90 the higher level has been decided, the lower level function is more like expected behaviors, and it also limits the designer‘s thinking. Figure 5.11 Two Possible GUI Entities 91 Figure 5.12 Class Diagrams Mappings of Two Possible GUI Entities From above GUI A, the designer also can define the type of similar objects by classifying them. The GUI A has three text-field instances for input and they all have the same behaviors. As a result, in the taxonomy space, all of them can belong to the ―TextField‖. For GUI B, the reused instances then become ―Button‖ and the way to lay out the class diagram is also different from GUI A. 92 Figure 5.13 The Third Level Zigzagging Under ScriptGenerator Under the ―ScriptGenerator‖, XMLHandler also has two possible structures after two dimensional specializing. Both of them (i.e., option A and B) come from the designer‘s abduction thinking. In option A, both XMLParser and DataCollection have the dependencies with TwoLayerData. As a result, in the design matrix of the option A, it indicates a coupled design. However, option B uses the general object for DataCollection and XMLParser; therefore, the dependencies between TwoLayerData and them can be removed. From option B‘s design matrix, it is a uncoupled design. To have a simpler design in the end, one must choose the functional uncoupling design. Hence, in this case, Option B is the choice for a good design. 93 Figure 5.14 Two Possible Structures XMLHandler 94 Chapter 6: Experimental Validation and Results 6.1 Validation Procedure The framework is evaluated primary with respect to complexity and efficiency in the real world software design. With aids from the Distributed Scalable Systems Division of Information Science Institute in University of Southern California, we collect whole development records from a software project, called Visualization of Attack Surfaces for Targeting (VAST). The VAST is an Ecilpse plug-in module that is implemented by JAVA. In major part of the validation, both SSDF and OO method are applied to design the VAST system. Then, two sets of design results are compared. Since the VAST project followed the rapid prototyping process and OO method, to understand the differences between OO method and SSDF, the original VAST design result is used for comparison. However, the original VAST development contains design, implementation, debugging and testing activities so it cannot be directly used for comparison. To have an objective and fair comparison, we firstly identify the design phase from the VAST development history and the customer needs from various meeting notes and logs. The designer who uses SSDF can work on the same customer 95 needs to design the VAST system within the same period. Then, these two results can be compared. However, the challenge in this comparison process is how to design, control, and perform the experiments. Hence, we start from a case study, learn from the study, and then make an experiment. Table 6.1 shows the step-by-step procedure regarding the proposed framework evaluation. Table 6.1 Validation Procedure Step Task Description 1 Project Data (VAST) Collection 2 Preliminary Data Analysis on VAST Project I. Determine the VAST project’s design period for comparison II. Determine the requirements for design experiment 3 Case Study I. Develop the measurement methods II. Understand the subjects for later experiment design 4 Experiment Design and Perform Experiments 5 Experiment Data Analysis and Validation In the later sections, we will explain the details of each step. 6.2 Measurement Methods Two measurements – complexity and number of iterations will be computed from our raw results. For measuring these two variables, we develop a graph-based method. 96 Because the design results are all in UML format, the graph-based method converts the UML diagrams into graphs, which represent the structural relationship between the internal elements, and then measures the graph. The complexity can be directly calculated from the graph, but the way to get the number of iterations is indirect. The graph is just an aid to calculate the amount of iterations. According to our definition, iteration is counted when the system structure changes considerably. Hence, the graphs of two consecutive iterations should be very different. We developed a graph similarity method to detect those similar results from the whole design history and then iterations are extracted after eliminating those similar ones. Design Result (UML) Graph Transformation Graph Graph-based Measurement Complexity Graph-based Similarity Iteration Figure 6.1 Process of Producing Complexity and Iteration The following three sections describe the graph-based methods. 97 6.2.1 Graph Transformation A "graph" refers to a collection of vertices (or nodes) and a collection of edges that connect pairs of vertices. It is commonly used to express the natural or man-made structure. To measurement the complexity of the result, the graph method is proposed. The graph transformation converts the UML class diagram into a typed directed graph that is called a ―class graph‖. To perform this graph transformation, we firstly need to define the expression of the graph. Given a directed labeled graph G, G can be defined as the following equation. G = (V, E, s, t, l, m) Where V is a finite set of nodes; E is a finite set of edges; s, t, : E → V assign a source node s(e) and a target node t (e) to every e ∈E; l : a node label l(v) to every node v in V; and m: an edge label m(e) to every edge e ∈E. Based on Tsiolakis (2000) and Zhou (2003), there are three key transformation rules: (1) converting classes into nodes; (2) making relations between classes into typed connections between nodes; (3) eliminating super nodes. Therefore, as shown in Figure 6.2 and 6.3, nodes represent the classes and edges represent relations between 98 classes. Since a class diagram contains multiple kinds of relations, we consider each relation‘s characteristics and convert into the graph. ClassName (visibility) attribute name: type express (visibility) operation name(parameter list): type expression Class Name= “ Pa c ka g e: C l a ssN a m e ” Attributes = {(visibility, attribute name, type express)} Operations = {(visibility, operation name, parameter list, type expression)} Figure 6.2 Transform classes into nodes in the graph ClassName ClassName Associate Name Visibility 1 Role1 n..m Visibility 2 Role2 o..p Class Class Associate Type = association Name = Association Name SourceRole = role1 DestinationRole = role2 SourceVisibility = visibility1 DestinationVisibility = visibility2 Associate Type = association Name = Association Name SourceRole = role2 DestinationRole = role1 SourceVisibility = visibility2 DestinationVisibility = visibility1 Figure 6.3 Transform associations into bi-direction edges in the graph Inheritance relations should be considered differently in our graph since inheritances exist in our taxonomy space in our synthesis-based design approach. Besides, super 99 (or abstract) classes usually maintain the outline of the sub-classes; therefore, in a system, the generalization between super and sub classes should not be considered as an edge in the class graph. However, we should take those inherited attributes and operations from the super class and put them into its sub classes (see Figure 6.4). SuperClass +attribute1: type #attribute2: type -attribute3: type +operation1(): type #operation2(): type -operation3(): type SubClass1 +subAttribute1: type +subOperation1(): type SubClass2 +subAttribute: type +subOperation(): type SuperClass Class Name= SubClass1 Attributes= {(+, subAttribute1, type), (+, attribute1, type), (#, attribute2, type), (-, attribute3, type)} Operations = (+, subOperation1, {}, type), (+, operation1, {}, type), (#, operation2, {}, type), (-, operation3, {}, type)} Class Name= SubClass2 Attributes= {(+, subAttribute1, type), (+, attribute1, type), (#, attribute2, type), (-, attribute3, type)} Operations = (+, subOperation1, {}, type), (+, operation1, {}, type), (#, operation2, {}, type), (-, operation3, {}, type)} Figure 6.4 Inheritance transformations SuperClass SubClass Class Class Class Name=SuperClass Name=SubClass Class Name=Class Association Name=association1 Association Name=association1 Association Name=association1 Figure 6.5 Inheritance transformations for associations 100 The super class should not be considered as a node in a graph during the transformation since its details are put into its sub classes and the generalization relation is not an edge in the graph. Therefore, if there is an association between a super class and a class like Figure 6.5, due to the elimination of the super class and generalization, the association should be changed to the sub class from the super class. 6.2.2 Graph-based Similarity By calculating the graph-based similarity between the design outcomes of two sequential iterations, we can decide if the later one has a significant change. If the later iteration has significant modifications, we regard the later iteration as a meaningful one. According to Le, Ho and Phan (2004), to obtain the graph-based similarity between G and G‘, we firstly need to look for the non-overlap connected common sub-graphs (NOCCS) between them and then use NOCCS to calculate the similarity. We demonstrate how to derive the definition of the similarity in the following. 101 Given the graph G =<V, E, u, v>, the other graph G’ =<V’, E’, u’, v‘> and their common sub-graph G i =<V i , E i , u i , v i >. T is a set of sub-graphs of G, and T = {G i =<V i , E i >: i =1, 2, ...} is called a set of non-overlap connected common sub-graphs (NOCCS) of G, denoted by π(T, G). Therefore, the similarity score between G and G’, denoted by Sim(G, G'), is defined based on the weights of sets of NOCCS of G and G’. The weight of each set is built on the weights of its member sub-graphs, which depend on the nodes and edges of the sub-graphs. Given a sub-graph G i =<V i , E i > of graph G =<V, E>. For each node v of Vi, we define its weight with respect to G i and G, denoted τ(v, G i , G), as the ratio between the number of edges from v in G i and that in G. Therefore, τ(v, G i , G)= |{v‘ : (v, v‘) ∈ E i )}| / |{v’ : (v, v’) ∈ E)}| 0 ≤ τ(v, G i , G) ≤ 1 Hence, the weight τ(v, G i , G) is clearly proportional to the number of common edges between G i and G at node v. The weight of sub-graph G i with respect to graph G can then be defined as the sum of weights of nodes in V i , denoted by ρ(G i ,G), which is i V v i i G G v G G ) , , ( ) , ( 102 Given T = {G i =<V i , E i >: i =1, 2, ...} is a set of NOCCS of G and G’. Define the weight δ(T), ) , ( ) , ( )) , ( ) , ( ( ) ( G G G G G G G G T i G i i The sum of products of weights of sub-graphs G i with respect to G and G’ divided by the product of weights of G and G’ with respect to themselves - then it is normalized. As a result, the definition of similarity Sim(G, G’) is ) , ( & ) , ( : ) ( max ) , ( G T G T T G G Sim T This simply means the Sim(G, G') is the maximum weight of all possible sets of NOCCS of G and G’. 6.2.3 Graph-based Measurement The class diagram shows a system's static structure. A class is also a specification of a set of objects that are instances of it. Elements in a class diagram include classes and their inter-relations. The relationships are divided into associations and generalizations. Special kinds of associations are compositions and aggregations. A class consists of a name, a set of attributes, and a set of operations. Every class specifies a set of objects called the instances of it. An association end is also an 103 element of class diagrams, and it contains some information such as the role a class plays in the corresponding association or its multiplicity. In a class, the attributes and operations have visibilities that indicate if they can be accessed by other classes. Basic class diagram metrics counts the number of elements in a class diagram. Since a class has two primary parts: operations (or methods) and attributes, those two parts can be sum up. Total number of classes can also be calculated, and it can also be one way to show the size of the static structure of the system. Similar to the total number of classes, summing up the total number of associations or aggregations is also a method to know how many numbers of inter-dependences among classes can be measured. These basic metrics based on the UML class diagram are defined in Table 6.2. Table 6.2 Metric Definition Type of Metrics Metric definition Size metrics Number of Classes (NC). The total number of classes. Number of Attributes (NA). Size metrics The total number of attributes. Number of Operations (NO). The total number of operations Structural complexity metrics Number of Associations (NAssoc).The total number of associations 104 However, basic metrics cannot truly reflect the structural complexity in a software design. The class diagram contains different types of classes, like abstract classes or interfaces, which should not be treated the same as a normal class. Therefore, one should transform the class diagram into a more generic graph to calculate the structural complexity. Besides, different relationships have different impacts on the degree of dependency among classes. As a result, we can assign different weights to different kinds of dependencies. We suppose t i to be common association (i.e., dependency, link, and etc.), association class, and aggregation, composition, and also suppose w i to be their weights, respectively. For a class diagram, we use a matrix W(i, j) to represent the weights of the edges in the corresponding graph G. For any two nodes n i and n j in G, it is easy to know that the weight between n i and n j is W n i , n j = w ij , if a relation in n i , n j exists 0 The weight of different dependences in a class graph can be different. Aggregations and compositions are particular cases of associations - they represent part-of relations. Since, in these part-of relations, every object that composes the object O can be 105 thought as an independent functional unit inside O and these two relations are also what we intend to form, aggregations and compositions should have very light weight (i.e., approaching zero) while we calculate the complexity. We define a full connected graph G f as a node n i connects all the other nodes in bi- direction. Since G f can be represented in a matrix G f (i, j), the complexity C l calculated based on inner connections can be represented as the following equation, C l = W∙ G f G f In our upcoming measurement, we do not specify the weight for different relations. The number of connections in a graph can also be a measurement to determine the complexity. We consider each association the same so that the structural complexity can be calculated by using non-weighted measurement. Hence, the equation becomes C l = G f G f We use the above equation for all the structural complexity measurements in our experimental validation. 106 6.3 Preliminary Data Analysis and Case Study 6.3.1 Data Analysis The original VAST version history contains completed development phases. From preliminary data analysis, each design period can be detected and defined. To do this, we collect all the meeting notes from the online wiki website and the entire 841 versions from Subversion - the version control repository. After both kinds of data are collected, we start to analyze all the meeting notes and source codes. During the analysis, we eliminate insignificant versions from our collected dataset in order to determine key versions that represent the critical time points of the development. Those key versions are used to identify different development phases, such as customer needs, system design, implementation, or testing. Finding these phases is the key step in our validation and comparison process because the number of iterations and the system complexity are calculated based on those versions within the design period that we just specify. 107 1.4 Calculate Direct and Relative Changes 1.3 Measure Source Codes VAST Project Data Program for Counting Line of Source Code and Number of Classes Lines of Source Codes and Number of Classes 1.1 Data Collaction Data Retrieval Program All Versions 1.2 Analyze Meeting Notes All Meeting Notes 1.5 Determine the Period for Comparison Time Points of Key Prototypes Key Versions in the Period Tables of Changes and Measurement Plots Thresholds Formulas of Direct and Relative Changes Sorting Figure 6.6 Process of Preliminary Data Analysis While studying the meeting notes, we distinguish three kinds of descriptions except the logs of task assignment. They are functional description, modification description, and description about testing or debugging of a function. We can identify those time points, which customers or developers demand a massive modification of the system. By knowing those time points, we can discover which version may have substantial changes. Using source code analysis to determine the significant structural change of the system is truly different from meeting note analysis. We write a Perl script to count the 108 amount of source lines of codes (SLOC) and number of classes at each version, and then plot the figures respectively (see Figure 6.7 and 6.8). One characteristic of the key version in the source code analysis is that the amount of code change is substantial. Therefore, by comparing the number of source line of code (SLOC) in two sequential versions, those key versions can be identified. More importantly, in a source analysis diagram (e.g., SLOC vs version), the key version points can match the shape of the curve. Three methods are applied to capture significant changes. The first method calculates the slop change (SC) against three consecutive versions. SC = (V n+1 - V n ) / (V n - V n-1 ) V n is the measured value (e.g., SLOC in the SLOC vs version diagram) at version n, V n+1 represents is the measured value at version n+1, and V n-1 represents is the measured value at version n-1. This SC represents the angle between two tangents from two sequential versions. Once the SC exceeds the threshold, the key versions can be extracted. The second method 109 extracts key versions based on the calculation of the relative difference RC between two sequential versions. We can also setup a threshold to determine if a version is the key version. The equation below is to calculate the relative changes RC. RC = (V n - V n-1 ) / V n-1 V n is the measured value (e.g., SLOC in the SLOC vs version diagram) at version n, and V n-1 represents is the measured value at version n-1. Once RC is larger than the threshold 25%, we think that version n contains enormous structural change. The third method is similar to the second one. Instead of calculating the relative change, we compute the direct difference based on a normalized curve. The normalized values are calculated by dividing the measured value like the number of SLOC by the maximum value at Y-axis on each point. After we get the normalized values, we can direct calculate the difference using the formula below. DC = (V n - V n-1 ) DC is direct change, V n is the normalized value at version n, and V n-1 represents the normalized value at version n-1. 110 In order to avoid missing any significant modifications or possible key versions, the union of all the above three method‘s results is the entire key version set. Figure 6.9 shows that, according to both meeting note and source code analyses, the whole development history can be divided into four phases: (1) customer needs to requirements (from version 1 to 173) (2) developer‘s learning and research (from version 174 to 264) (3) implementation and testing (from version 264 to 625) (4) debugging (from version 626 to the end of development - version 841). The first phase is used for later validation. Figure 6.7 Number of SLOC at Each Version 0 2000 4000 6000 8000 10000 12000 14000 0 100 200 300 400 500 600 700 800 Number of SLOC Version SLOC Matching Curve 111 Figure 6.8 Number of Classes at Each Version Figure 6.9 Different Development Phases 0 20 40 60 80 100 120 140 0 100 200 300 400 500 600 700 800 Number of Classes Version Class Matching Curve 112 6.3.1 Case Study The case study is considered as a preparation of the experimental validation. Hence, the case study result is used for us to understand the way to design and carry out the experiment. 6.3.1.1 Case Study Plan The case study contains two stages. The first stage uses a small-scale and limited system design. The second stage asks the designer to design a real world project. The first part not only investigates the trend of the comparison in relatively small- scaled system, but also lets the subjects learn the SSDF. The scale of the first stage system is regarded small and limited because the workload is estimated around five hours per person, the guideline already provides a web-based user interface, and the programming language is required to use JAVA. The participant only needs to consider the back-end (server side) function. We setup two variables, graph-based complexity and time, for measuring the complexity and efficiency, respectively, at the end of the design, so that we can understand if both measurements are capable enough to represent the validations. The first stage design is limited and simpler. The 113 completion criterion is achieved when the result satisfies all the customer's needs. If a subject's design is disqualified, they have to keep doing the design until all the functions are fulfilled. To know participants‘ understanding of the SSDF, we study their performance improvement. During the study, we create a blind test to ensure if the performance is not improved due to the information leaking. In the blind test, two sets of customer needs are setup for two groups of participants, and these two different sets of needs should result in equivalent system structure, where the designer does not know about. In other words, during the test, one group uses SSDF to redo the design, but the other group performs SSDF on the new system design. Both designs are assumed to result in identical system structure. If the complexity of both results are very different, this implies that the participants know our measurement goal and intend to perform better when they use SSDF. However, how to produce another set of customer needs for this blind test is challenging. We develop a method, which can create two sets of customer needs 114 sharing with two identical structures. In our method, these two sets of customer needs are called Structural Identical Customer Need‘s Descriptions (SICND). The details of generating SICND from one system‘s customer needs will be discussed in section 6.3.1.2. Table 6.3 Two Stage Case Study Plan Stage Objective Measurement Point of Completion 1 Small-scaled, more limited system design For Subject to Learn the SSDF Complexity: Graph-based complexity Efficiency: Time of performing the design The design fulfills the customer's needs 2 Investigate SSDF in a real world design Apply existing project data Complexity: Graph-based complexity Efficiency: Number of iterations A prototype fulfills the customer's needs The second stage‘s purpose is to understand SSDF‘s performance in a larger scale system design, so this stage uses the VAST project data. From our preliminary data analysis, we specify a part of its version history for calculating the number of iterations. Each participant performs the SSDF to do the same VAST system design. In the end, their results are compared with the original VAST project data that we retrieved. The design guide for both stages of the case study can be seen in Appendix. 115 6.3.1.2 SICND method SICND is a new term that we introduce in our approach. When a pair of customer needs looks very different but results in two software systems with similar functionality or identical structure, we regard these two sets of descriptions as SICND. However, in the real world, two structures may not be perfectly identical. Hence, we need to setup a threshold to determine if the structures of two resulting systems are identical during their measurement. For example, A and B are two sets of customer need’s descriptions. Designers build two systems X and Y according to A and B, respectively. After X and Y are both measured on their structures, if similarity of X and Y is larger than threshold (or difference of X and Y is smaller than threshold), we consider A and B are SICND. Determine the Verb and Noun Combine Template and Keywords Mapping Keywords Additional Annotation or Modification Template Keywords Draft of Structural Identical Description Description of Customer Needs Structural Identical Description of Customer Needs New Keywords Syntax Figure 6.10 IDEF0 Diagram of Creating Structural Identical Customer Need’s Description 116 To create the SICND, we firstly specify an original customer need’s description. The original needs may include textual descriptions and user interface prototypes. Then, a template and its mapping keyword set are extracted from the original customer needs. Based on the template and keyword set, the structural identical customer need’s description can be created. When original descriptions come along with user interface prototypes, the keyword set can be used to replace the corresponding word in the original user interface. For example, a word “user” has its mapping keyword “product”. In the user interface, when the label is “user”, it has to be replaced by the word “product”. We analyze the original customer need's description and extract the verbs, nouns and the rest of the sentences base on syntax analysis. Like Figure 6.11, the sentence that describes the customer need can be broken down into different categories. Figure 6.11 Syntax Analysis on the Sentence One user ID can only join one group. Noun Noun Quantifier Quantifier Verb Aux 117 Figure 6.12 The Mapping Sentence A template is created primary by removing the nouns in the sentence. The nouns are the keywords that wait for mapping to a new set of words (nouns). For example, in Figure 6.12, the keywords are "user", "ID", and "group" and they can be mapped to "product", "ID", and "category" in the new description. The new words in the same set should be selected based on the related semantics. For instance, "product" and "category" are related because products can be classified into categories. The verbs in the sentence may need to be changed according to the mapping keywords (nouns) that we selected since the sentence has still to be understandable by the designer. Once those mapping keywords have been filled into the template, to make the new customer need's description understandable, some additional annotations and minor modifications need to be applied. Figure 6.12 also demonstrates the sentence from the template, the keywords, and the result after the keywords have been replaced. One user ID can only join one group. One product ID can only join one category. 118 6.3.1.3 Design Tasks In the first stage, we specify a login system and then use SICND to generate a product system‘s customer need descriptions. Both of the descriptions are in Table 6.4 and 6.5. Two guides are created for participants to do the case study and the details of the guides are shown in Appendix. Table 6.4 Customer Need ’s Description of Login System Customer's Needs of Login System: 1 The system only has three kinds of users (roles) - "admin", "user", "group manager". 2 One user ID can only associate with one role. 3 The user ID is unique as a key to identify users. 4 One user ID can only join one group. 5 The group name can be changed while admin click “edit” when the user is in the "admin" page.. 6 A group only has one group manager. The group manager is assigned by admin while the group is being created. 7 The system should respond login error to users when they input the wrong account ID. But, once logging in is successful, the system should forward the user to his/her individual homepage according to his/her role. 8 A normal user's homepage contains a user's group names, account IDs, and a personal information edit button. A normal user can only edit his/her information in the user's page. 9 A group manager's homepage contains user's group name, account ID, a personal information edit button, and a list of group member's account ID with an information edit button. Users can edit those users that belong to the group in the group manager page. 10 In admin page, users (admin) can remove, edit, or add a group. 11 An admin can remove a user's account from the system, can also remove and add users from a group, and can also edit a user’s information. 12 Users can log out from their homepage. 119 13 New users need to register their information. It must contain their account ID, password, email, name, and address. After saving it, the system should show a successful page. The customer need‘s descriptions are extracted from the original VAST project data, and those are transformed into the design guideline for the case study design. The details of the design task and guideline are shown in Appendix Table 6.5 Customer Need ’s Description of Product System Customer's Needs of Product System: 1 The system only has three kinds of levels (layers)- "all", "product", and "category". 2 One browsing ID can only associate with one level. 3 The browsing ID is a unique key to identify layers. For example, one product only has one product key (PK or product ID), and one category only has one category key (CK or category ID). 4 One product ID can only join one category. 5 The category name can be changed while admin clicks “edit” when the user is in the "all" page. 6 A category only has one name. The category name is assigned while the category is being created. 7 The system should respond key-in errors to users when they input the wrong browser ID. But, once key-in is successful, the system should forward the user to the corresponding page according to the appropriate level. 8 A normal product homepage contains a product name, browse ID (product ID), photo, manufacturer, material, and a product information edit button. A normal user can only edit product information in the product's page. 9 A category homepage contains a category name, browse ID (category ID), a category information edit button, and a list of product IDs with an information edit button. Users can edit those products that belong to the category in the category page. 10 In browse all pages, users can remove, edit, or add a category. 11 In browse all pages, users can remove products from the system, can also remove and add a product from a category, and can also edit a product's information 120 12 Users can leave from the homepage. 13 New products need to key-in their information. They must contain their level ID (product ID), name, photo URL, manufacturer, and material. After saving it, the system should show a successful page. 6.3.1.4 Results and Analysis There were seventeen participants in the first stage; however, only seven people continued to the second stage. Table 6.6 shows the statistics of the background survey of the all the participants. From the statistics, we can understand that all participants‘ programming experiences are more than two years. According to average system design and architecture years, we understand that most of the participants are not particularly experienced in doing the system design. Table 6.6 Case Study Participant’s Background Statistics Maximum Minimum Average Standard Deviation Years of Programming Experience 8 2 4.08 1.30 Years of System Analysis and Design 5 0 1.85 1.6 Years of Architecture Experience 3 0 0.65 0.9 Years of Detailed Design Experience 4 0 1.44 1.38 Years of Learning UML 5 0 1.94 1.46 Years of OOAD Experience 5 0.5 2.38 1.35 We summarize the discussions and results of each stage in the following. First stage results and discussions: (1) The SSDF can help the designer to create less complex design because all the complexity reduction rates are positive. From the blind and non-blind groups 121 average complexity reduction percentage, both results are very close but the non- blind group is a little higher. We cannot exclude the information leaking or intention of reducing the complexity when the designer is re-doing the same design. (2) Time is not a good measurement for validating the framework‘s efficiency improvement. Even thought the SSDF was applied, the designer still needed to spend a certain amount of time to think and work on the design. The time reduction is not obvious. Figure 6.13 OOAD Experience VS Complexity Reduction Rate 0 20 40 60 80 100 0 2 4 6 8 Reduction (%) Years OOAD Linear (OOAD) 122 (3) Figure 6.13 implies that a more experienced designer can use OOAD to create a system that already reduces certain level of structural dependency. As a result, more experienced designers have less improvement on the system complexity. (4) Programming experience may not be highly influential in designing a system. From Figure 6.14, the complexity reduction does not reach higher when the designer has more years of programming experience. Figure 6.14 Programming Experience VS Complexity Reduction Rate (5) The linear regression line in Figure 6.15 indicates that the SSDF should be design representation neutral. System complexity improvement is not affected by the designer‘s UML experience. R² = 0.0353 0 20 40 60 80 100 0 2 4 6 8 Reduction (%) Years Prog Linear (Prog) 123 Figure 6.15 UML Experience VS Complexity Reduction Rate Second stage results and discussions: (1) According to the complexity of the system each iteration (see Figure 6.16), the complexity of the OO design in the original VAST project changes dramatically before the fifth iteration. The SSDF results that collected from the designers decrease smoothly at each iteration. This result implies that the SSDF should manage the system complexity better. (2) Table 6.7 shows the statistics of the comparison in terms of the number of iterations. From the statistics, all participants can reach the final state of the system design using fewer numbers of iteration. From this fact, we can conclude that the SSDF improves the efficiency. R² = 0.0025 0 20 40 60 80 100 0 1 2 3 4 5 6 Reduction (%) Years UML Linear (UML) 124 Figure 6.16 Average Complexity Each Iteration in Case Study Figure 6.17 Number of Iterations Used to Finish the Design in Case Study 125 Table 6.7 Number of Iterations Between Baseline and Participants VAST Project 10 Case Study Average Number of Iterations 6.28 Maximum Number of Iterations 9 Minimum Number of Iterations 5 Standard Deviation 1.63 (3) Ten participants did not continue to the second stage of the case study. From Table 6.8, the non-drop group is more experienced than drop group in software architecture and OOAD, but drop group is better in requirement (system analysis and design) and detailed design. This may imply three things: a designer who has fewer years of OO experiences or seldom does conceptual or abstract design considers that it is more difficult to apply the SSDF, non-drop group participants are just simply more motivate, or non-drop group is more willing to continue the case study due to receiving more training. However, from our observations, the non-drop group indeed had stronger motivation because they asked for more related materials. We cannot exclude if the SSDF requires more OOAD or abstract design experiences. Moreover, from Figure 6.18, we think that training efforts that the mentor puts should not be the reason of participant‘s discontinuing because the drop group receives more training time and discusses more via emails, or both groups get almost same amount of training and discussion (according to P > 0.05). 126 Hence, to conclude why subjects continue, the reason could be their better OO and abstract design experiences or their stronger motivation. Table 6.8 Background Statistics of Drop and Non-drop Group Maximum Minimum Average Standard Deviation Group ND D ND D ND D ND D Years of Programming Experience 5 8 3 2 4.07 4.1 0.6 1.66 Years of System Analysis and Design 3 5 0 0 1.5 2.5 1.19 1.77 Years of Architecture Experience 3 2 0 0 1.36 0.45 1.02 0.64 Years of Detailed Design Experience 4 4 0 0.5 1.78 1.8 1.52 1.29 Years of Learning UML 3 5 0 0.5 1.92 2.1 1.37 1.43 Years of OOAD Experience 4 5 1.5 0.5 2.79 2.45 1.15 1.34 Figure 6.18 Normalized Average Training and Performance between Non-Drop and Drop Groups 1 0.68 0.88 1 0.94 1 Non-Drop Drop Complexity Reduction Number of Iteractions Training Time P = 0.061 P = 0.56 P = 0.808 60.1 % 41 % 12.5 Times 14.2 Times 269.6 Mins 286.4 Mins 127 6.4 Experiment Design This section presents the experimental design for evaluating the framework with respect to the two hypotheses – system complexity reduction and design efficiency improvement. Similar to our case study, in the validation, three experiments are executed. The first one – experiment A evaluates the complexity reduction in a small scale design and the others – experiment B and C are to validate efficiency and complexity in the VAST project. The design tasks in experiment A, B and C have the requirements that we have specified and used in prior case study. Table 6.9 shows the details of the experiment A. In experiment A, the group A2 is the blind group. After the results of A1 and A2 are compared, if A1’s result is significantly better than A2, we suspect that there is any information leaking or subjects intend to perform better. Table 6.10 shows the control variables we employ in the both groups in the experiment B. Table 6.11 demonstrates the behaviors that are investigated. Experiment C is a special one in which no subjects use the SSDF method, thus, subjects are asked to do the design by using OO design method in VAST project. Group C’s result is for comparing with 128 group B2’s after experiment C. because both groups are under the same conditions. According to experiment B and C, we can understand if both complexity and efficiency are still improved by using SSDF in the VAST project. In addition, at the end of the experiment A and B, a questionnaire regarding the comments about two methods is given to subjects. Hence, we can understand each participant’s subjective opinions after they apply both methods. Table 6.9 Experiment A Setting Group A1 Group A2 (Blind) Requirements Fixed Requirements Fixed Requirements Method OOAD SSDF SSAD SSDF Design Task Login system Login system Login system Product system Validation Complexity Complexity Comparison Complexity Reduction Between A and B Table 6.10 Experiment B Setting Experiment Original VAST Project Group B1 (NCR-SF) 1 Group B2 (CR-SF) 2 Project Team Time Constraints Weekly releases To deliver design in a short time Weekly releases To deliver design in a short time Release weekly To deliver design in a short time Requirements Fixed requirements Requirements change weekly Requirements change based on customer’s feedbacks weekly 1 NCR stands for non changing requirements and SF means Synthesis-based Design Framework 2 CR stands for changing requirement 129 Table 6.11 Rationale of Experiment B Group B1 Group B2 VAST Project Team Group B1 If changing requirements affect the design framework performance If the design framework produces less complex result more efficiently Group B2 If the design framework produces less complex result more efficiently Table 6.12 Experiment C Setting Experiment Original VAST Project Group B2 (CR-SF) Group C (CR-OO) Project Team Time Constraints Weekly releases To deliver design in a short time Weekly releases To deliver design in a short time Release weekly To deliver design in a short time Requirements Requirements change weekly Requirements change weekly Requirements change based on customer’s feedbacks weekly Method SSDF OOAD OOAD 6.4.1 Group Division Method From the case study, we know that the SSDF can improve the efficiency and complexity but we cannot ensure whether the improvement is achieved by the framework or by the subject’s personal design capability. To eliminate the factor of personal design ability, we randomize subjects into two groups. Hence, two groups have almost the same level of design ability. How to organize groups in all the 130 experiments are explained as follows Experiment A Step 1: All subjects do the OO design. Like our case study, each subject does the login system design. After the design has been done, determine the design capability of each subject. Step 2: All subjects learn the SSDF. Exams and trainings are given to make sure each subject have good understanding of the framework. Then, half of subjects to re-do the login system design. But, the other half applies the SSDF doing the product system design, the same as the one in the case study. Two groups are divided by the way of mixing high and low design capability subjects in a group. The design capability is based on their OO design performance, exam, and background survey. Experiment B Step 1: Since the second design task in the experiment A is also treated as a test of SSDF design capability for each subject, we list subjects according to their SSDF design capability. 131 Step 2: We distribute subjects randomly in two groups, and ensure that skillful and less skillful ones are mixed in a group. By doing this, two groups should have equal SSDF design capability before they start to do the VAST project design. Experiment C This group only applies OO method to the VAST project. Therefore, it is not necessary to divide subjects into groups. All subjects learn the customer needs of the VAST project and they do not have to learn the SSDF. The design results of this group are compared with group B2’s results (i.e. experiment B’s SSDF design outcome under the changing requirement condition). 6.4.2 Threats to Validity There are some potential threats to the validity of the research and these threats can be classified as controllable and uncontrollable ones (Campbell and Stanley 1963). Through careful treatments, we can reduce those influential factors in the experiment. This section attempts to identify the most significant threats to the validity of our design framework and also outline the way to reduce them. (1) Controllable Threats Threat: Learning Curve. 132 There is a learning curve for candidates using the SSDF framework. Threat Control: Subjects were provided with enough reading materials and one hour training before they start to use the framework to do the design in the experiment. To ensure subject’s understanding of the framework, during the first step of the whole experiment, tests and questionnaires are applied. The final design task in the first step of the experiment is also the evaluation for determining if the subject can continue to the second step of the experiment. Threat: Personal Ability. The SSDF shows that the improvement is not due to the framework but is from the subject’s original design ability. Threat Control: Subject’s OO design ability as well as their SSDF design ability are also evaluated before subjects are divided into two groups. Skillful and less skillful subjects are intentionally mixed in one group, and two groups have almost equal design ability and years of developmental experiences. In this way, we can reduce the potential performance influences from subject’s original ability. Threat: Result Reporting. Fewer numbers of design result submissions may lead to inaccurate measurement of 133 efficiency. Threat Control: All subjects are asked to save all the possible versions. Even when interruptions occur and subject’s design is not completed, they still have to save the result into one version. Moreover, we determine the number of iterations based on all the versions that subjects commit. If a version is significantly different from its prior version, the version is the outcome after the subject finishes one iteration cycle. However, the whole version history may include some versions that are not significant enough to be thought as iteration. The graph-similarity method is applied for determining which version should be determined as iteration. Threat: Treatment Leakage. It is possible that subjects know that their results will be used for comparing to the original VAST OO design result or one subject learns from the other one’s design. Threat Control: During the experiment, all subjects perform the design individually so they do not know each other’s exercise. They are only told to learn the new design framework and to utilize it. None of them knows any existing VAST design result. 134 (2) Uncontrollable Threats Threat: Non-representativeness of Projects. There is a threat that the design project is not representative in the real world. Threat Reduction: The VAST design is the real world OO project development. Although it was developed by a six people team, the major design and the software architecture was done by only one developer. All the other team developers just followed the design and implemented the details. The VAST project also used rapid prototyping and was released weekly. This is the target data that we would like to use for validation. The only limitation is that just one VAST project is used. It is an Eclipsed plug-in. Hence, the validation does not cover system designs in various application domains, for example, embedded system. As a result, we also employ two small scale web-based systems, login and product systems, to mitigate this threat. Threat: Non-representativeness of Subjects. The experimental candidate pool consists of almost full-time master’s level students. There is a possibility that this pool does not represent the industry candidates. Threat Reduction: Many of the experimental subjects are full-time students who come to school with a certain amount of industry experiences. Subject’s work 135 experiences are collected by online survey form before the experimental validation. The experience information for the two groups has been reported alongside experiment results. 6.5 Experiment Result and Discussion In this section, we show the results and discussions of the research validations. 6.5.1 Experiment A This background data are collected from the online survey form (see Appendix G). All subjects filled out in the beginning of the experiment. There were 41 master students voluntarily joining the experiment. Most of them have several years of industrial experiences. Figure 6.19 show that subjects with more years of design experiences have better OO design performance. Table 6.13 indicates that three linear regressions are significant (Significance F<0.05). This supports our assumption of the grouping method in experiment A because the subject‘s design ability should reflect on their experience. 136 Figure 6.19 Design Experience vs Complexity Reduction of OO Design Result Table 6.13 Significance F Value of Linear Regression Significance F OOAD 0.000405 (< 0.05) SAD 0.024157 (< 0.05) DD 0.044663 (< 0.05) According to subject‘s design ability, subjects are divided into two groups. Group A1 has 21 subjects and group A2 has 20 subjects. The background statistics for both groups are shown in Table 6.14. Comparing both groups, average years that are measured in all design experiences are remarkably close. 137 Table 6.14 Background Statistics of A1 and A2 Groups Maximum Minimum Median Average Standard Deviation Group A1 A2 A1 A2 A1 A2 A1 A2 A1 A2 Years of Programming 5 8 1 1 4 4 3.93 3.78 1.10 1.66 Total SAD years 5 5 0 0 1.5 2 2.05 2.18 1.52 1.49 Total Arch Years 2 4 0 0 0.5 1 0.81 1.1 0.64 1.03 Total DD Years 4 3 0 0 1 1 1.48 1.33 1.4 1.03 Total UML Years 5 3 0 0 2 1.75 1.76 1.8 1.44 1.11 Table 6.15 shows the SSDF design performance between non-blind group (A1) and blind group (A2). In the Table 6.15, the measurement is the complexity reduction percentage calculated from the complexity difference between OO and SSDF results in each group. Non-blind group performs better, but the difference is not significant (P=0.461 > 0.05). This result means that, during the experiment, no design information is leaking, the improvement is not from the familiarity of the system due to re-designing, and subjects should not receive any hint about system complexity‘s improvement. Table 6.15 Performance Difference between Blind and Non-Blind Groups Group Maximum Minimum Median Average Standard Deviation P(T<=t) A1 82% 18.18% 44.28% 43.79% 14.95% 0.461 A2 (Blind) 78.57% 12.82% 55.61% 47.94% 20.25% 138 Table 6.16 Complexity Improvement Complexity Maximum Minimum Median Average Standard Deviation P(T<=t) OO 0.667 0.098 0.286 0.305 0.161 5.76E-09 SSDF 0.309 0.049 0.144 0.152 0.067 All subjects had improvements on complexity reduction while they were using the framework. In Table 6.16, the average complexity of OO is 0.305 that is more than SSDF‘s value 0.152 and this improvement is significant (P=5.76E-09 < 0.05). The experiment A concludes that the SSDF can help OO method to reduce the system complexity. From the very small P value, this could also indicate two things: (1) the improvement in small scale experiment is very obvious; (2) the measurement method is very sensitive. 6.5.2 Experiment B Total 41 subjects were divided into two groups, B1 and B2, based on their SSDF design ability. Group B1 had 21 subjects and group B2 had 20 subjects. Table 6.17 shows the background analysis from two groups. To be noted that, the experiences in almost all the categories are more than one year on average, both group‘s average time of OOAD experiences are even above two years, but average time of the architecture related experiences in both groups are less than one year. 139 Table 6.17 Background Statistics of B1 and B2 Groups Maximum Minimum Median Average Standard Deviation Group B1 B2 B1 B2 B1 B2 B1 B2 B1 B2 Years of Programming 5 8 1 1 4 4 3.88 3.83 1.12 1.65 Total SAD years 5 5 0 0 2 2 2.17 2.05 1.53 1.48 Total Arch Years 2 4 0 0 1 0.75 0.93 0.98 0.64 1.06 Total DD Years 4 4 0 0 1 1 1.38 1.43 1.36 1.09 Total UML Years 5 3 0 0 1.5 2 1.79 1.78 1.41 1.14 OOAD Years 6 6 0.5 0 2 2 2.57 2.28 1.56 1.49 From Table 6.18, both groups have fewer numbers of iterations than VAST project. This implies that the SSDF has better efficiency than OO design method. Table 6.18 Statistics of Number of Iterations in Group B1 and B2 Group Maximum Minimum Average Standard Deviation VAST B1 8 5 6.67 1.017 10 B2 9 5 6.7 1.342 Figure 6.20 demonstrates the frequency diagram in group B1 and B2. Comparing to group B2, group B1 uses less iteration to finish the design on average, but it is not significantly better (P=0.928 > 0.05). This explains the factor of requirement changing may not affect the efficiency; that is, SSDF can improve efficiency almost at the same level, whether customer needs change or not. The alternative explanation on this is that the type of customer change does not have any impact on the design. Therefore, the efficiency difference between two groups does not occur. 140 Figure 6.20 Number of Iterations Used to Finish the Design Figure 6.21 shows that the average complexity at earlier iteration is higher in B2 group. This means that the factor of customer need change influences the converging speed. Until sixth iteration, the average complexity is almost the same in both groups. In Figure 6.21, the requirement change area shows when requirement change is ended. After requirement change ends, the average complexity starts to converge. Figure 6.22 and 6.23 demonstrate that SSDF can manage complexity well since both groups‘ complexity value is lower than the VAST project every iteration cycle. Even most maximum complexity value in both groups is lower than VAST project. 141 Figure 6.21 Average Complexity at Each Iteration (B1 and B2) Figure 6.22 Average Complexity at Each Iteration (VAST and B1) Requirement Change End 142 Figure 6.23 Average Complexity at Each Iteration (VAST and B2) Figure 6.24 Maximum Complexity Variation at Each Iteration 143 To understand better on complexity management in design methods, we define complexity variation as the following equation. Complexity Variation = (Complexity at Iteration i) - (Complexity at Iteration i-1) Figure 6.24 shows the maximum complexity variation each iteration. The complexity change in group B1 and B2 is not larger than VAST project. In addition, even in the later iteration, VAST project still has significantly complexity change. Alternative Explanation The VAST team has not been told to minimize the system complexity as a goal during the design. As a result, the complexity change is relatively higher. However, during the experiment B, no subjects have been told to minimize the complexity as their design goal as well. Instead, the subject used the design matrix to manage the complexity in the entire design. Hence, it is very possible that the OO method is lacking the capability to handle the complexity during the design like SSDF. The other reason for VAST team‘s relative poor result is that team members may spend more time to figure out the changing customer needs and the problem they deal with. The subjects in the experiment may not need that. However, the group B2 is also under the changing requirement condition. Although the changing requirement 144 condition cannot perfectly reflect the real situation that VAST team faces, this validity risk has been possibly reduced. 6.5.3 Experiment C Group C‘s result is used to compare with group B2‘s. Table 6.19 shows the background statistics of experiment B‘s group B2 and experiment C‘s group C. The experiences between these two groups are also real close. This statistics shows that the result should be avoided from the influences of personal original design ability. Table 6.19 Background Statistics of Group B2 and C Maximum Minimum Median Average Standard Deviation Group B2 C B2 C B2 C B2 C B2 C Years of Programming 8 7 1 1 4 4 3.83 3.7 1.65 1.76 Total SAD years 5 5 0 0.5 2 2 2.05 1.95 1.48 1.4 Total Arch Years 4 4 0 0 0.75 1 0.98 1.1 1.06 1.17 Total DD Years 4 2 0 0 1 1 1.43 0.95 1.09 0.8 Total UML Years 3 3 0 0 2 2.25 1.78 1.85 1.14 1.22 OOAD Years 6 4.5 0 0 2 1.5 2.28 2.05 1.49 1.55 To validate the efficiency, we also compare the number of iterations between group B2 and C. Table 6.20 shows that B2 uses less iteration to finish the system design, and the result is significant because P-value is less than 0.05 (alpha value). This result compliments using only original VAST data in the validation process. 145 Table 6.20 Statistics of Number of Iterations in Group B2 and C Group Maximum Minimum Average Standard Deviation P Value VAST B2 9 5 6.7 1.342 0.00371 (<0.05) 10 C 12 5 9.9 1.524 In Figure 6.25, comparing between group B2 and C, the average complexity in group C is always higher than group B2‘s. Besides, the average complexity in group C before iteration four is up-and-down. This behavior is particular obvious during the changing requirement period. This implies that the SSDF handles the complexity better than OO method in the design, especially in the earlier stage. Figure 6.25 Average Complexity at Each Iteration (B2 and C) Requirement Change End 146 6.5.4 Survey and Framework Feedbacks Our survey collected feedbacks from subjects. Table 6.21 shows the survey results. In subject’s personal opinions, SSDF helps the design in many ways, such as shortening the design time, manage the complexity, layout the constraints and etc; overall, subjects think SSDF is more helpful than OO method. However, subjects also feel that it is a little bit difficult to use SSDF. Table 6.21 Statistics of Subject’s Survey OO SSDF Create objects and their classes. (0-5) 3.241 4.17 Layout the relations between objects (classes). (0-5) 3.94 4.375 Decompose the system. 2.69 4.77 Shorten your designing time. (0-5) 3.23 4.4 Specify the functional requirements for your design. (0-5) 3.31 4.33 Specify the constraints for your design. (0-5) 3.94 4.48 Manage the complexity. (0-5) 3.08 3.48 Difficulty to apply the method. (0-5) 2.54 3.33 Overall, help on designing your software system. (0-5) 3.10 4.64 One question was designed to obtain overall comments about the design framework from experiment subjects. The following highlights some feedbacks provided by subjects. “I think the SSDF framework for laying out the constraints, elements…etc. is helpful. In my old experiences, I did not know any ways to understand relations between constraints or elements. I also realize the dependency between objects. This helps a 147 lot.” “Sometimes, it is hard to decompose the system. With SSDF, there is a systematic way to do that. I think using SSDF also helps to know the relations between what and how during the design. I have never learned this before even I have used OO design for several years” “The design matrix helps me to trace many relations. It also helps to trace the requirements. I can know which component fulfills which requirement.” 148 Chapter 7: Contributions and Future Direction Concluding this dissertation, we provide a detailed list of this work‘s contributions and future research directions. 7.1 Contributions The contributions of this research are summarized as follows: (1) According to our validation results, the synthesis-based design framework can provide a better way to manage the complexity of the software system during the design. This conclusion may not be extended to the fact of designing all kinds of systems; however, the primary concept of the framework, design synthesis reasoning method, should still benefit the traditional OO design method regarding handling the complexity during the design process and reducing the complexity of the software design at the end. (2) Our validation shows that the design framework is more efficient than the traditional OO design method. The designer needs fewer numbers of iteration to finish the software system design by using the framework. This conclusion may be limited to designing certain software systems. The OO design method can still be inspired by the design framework. 149 (3) We clarify function and behavior in designs. That is where most current design approaches do not pay attention to and really misuse. The design framework combines the benefits of these two concepts to assist the software designers to specify the functional requirements during the conceptual stage. (4) We suggest a graph-based complexity measurement. This method can allow users to calculate the complexity by investigating the UML class diagram. (5) Past research only discussed the way to investigate performance among various groups within one design approach, and there is no way to compare multiple design approaches. We have developed a method that can objectively compare different design approaches. 7.2 Fundamental Weakness and Limitation 7.2.1 Fundamental Weakness Ideally, during the case study and experiments, the completion of the subject’s design assignments should be assessed independently. The one who holds the experiments and works with subjects should not be the one who determines if the subject’s design results satisfy completion criteria. More importantly, the assessment should be performed by a group of evaluators so that the decision can be made objectively. 150 However, in real situation, the assessment is not done independently during the case study and experiments. The execution of the experiment and the assessments are done by the same person. This leads a weakness – subjectivity of the study. Our validation could be influenced by this subjective assessment. 7.2.2 Framework Limitation The follows are some limitations of this framework: (1) The framework is limited in the software systems that are developed by OO programming language. The usage has not been extended to those function- based programming language. (2) The framework is validated under web-based system designs and the desktop tool built on top of JAVA virtual machine. Both of them are high level and non-distributed applications. It is possible the framework might not work well in operation system level programs, embedded systems, or some distributed software systems. These require further validation to make sure that the framework can improve performance across various application domains. 151 (3) The framework has not been used by multiple users. We are not sure that the framework can still be performed well under team design collaboration. This also requires further evaluations. (4) An assumption, designing a new system or re-doing a design, is in the framework. It is possible that the framework cannot be helpful while the developer is doing system maintenance or modifications. 7.2.3 Validation Limitation The empirical validation of this framework was primarily limited by the available data and experimental subjects. Three experiments (A, B and C) were run mostly by student volunteers. Only one real world project is employed. Given these constraints, the validation could have some drawbacks: (1) The validation of this research has primarily involved web technology and software development tool‘s plug-in projects. This framework still needs assessments in other project types. (2) There is one designer who applies SSDF. In the industry, software systems are usually designed or developed through team work. The validation does not consider this situation. 152 7.3 Future Work The future extensions of this research are as follows: (1) We can research various quality attributes from the design result of the framework. Hence, we can know if the framework can also improve different quality attributes. Besides, we can investigate how the system complexity affects other quality attributes. By studying these, our design framework will be further improved. (2) The SSDF still needs to be further investigated under the collaboration among multiple designers. (3) The SSDF can be further developed to satisfy the system that is designed by non-OO method, such as structural method. (4) Graph-based complexity metrics can be studied in various aspects to calculate the complexity of the software system. a. The complexity measurement can consider the differences of various associations. We can add different weight for different relation and the value of the weight should be further researched. 153 b. In addition to considering static behaviors in the UML diagrams, dynamic behaviors, such as the sequence diagram, should also be considered in graph-based complexity measurement. Lastly, a tool can be implemented for handling the whole SSDF design process. In the research, we use multiple tools to do the design. Having such a tool, SSDF could be more applicable to the industry. 154 Bibliography 1. A von Knethen (2001), ―Change-Oriented Requirements Traceability: Support for Evolution of Embedded Systems.‖, 18th IEEE International Conference on Software Maintenance, pp.0482. 2. Akram I. Salah (2002), "Engineering an Academic Program in Software Engineering." 35th Annual Midwest Instruction and Computing Symposium. 3. Aldrich J., Garlan D., Schmerl B., Shaw M., Wing J. (2006). ―Software Engineering Research in the Computer Science Department of at Carnegie Mellon.‖ A web tutorial, http://www.csd.cs.cmu.edu/research/areas/softeng/. 4. Alexander, C. (1979), The Timeless Way of Building, Oxford University Press. 5. Alexander, E.R. (1979), "The Design of Alternatives in Organizational Contexts: A Pilot Study." Admin. Sci. Quart., 24, p.382-404. 6. America, P., Obbink, H., Muller, J. and van Ommering R.(2000), "COPA: A Component-Oriented Platform Architecting Method for Families of Software Intensive Electronic Products." The First Conference on Software Product Line Engineering. 7. Andrade Jr., J.S., Wainer, I. and Moreira, J.E. (1995), "Self-organized criticality in the El Nino southern oscillation." Physica, A 215(3), pp.331-338. 8. Atkinson, C. et al. (2002), Component-based product line engineering with UML. Addison-Wesley, London, New York. 9. Atkinson, C., Bayer, J. and Muthig, D. (2000), "Component-Based Product Line Development. The KobrA Approach." Proceeding of First Software Product Lines Conference (SPLC1). P.hoe, Ed. Kluwer Academic Publishers, Boston, pp.289-309. 10. Atkinson, C., and Muthig, D. (2002), "Component-based productline engineering with the UML (tutorial)." Proc. of the 7th International Conference on Software Reuse, Berlin. 11. B. Paech, A. Dutoit, D. Kerkow, and A. von Knethen (2002), ―Functional requirements, non-functional requirements, and architecture should not be separated.‖ REFSQ Essen. 12. Bak, P., Chen, K. (1999), "Self-organized criticality." Sci. Am., 46-53. 155 13. Barnes, T., Duncan, J. (1991), Writing Worlds: Discourse, Text, and Metaphor. Routledge, London, UK. 14. Barry W. Boehm (1975), ―Software Design and Structuring. Practical Strategies for Developing Large Software Systems‖. Addison-Wesley. 15. Basili, V. R. (1980), "Qualitative software complexity models: a summary in tutorial on models und methods for software management and engineering." IEEE Computer Society Press, Los Alamitos, CA. 16. Bernhard Turban, Markus Kucera, Athanassios Tsakpinis, and Christian Wolff (2009), ―Bridging the requirements to design traceability gap.‖ Intelligent Technical System, Chapter 20, pp. 275-288, Springer. 17. Biddle, R. and Tempero, E. (1996), "Explaining inheritance: a code reusability perspective." Proceedings of the twenty-seventh SIGCSE technical symposium on Computer science education. 18. Brown A.W. and Short K. (1997). ―On components and objects: the foundations of component-based development.‖ In Proceedings of 5th International Symposium on Assessment of Software Tools and Technologies, Pittsburgh, PA, pp.112-121, California. 19. Brown A.W. and Wallnau K.C. (1998). ―The current state of CBSE. ― IEEE Software, 15(5), pp.37-46. 20. Brown, C. (1995), Chaos and Catastrophe Theories. Sage, Thousand Oaks, CA. 21. Byrne, D. (1997), "Chaotic places or complex places: Cities in a post-industrial era." In: Westwood, S., Williams, J. (Eds.), Imagining Cities, Routledge, London, pp.50-72. 22. Campbell, D. and Stanley, J. (1963). ―Experimental and Quasi-experimental Designs for Research‖. Chicago, IL: Rand-McNally. 23. Carmichael, A. R. (1992), ―Defining Software Architectures Using the Hierarchical Object-oriented Design Method (HOOD).‖ Annual International Conference on Ada. pp. 211-219. 24. Chaitin, G. J. (1990), "Information, Randomness, and Incompleteness." World Scientific, Singapore. 25. Chakrabarti, A. and Bligh T. P. (2001), "A scheme for functional reasoning in conceptual design." Design Studies. 26. Cherniavsky, J. C. and Smith, C. H. (1991), "On Weyuker‘s axioms for software complexity measures." IEEE Trans. Soft. Eng., Vol.17, No.6, pp. 636-638. 156 27. Christof Ebert (2005), Systematisches Requirements Management. 28. Chu, D., Strand, R. and Fjelland, R. (2003), ―Theories of complexity: common denominators of complex systems.‖ Complexity, Vol.8, No.3. 29. Cook, C. and Churcher, N. (2005). ―Modelling and measuring Collaborative Software Engineering‖, In Proceedings of the Twenty-Eighth Australasian Conference on Computer Science, Vol. 38 (Newcastle, Australia). V. Estivill- Castro, Ed. ACM International Conference Proceeding Series, vol. 102. Australian Computer Society, Darlinghurst, Australia, 267-276. 30. Coulter, N. S. (1983), ―Software science and cognitive psychology.‖ IEEE Transaction Software Engineering, Vol. 9, No.2, pp 166-171. 31. Correig, A.M., Urquizu, M. and Vila, J. (1997), "Aftershock series of event February 18, 1996: an interpretation in terms of self-organized criticality." J. Geophys. Res., B 102 (12), pp.27407-27420. 32. Curtis B., ―By the way, did anyone study any real programmers?‖ Workshop on empirical studies of programmers on empirical studies of programmers, p.256- 262, June 1986, Washington, D.C., United States 33. Curtis B., Krasner H., Iscoe N. (1988), ―A field study of the software design process for large systems.‖ Communications of the ACM, v.31 n.11, p.1268- 1287. 34. Daniel E. Whitney (1996), ―Why mechanical design cannot be like VLSI design.‖ Research in Engineering Design. September 1996. 35. David L. Parnas (1972),‖ On the Criteria to Be Used in Decomposing Systems into Modules.‖ Communications of the ACM. 1972. 36. David L. Parnas, Paul C. Clements and David M. Weiss (1985), ―The Modular Structure of Complex Software Systems.‖ IEEE Transactions On Software Engineering. Vol. SE-11. no. 3. 37. David Budgen (2003), Software Design. 2nd ed.. Addison Wesley. 38. David G. Ullman (2003), The Mechanical Design Process. 3rd ed. McGraw-Hill. 39. Davis, J. S. (1984), "Chunks: a basis for complexity measurement." Process and Management, Vol.20, No.1, pp. 119- 127. 40. DeMarco, T. (1978), Structured Analysis and Systems Specififcation. Prentice Hall. 157 41. Edward Colbert (1994), Requirements Analysis Using the Object–Oriented Software Development method. Object Development Methods, Andy Carmichael ed.. SIGS Books. 42. E .Yourdon (1989), Modern Structured Analysis. Yourdon Press. 43. Evangelist, M. (1984), ―An analysis of control flow complexity.‖ COMPSAC, pp388-396. 44. Felix Bachman, Len Bass, Gary Chastek, Patrick Donohoe and Fabio Peruzzi (2000), ―The Architecture Based Design Method.‖ Technical Report CMU/SEI- 2000-TR-001. 45. Frederick Brooks (1995), The Mythical Man-Month: Essays on Software Engineering, 20th Anniversary Edition. Addison-Wesley. 46. Finger, S. and Dixon, J. R. (1989), ―A Review of Research in Mechanical Engineering Design. Part I: Descriptive, Prescriptive, and Computer-Based Models 47. Garlan D,, Allen R. and Ockerbloom J. (1995), ―Architectural mismatch: why reuse is so hard.‖ IEEE Software, 12(6), 17-26. 48. Gero, J. S. and Kannengiesser, U. (2002), ―The Situated Function-Behaviour- Structure Framework.‖ Artificial Intelligence in Design, Kluwer, Dordrecht, pp. 89-104. 49. G. Pahl, W. Beitz, J. Feldhusen and K.H. Grote (2007), Engineering Design: a systematic approach. 3rd ed. Springer. 50. Grady Booch, Robert A. Maksimchuk, Michael W. Engel, Bobbi J. Young, Jim Conallen, Kelli A. Houston (2007), Object-Oriented Analysis and Design with Applications. 3rd ed., Addison-Wesley. 51. Gaudi System Architecting homepage, http://www.gaudisite.nl/. 52. Hainaut, J-L., Hick, M., Englebert, V., Henrard, J. and Roland, D. (1996), "Understanding the implementation of IS-A relations." 15th International Conference on Conceptual Modeling Proceedings, Lecture Notes in Computer Science 1157, Springer-Verlag, Berlin, Germany, pp. 42–57. 53. Hansen, W. J. (1978), "Measurement of program complexity by the pair (cyclomatic number, operator count)." ACM SIGPLAN Notices, pp. 29-33. 54. Heineman G.T. and Council W.T. (2001), Component-based Software Engineering: Putting the Pieces Together. Addison-Wesley. 158 55. Henderson-Sellers, B., Pant, Y. R. and Verner, J. M. (1993), "Cyclomatic complexity: theme and variations." Australian J. of Information Systems, Vol.1, No.1, pp. 24-37. 56. Hayles, N.K. (1991), "Complex dynamics in literature and science." In: Hayles, N.K. (Ed.), Chaos and Order: Complex Dynamics in Literature and Science. University of Chicago Press, Chicago, IL, pp. 1-33. 57. Henry, S. and Kafura, D. (1981), "Software structure metrics based on information flow." IEEE Transaction Software Engineering, Vol. 7 No. 5 pp.510-518. 58. Henry, S. and Salig, C.(1990), "Predicting source-code complexity at the design stage." IEEE Software, Vol.7, No.2, pp.36-44. 59. Holland, J. (1995), Hidden Order, Addison-Wesley & Son, New York. 60. Holland, J. H. (1992), Complex adaptive systems. Daedalus 121 (1), pp.17-30. 61. Holling, C. S. (1978), Adaptive Environmental Assessment and Learning. Wiley, London. 62. Holling, C. S. (1995), Sustainability: The cross-scale dimension. In:Murasinghe, M., Shearer, W. (Eds.), Defining and Measuring Sustainability. The World Bank, Washington, DC, pp.65-75. 63. Hubka, V., & Eder, W.E. (1996), Design Science: Introduction to the Needs, Scope and Organization of Engineering Design Knowledge, London: Springer- Verlag. 64. H. Gomaa (1993), Software Design Methods for Concurrent and Real-Time Systems. Addison Wesley. 65. Halstead, M. H. (1987), Elements of software science, Elsevier. 66. I. Jacobson, M. Christerson, P. Jonsson, and G. Overgaard (1992), Object- Oriented Software Engineering: A Use Case Driven Approach. Addison Wesley. 67. Jackson, D. (2004), "Module Dependences in Software Design Export." In Radical Innovations of Software and Systems Engineering in the Future, pp. 198-203. 68. Jiang, H. and Yen, C.-C. (2009), "Protocol analysis in design research: a review." International Association of Societies of Design Research. 69. Jeanne, O. (1997), "Are currency crises self-fulRlling? A test." J. Int. Econ. 43(3/4), 263-286. 159 70. Johnson, R. E. and Foote, B. (1988), "Designing reusable classes." Journal of object-oriented programming. 71. Katz, R. H. (1985), Information Management for Engineering Design Applications, New York: Springer-Verlag. 72. Kazman, R. (2005), ―The essential components of software architecture design and analysis.‖ 12th Asia-Pacific Software Engineering Conference,15-17. 73. Kazman, R., Klein, M., Barbacci, M., Longstaff, T., Lipson, H., Carriere, J. (1998), ―The architecture tradeoff analysis method‖, Engineering of Complex Computer Systems.‖ ICECCS '98. Proceeding, pp. 68–78. 74. Kearney, J. K., Sedlmeyer, R. L., Thompson, W. B., Gray, M. A. and Adler, M. A. (1986), "Software complexity measurement." Communication ACM, Vol.29, No.11, pp.1044-1050. 75. Keen, S. (1997), "From stochastics to complexity in models of economic instability." Nonlinear Dynamic Psychology and Life Science, 1(2), pp.151-171. 76. Sullivan, K. J., Griswold, W. G., Cai and Y. Hallen, B. (2001), ―The structure and value of modularity in software design.‖ Proceedings of the 8th European software engineering. 77. Kikuchi, M. and Nagasaka, I., "On the Three Axioms of General Design Theory." Proceedings of IWES. 78. Kim, J. and Lerch F. J. (1992), "Towards a model of cognitive process in logical design: comparing object-oriented and traditional functional decomposition software methodologies." Proceedings of the SIGCHI conference on Human factors in computing systems, Conference on Human Factors in Computing Systems. 79. Khoshgoftaar, T. M., Munson, J. C., Bhattacharya, B. B. and Richardson, G. D.(1992), "Predictive modeling techniques of software quality from software measures." IEEE Trans. Soft. Eng., Vol.18, No.11, pp. 979-987. 80. Krider, R. E. and Weinberg, C. B. (1997), "Spatial competition and bounded rationality: retailing on the edge of chaos." Geogr. Anal. 29 (1), pp.17-34. 81. Kruchten P. (2004), ―An Ontology of Architectural Design Decisions in Software-Intensive Systems.‖ 2nd Groningen Workshop on Software Variability. 82. Lakshmanan, K. B., Jayaprakash, S. and Sinha, P. K. (1991), "Properties of control flow complexity measures." IEEE Trans. Soft. Eng., Vol.17, No. 12, pp. 1289-1295. 160 83. Lansing, J. S. and Kremer, J. N. (1993), "Emergent properties of Balinese water temple networks: coadaptation on a rugged Rtness landscape." Am. Anthropol., 95 (1), pp.97-114. 84. Le, S. Q., Ho, T. B. and Phan, T. H. (2004), "A novel graph-based similarity measure for 2D chemical structures." Genome Informatics, 15 (2): pp.82-91. 85. Lee Y., Choi H.-J. (2005) ‗Experience of Combing Qualitative and Quantitative Analysis Methods for Evaluating Software Architecture.‘ Proceedings of the Fourth Annual ACIS International Conference on Computer and Information Science (ICIS 2005), 152-157. 86. Lowe, J. (1985), The Dynamics of Collapse: A Systems Simulation of the Classic Maya Collapse. University of New Mexico Press, Albuquerque, NM. 87. Lu, S. C-Y., Suh, N.P. (2009), ―Complexity in Design of Technical Systems.‖ Annals of the CIRP, Vol. 5, 8/1/2009, pp. 241-245. 88. M. Lindvall. (1994) ―A study of traceability in object-oriented systems development.‖ Licenciate thesis, Linkoping University, Institute of Technology, Sweden. 89. Mata-Toledo, R. A. and Gustafson, D. A. (1992), "A factor analysis of software complexity measures." J. Systems and Software, Vol.17, pp. 267-273 90. McCabe, T. J. (1976) ―A complexity measure‖ IEEE Transaction Software Engineering, Vol. 2, No.4, pp 303-320. 91. Mezentsev, A. A. (1996), ―A methodology of complicated mechanical systems virtual prototyping.‖ Intelligence and Systems, IEEE International Joint Symposia, pp.208 – 214 92. Michael Jackson (1994), ―Jackson Development Methods: JSP and JSD.‖ Encyclopaedia of Software Engineering, John J Marciniak ed, Vol. I pp 585-593. John Wiley & Sons. 93. Michael Jackson (2000), ―The Origins of JSP and JSD: a Personal Recollection.‖ IEEE Annals of Software Engineering, Vol. 22 no. 2. 94. MacDonnell, S. G. (1991), "Rigor in software complexity measurement experimentation." J. Systems and Software, Vol.16, pp. 141-149. 95. May, R. (1976), "Simple mathematical models with very complicated dynamics." Nature 261, 459-467. 96. Mainzer, K. (1996), Thinking in Complexity: The Complex Dynamics of Matter, Mind, and Mankind. Springer, New York. 161 97. Matinlassi, M. (2004), ―Comparison of Software Product Line Architecture Design Methods: COPA, FAST, FORM, KobrA and QADA.‖ Proceeding of the 26th International Conference on Software Engineering. 98. Meyer, B. (1987), "Reusability: The case for object-oriented design." IEEE Software. 99. Motoshi Saeki (1994), ―Software specification & design methods and method engineering.‖ International Journal of Software Engineering and Knowledge Engineering. 100. Motschnig-Pitrik, R. (1993), ―The Semantics of Parts Versus Aggregates in Data/Knowledge Modelling.‖ Proceedings of Advanced Information Systems Engineering. 101. Myers, G. J. (1977), "An extension to the cyclomatic measure of program complexity." SIGPLAN Notices, pp.61-64. 102. Neill, T. M., Gero, J. S., and Warren, J. (1998), "Understanding conceptual electronic design using protocol analysis." Research in Engineering Design. 103. Nijkamp, P., Reggiani, A. (1990), "Logit models and chaotic behaviour: a new perspective." Environ. Planning, A2(11), 1455-1467. 104. Niklaus Wirth (1971), ―Program Development by Stepwise Refinement.‖ ACM Communications, vol. 14, no. 4. 105. Norton, B.G., Ulanowicz, R.E. (1992), "Scale and biodiversity policy: a hierarchical approach." Ambio Vol.21, pp.244-249. 106. O. Gotel, and A. Finkelstein (1994), ―An analysis of the requirements traceability problem.‖ Proceeding of First International Conference on Requirement Engineering, pp.94-101. 107. Offutt, J., Harrold, M. J. and Kolte, P. (1993), "A Software Metric System for Module Coupling." The Journal of Systems and Software, Vol. 20, No 3, pp. 295—308. 108. Oviedo, E. I. (1980), "Control flow, data flow and program complexity." Proc. COMPSAC, pp. 146-152. 109. Peirce, C. S. (1960), Collected papers of Charles Sanders Peirce. Cambridge: Harvard University Press. 110. Pecknold, S., Lovejoy, S., Schertzer, D., et al. (1997), "Multifractals and resolution dependence of remotely sensed data: GSI to GIS." In: Quattrochi, 162 D.A., Goodchild, M.F. (Eds.), Scale in Remote Sensing and GIS. Lewis, New York, pp.361-394. 111. Purhonen, A., Niemela, E., and Matinlassi M. (2004), "Viewpoints of DSP Software and Service Architectures." Journal of Systems and Software, Vol.69, pp. 57-73. 112. Poshyvanyk, D. and Marcus, A. (2006), "The Conceptual Coupling Metrics for Object-Oriented Systems." The 22nd IEEE International Conference on Software Maintenance (ICSM'06). 113. Pool, D. (1994), "Representing Diagnosis Knowledge." Annals of Mathematics and Artificial Intelligence, Vol. 11, pp.33-50. 114. Renfrew, C., Cooke, K. L. (1979), Transformations: Mathematical Approaches to Cultural Change. 115. Richard N. Taylor, Andre van der Hoek (2007), ―Software Design and Architecture Once and Future Focus of Software Engineering.‖ FOSE. 116. Saaty T. L. (1980), The Analytic Hierarchy Process. New York: McGraw-Hill. 117. Savit, R. (1992), Chaos on the trading floor, in Hall, N. (ed.), The New Scientist Guide to Chaos. London: Penguin Books, 174-183. 118. Sage, A. P. (1992), Systems Engineering, John Wiley and Sons, New York. 119. Scheinkman, J. A. and Woodford, M. (1994), "Self-organized criticality and economic fluctuations." Am. Econ. Rev., 84(2), pp.417-421. 120. Schieve, W.C. and Allen, P.M. (1982), Self-Organization and Dissipative Structures: Applications in the Physical and Social Sciences. University of Texas Press, Austin, TX. 121. Simon, H.A. (1981), The Sciences of the Artificial, 2 ed., Cambridge, MA: MIT Press. 122. Siricharoen, W. V. (2007), "Ontologies and Object models in Object Oriented Software Engineering." IAENG International Journal of Computer Science. 123. Shen, V. Y., Conte, S. D. and Dunsmore, H. E. (1983), ―Software science revised: a critical analysis of the theory and its empirical support.‖ IEEE Transaction Software Engineering, Vol. 9, No. 2, pp. 155-165. 124. Shepperd, M. (1990), "Early life-cycle metrics and software quality models" Information and Software Technology, Vol.32, No.4, pp. 31l-316. 125. Sarkar, S., Dong, A. and Gero, J. S. (2009), ―A problem decomposition method for conceptual design.‖ ICoRD-09. 163 126. Suh, N.P., (1990). The Principle of Design. Oxford University Press, Oxford. 127. Suh, N.P.(2001), Axiomatic Design: Advances and Applications. Oxford University Press. 128. Szyperski, C. (1998), Component Software: Beyond Object Oriented Programming, Addison-Wesley. 129. Taylor, E.S. (1959), M.I.T. Report, Cambridge, MA: MIT Press. 130. Takeda, H., Veerkamp, P., Tomiyama, T. and Yoshikawa, H. (1990), ―Modeling Design Process.‖ AI Magazine 131. Tian, J. and Zelkowitz, M. V. (1992), "A formal program complexity model and its application." J. Systems and Soffware, Vol.17, pp. 253-266. 132. Tomiyama, T., Kiriyama, T., Takeda, H. and Xue D. (1989), ―Metamodel: A Key to Intelligent CAD Systems.‖ Research in Engineering Design, Springer- Verlag. 133. Tsiolakis, A. (2000), "Consistency Analysis of UML Class and Sequence Diagrams based on Attributed Typed Graphs and their Transformation." ETAPS Workshop On Graph Transformation Systems. 134. Treude, C., Berlik, S., Wenzel, S., and Kelter U. (2007), "Difference computation of large models." Proceedings of the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering. 135. Reich, Y. (1995), ―A Critical Review of General Design Theory.‖ Research in Engineering Design, Vol.7, 1-18. 136. Remco C. de Boer and Hans van Vliet (2009), "On the similarity between requirements and architecture." Journal of Systems and Software, pp.544-550. 137. Roger S. Pressman (2004), Software Engineering – A Practioner‘s Approach. McGraw-Hill. 138. Rufai, R. A. (2003), "New structural similarity metrics for UML models." Master Thesis. King Fahd University of Petroleum and Minerals. 139. Visser, W. (2009), "Design: one, but in different forms." Design Studies. 140. Warf, B. (1993), "Postmodernism and the localities debate: ontological questions and epistemological implications." Tijdschrift voor Economische en Sociale Geografie, 84(3), 162-168. 164 141. Wieringa, R. J. (1998), "Traceability and Modularity in Software Design." Proceedings of the 9th international workshop on Software specification and design. 142. Weaver W. (1948), "Science and Complexity", American Scientist, 36: 536. 143. Weiss, D. and Lai, R. C.-T. (1999), Software product-line engineering: a family- based software development process. Addison-Wesley. 144. Weyuker, E. J. (1988), "Evaluating software complexity measures." IEEE Trans. Soft. Eng., Vol.14, No.9, pp. 1357-1365. 145. White, R., Engelen, G. (1993), "Cellular automata and fractal urban form: a cellular modelling approach to the evolution of urban land-use patterns." Environ, Planning D25, 1175-1199. 146. Wong, D. W. S. and Fotheringham, A. S. (1990), "Urban systems as examples of bounded chaos: exploring the relationship between fractal dimension, rank-size, and rural-to-urban migration." Geogr. Anal., B 72(3), pp.89-99. 147. Woodfield, S. N., Shen, V. Y. and Dunsmore, H. E. (1981), "A study of several metrics for programming effort." J. Systems and Softare, Vol. 2, pp. 97-103. 148. Yuming, Z. and Baowen, X. (2003), "Measuring structure complexity of UML class diagrams." Journal of Electronics, Vol. 20 No.3. 149. Zuse, H. and Bollman, P. (1989), "Software metrics: using measurement theory to describe the properties and scales of static-software complexity metrics." Sigplan Notices, Vol.24, No.8, pp.23-33. 165 Appendix A. Software Design – Login System (OO) Objective: To understand the participant's capability of UML and software system design. Tool: ArgoUML is suggested but Rational Rose is also acceptable. (ArgoUML download link - http://argouml-downloads.tigris.org/) Customer's Needs: You are designing the backend of a JAVA-based website login system. Your customer needs the following functionalities. You can check the prototype website figures below. 1) The system only have three kinds of users (roles)- "admin", "users", "group managers". 2) One user ID can only associates with one role. 3) The user ID is unique as a key to identify users. 4) One user ID can only join one group. 5) The group name can be changed while admin click ―edit‖. 6) A group only has one group manager. The group manger is assigned by admin while the group is being created. 7) The system should response login error to users. But, once logging in is successful, the system should forward the user to his/her individual homepage according to his/her role (see Fig. 1, 2). 8) A normal user's homepage contains user's group name, account ID, a personal information edit button. A normal user can only edit his/her information (see Fig 5). 9) A group manager's homepage contains user's group name, account ID, a personal information edit button, and a list of group member's account ID with an information edit button. A group manager can edit those users that belong to his/her group (see Fig.6). 10) An admin can remove, edit, or add a group (see Fig. 7). 11) An admin can remove user's account from the system, can also remove and add user from a group, can also edit user‘s personal information (see Fig. 7). 12) All users can log out from their homepage (see Fig. 5, 6, 7). 166 13) New user need to register his/her information. It must contain his/her account ID, password, email, name, address. After saving it, the system should show successful page (see Fig. 3, 4). The follows are assumptions or hints for this system design. 1) All users‘ data can be stored in any kind of storage system, for example, database, text file, list,…etc. It is up to your design. 2) Assume ―admin‖ is fixed and saved in the system. You DO NOT create a user to be admin. 3) You do not have to concern too much about the web page manipulations. You are only designing the backend. That is used to support the web UI‘s manipulations. Again, do not think something like pop-up windows for confirmation…etc. That belongs to front end (web UI). 4) Something like successful or error page, you can assume return ―True/False‖ after the method calls in your backend design. It is not necessary to mention anything about webpage UI. What to do: You need to select one of above UML design tool and draw your class diagrams and corresponding sequence diagrams. Remember that there is no need for other diagrams, for example, use case or activity diagram. A text file, "readme.txt", may contain the following information: 1) Your explanation if needed. 2) Any assumptions if you have (you can annotate them in the diagrams, not in the text file) 3) The time that you spend in this test. Notes: If you have any questions regarding customer's needs, please feel free to ask. There is no correct or wrong, good or bad design. Just be honest with it. However, we certainly can understand if the design is reasonable or not. 167 Fig 1 Login Page Fig.2 Login Error Fig 3 Registration Page Fig.4 Registration Successful Page 168 Fig 5 Normal User Homepage Fig.6 Group Manager Homepage Fig 7 Admin Homepage 169 B. Software Design – Login System (SSDF) Objective: To understand the participant's capability of using Synthesis-based Software Design Framework. Tools: We do not have a specific tool so far for you to address the framework so you need to use two tools (1) and (2) to express the hierarchy and the Design Matrix individually. (1)Drawing tool (for hierarchy): XMind (http://www.xmind.net/downloads/) or MS Visio (2)Design Matrix tool: MSExcel or MS Visio (3)UML Tool: ArgoUML or Rational Rose or StarUML What to do: You need to REDO the design test #1 (see below) by using the Synthesis-based design framework. You need to draw your class diagrams and corresponding sequence diagrams. Remember that there is NO NEED for other diagrams, for example, use case or activity diagram. Your "Design Matrix" excel file and "FR and Entity hierarchy diagram" (XMind file). A text file, "readme.txt", may contain the following information: 1) Your explanation if needed. 2) Any assumptions if you have (you can annotate them in the diagrams, not in the text file) 3) The time that you spend in this test. How long did you learn to use the tool? How long did you review the concepts? How long did you modify or do the design? Customer's Needs: You are designing the backend of a JAVA-based website login system. Your customer needs the following functionalities. You can check the prototype website figures below. 1) The system only have three kinds of users (roles)- "admin", "users", "group managers". 2) One user ID can only associates with one role. 3) The user ID is unique as a key to identify users. 4) One user ID can only join one group. 5) The group name can be changed while admin click ―edit‖. 170 6) A group only has one group manager. The group manger is assigned by admin while the group is being created. 7) The system should response login error to users. But, once logging in is successful, the system should forward the user to his/her individual homepage according to his/her role (see Fig. 1, 2). 8) A normal user's homepage contains user's group name, account ID, a personal information edit button. A normal user can only edit his/her information (see Fig 5). 9) A group manager's homepage contains user's group name, account ID, a personal information edit button, and a list of group member's account ID with an information edit button. A group manager can edit those users that belong to his/her group (see Fig.6). 10) An admin can remove, edit, or add a group (see Fig. 7). 11) An admin can remove user's account from the system, can also remove and add user from a group, can also edit user‘s personal information (see Fig. 7). 12) All users can log out from their homepage (see Fig. 5, 6, 7). 13) New user need to register his/her information. It must contain his/her account ID, password, email, name, address. After saving it, the system should show successful page (see Fig. 3, 4). The follows are assumptions or hints for this system design. 1) All users‘ data can be stored in any kind of storage system, for example, database, text file, list,…etc. It is up to your design. 2) Assume ―admin‖ is fixed and saved in the system. You DO NOT create a user to be admin. 3) You do not have to concern too much about the web page manipulations. You are only designing the backend. That is used to support the web UI‘s manipulations. Again, do not think something like pop-up windows for confirmation…etc. That belongs to front end (web UI). 4) Something like successful or error page, you can assume return ―True/False‖ after the method calls in your backend design. It is not necessary to mention anything about webpage UI. How to use XMind for FR and E Hierarchies: 171 (1) Start the XMind and your tool should like Figure 8 (2) Double click the "Sheet" bar on the bottom of the sheet and then create "Sheet 2" (see Figure 9) Fig 8 Initial Page Fig.9 Creating Sheet 2 (3) Click the box "Central Topic" and then change structure to "Org (down)" (see Figure 10). (4) Add sub-nodes from top toolbar (see Figure 10). 172 Fig 10 Adding Sub-nodes and Changing the Structure How to use Excel for Design Matrix: (1) Start the Excel and use the table put both FR and E on the left and top side. (2) Mark the coupling by input "X". 173 C. Software Design – Product System Objective: To understand the participant's capability of using Synthesis-based Software Design Framework. Tools: We do not have a specific tool so far for you to address the framework so you need to use two tools (1) and (2) to draw the hierarchy and the Design Matrix individually. (1)Drawing tool (for hierarchy): XMind (http://www.xmind.net/downloads/) or MS Visio (2)Design Matrix tool: MSExcel or MS Visio (3)UML Tool: ArgoUML or Rational Rose or StarUML Customer's Needs: You are designing the JAVA-based website product module on the server-side (backend). Your customer needs the following functionalities. You can check the prototype website figures below. 1) The system has three kinds of levels (layers) for users to browse the product information. 2) The highest level (layer) is "all", the second level is "category", and the lowest level is "product". 3) One product only has one product key (PK or product ID), and one category only has one category key (CK or category ID). 4) Each product belongs to only one category. 5) An key (AK) is used for highest level and it is fixed (preset) in the system and also initialized by the system. 6) The category name can be changed while the user click ―edit‖ after he/she is in the browse all page. 7) The system should response key-in error to users when they input the wrong PK, CK or AK (see Fig. 1, 2). 8) When inputting a correct key, the system should forward the user to product, category, or browse all information page according to the key. 174 9) A product information webpage contains product name, product key (PK), photo, manufacturer, material, and a product information edit button. Users can only edit the product information (see Fig 3) in the product page. 10) A category webpage contains category name, category key, a category information edit button, and a list of product names with an information edit button. Users can edit those products that belong to the category in the category page(see Fig.4). 11) Users can remove, or edit a category (see Fig. 5) in browse all page. 12) In browse all page, users can remove product from the system and can also edit a product's information (see Fig. 5). 13) New product need to key-in its information. It must contain his/her product key, name, category key (assume users already know this and assign category here), photo URL, manufacturer, and material. After saving it, the system should show successful page (see Fig. 6, 7). 14) A category only has a name. The category is assigned by users while the category is being created after users submit it in the new category page (see Fig. 8). The follows are assumptions or hints for this system design. 1) All data can be stored in any kind of storage system, for example, database, text file, list,…etc. It is up to your design. 2) You do not have to concern too much about the web page manipulations. You are only designing the backend. That is used to support the web UI‘s manipulations. Again, do not think something like pop-up windows for confirmation…etc. That belongs to front end (web UI). 3) Something like successful or error page, you can assume returning ―True/False‖ after the method calls in your backend design. It is not necessary to mention anything about webpage UI. What to do: You need to draw your class diagrams and corresponding sequence diagrams. Remember that there is NO NEED for other diagrams, for example, use case or activity diagram. 175 Your "Design Matrix" excel file and "FR and Entity hierarchy diagram" (XMind file). A text file, "readme.txt", may contain the following information: 1) Your explanation if needed. 2) Any assumptions if you have (you can annotate them in the diagrams, not in the text file) 3) The time that you spend in this test. How long did you learn to use the tool? How long did you review the concepts? How long did you modify or do the design? Fig 1 Product Page Fig.2 No Such Key Error Fig 3 Product Information Page Fig.4 Category Page 176 Fig 5 Browse All Page Fig.6 Product Information Entry Page Fig 7 Successfully Entry Page Fig 8 New Category Page How to use XMind for FR and E Hierarchies: (1) Start the XMind and your tool should like Figure 8 (2) Double click the "Sheet" bar on the bottom of the sheet and then create "Sheet 2" (see Figure 9) 177 Fig 8 Initial Page Fig.9 Creating Sheet 2 (3) Click the box "Central Topic" and then change structure to "Org (down)" (see Figure 10). (4) Add sub-nodes from top toolbar (see Figure 10). Fig 10 Adding Sub-nodes and Changing the Structure 178 How to use Excel for Design Matrix: (1) Start the Excel and use the table put both FR and E on the left and top side. (2) Mark the coupling by input "X". 179 D. Customer's Needs of the Visualization of Attack Surfaces for Targeting (VAST) Overall Summary: VAST is an Eclipse plug-in project. It provides annotation and code browsing analysis functions to a standard Eclipse tool. VAST can use the Eclipse's application program interfaces (APIs) and offer the needed functionalities. Terminologies: Eclipse - a JAVA development tool. Breadcrumb - Breadcrumbs (BC) are ―dots‖ on a map. The BC map is a grid that aligns each column to a set of tabbed views in a window. BC‘s are grouped into ―thoughts‖ which are linked by ―edges‖. They grow forwards or backwards across the grid along multiple rows. Edges are introduced to extend an existing thought. A new thought will be introduced with a BC with no connecting edge. Each thought occupies a distinct horizontal strip of the grid. View in Eclipse - a view is a panel that shows information in the Eclipse. It often provide additional information except source codes, such as the file system, debug panel...etc. Code Editor in Eclipse - a panel that show source codes and users can edit their codes there. List of Customer Needs: VAST plug-in works on Linux 32bit libraries (including X, Gnome, gtk libraries) and, using 32 bit Java and 32bit Eclipse Breadcrumbs can be placed in a view (see Fig. 2). The name that is displayed on a breadcrumbs should be the filename that the users have browsed. The breadcrumbs have the forward and backward links. Click them can extend to another breadcrumb (file/callee). The forward and backward links are based on the method calls. The whole breadcrumbs mean a "call graph". Users should hide, unhide, and delete (clear) the graph. A breadcrumb can be annotated as three levels of risk. The capability of stopping/pausing the recording of the breadcrumbs. The active breadcrumb node should be highlighted. A row of breadcrumbs is a thought, should be called a trail. It can be hided and unhided. Users can use "select all" to pick the whole breadcrumb row/trail. 180 When a breadcrumb is activated, the corresponding file should be open in the code editor and also be focused. When multiple nodes in the breadcrumbs are shown, the corresponding code editors are open. Therefore, they are multiple columns (see Fig.1). The subtree of a node in breadcrumb trail can be deleted when users activate prune operation. The subtree of a node in breadcrumb trail can be collapsed or expanded while users activate it. Users can annotate a group of related nodes/breadcrumbs (e.g., a call subtree) Ability to start/stop recording navigation Ability to start new navigation graph Ability to hide unimportant parts of the navigation graph. From there onwards a user may decide to navigate into function calls or out to function callees. Fig. 1. Multiple Column Code Editor 181 Fig 2. Breadcrumbs View E. SSDF Design Procedure of the Visualization of Attack Surfaces for Targeting (VAST) Goal: The goal of this study is to understand if a new design framework can help the designer to create a better design. The subjects of this study will be asked to do a project design by using the new framework, and then keep the logs of the subject‘s design. The records will be analyzed later on to show the improvement. Prerequisite: (1) Familiar with the Synthesis Design Framework (see attached document) (2) UML representation (3) Programming language Tool Used During the Study: (1) UML tool: Argouml (http://argouml.tigris.org/) or Rational Rose (http://www- 01.ibm.com/software/awdtools/modeler/swmodeler/?S_TACT=105AGX23&S_CMP=ROSE) (2) Suggested tree graph drawing tool: MS Visio or XMind (http://www.xmind.net/downloads/).or VGJ 182 (http://www.eng.auburn.edu/department/cse/research/graph_drawing/graph_drawing.html). (3) Design Matrix tool: MS Excel Background Survey: Please go to the homepage to fill out the survey form - http://sun.usc.edu/exp/ss.html Customer Needs of the Project: Please refer to the "Customer Needs Document" (see attached document) Before You Start: You will do a software system design according to the customer needs. Please fully understand the customer needs before starting your design. The whole case study may take you around four weeks (not full-time working, about 2 or 3 hrs per day). You do not have to perform the design continuously to the end. Periodically doing and resting is acceptable. We will collect your design results (each version's class diagrams, your Functional Requirement and Entity hierarchy diagrams, and design matrix) To-do: (1) Please draw class diagrams and sequence diagrams. Use your sequence diagrams to make sure your class diagrams is appropriate. You must use ArgoUML or Rational Rose tool. We will collect each version of your UML diagrams. (2) Keep the design history: Please save your class diagram in the following situations. a. Every time you have a revision. For example, after checking coupling in design matrix, you revise your design. b. Every time you temporarily end your design work. c. Any big change that you have make. d. Anytime you want to save a version. You can just save the file by using "Save As" function. Please keep the version in the format "filename_v0, filename_v1...etc." (3) Design Matrix: 183 Please use MS Excel you keep your design matrix. The design matrix could be very big but Excel should be enough for this. It is not necessary to keep every version unless you re-do a large part of the design. (4) FR and Entity hierarchy: Please use MS Visio, XMind, or VGJ ((2) in tools section above) to draw these two tree graphs. If not, using hand drawing is also acceptable but please keep it for us to collect. Also, you do not have to keep every version unless you re-do a large part of the design. F. OO Design Procedure of the Visualization of Attack Surfaces for Targeting (VAST) Goal: The goal of this study is to understand OOAD in the design of a real world project. The subjects of this study will be asked to keep the logs of the subject‘s design. The records will be analyzed later. Prerequisite: (1) UML representation (2) Programming language Tool Used During the Study: UML tool: Argouml (http://argouml.tigris.org/) or Rational Rose (http://www- 01.ibm.com/software/awdtools/modeler/swmodeler/?S_TACT=105AGX23&S_CMP=ROSE) Background Survey: Please go to the homepage to fill out the survey form - http://sun.usc.edu/exp/ss.html Customer Needs of the Project: Please refer to the "Customer Needs Document" (see attached document) Before You Start: You will do a software system design according to the customer needs. Please fully understand the customer needs before starting your design. The whole case study may take you around four weeks 184 (not full-time working, about 2 or 3 hrs per day). You do not have to perform the design continuously to the end. Periodically doing and resting is acceptable. We will collect your design results (each version's class diagrams, sequence diagrams) To-do: (1) Please draw class diagrams and sequence diagrams. Use your sequence diagrams to make sure your class diagrams is appropriate. You must use ArgoUML or Rational Rose tool. We will collect each version of your UML diagrams. (2) Keep the design history: Please save your class diagram in the following situations. a. Every time you have a revision. For example, after checking coupling in design matrix, you revise your design. b. Every time you temporarily end your design work. c. Any big change that you have make. d. Anytime you want to save a version. You can just save the file by using "Save As" function. Please keep the version in the format "filename_v0, filename_v1...etc." 185 G. Survey Form After you have done the final design practice, we would like to have your opinions. Questions of Synthesis-based Software Design Framework (SSDF) and Traditional OO Please indicate the level of performance (0:Not Useful/Not True, 2.5: Neutral, 5: Definitely Useful/Definitely True). 1 SSDF can provide a support for you to create objects and their classes. (0-5) 2 SSDF can help you to layout the relations between objects (classes). (0-5) 3 SSDF can assist you to decompose the system. 4 SSDF can help you to shorten your designing time. (0-5) 5 SSDF can assist you to specify the functional requirements for your design. (0-5) 6 SSDF can assist you to specify the constraints for your design. (0- 5) 7 SSDF can assist you to manage the complexity. (0-5) 8 It is difficult to apply SSDF. (0-5) 9 Overall, SSDF can help you to design your software system. (0-5) Why or why not? Please make comments. 1 OO can provide a support for you to create objects and their classes. (0-5) 2 OO can help you to layout the relations between objects (classes). (0-5) 3 OO can assist you to decompose the system. 4 OO can help you to shorten your designing time. (0-5) 5 OO can assist you to specify the functional requirements for your design. (0-5) 6 OO can assist you to specify the constraints for your design. (0-5) 7 OO can assist you to manage the complexity. (0-5) 8 It is difficult to apply OO. (0-5) 9 Overall, OO can help you to design your software system. (0-5) Why or why not? Please make comments.
Abstract (if available)
Abstract
Modern software systems have become increasingly complex due to heightened customer demands on more functions and reliability. When software engineers face the requirements of lower costs, shorter time, and easier maintenance, the challenges of designing complex software rise sharply. Complexity of a software system in traditional software design approaches can only be analyzed after the design is done. As a result, complexity is treated as an inherited property of engineered software systems which can only be reduced and managed by expensive and time-consuming iterative methods. In this research, we propose a new software design approach which treats complexity as an evolving property of the systems being designed. In this way, the software design will have the control of systems complexity during design, rather than only given the chance of analyzing it afterwards. By treating complexity as a design variable and suggesting a new synthesis design method, our research can systematically reduce and manage complexity when designing modern software systems to meet high customer needs. This fundamentally changes the notion of systems complexity and offers a better method for designing complex software. This research proposes a new software design framework, which is based on a synthesis theory, to complement the existing object-oriented design method. The theoretical backgrounds, the new synthesis design framework, and the process will be developed and then demonstrated with a series of comparative tests to validate our research results.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Building cellular self-organizing system (CSO): a behavior regulation based approach
PDF
A synthesis reasoning framework for early-stage engineering design
PDF
Managing functional coupling sequences to reduce complexity and increase modularity in conceptual design
PDF
Massive user enabled evolving web system
PDF
Designing an optimal software intensive system acquisition: a game theoretic approach
PDF
Dynamic social structuring in cellular self-organizing systems
PDF
Behavioral modeling and computational synthesis of self-organizing systems
PDF
Using organized objectives to structure arguments for collaborative negotiation of group decisions in software design
PDF
Design-time software quality modeling and analysis of distributed software-intensive systems
PDF
A user-centric approach for improving a distributed software system's deployment architecture
PDF
Software quality analysis: a value-based approach
PDF
Composable risk-driven processes for developing software systems from commercial-off-the-shelf (COTS) products
PDF
A biologically inspired DNA-based cellular approach to developing complex adaptive systems
PDF
Prediction of energy consumption behavior in component-based distributed systems
PDF
Learning to adapt to sensor changes and failures
PDF
Automatic conversion from flip-flop to 3-phase latch-based designs
PDF
Studies into computational intelligence approaches for the identification of complex nonlinear systems
PDF
A meta-interaction model for designing cellular self-organizing systems
PDF
Assessing software maintainability in systems by leveraging fuzzy methods and linguistic analysis
PDF
Extraction of preferential probabilities from early stage engineering design team discussion
Asset Metadata
Creator
Chang, Hung-Fu
(author)
Core Title
A synthesis approach to manage complexity in software systems design
School
Viterbi School of Engineering
Degree
Doctor of Philosophy
Degree Program
Computer Science
Publication Date
07/24/2013
Defense Date
05/03/2013
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
complex system,design method comparison,design synthesis,graph-based complexity,OAI-PMH Harvest,object-oriented system design method,synthesis-based design framework
Format
application/pdf
(imt)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Lu, Stephen C.-Y. (
committee chair
), Boehm, Barry W. (
committee member
), Jin, Yan (
committee member
), Neches, Robert (
committee member
)
Creator Email
hungfuc@gmail.com,hungfuch@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c3-299398
Unique identifier
UC11295005
Identifier
etd-ChangHungF-1838.pdf (filename),usctheses-c3-299398 (legacy record id)
Legacy Identifier
etd-ChangHungF-1838.pdf
Dmrecord
299398
Document Type
Dissertation
Format
application/pdf (imt)
Rights
Chang, Hung-Fu
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
complex system
design method comparison
design synthesis
graph-based complexity
object-oriented system design method
synthesis-based design framework