University of Southern California Dissertations and Theses

Software connectors for highly distributed and voluminous data-intensive systems

(USC Thesis Other)

Software connectors for highly distributed and voluminous data-intensive systems

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content SOFTWARE CONNECTORS FOR HIGHLY DISTRIBUTED AND VOLUMINOUS DATA-INTENSIVE SYSTEMS. by Christian Alan Mattmann A Dissertation Presented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (COMPUTER SCIENCE) December 2007 Copyright 2007 Christian Alan Mattmann Dedication To my father John, who passed before he was able to see me finish this long journey. To Chica, our favorite pup who is now roaming the heavens. And most of all, to my beautiful, patient, caring wife Lisa, my best friend, whose personal sacrifice over the past five years will never be forgotten. ii Acknowledgments There are many people to thank, so I’ll jump right to it. I am especially indebted to my advisor, Dr. Nenad Medvidovic. His guidance and tutelage has empowered me to enjoy writing, to strive for excellence and scholarly merit, and to always work hard. Neno was always was willing to stay up late into the night with me working on papers and proposals. This type of effort fromanadvisorwasinfinitelyinspiring. NotonlydoIappreciatehisguidance, but I have grown quite fond of Neno outside of the work arena. I am sure we’ll spend timewatchingUSCfootballintheyearstocomeandIlookforwardtomaintaining our friendship. Thanks, Neno! I am grateful to the members of my dissertation committee. To Dr. Horowitz and Dr. Boehm and Dr. Gupta, I appreciate your willingness to understand my research, and to help me revise my thesis. Its quality was no doubt greatly improved by your suggestions and feedback. I’ve had the pleasure of working with numerous world-class researchers during my time at USC and I’m glad to call them my friends. Thanks to the members of theSoftwareArchitectureResearchGroup, includingNikunjMehta, MarijaMikic- Rakic,RoshanakRoshandel,VladimirJakobac,SamMalek,DavidWoollard,Somo Banerjee, Chiyoung Seo, George Edwards, Yuriy Brun, Daniel Popescu and the rest of the folks that made my stay as a USC graduate student all the better. I’d iii also like to thank Jesal Bhuta for his collaboration and effort in getting our two research frameworks integrated. Jesal is an excellent friend and his research was particularly inspiring. My stay at USC was paralleled by my full-time job as a software engineer at NASA’s Jet Propulsion Laboratory. Many folks at JPL contributed to the success of my research, and to the ultimate production of my thesis. ToDanCrichton,Iamespeciallythankfulforeverythingthatyouhavedonefor me the past 4 years. Fondly known as my “JPL advisor”, Dan took me under his wingandchallengedmetoreachforthestarsinalltheprojectsthatIwasinvolved in at JPL. I’d also like to graciously acknowledge all of the funding support Dan has provided me, allowing me to explore my ideas and flush out the crux of my thesis. My thesis topic in fact grew out of a conversation I had one afternoon with Dan about a large problem plaguing NASA’s Planetary Data System. On the subject of the Planetary Data System, I would like to express my gratitude to the project’s system engineer, Steve Hughes. Steve is a world class computer scientist, and was an exemplary colleague and friend throughout the duration of my dissertation work. Steve was always willing to digest and vet my ideas and for that I will be forever grateful. I look forward to working with Steve in the years to come. I’d like to thank Dr. Robert Raskin, who was responsible for originally hiring me at JPL, and whose research and work motivated me to believe that getting a Ph.D. was possible. Rob gave me many interesting and challenging problems to work on, and introduced me to the wonderful environment that is JPL — something that I will appreciate for the rest of my life. I would like to graciously acknowledge the funding support provided by Rob in my initial years at JPL. iv To my office mate, Paul Ramirez, thanks so much for always listening to my rantings about research and life as a graduate student, long after you finished your graduate degree at USC. Thanks also for serving as a sounding board for work topics, frustrations, and my kooky ideas in general. The past few years have allowed us to become great friends and I look forward to working with you in the years to come. Several other folks had a hand in my research in some form or fashion. I’d like to thank Dana Freeborn, my group supervisor, for allowing me to spend time working on my research. To Sean Kelly, I’m especially grateful to have you as a friend and as a colleague. Your wit and style have had a great influence in the way that I conduct myself both personally and professionally. Last,butcertainlynotleast,I’dliketothankmybeautifulwifeLisa,forputting up with 4 years of staying up late at night, and working on the weekends, when we could have been out having fun. Lisa, you are my best friend. I love you bud. I look forward to spending the rest of our lives together, no longer beleaguered by having two full time jobs. I’m so proud to say, I made it! v Table of Contents Dedication ii Acknowledgments iii List of Tables x List of Figures xi Abstract xiv Chapter 1: Introduction 1 1.1 Research Problem and Importance . . . . . . . . . . . . . . . . . . 1 1.2 Research Statement and Hypotheses . . . . . . . . . . . . . . . . . 6 1.3 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.4 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.5 Organization of the Dissertation . . . . . . . . . . . . . . . . . . . . 13 Chapter 2: Background and Related Work 14 2.1 Software Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.2 Classifying Software Connectors . . . . . . . . . . . . . . . . . . . . 15 2.2.1 Combining Software Connectors . . . . . . . . . . . . . . . . 17 2.3 Detecting Architectural Mismatch . . . . . . . . . . . . . . . . . . . 21 2.4 Middleware Technologies and Software Connection. . . . . . . . . . 23 2.5 Information Dissemination Systems . . . . . . . . . . . . . . . . . . 24 2.6 Evaluating and Testing Software Connectors . . . . . . . . . . . . . 27 2.7 Selecting Software (COTS) Components . . . . . . . . . . . . . . . 30 Chapter 3: Motivating Experience 32 3.1 OODT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.1.1 The Planetary Data System . . . . . . . . . . . . . . . . . . 42 3.1.2 The Early Detection Research Network . . . . . . . . . . . . 47 3.1.3 The Orbiting Carbon Observatory . . . . . . . . . . . . . . . 52 vi 3.2 GLIDE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 3.2.1 Mobile Media Sharing System . . . . . . . . . . . . . . . . . 61 Chapter 4: Understanding Distribution Connectors 66 4.1 Event-based Distribution Connectors . . . . . . . . . . . . . . . . . 67 4.2 Grid-based Distribution Connectors . . . . . . . . . . . . . . . . . . 69 4.3 Client/Server Distribution Connectors . . . . . . . . . . . . . . . . 71 4.4 P2P Distribution Connectors. . . . . . . . . . . . . . . . . . . . . . 73 Chapter 5: The Distribution Connector Selection Problem 75 5.1 Problem Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . 77 5.1.1 Total Volume . . . . . . . . . . . . . . . . . . . . . . . . . . 78 5.1.2 Delivery Schedule . . . . . . . . . . . . . . . . . . . . . . . . 79 5.1.3 Performance Requirements . . . . . . . . . . . . . . . . . . . 80 5.1.4 Number of Users . . . . . . . . . . . . . . . . . . . . . . . . 81 5.1.5 Number of User Types . . . . . . . . . . . . . . . . . . . . . 82 5.1.6 Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 5.1.7 Geographic Distribution . . . . . . . . . . . . . . . . . . . . 84 5.1.8 Access Policies . . . . . . . . . . . . . . . . . . . . . . . . . 85 5.2 Problem Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Chapter 6: DISCO: A Framework for Software Connector Classifi- cation and Selection 88 6.1 Motivating Example . . . . . . . . . . . . . . . . . . . . . . . . . . 89 6.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 6.3 Building the Connector Knowledge Base . . . . . . . . . . . . . . . 94 6.3.1 Distribution Connector Profiles . . . . . . . . . . . . . . . . 97 6.3.2 Distribution Scenarios . . . . . . . . . . . . . . . . . . . . . 102 6.4 Developing Connector Selection Algorithms . . . . . . . . . . . . . 103 6.4.1 Connector Rank Lists . . . . . . . . . . . . . . . . . . . . . 106 6.4.2 Bayesian Algorithm . . . . . . . . . . . . . . . . . . . . . . . 107 6.4.3 Score-based Algorithm . . . . . . . . . . . . . . . . . . . . . 112 6.4.4 Optimizing Algorithm . . . . . . . . . . . . . . . . . . . . . 117 6.5 Performing Connector Selection Analysis . . . . . . . . . . . . . . . 120 6.5.1 Appropriate and Inappropriate Bins . . . . . . . . . . . . . . 121 6.5.2 Answer Key . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 6.5.3 Exhaustive K-Means Connector Clustering Algorithm . . . . 123 6.5.4 Precision-Recall Analysis . . . . . . . . . . . . . . . . . . . . 125 6.6 Validating selections against performance dimensions . . . . . . . . 127 6.6.1 Distribution Connector Performance Profiles . . . . . . . . . 127 vii Chapter 7: Tool Support 130 7.1 Extension Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 7.2 Framework Architecture . . . . . . . . . . . . . . . . . . . . . . . . 131 7.3 GUI Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 7.4 XML Data Formats . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 7.5 Command Line Tools . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Chapter 8: Evaluation 141 8.1 Connector Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 8.1.1 Expert Survey . . . . . . . . . . . . . . . . . . . . . . . . . . 143 8.1.2 Software Architecture Class Survey . . . . . . . . . . . . . . 145 8.2 Connectors Studied and their Variations . . . . . . . . . . . . . . . 150 8.3 Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 8.3.1 Sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 8.3.2 Relaxing Scenario Requirements . . . . . . . . . . . . . . . . 170 8.4 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 8.4.1 Time Complexity of Bayesian algorithm . . . . . . . . . . . 173 8.4.2 Time Complexity of Score-based algorithm . . . . . . . . . . 175 8.4.3 Time Complexity of Optimizing Algorithm . . . . . . . . . . 176 8.4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 8.5 Expressiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 8.6 Revising Connector Selections using Performance Data . . . . . . . 183 8.7 Integration with COTS assessment framework . . . . . . . . . . . . 186 Chapter 9: Conclusion and Future Work 191 9.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 9.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 9.2.1 Investigation of Architectural Mismatch . . . . . . . . . . . 194 9.2.2 Ability to Dynamically Redeploy Connectors . . . . . . . . . 195 9.2.3 Incorporation of other QoS attributes . . . . . . . . . . . . . 196 9.2.4 Evaluation of Expert Domain Knowledge Quality . . . . . . 197 9.2.5 Automatic Generation of Expert Domain Knowledge . . . . 199 9.2.6 Development of New Connector Selection Approaches . . . . 199 9.2.7 Connector Integration . . . . . . . . . . . . . . . . . . . . . 200 Bibliography 203 Appendices 213 Appendix A: Distribution Connector Profiles 213 A.1 Client Server Distribution Connectors . . . . . . . . . . . . . . . . . 213 A.1.1 File Transfer Protocol (FTP) . . . . . . . . . . . . . . . . . 213 viii A.1.2 Secure Copy Protocol (SCP) . . . . . . . . . . . . . . . . . . 216 A.1.3 Common Request Broker Architecture: CORBA . . . . . . . 219 A.1.4 REST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 A.1.5 Java Remote Method Invocation (RMI) . . . . . . . . . . . . 225 A.1.6 Simple Object Access Protocol (SOAP) . . . . . . . . . . . . 228 A.1.7 Commercial UDP Technology . . . . . . . . . . . . . . . . . 231 A.1.8 UDP-based FTP with Multicast (UFTP) . . . . . . . . . . . 234 A.2 Grid Distribution Connectors . . . . . . . . . . . . . . . . . . . . . 237 A.2.1 bbFTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 A.2.2 GridFTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240 A.3 Event Distribution Connectors . . . . . . . . . . . . . . . . . . . . . 243 A.3.1 GLIDE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 A.3.2 Sienna . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 A.4 P2P Distribution Connectors. . . . . . . . . . . . . . . . . . . . . . 249 A.4.1 Bittorrent . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 Appendix B: Software Connector Survey 253 B.1 General Questions. . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 B.2 Distribution Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . 256 B.3 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 B.3.1 Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274 B.3.2 Connector Descriptions . . . . . . . . . . . . . . . . . . . . . 275 Appendix C: DISCO GUI in Action 278 C.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278 C.2 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 C.3 GUI in Action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 ix List of Tables 6.1 Data Access DCP metadata. . . . . . . . . . . . . . . . . . . . . . . 98 6.2 Stream DCP metadata. . . . . . . . . . . . . . . . . . . . . . . . . . 99 6.3 Distributor DCP metadata. . . . . . . . . . . . . . . . . . . . . . . 100 6.4 Procedure Call DCP metadata. . . . . . . . . . . . . . . . . . . . . 101 6.5 Event DCP metadata. . . . . . . . . . . . . . . . . . . . . . . . . . 102 6.6 Arbitrator DCP metadata. . . . . . . . . . . . . . . . . . . . . . . . 103 6.7 Example distribution scenario. . . . . . . . . . . . . . . . . . . . . . 104 6.8 Partial DCPs of two postulated connectors.. . . . . . . . . . . . . . 106 6.9 Example domain profile knowledge for the Bayesian algorithm. . . . 108 6.10 Data from a score function relating num users and efficiency for six distribution connectors. . . . . . . . . . . . . . . . . . . . . . . . . . 114 6.11 Data from four score functions for two postulated connectors. . . . 116 8.1 Confusionmatrixusedinprecision-recallanalysisofthreeconnector selection algorithms with varying amounts of knowledge. . . . . . . 161 8.2 Fine-grained versus coarse-grained quantization of DCP properties and score functions and its implication on connector clustering. . . 169 8.3 Relationship between connector families and distribution scenario dimensions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 C.1 DISCO GUI requirements. . . . . . . . . . . . . . . . . . . . . . . . 280 x List of Figures 1.1 PDS Archive Volume Growth. . . . . . . . . . . . . . . . . . . . . . 3 1.2 OCO Ground Data System Archive Volume Growth (adapted from B. Chafin [Cha07]). . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 Software Architectural Model for Distribution Technologies . . . . . 8 2.1 Classification of Data Access Connector Type, taken from [MMP00] 15 2.2 Lopes et al.’s general connector model, adapted from [LWF03] . . . 20 2.3 Higher order connector example taken from [LWF03] . . . . . . . . 21 3.1 ThePlanetaryDataSystem(PDS)OODTArchitectureInstantiation. 36 3.2 Overview of the EDRN Knowledge Environment. . . . . . . . . . . 48 3.3 The Orbiting Carbon Observatory Process Control System Archi- tecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.4 Architecture diagram of GLIDE showing its PRISM and OODT foundation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 3.5 Mobile Media Application Architecture.. . . . . . . . . . . . . . . . 62 4.1 General Model for Distribution Connectors . . . . . . . . . . . . . . 67 4.2 Event-based distribution connector activity diagram . . . . . . . . . 68 4.3 Grid-based distribution connector activity diagram . . . . . . . . . 70 4.4 Client/server-based distribution connector activity diagram . . . . . 72 4.5 P2P-based distribution connector activity diagram . . . . . . . . . . 74 5.1 Distribution scenario problem space . . . . . . . . . . . . . . . . . . 76 6.1 High level architecture of a movie distribution system.. . . . . . . . 89 xi 6.2 DISCO’s overall architecture. . . . . . . . . . . . . . . . . . . . . . 93 6.3 DCPs and what they describe. . . . . . . . . . . . . . . . . . . . . . 96 6.4 Approach to developing connector selection algorithms. . . . . . . . 105 6.5 SamplescorefunctionsrelatingNumUsersandefficiencyfor6ofthe 13 connectors in the knowledge base. . . . . . . . . . . . . . . . . . 113 6.6 An overview of connector selection analysis within DISCO. . . . . . 122 7.1 Implementation-level architecture for the DISCO framework. . . . . 132 8.1 Results of expert connector survey. . . . . . . . . . . . . . . . . . . 143 8.2 Results of software architecture class connector survey. . . . . . . . 146 8.3 Appropriateandinappropriateconnectorsidentifiedinsoftwarearchi- tecture class connector survey. . . . . . . . . . . . . . . . . . . . . . 148 8.4 Distribution Connectors Studied. . . . . . . . . . . . . . . . . . . . 150 8.5 Data Access DCP attributes within connectors studied. . . . . . . . 154 8.6 Stream DCP attributes within connectors studied. . . . . . . . . . . 155 8.7 Distributor DCP attributes within connectors studied. . . . . . . . 156 8.8 Event DCP attributes within connectors studied. . . . . . . . . . . 157 8.9 Procedure Call DCP attributes within connectors studied. . . . . . 158 8.10 Precision-recall analysis of the three connector selection algorithms demonstrating low, medium, and high knowledge for each selection algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 8.11 Empirical evaluation showing selection time within Bayesian algo- rithm using knowledge bases of 100, 1000, and 10000 DCPs. . . . . 174 8.12 EmpiricalevaluationshowingselectiontimewithinScore-basedalgo- rithm using knowledge bases of 100, 1000, and 10000 DCPs. . . . . 176 8.13 EmpiricalevaluationshowingselectiontimewithinOptimizingalgo- rithm using knowledge bases of 100, 1000, and 10000 DCPs. . . . . 177 8.14 Precision-recall analysis of the three connector selection algorithms using performance data to revise selections.. . . . . . . . . . . . . . 183 xii 8.15 Integration of DISCO with externally developed COTS interoper- ability evaluation framework. . . . . . . . . . . . . . . . . . . . . . 187 9.1 Thesis high level summary diagram. . . . . . . . . . . . . . . . . . . 192 C.1 Overview screenshot of the DISCO GUI. . . . . . . . . . . . . . . . 281 C.2 Screenshot of choosing selectors within the DISCO GUI. . . . . . . 281 C.3 Screenshot of adding scenarios to run connector selection. . . . . . . 282 C.4 Screenshot of specifying a custom distribution scenario. . . . . . . . 283 C.5 Screenshot of connector selection for a single scenario. . . . . . . . . 284 C.6 Screenshot of connector selection results. . . . . . . . . . . . . . . . 284 xiii Abstract Data-intensive systems and applications transfer large volumes of data and meta- data to highly distributed users separated by geographic distance and organiza- tional boundaries. A dominating factor in these large volume data transfers is the selection of the appropriate software connector that satisfies user constraints on the required data distribution scenarios. This task is typically accomplished by consulting “gurus” who rely on their intuitions, at best backed by anecdotal evidence. In this dissertation we motivate, present and evaluate a software architecture- based systematic framework for selecting software connectors based on eight key dimensions of data distribution that we use to represent the data distribution scenarios. Our framework, dubbed DISCO, accurately, efficiently, and reliably captures a guru’s domain knowledge and allows a user to automatically leverage that knowledge to drive connector selection. In addition, DISCO affords a user theabilitytovalidateaguru’sdomainknowledgeagainstactualperformancemea- surementsoftheconnectorsintheareasofefficiency, scalability, dependabilityand consistency. We provide a set of models, algorithms, techniques and tools to represent data distributionscenarios,classifyandselectconnectorsandexplorethetradeoffspace xiv when architecting large scale data distribution systems. To date, 13 real-world connectors across four connector families have been explored using our framework. We validate our framework empirically and qualitatively, employing 30 data distribution scenarios gleaned from three real-world projects spanning planetary science, cancer research and earth science at NASA’s Jet Propulsion Laboratory. We use a number of measures of accuracy including precision, recall and error rate. We also provide theoretical performance analysis of our connector selection algorithms. We report empirical performance measurements of the 13 connectors and use the data to revise and validate our precision measurements. In addition to ourvalidation,wehaveintegratedDISCOasa“plug-in”piecetoanindependently- developed COTS interoperability assessment framework, providing more feedback for a second use-case of the tool. We conclude the dissertation with a set of open research questions that will frame our future work. xv Chapter 1 Introduction 1.1 Research Problem and Importance The terabyte- and petabyte-scale volumes of data [CFK + 00] generated by mod- ern data providers pose a significant challenge in terms of selecting appropriate interconnection/distribution mechanisms (i.e., software connectors) for data deliv- ery. The ideal software connector must deliver data from providers in a consistent, efficient, scalable, and dependable fashion. In general, end users of large scale dis- tributionsystemsexpect(near)real-timeaccesstodataonceithasbeengenerated, processed sufficiently, and packaged for distribution. This data must be delivered toendusersinalargenumberofdifferentscenariosinthedatadistributiondomain (referred to as data distribution scenarios throughout this dissertation). Each sce- nario specifies multiple data distribution dimensions. For example: • a large volume of data (e.g., on the order of a terabyte) of a single type may need to be delivered over the WAN to a small number of users, possibly across multiple delivery intervals; • a similar volume of data may need to be delivered to a different set of users over the LAN in a single delivery interval; • a medium volume of data (e.g., on the order of 10 gigabytes) of different types may need to be delivered to many users and user types; 1 • a smaller volume of data (e.g., on the order of a 100 megabytes) may need to be delivered across a WAN to a very large number of users of the same type, but with stringent performance requirements; and so forth. There are many existing software connector classes for data distribution (e.g., peer-to-peer or P2P, grid technologies, publish-subscribe, and so on). All of them have different characteristics that impact their performance in a given scenario. In other words, as we will elaborate later in this dissertation, it does matter what connectororcombinationofconnectorsisselectedtoaccomplishadatadistribution scenario such as those postulated above. The state-of-the-art in selecting such connectorsrightnowunfortunatelyremainsanartform,andfrequentlyboilsdown togoingwitha“hunch”orconsulting“gurus”. ThisiscertainlythecaseatNASA’s JetPropulsionLaboratory;ourrecentinformalsurveyofcolleaguesinseveralother large organizations suggests that this is true in their organizations as well. Both“hunches”and“gurus”maybeunreliable,however. Forexample,“gurus” may not be sufficiently informed about the possible options at their disposal, may haveasingle“pet”solution,ormayhaveavestedinterestinpromotingoneormore technologies. Furthermore, if there does not appear to be a good solution for the givenscenario,thelikelybestrecommendationwillbetogowithoneoftheexisting technologies anyway. We have recently witnessed first-hand a manifestation of this problem: we asked an acknowledged expert in the field of large-scale data distribution–who has been instrumental in developing a technology we consider in this dissertation and has published several authoritative works on this subject– to complete a questionnaire [Mat07] trying to assess the suitability of a set of widelyusedcandidatetechnologies,includinghisown,forasetofdetailedscenarios similar to those outlined above; the expert declined due to “a lack of expertise”. 2 Figure 1.1: PDS Archive Volume Growth. This is just one example to illustrate the point that a principled solution to theproblemofselectingthemostappropriateconnectorclassesthatcaneffectively addressaparticulardistributionscenario,andthatareatthesametimecompatible with existing data distribution architectures and systems, currently does not exist. Atthesametime,thedatadistributionproblemsthatwouldrequiresuchasolution are growing in numbers, scale, and complexity. One source of motivation for this dissertation has been our recent work in NASA’s Planetary Data System [HM98] (PDS). The PDS is NASA’s archive for planetarysciencedata. AllNASAplanetarysciencemissionsarerequiredtoensure that the data generated by their scientific instruments is formatted in such a way thatthePDScanreceiveit, catalogit, andmakeitavailablefordistributiontothe larger planetary science and educational community. A recently launched NASA planetary science mission, the Mars Reconnaissance Orbiter (MRO), will increase 3 OCO GDS Data Volumes 0 20 40 60 80 100 120 140 2/1/10 6/1/10 10/1/10 2/1/11 6/1/11 10/1/11 2/1/12 6/1/12 10/1/12 2/1/13 6/1/13 10/1/13 2/1/14 6/1/14 10/1/14 Date Volume (TiB) On-line storage available Total data volume Figure 1.2: OCO Ground Data System Archive Volume Growth (adapted from B. Chafin [Cha07]). the size of the current PDS archive from 20 terabytes (TB), to over 200 terabytes. This is a ten-fold increase (shown partially in Fig. 1.1) in the total data volume of the planetary archive to date (nearly 30 years), just from a single mission. The requirements of PDS mandate that the large volumes of data generated by MRO must still be distributed from the PDS to the planetary science community in the same fashion as is done today. Current knowledge is simply inadequate to address the projected volume increase from MRO and similar missions. There are other large-scale data systems that are facing similar data distribu- tion problems. For example, NASA’s Orbiting Carbon Observatory (OCO) earth science mission [OCO07] is set to produce nearly 150 terabytes of data (as shown in Fig. 1.2) from instruments deployed onto the OCO spacecraft which will launch in 2009. Once the OCO spacecraft is launched, the OCO ground data system will 4 need to process, archive, and distribute the 150 terabytes of data from computers inside JPL across a LAN to the Physical Oceanographic Data Active Archive Cen- ter (PO.DAAC) system at JPL. PO.DAAC will then further distribute the OCO data to the earth science community. OCO engineers are in the midst of design- ing a distribution system based on the Object Oriented Data Technology (OODT) frameworkforsciencedatadistribution[MCMH06];howeverOODThasneversup- porteddistributionofsuchalargedatavolume,andclearlywillrequireameansfor understanding the appropriate distribution connectors to select and use together in support of the OCO data distribution task, and what tradeoffs the selection of each induces on the system’s requirements, performance, and maintenance. On the other side of the scientific spectrum, the National Cancer Institute (NCI)’s Early Detection Research Network (EDRN) is a large-scale research net- work comprising 31 cancer research institutions sharing and distributing data in support of the early detection of cancer. Over the course of the next few years EDRNwillbecollectinghundredsofgigabytes(soonterabytes)ofcancerbiomarker proteomics data, including SELDI [SEL07], Q-Tof [Lee07], and LCT [Lee07] mass spectrometry, in search of biomarkers for the early detection of cancer. The pro- teomics data sets will need to be processed, archived, and distributed across the institutions participating in the EDRN, which span large geographic distances across the United States. An understanding of the appropriate software connec- tors and their support for different data distribution dimensions of interest would significantly aid the development of the EDRN distribution system for the pro- teomic data sets. 5 1.2 Research Statement and Hypotheses The preceding motivational scenarios beget the question that frames the problem area for our research: How do we distribute such voluminous data sets in a manner that is performant, and that adheres to a user’s requirements as well as the overall data system architecture? If we ask ourselves “which connector technology is most appropriate for a par- ticular data distribution scenario?” we must qualify the scenarios in which a data movement technology is appropriate. As mentioned previously, we must under- stand if a data movement technology is appropriate in larger volume scenarios, across the WAN; in medium volume scenarios, adhering to many users, user types, and data types; in larger volume scenarios, but over a single interval, across the LAN;andsoon. Thesetypesofquestionsimplytheneedforalanguagefordescrib- ingdatadistributionscenarios. Additionally,theremustbeawayofunderstanding how the available OTS data movement technologies map to the distribution sce- narios described by the designers of data distribution systems. If we restrict our focus to the question, “Which connector technology is the most appropriate?”, we arrive at some interesting observations: 1. It is a very difficult question to answer in a general case: some connectors arehighlyefficient,yetnotverydependable; somearehighlydependableand consistent, but neglect efficiency. 2. There has been little work on characterizing the space of distribution sce- narios that exist. What are the important dimensions? What are the users concerned with, and how do we map those user concerns to the appropriate connector(s) available? 6 3. Even when we have chosen a connector, how was it selected? What are the entry points into the selection? Where does the user’s knowledge come in versus the software architect’s knowledge? Further muddying the waters is the fact that there are a number of available off-the-shelf (OTS) technologies that claim to handle large scale data movement, includingcommercial(e.g.,Bittorrent,SOAP/WebServices)andopensource(e.g., UFTP, GridFTP, bbFTP) technologies. The sheer scope of the trade-off space alone makes a manual approach to selecting the appropriate technology unwieldy. We will demonstrate in this dissertation that the study of software architecture [PW92,SG96]canaidusinprovidingaflexible,principledsolutiontoselectingthe appropriate connectors for large-scale data distribution. A software architecture consists ofcomponents (the units of computation in asoftware system), connectors (facilities that model the interactions between software components), and configu- rations (assemblages of components and connectors, and the rules that guide their composition) [MT00]. Software connectors provide us with appropriate model- ing facilities for, and appropriate understanding of, large scale data distribution technologies 1 , as we depict graphically in Fig. 1.3. In this dissertation, we leverage one of the widely accepted models of software connectors, proposed by Mehta et al. [MMP00] to classify distribution connectors using standard metadata. In addition, we present a language for describing distri- bution scenarios based on eight key dimensions of data distribution taken from a thorough literature review, and in the context of our work at NASA’s Jet Propul- sion Laboratory. Using these classifications, and language, we have developed complementary selection algorithms that map the connectors (and their meta- data) to data distribution scenarios (expressed using our language), and that map 1 which we refer to as “data distribution connectors”, or “distribution connectors” for short 7 Data Producer Data Producer Data Producer Software Component Data Producer Data Producer Data Producer Data Producer Data Consumers ???? data data Software Component data data Software Connector Figure 1.3: Software Architectural Model for Distribution Technologies system performance requirements to both the connectors and scenario dimensions. Our framework, dubbed DISCO (for Data-Intensive Software COnnectors), has been implemented as a Java-based API and tool suite, and has proved valuable in providing a means for formally understanding the relationship between differ- ent data distribution scenarios and the connector technologies that exist. We view thistobeanimportantinitialstepinalleviatingtheneedtorelyonorganizational gurustomakesuchdecisions, andinformulatingaprincipledapproachtosoftware connector selection. The hypotheses that underlie our research fall naturally out of the above dis- cussion: Hypothesis 1 Eight key dimensions of data distribution can describe the wealth of data distribution scenarios that exist. Each of the eight dimensions are 8 interrelated, and have different levels of influence on the types of connectors that suit their constraints. The distribution scenario dimensions, the avail- ableclassesofdistributionscenarios, andperformancerequirementsonthose scenarios are highly interdependent. Hypothesis 2 Data movement technologies can be modeled expressively and accurately using an explicit software connector model. Hypothesis 3 Given a set of≥ 1 OTS distribution connectors, and distribution scenarios and performance requirements that must be satisfied, an efficient algorithm can calculate within 20% precision of a guru’s domain knowledge as to what connectors to select to meet the requirements. Hypothesis 4 We can accurately capture empirical performance measurements in the areas of consistency, dependability, efficiency, and scalability for each connector, and use such measurements to validate connector selections. 1.3 Assumptions The framework for software connector understanding and selection described in this dissertation is guided by the following assumptions: • Our research is focused specifically on software connector technologies, more generally referred to as “middleware”. Middleware is a higher level imple- mentation construct built on top of the underlying operating system and hardware platform. Our methods and techniques for selecting software con- nectorsisfocusedonmiddleware(andanyotherencapsulatingtechnologies), rather than the underlying hardware platforms and organization. 9 • Our research does not consider changing the underlying networking infras- tructure. Thetechniquesandprinciplesdescribedinthisdissertationassume that the data distribution technology is changeable, and the hardware is not. • Our research assumes that implementation costs for adopting a particular connector technology are reasonable. In other words, all connector technolo- gies are assumed to be acceptable to an organization, and cost agnostic. • Our research assumes that a distribution connector technology’s protocol (communication mechanism) cannot be changed. Instead of changing a particular distribution connector technology’s protocol, another distribution connector (with a different protocol) may be chosen. 1.4 Contributions This dissertation in general provides a principled approach to understanding how software connectors apply to different distribution scenarios. The contributions of the dissertation are six-fold: • The definition and implementation of the architecture of two data dissem- ination technologies, OODT, a data grid technology developed at NASA’s Jet Propulsion Laboratory (JPL), and GLIDE, a light-weight mobile data grid infrastructure developed at USC and JPL. Both of these technologies provide background understanding and the motivating experience for the construction of DISCO. • Definitionofadatadistributionscenariolanguage,forexpressingdistribution scenariorequirements. Thelanguageisbasedoneightkeydimensionsofdata distribution. 10 • Classification model for data distribution connectors, and detailed classifica- tions of 13 major OTS data distribution connectors. • A dynamic software technology for connector classification and selection, accompanied by tool support for exploring the trade-off space of connectors and distribution scenarios. • Empirical connector quality-of-service (QoS) performance evaluation in the areas of consistency, dependability, efficiency, and scalability. • Integration of our software technology with a real-world COTS interoper- ability evaluation framework, demonstrating our technology’s utility in a systems-of-systems environment, and furthermore, its applicability in the area of COTS interoperability evaluation. Theclaimsofthisdissertationhavebeenexaminedanditscontributionsdemon- strated in several controlled experiments, and via the evaluation of 30 real-world distributionscenariostakenfromtheaforementionedthreeJPLprojects, spanning three scientific domains: cancer research, planetary science, and earth science. The evaluation of the dissertation includes: 1. Accuracy experiments that identify the ability of our framework to capture a guru’s domain knowledge about the correct software connectors to select for distribution scenarios. 2. Experiments that empirically demonstrate our framework implementation’s scalability and utility in making real-time software connector selections for users and software systems, increasing the trade-off space as new connectors are identified. 11 3. A qualitative discussion of the expressiveness of our distribution scenario language based on eight key dimensions of data distribution. The discussion demonstrates our ability to describe the user-level needs in our framework, and to describe the data distribution problem space. 4. Empirical performance measurement in the areas of consistency, efficiency, dependability and scalability for 13 real world OTS distribution connectors, providing an empirical understanding of connectors that allows us to further refine gurus’ knowledge and validate their intuitions, alleviating the problem of “anecdotal evidence”. 5. The construction and qualitative evaluation of the integration of our frame- work with a separately developed COTS interoperability evaluation frame- workthatdemonstratestwokeypoints: (1)thatourframeworkcanbemade availableinasystems-of-systemsenvironmentasaplug-incapabilitytoother toolsandapplications; and(2)thatourframeworkishighlyapplicableinthe realm of COTS interoperability evaluation. 6. The generation of a set of statistics from two connector surveys sent to experts, novices, and practitioners in the field of data distribution. We pro- vide general statistics and motivation demonstrating the need for a frame- work such as ours, highlighting that the selection of connectors is something that students, software practitioners, and even experts are not easily able to do. 12 1.5 Organization of the Dissertation The remainder of this dissertation is organized as follows. Chapter 2 contains a literature review of the problem space, including background and related work in the areas of software architecture, software connector classification, selection, and combination, architectural mismatch, middleware, an overview of data dissemi- nation systems, and software connector testing. Chapter 3 describes in greater detail our motivating experience in the area of data distribution in the context of two real world data grid technologies: the Object Oriented Data Technology (OODT) project, and GLIDE, a grid-based, light-weight infrastructure for data- intensive environments. Chapter 4 describes the canonical four families of data distribution connectors we have identified, and their important properties. Chap- ter 5 introduces the data distribution connector selection problem, including its important dimensions, and its formal underpinnings. In Chapter 6, we discuss our framework (DISCO), and the main elements of our approach to connector clas- sification, selection, and evaluation. Chapter 7 describes the implementation of DISCO, including APIs and tool support allowing users to rapidly and accurately invoke DISCO, and evaluate distribution scenarios. Chapter 8 contains the val- idation experiments conducted, and Chapter 9 rounds out the dissertation with conclusions and future work. Thedissertationcontainsthreeappendices. Thefirstappendixcontainsfulllist- ingsofconnectormetadataforthe13connectorswestudied. Thesecondappendix contains a full listing of the software connector survey sent to students, practition- ers, and experts in the field of data distribution. The third appendix contains a discussion of the graphical user interface that we built to help evaluate the DISCO framework. 13 Chapter 2 Background and Related Work Our dissertation research is informed by and related to previous work in several areas: software architectures, and specifically software connectors; architecture- levelanalysis; middleware; informationdissemination; andperformanceevaluation of distributed computing platforms. In this chapter, each of these is discussed in greater detail. 2.1 Software Architecture Traditionally, software architecture has referred to the abstraction of a software system into its fundamental building blocks: software components, their meth- ods of interaction (or software connectors), and the governing rules that guide the composition of software components and software connectors (configurations) [PW92, MT00]. Software architecture has been recognized in many ways to be the linchpin of the software development process. Ideally, the software require- ments are reflected within the software system’s components and interactions; the components and interactions are captured within the system’s architecture; and the architecture is used to guide the design, implementation, and evolution of the system. Design guidelines that have been proven effective are often codified into architectural styles [Fie00, SG96], while specific architectural solutions (e.g., con- crete system structures, component types and interfaces, and interaction facilities) within specific domains are captured as reusable reference architectures [TTC95]. 14 Figure 2.1: Classification of Data Access Connector Type, taken from [MMP00] 2.2 Classifying Software Connectors Our principle source for the classification of software connectors has been the tax- onomy of software connectors by Mehta et al. [MMP00]. Although there are many other seminal works that provide a theoretical foundation for software connectors (e.g., [PW92, MT00, Sha93, ORT98, AG97, Arb04]) , our work has drawn heavily from this taxonomy. In particular, we have used the taxonomy to identify the set of basic connector types regularly composed into a data distribution connector, and the connector dimensions that are critical for data distribution. Mehtaetal.’sconnectortaxonomyidentifieseighttypesofsoftwareconnectors: (1) Procedure Call, (2) Event, (3) Data Access, (4) Linkage, (5) Stream, (6) Arbi- trator,(7)Adaptorand(8)Distributor. Eachidentifiedconnectortypeisclassified along a set of dimensions and sub dimensions, unique to each connector type. For each dimension, or sub dimension, valid values are identified. For example, in Fig. 2.1, the Data Access connector is classified along the Availability dimension, which has two sub dimensions, Transient and Persistent. For transient availability, the 15 data access connector can have any of the following values (possibly more than one): Register, Cache, DMA, Heap, or Stack. Mehta et al. employ their connector taxonomy through a set of illustrative examples, classifying the Linux file fa¸ cade connector, a shared memory access connector and a process scheduler connector. Mehta et al.’s taxonomy has been employed and studied by Kand´ e and Strohmeier [KS01], who propose a UML connector model based on Mehta et al.’s suggested eight connector types. Kand´ e and Strohmeier’s UML model distin- guishes between simple and complex connectors. Simple connectors define one type of role and have two connection points. A role is a channel that links the two connection points and represents the data exchanged between them. Connec- tion points model the interface method of the connector. The authors refer to connector interfaces as interaction services, and they identify a representative set of basic interaction services including stream, message and operational calls that map directly to Mehta et al.’s stream, event, and procedure call connector types described above. Complex connectors are compositions of two or more simple con- nector types. In complex connectors, simple connector types are linked together using connection protocols which bind two simple connector connection points. In[BP00],B´ alekandPl´ a˘ silarguethatMehtaetal.’staxonomydoesnotaddress the issue of how to design connector types, and that the “basic” connector types identified in the taxonomy are at different levels of granularity (e.g., procedure call vs. data access, or event vs. linkage). To address these issues, and to further strengthen the cause for recognizing connectors as first-class architectural entities, the authors propose a connector model that recognizes connectors at a single level of granularity—the software deployment phase of the software engineering lifecy- cle. In software deployment, the authors identify the deployment anomaly issue. Thedeploymentanomalyissueariseswhensoftwarecomponentsaredeployedonto 16 a target system, and a change in the communication mechanism between the com- ponents mandates a change in the system architecture. The authors suggest a connector as a first class entity that allows component interaction to evolve with- out component (or connector) architectural changes. B´ alek and Pl´ a˘ sil posit that the basic set of dynamic services that a connector should provide map directly to the key service categories in Mehta et al.’s taxon- omy: control and data transfer, interface adaptation and data conversion, access coordination and synchronization, communication intercepting and dynamic com- ponent linking. B´ alek and Pl´ a˘ sil’s connectors can be composed together to form even more complex behaviors required by the components they serve. In contrast toourdissertationwork,theworkbyB´ alekandPl´ a˘ silfocusesonformality,neglect- ing to apply their approach in the context of a real world project. Our approach on the other hand investigates the utility and usefulness of a connector model in the context of several large-scale real world data distribution scenarios. 2.2.1 Combining Software Connectors We note that, while our framework does not explicitly deal with means for con- nector composition, it is a logical extension to the body of work reported in this dissertation. In this section, we survey a representative body of existing work in low-level, and high-level connector composition. This section is meant to inform any future research that extends DISCO to support a means for connector com- position. Spitznagel and Garlan [SG01, SG03], outline a framework for composing and combining common software connectors (e.g., RPC and pipes), but focus on a class of connectors called wrappers. They model a connector as a six-tuple of concrete areas comprising: application code, communication libraries, low level 17 infrastructure services, data tables, policy, and formal behavior. In the authors’ model, connectors are combined via a set of extensible transformations, which are operators employed on any one of the six connector areas. Spitznagel and Garlan demonstratetheirapproachbycombiningconnectorssuchaskerberossecurityand RPC messaging using their base set of transformations. In addition, the authors show how certain measurable properties (e.g., deadlock detection, fault recovery) canbeprovablydeterminedwhencombiningwrappers. Theauthors’workinforms our work as its connector transformations can potentially be leveraged in any software technology that extends DISCO to implement connector composition. Also, the work proves the feasibility of ensuring desirable connector properties are met during composition The main advantage of the authors’ work is the definition of a primitive set of connector transformations that can be used to compose basic connector functionalities; however, the main drawback of the authors’ work is that the connector transformations are stringently defined in terms of the authors’ connector model (recall the six concrete areas defined above), and not generic. Arbab’s work on the Reo[Arb04]channelcoordination systemandthe workby Lau et al. [LEW05] are supplementary theoretical models for combining connec- tors. Bothconnectormodelsutilizethenotionofexogenousconnectors. Exogenous connectors control the execution of a program, rather than just the communica- tion. Each of these connector models provides the ability to construct more com- plex connectors out of a limited number of primitive connector types. This relates to our dissertation work in that we leverage Mehta et al.’s taxonomy to identify the underlying primitive connector constituents of a data distribution connector. WhileArbabandLauetal. bothfocusonidentifyingmeansofcomposition,inour work, we focus more on the identification of primitive connector types belonging 18 to a data distribution connector family, concerning ourselves with the important metadata to capture for each. Wohlstadter and Devanbu [WD01] show how so-called “higher order connec- tors” can be developed by composing simpler connector interactions such as asyn- chronous channels and shared memory access. The simpler connector interac- tions are used as parameters to a “higher order” connector function. The higher order connector function composes the functionality of the lower level connectors. Wohlstadter and Devanbu use a functional language called Haskell to demonstrate the applicability of their work; however, they note their initial examples are “sim- ple”, and meant to be instructive and used as a starting point for the definition of more useful connector compositions such as encryption, compression, and logging. Lopes et al. [LWF03] expand the work of Garlan and Spitznagel, and in the spirit of Wohlstadter and Devanbu, formalize the notion of higher order architec- tural connectors. Lopes et al. state that in general, a connector is made up of two concrete parts: roles, which define the interaction mechanisms that the connector participates in at each end of an interaction, and glue code, which is the functional code that performs the actual interaction (as shown in Fig. 2.2). This definition of a connector is similar to that of Kand´ e and Strohmeier, described above. Lopes et al. define a “higher order” connector to be a connector composed from pos- sibly many connector constituents cast into the glue portion of the higher order connector. To illustrate their model, the authors introduce a simple example of a connector C which acts as an asynchronous message transfer connector (shown in Fig. 2.2). C has two roles: the first role being a sender of messages, and the other being a receiver. Component A, the sender of messages through C, binds to the sender role of C, and a component B, the receiver of messages from C, binds to the receiver role of C. In between C’s roles, the glue that actually implements 19 A C B sender receiver buffer Components Roles Glue Connector Figure 2.2: Lopes et al.’s general connector model, adapted from [LWF03] the message transfer is a FIFO queue implementation that marshals the messages sent through the sender role, to the receiver role. Supposingthatauserwantedtoaddcompressiontotheexistingmessagetrans- ferconnectorC,theauthorssuggestthatthecompressionconnectorcouldbeadded by creating a new compositional connector D. D would have two roles, compress and decompress. D’s glue code would be formulated by utilizing the original con- nector C (shown in Fig. 2.3). The authors expand on this initial example, and go on to formalize the notion of higher-order, compositional connectors using both CommUnity, which is a parallel programming design language developed by the authors, and also by providing formal algebraic semantic specifications for connec- tor compositionality. Akin to B´ alek and Pl´ a˘ sil, Lopes et al. focus on formality, and thus define a specification for combining connectors according to their model. 20 D A C B sender receiver buffer Components Roles Glue Connector compress decompress Figure 2.3: Higher order connector example taken from [LWF03] 2.3 Detecting Architectural Mismatch Garlan et al. [GAO95] defined the problem of architectural mismatch. Architec- tural mismatch is observed to be a type of mismatch in which a “reusable part makesassumptionsaboutthestructureoftheapplicationinwhichitistoappear”. The authors first faced the architectural mismatch problem when attempting to construct Aesop, a suite of software architectural modeling tools, using a base set of four reusable off-the-shelf (OTS) components. The authors classify the four majorformsofarchitecturalmismatchencountered: componentassumptions, con- nector assumptions, topology assumptions, and construction process assumptions, or assumptions about the way in which the components and connectors were to fit together. The major connector assumptions encountered by the authors included protocol assumptions, which involved incorrect assumptions about the manner in which connectors communicate, and data model assumptions, which involved incorrect assumptions about the types of data exchanged between connectors. 21 Complementary to Garlan et al.’s work on architectural mismatch was the dis- sertation work by Gacek [Gac98]. Gacek delves into the problem of architectural mismatch when composing systems (and their subsystems). Gacek extends the work of Abd-Allah [AA96] by enriching his conceptual feature set that describes a system’s architecture. Abd-Allah’s original set consisted of architectural features such as dynamism, concurrency, and distribution. Gacek lengthens Abd-Allah’s set from 7 to 14 dimensions, adding new dimensions such as control unit, and pre- emption, while refining and revising some of the others. From Gacek’s 14 dimen- sions, 46 architectural mismatch examples are identified. Combining and selecting distribution connectors can benefit from a method for identifying mismatches; a natural extension to DISCO to support connector composition would leverage the foundational work by Gacek to provide a similar approach. ArelatedapproachtoarchitecturalmismatchidentificationisthatoftheArchi- tectural Trade-off Analysis Method [KKB + 98], or ATAM, developed at Carnigie Melon’s Software Engineering Institute (SEI). ATAM is a human-centric process for identifying risks in a software design early on in the development lifecycle. The process involves the gathering of a set of software architects designing the system, the stakeholders of that system, and a separate independent architecture evalua- tion team. The output of an ATAM analysis includes: (1) a set of risks for the system architecture, (2) a set of non-risks for the system architecture, (3) architec- tural styles identified within the system, (4) a model of the driving requirements for the system, (5) a set of scenarios used to evaluate the architecture, and (6) an analysis of quality attributes relevant to the architecture. TheapproachtakenbyDISCOissimilartothatofATAM.DISCOidentifiesthe architecturalstylesusedbytheconnectors(intheformofidentifyingaconnector’s family), and it utilizes scenarios (distribution scenarios) to evaluate the quality of 22 particular connector selections for an architecture. The output of DISCO is a rank set of connectors and their applicability to a data distribution architecture and its different distribution scenarios. Where DISCO differs from ATAM is that the goal of DISCO is to provide automatic guidance, based on the codification of domain expertise, as well as distribution scenarios, and connector models. The resultant system has the ability to automatically do what an approach such as ATAM still requires a human-in-the-loop to accomplish. 2.4 Middleware Technologies and Software Con- nection Middleware [Emm00a] and software bus [MHO96] technologies have often been thought of as implementation-level artifacts of architectural connection [MDT03]. This is also evidenced by the fact that many data distribution connectors (e.g., OODT [MCMH06], GridFTP [FKT01]) are implemented by leveraging capabil- ities from existing middleware technologies. Although there have been several recent attempts to classify and compare middleware technologies (see Emmerich [Emm00b] and Zarras [Zar04]), the work to date has been very broad. It has focused on defining the general requirements of distributed applications that must runontopofthemiddleware,andthenmappingthoserequirementstotheservices each middleware provides. General application requirements such as transparency, scalability, performance, and modularity have been mapped to basic middleware services including repository, coordination and security services. Inthisdissertation,wefocusondatadistributionconnectorswhicharetypically built as applications on top of middleware infrastructures. Therefore, the connec- tors we have studied (e.g., Bittorrent, GridFTP, HTTP/REST) can be thought of 23 as a particular family of applications that run on top of middleware. This data distribution application family induces its own set of requirements, and requires its own set of services that satisfy those requirements, which may or may not be directly provided by the underlying middleware. Our work provides a body of knowledge and software capability that rapidly and effectively compares and classifies how different connectors affect the data distribution application family along its dimensions such as total volume, number of sites, and number of users. The lack of a detailed comparison of middleware along these lines has hindered the development of a single reusable architectural solution to the data distribution problems described in the introductory chapter of this dissertation. Our work is a first step in addressing this missing element. The domain knowledge reported on in this dissertation can be used to select appropriate middleware technologies for plugging together existing, and constructing new, data distribution connectors and architectures. 2.5 Information Dissemination Systems Many information dissemination systems (also referred to as dissemination based informationsystems,orDBIS)havebeenstudiedoverthecourseofthepastdecade, exploring tradeoffs of data delivery methods, such as push versus pull, unicast versusbroadcast,andthelike. ThemostnotablewastheseminalstudybyFranklin and Zdonik [AAB + 99, FZ97] who propose the DBIS toolkit for understanding and constructing dissemination based information systems. The authors identify disseminationbasedinformationsystemstobesystemsthatinvolvethedeliveryof data from a set of sources to a set of interested consumers, using the appropriate delivery methods, schedules, and network paths. 24 In their attempt to understand dissemination based information systems, the authors explore the characteristics of the domain, and present an initial classifica- tion of data delivery methods along a set of dimensions that include: client pull versus server push, aperiodic versus periodic, and unicast versus 1-N. The data delivery methods discussed includes request/response, polling, publish/subscribe and broadcast disks. After classifying the data delivery methods using their ini- tial framework, the authors discuss the characteristics of a more general DBIS architecture and propose the basic nodes, or components of the architecture which include (1) data sources, or providers of data, (2) clients, which are consumers of data, and (3) information brokers, which provide value added information to the data that is delivered between sources and clients. Other components in a DBIS include (4) value-added nodes, which may filter data, or add metadata to data that is on the delivery path, and (5) recoverable nodes, that are able to guaran- tee some degree of reliability of data transfer and of system reliability. Franklin and Zdonik also identify that a DBIS system should handle caching of data at various nodes throughout the system, and that a DBIS should be able to employ different caching mechanisms including the least recently used (LRU) mechanism, invalidating and prefetching. Using their basic definition of a DBIS, the authors demonstrate how several representative dissemination information systems in exis- tence circa 1997, including the Pointcast system [Poi07], the authors’ own work on broadcast disks [AAFZ95], and the SIFT [YGM95] system, can be classified using their DBIS framework and architecture. The work by Franklin and Zdonik provided an initial starting point to our classification and understanding of distribution connectors, while at the same time bringing to light several distinct differences between their work and our own con- tributions. Franklin and Zdonik’s classification framework neglects to focus on 25 several important data dissemination dimensions, which we have chosen to include in our distribution connector framework. This includes dimensions such as the total volume of data, the number and types of access policies which are present at the sources and consumers of data in the system 1 , and delivery schedule, which can be a much richer function than just aperiodic versus periodic delivery. Our data distribution scenario language captures a data delivery schedule using three sub dimensions: volume per interval, number of intervals and timing per interval. Additionally, our work takes into account the different types of sources of data (e.g., long term archives and scientific instruments) and consumers of data (e.g., scientists and educators) on the one hand, and the different types of data itself transferred via the distribution connectors on the other, both of which may affect the additional data distribution dimensions. We note here that while we may refer to the software architectural understanding of how data is transferred among soft- ware components via software connectors as “data distribution”, the terminology is widely referred to in other disciplines (e.g., databases, computer networking) as “data delivery” or “data dissemination”. More importantly, however, our dissertation work takes a top-down approach; while Franklin and Zdonik’s work is really bottom-up. Franklin and Zdonik took the initiative to propose the entire architecture of a DBIS system; this gave them much flexibility in terms of being able to state what the components in the sys- tem should be, what their interconnection mechanisms (e.g., delivery mechanisms) should be, and how the components should work together. In contrast, our work involves discerning how to take existing pieces of data delivery software, model them as asoftware connector, and then identify how (and which of) those pieces of 1 Franklin and Zdonik do not provide any advice or suggestions for how to deal with data distribution across heterogeneous, so-called virtual organizations 26 software can work together and then test and validate that the connectors actually are able to perform the desired data delivery. Our work involves several facets of user interaction as well, including the specification of different “distribution scenarios” (using our scenario language), which ultimately involve semantically rich constraints on our data distribution dimensional model. Users of DISCO, our framework, may also identify what performance constraints they are interested in maintaining through the data delivery (e.g., consistency, scalability, etc.). Ulti- mately, our work contributes a software technique and accompanying technology capable of allowing a user to make informed decisions of the appropriate distribu- tion connectors to select and use in support of any distribution scenario that can be modeled using our general model of data distribution. 2.6 Evaluating and Testing Software Connectors Rosenblum[Ros98]proposesaformalapproachforincorporatingarchitecturalenti- tiesintosystemstesting. Theauthor’sformalapproachcanbeusedtodetermineif appropriate test cases are being selected in order to induce coverage of a software component’s exposed interfaces. Rosenblum identifies that, while testing at the system architecture development stage will not eliminate the need for the system testing stage in the software engineering process, architectural models are well positioned to test the system before deployment, greatly reducing cost in fixing errors that are propagated to the implementation stage of the software lifecycle. Rosenblum’s model is particularly geared towards software components, neglecting to focus on other architectural elements such as software connectors and configurations. The author does note, however, that some connector mod- els that exposed static external interfaces (e.g., Wright) may be described and 27 tested as components using his approach. However, other connectors that pro- vide dynamic interfaces (such as those used in C2 [TMA + 96]) are not able to be modeled using the author’s approach. Rosenblum’s approach is focused on under- standing how architecture affects test cases prior to system deployment, whereas our work focuses on testing software connectors that have already been deployed, and on issues in incorporating test attributes into connector models for use in performance testing. Verdickt et al. [VDGD05] propose an XML model for software connectors (in theformofmiddlewares)that(semi-)automaticallytakesintoaccountperformance attributes that would be useful for testing and validating properties of the middle- ware. The authors study the popular CORBA, J2EE, and Web Services middle- ware platforms and propose several important performance attributes that should be taken into account when testing software connectors, including (1) middleware initialization time, (2) middleware destroy time, (3) stub time (the client-side over- head when using the middleware and for performing a call), (4) skeleton time (the server side overhead when using the middleware and for performing a call), (5) service initialization overhead (on both the client and the server) and (6) service invocation overhead (on both the client and the server). The properties more or less all fall under the generic umbrella of efficiency and scalability, both of which wehaveselectedasimportantperformancerequirements(orattributes)inourown model. Verdickt et al. show how middleware-independent models can be (semi- )automatically tailored into “middleware-aware” models by a system architect to include the aforementioned performance attributes. Denaro et al. [DPE04] study how to incorporate performance attributes induced by software connectors at the early stages of software development. The authors’ approach is meant to help guide critical architectural choices that must 28 bemadewhenselectingmiddlewareinfrastructurestosatisfysystemrequirements. The key observation made by the authors is that middleware solutions selected in systems development greatly influence the performance properties of the system architecture. Denaroetal.’sapproachcanbesummarizedinfoursteps: (1)generateuse-cases that describe the performance requirements of the distributed system, (2) select software connectors that satisfy the use-case scenarios, (3) generate component “stubs” (for those system components not yet test-ready in the early stages of softwaredevelopment)thatcanbeusedinsimulationofthesystem’sperformance, and (4) deployment and testing. This last step has four steps of its own: (a) deploy the system, (b) create appropriate stress-testing generators, (c) initialize and deploy necessary test data, (d) report and evaluate performance metrics. The approach taken by Denaro et al. is quite similar to the approach we adopted for DISCO. Denaro et al.’s “use-cases” map to our “distribution sce- narios”, and the distribution connector selection algorithms that we developed (reported on in Chapter 5) were guided at least in some part by the authors’ own proposed algorithm. Their algorithm consists of the following steps: identify con- nector performance attributes that are important to test, classify connectors using those attributes (the authors suggest constructing a “distributed interaction tax- onomy”), and then select connectors which satisfy the constraints on the use cases (or distribution scenarios). 29 2.7 Selecting Software (COTS) Components Several approaches for the selection of COTS components have been developed over time. We consider three of them that are most closely related to our research below. The approach by Mancebo et al. [MA05] focuses on selecting COTS compo- nents during the architecture design phase of software development. The authors focus on three key trade dimensions that are important in selecting COTS compo- nents in an architecture: (1) requirements satisfaction, (2) architectural assump- tions, and (3) provided and required COTS component functionality. They con- struct matrices that evaluate COTS components for an exact fit along these three dimensions, and use that data to guide the selection. Their approach relies on the knowledge of the “key architectural assumptions” that a component makes. Our approach considers similar trade dimensions; however, in contrast to Mancebo et al., ourapproachusestheconnectorinformationasameansofrankingconnectors, allowing comparison of “how well” a connector fits a particular scenario. Alves et al. [AFCF05] developed a goal-based quality model approach for assessing COTS components, and their applicability to the architecture of a soft- ware system. Similar to our score-based algorithm discussed in Chapter 5, their approach requires the definition of “quality functions” that map a user’s satisfac- tionofaparticularCOTScomponenttoagivengoalorrequirementforthesystem. In contrast to our approach, this approach is entirely black box, focusing only on the observable properties of a COTS component, and not its internal architecture. We have recently completed a study that integrated DISCO with a framework for COTS component selection for interoperability assessment [BMMB07] devel- opedatUSC.Inourstudy(discussed furtherin Chapter7), we developedaCOTS componentandconnectorselectionapproachthatutilizedCOTSdefinitionmodels 30 and integration assessment rules to determine COTS components that could be applied to implement architectures, increasing the chances of interoperability with other components in the system. The output of the COTS selection framework is an interoperability assessment report that details architectural mismatches and level of interoperability for COTS components. The framework utilized DISCO as a level of service extension, a means for handling architectures involving large- scale, distributeddataconnectors. DISCO’sconnectorrankingswereobtainedand included in interoperability reports for large data system architectures. 31 Chapter 3 Motivating Experience Over the last four years, our work has focused on all aspects of the design and construction of large-scale, distributed data-intensive systems [BMMG05, MCH + 04, MRCH04, MFC03, MMB + 05, MCMH06, MM07, MHM03], across sev- eral highly heterogeneous domains: mobile computing [MMB + 05, MM07], search engines [RM04], scientific data distribution systems for planetary science and can- cer research at JPL [MCH + 04, MRCH04, MFC03, MHM03, MMRJ05] and grid computing systems [MRCH04, MMB + 05, MM07, MMRJ05]. Aparticularfocusofourworkhasbeenonunderstandingthesoftware architec- ture of these classes of software systems [MCH + 04, MCMH06, MMRJ05, MS03], and classifying and comparing their implementation properties to that of their intended architectures [MMRJ05]. We have also focused on means for understand- ing, representing and distributing metadata across each of the above mentioned systems, particularly focusing on: semantic interoperability [HCK + 04, HCKM05, RPM04], search and concept extraction [RM04], and intelligent resource discovery [MS03, HCKM05, HCK + 05]. In the following chapter, we detail our experience working on the Object Ori- ented Data Technology (OODT) [MCH + 04, MCMH06] project at JPL, and our experiencedevelopingtheGLIDElightweightgridinfrastructureformobiledevices [MMB + 05] at USC and JPL. Within the context of OODT, we discuss three par- ticular representative deployments of the system: NASA’s Planetary Data Sys- tem(PDS)[HM98,HCK + 04,HCKM05,HCK + 05], theNationalCancerInstitute’s 32 Early Detection Research Network (EDRN), and NASA’s Orbiting Carbon Obser- vatory (OCO) earth science mission. Within the context of GLIDE, we describe a mobile media sharing application that we constructed to share music files between mobile, distributed computing platforms. Our experience deploying OODT and GLIDE helped to motivate the need for our dissertation work, as each of these projects involves distributing (repre- sentatively) large amounts of data to different users, and software systems. The respective architectures of each of the three systems built using OODT and the one system built using GLIDE has provided us with a wealth of real world data distribution scenarios that must be supported not only in these systems, but more generally, in similar data distribution systems across many other domains (e.g., astrophysics, earthquake simulations, proteomics, and so on). We have identi- fied and captured a canonical set of thirty such data distribution scenarios that illustrate the high degree of complexity, and variation in the different scenario requirements (such as total volume of data, to be distributed, the number of users to send the data to, and so forth). These scenarios will form the crux of our eval- uation strategy for DISCO, our decision making framework for software connector selection that is explained in Chapter 6, and will be used to motivate, explain and validate the remainder of this dissertation. 3.1 OODT TheObjectOrientedDataTechnology(OODT)projectgrewoutofNASA’sOffice of Space Science’s desire to explore construction of an end-to-end software frame- work that would lower the cost of distributing and managing scientific data, from the inception of data at a science processing center to its ultimate arrival on the 33 desks of interested users. Because of increasing data volumes, the framework had to be scalable and have native support for evolution to hundreds of sites and thousands of data types. Additionally, the framework had to enable the virtual- ization of heterogeneous data (and processing) sources, and to address wide-scale (national and international) distribution of data. The framework needed to be flexible: it needed to support fully automated processing of data throughout its lifecycle, while still allowing interactivity and intervention from an operator when needed. Furthermore because data is itself distributed across NASA agencies, any software framework that distributes NASA’s data would require the capability for tailorable levels of security [BMMG05] and for varying types of users belonging to multiple organizations. There were also miscellaneous issues of data ownership that needed to be over- come. Ultimately, because NASA’s science data is so distributed, the owners of data systems (e.g., a Planetary Science Principal Investigator) feel hard pressed to control their data, as the successful operation and maintenance of their data systems are essential services that they provide. As such, any framework that vir- tualizes science data sources across NASA should be transparent and unobtrusive: it should enable dissemination and retrieval of data across data systems, each of which may have their own external interfaces and services; at the same time, it should enable scientists to maintain and operate their data systems independently. Finally, to lower costs, once the framework was built and installed, it needed to be reusable, free, and distributable to other NASA sites and centers for use. Overthepastsevenyearswehavedesigned,implementedanddeployedaframe- workcalledOODTthathasmettheserigorousdemands. Aparticularfocusofour work has been that of designing OODT’s architecture. OODT’s architecture is a referencearchitecturethatisintendedtobeinstantiatedandtailoredforuseacross 34 science domains and projects. The reference architecture comprises several com- ponents and connectors. A particular instance of this reference architecture, that of NASA’s Planetary Data System (PDS) project, is shown in Fig. 3.1. OODT is installedonagivenhostinsidea“sandbox”,andisawareofandinteractsonlywith the designated external data sources outside its sandbox. OODT’s components are responsible for delivering data from heterogeneous data stores, identifying and locatingdatawithinthesystem, andingestingandprocessingdataintounderlying data stores. The connectors are responsible for integrating OODT with heteroge- neous data sources; providing reliable messaging to the software components; mar- shalling resource descriptions and transferring data between components; trans- actional communication between components; and security related issues such as identification, authorization, and authentication. Below we discuss OODT’s com- ponents and connectors in greater detail. The Product Server is used to retrieve data from heterogeneous data stores. The product server accepts a query structure that identifies a set of zero or more products which should be returned the issuer of the query. A product is a unit of data in OODT and represents anything that a user of the system is interested in retrieving: a JPEG image of Mars, an MS Word document, a zip file containing text file results of a cancer study, and so on. Product servers can be located at remote data sites, geographically and/or institutionally disparate from other OODT components. Alternatively, product servers can be centralized, located at a single site. The objective of the product server is to deliver data from otherwise heterogeneousdatastoresandsystems. Aslongasadatastore(orsystem)provides some kind of access interface to get its data, a product server can “wrap” those interfaces with the help of Handler connectors described below. 35 PDS Query Ser ver m essaging layer (H T TP) Dataset/Product Catalog Pr ofile Server Pr ofile Ser ver Profile Server Pr ofile Ser ver ... ….. ….. ... pds .jpl.nasa.gov (Linux ) Legend : OODT Component Dat a /metadata store OODT Connector Hardware host OODT controlled portion of machine Product Ser ver Profile Server DBMS (PDS Metadata ) themis .asu.edu (Linux ) data/contr ol flow Black Box Filesyst em (PDS Products ) PDS Profile Handler PDS Quer y Handler DB /Filesys /Data System (PDS metadata ) PDS Pr ofile Handler ( Oracle ) OODT “Sandbox” OODT “Sandbox” Product Ser ver Profile Server anot her .pds .server ( Anot herOS ) Filesystem (PDS Product s ) PDS Profile Handler PDS Quer y Handler DB /Filesys /Data Syst em (PDS met adat a ) OODT “Sandbox” Catalog and Archive Ser ver Filesyst em (PDS Products ) Other Applications pds .jpl.nasa.gov (Linux ) Other Applications PDS Portal (Quer y Client ) user host Profile Client Pr oduct Client Figure 3.1: The Planetary Data System (PDS) OODT Architecture Instantiation. The Product Client component communicates with a product server via the Messaging Layer connectors described below. A product client resides at the end- user’s (e.g., scientist’s) site. It must know the location of at least one product server, and the query structure that identifies the set of products that the user wants to retrieve. At the same time, it is completely insulated from any changes in thephysicallocationoractualrepresentationofthedata;itsonlyinterfaceistothe product server(s). Many product clients may communicate with the same product server, and many product servers can return data to the same product client. This adds flexibility to the architecture without introducing unwanted long-term dependencies: a product client can be added, removed, or replaced with another 36 one that depends on different product servers, without any effect on the rest of the architecture. The Profile Server manages resource description information, i.e., metadata, in a system built with OODT. Resource description information is divided into three main categories: 1. Housekeeping Information – Metadata such as ID, Last Modified Date, Last Revised By. This information is kept about the resource descriptions them- selves and is used by the profile server to inventory and catalog resource descriptions. This is a fixed set of metadata. 2. Resource Information – This includes metadata such as Title, Author, Cre- ator, Publisher, Resource Type, and Resource Location. This information is kept for all the data in the system, and is an extended version of the Dublin Core Metadata for describing electronic resources [DCM99]. This is also a fixed set of metadata. 3. Domain-Specific Information – This includes metadata specific to a par- ticular data domain. For instance, in a cancer research system this may include metadata such as Blood Specimen Type, Site ID, and Protocol/S- tudy Description. This set of metadata is flexible and is expected to change. As with product servers, profile servers can be decentralized at multiple sites or centralized at a single site. The objective of the profile server is to deliver metadata that gives a user enough information to locate the actual data within OODT regardless of the underlying system’s exact configuration, and degrees of complexity and heterogeneity; the user then retrieves the data via one or more product servers. Because profile servers do not serve the actual data, they need not have a direct interface to the data that they describe. In addition to the 37 complete separation of duties between profile and product servers, this ensures their location independence, allows their separate evolution, and minimizes the effects of component and/or network failures in an OODT system. Profile Client components communicate with profile servers over the messaging layer connectors. The client must know the location of the profile server, and must provide a query that identifies the metadata that a user is interested in retrieving. There can be many profile clients speaking with a single profile server, and many profile servers speaking with a single profile client. The architectural effects are analogous to those in the case of product clients and servers. The Query Server component provides an integrated search and retrieval capa- bilityfortheOODTreferencearchitecture. Queryserversinteractwithprofileand product servers to retrieve metadata and data requested by system users. A query server is seeded with an initial set of references to profile servers. Upon receiving a query from a user, the query server passes it along to each profile server from its list, and collects the metadata returned. Part of this metadata is a resource location in the form of a URI [BLFM98]. A URI can be a link to a product server, to a web site with the actual data, or to some external data providing system. This directly supports heterogeneity, location transparency, and autonomy of data providers in OODT. AnothernovelaspectofOODT’sarchitectureisthatifaprofileserverisunable to service the query, or if it believes that other profile servers it is aware of may contain relevant metadata, it will return the URIs of those profile servers; the query server may then forward the query to them. As a result, query servers are completely decoupled from product servers (and from any “exposed” external datasources), andarealsodecoupledfrommostoftheprofileservers. Inturn, this lessens the complexity of implementing, integrating, and evolving query servers. 38 Once the resource metadata is returned, the query server will either allow the user herself to use the supplied URIs to find the data in which she was interested (interactive mode), or it will retrieve, package, and deliver the data to the user (non-interactive mode). As with the product and profile servers, query servers can be centrally located at a single site, or they can be decentralized across multiple sites. Query Client components communicate with the query servers. The query client must provide a query server with a query that identifies the data in which the user is interested, and it must set a mode for the query server (interactive or non-interactivemode). Thequeryclientmayknowthelocationofthequeryserver that it wants to contact, or it may rely on the messaging layer connector to route its queries to one or more query servers. The Catalog and Archive Server (CAS) component in OODT is responsible for providing a common mechanism for ingestion of data into a data store, including anyprocessingrequiredasaresultofingestion. Forinstance, priortotheingestion of a poor-resolution image of Mars, the image may need to be refined and the resolution improved. CAS would handle this type of processing. Any data ingested into CAS must include associated metadata information so that the data can be cataloged for search and retrieval purposes. Upon ingestion, the data is sent to a data store for preservation, and the corresponding metadata is sent to the associated catalog. The data store and catalog need not be located on the same host; they may be located on remote sites provided there is an access mechanism to store and retrieve data from each. The goal of CAS is to streamline and standardize the process of adding data to an OODT-aware system. Note that a system whose data stores were populated prior to its integration into OODT can still use CAS for its new data. Since the CAS component populates data 39 stores and catalogs with both data and metadata, specialized product and profile server components have been developed to serve data and metadata from the CAS backend data stores and catalogs more efficiently. Any older data can still be served with existing product and profile servers. The Archive Client component communicates with CAS. The archive client must know the location of the CAS component, and must provide it with data to ingest. Many archive clients can communicate with a single CAS component, and vice versa. Both the archive client and CAS components are completely inde- pendent of the preceding three pairs of component types in the OODT reference architecture. Handler connectors are responsible for enabling the interaction between OODT’s components and third-party data stores. A handler connector performs the transformation between an underlying (meta-)data store’s internal API for retrieving data and its (meta-)data format on the one hand, and the OODT sys- tem on the other. Each handler connector is typically developed for a class of data stores and metadata systems. For example, for a given DBMS such as Oracle, and a given internal representation schema for metadata, a generic Oracle handler connector is typically developed and then reused. Similarly, for a given filesystem scheme for storing data, a generic filesystem handler connector is developed and reused across like filesystem data stores. Each profile server and product server relies on one or more handler connec- tors. Profile servers use profile handlers, and product servers use query handlers. Handler connectors thereby completely insulate product and profile servers from the third-party data stores. Handlers also allow for different types of transforma- tions on (meta-)data to be introduced dynamically without any effect on the rest of OODT components. For example, a product server that distributes Mars image 40 data might be serviced by a query handler connector that returns high-resolution (e.g., 10 GB) JPEG image files of the latest summit climbed by a Mars rover; if the system ends up experiencing performance problems, another handler may be (temporarily) added to return lower-resolution (e.g., 1 MB) JPEG image files of the same scenario. Likewise, a profile server may have two profile handler connec- tors, one that returns image-quality metadata (e.g., resolution and bits/pixel) and another that returns instrument metadata about Mars rover images (e.g., instru- ment name or image creation date). The Messaging Layer connector is responsible for marshalling data and meta- data between components in an OODT system. The messaging layer must keep trackofthelocationsofthecomponents,whattypesofcomponentsresideinwhich locations, and if components are still running or not. Additionally, the messag- ing layer is responsible for taking care of any needed security mechanisms such as authentication against an LDAP directory service, or authorization of a user to perform certain role-based actions. The messaging layer in OODT provides synchronous interaction among the components, and some delivery guarantees on messages transferred between the software components. Typically in any large-scale data system, the asynchronous mode of interaction is not encouraged because partial data transfers are of no use to users such as scientists who need to make analysis on entire data sets. The messaging layer supports communication between any number of connected OODT software components. In addition, the messaging layer natively supports connections to other messaging layer connectors as well. This provides us with the ability to extend and adapt an OODT systems architecture, as well as easily tailor the architecture for any specific interaction needs (e.g., by adding data encryption and/or compression capabilities to the connector). 41 The OODT framework has been used both within and outside NASA. JPL, NASA’s Ames Research Center, the National Institutes of Health (NIH), the National Cancer Institute (NCI), several research universities, and U.S. Feder- ally Funded Research and Development Centers (FFRDCs) are all using OODT in some form or fashion. OODT is also available for download through a large open-source software distributor [Ope05]. OODT components are found in plan- etary science, earth science, biomedical, and clinical research projects, as we will elaborate in the remainder of this chapter. 3.1.1 The Planetary Data System One of the flagship deployments of OODT has been for NASA’s Planetary Data System (PDS) [HM98]. PDS consists of seven “discipline nodes” and an engineer- ing and management node. Each node resides at a different U.S. university or government agency, and is managed autonomously. For many years PDS distributed its data and metadata on physical media, primarilyCD-ROM.EachCD-ROMwasformattedaaccordingtoa“home-grown” directory layout structure called an archive volume, which later was turned into a PDS standard. PDS metadata was constructed using a common, well-structured set of 1200 metadata elements, such as Target Name and Instrument Type that were identified from the onset of the PDS project by planetary scientists. Beginning in the late 1990s the advent of the WWW and the increasing data volumesofmissionsledNASAmanagerstoimposeanewparadigmfordistributing data to the users of the PDS: data and metadata were now to be distributed electronically, via a single, unified web portal. The web portal and accompanying infrastructure to distribute PDS data and metadata, dubbed PDS-D, was built in 2001 using OODT in the manner depicted in Fig. 3.1. 42 Our core experience in data distribution has been within the context of the PDS.BecausePDShasevolvedfromthe“olddays”ofdistributingdataonphysical media to an online distributed data system, it regularly involves the dissemination ofdatainternallyfromonenodetoanother, fromnodestotheexternalworld(e.g., educators, other institutions), and the like. Using OODT, we have constructed systems to deal with several data distribu- tion scenarios within the PDS. Nine such representative scenarios are presented below, discussed, where possible, in the context of Fig. 3.1, and the PDS data distribution system. The labels beginning with the letter S below, are meant to tie the scenario descriptions in natural language to their actual representations using our scenario language discussed in Chapter 5. The actual numbered repre- sentations (using our scenario language) of all scenarios for each of the following OODT systems discussed (including PDS) are provided in Appendix II. S1 The PDS engineering node is required to delivery monthly backups amounting to 100s of GBs of information to the National Space Science Data Center (NSSDC) from JPL in Pasadena, across the WAN to the NSSDC in Mary- land. The WAN network can support delivery intervals of up to 10 GBs, and the users at the the NSSDC have expressed their desire to receive the back- ups in at most 10 delivery intervals. Consistency and dependability of the transfer are the prime concerns as the data is being prepared for long-term preservation. S3 Tensofthousandsofusers(includingscientists,educatorsandthegeneralpub- lic) would like to download the latest data from CASSINI, a mission sent to observe Jupiter some time ago. The latest delivery from the CASSINI mis- sion includes metadata describing 100s of GBs of images, and other scientific measurements of Jupiter’s atmosphere. The large variety of users will access 43 the CASSINI data from the WAN, with varying network connectivity levels including users on dedicated gigabyte network links, users on home internet service providers, and users at different universities and schools across the U.S., and also internationally. The data can be accessed using the PDS- D profile servers and product servers, and on average, the PDS developers expect that a user will take no more than 10 intervals to download the data she wants. Scalability of the distribution system is the most important ele- ment of the transfer. S9 More than 100 GB of PDS SPICE kernel datasets (that describe planetary missionnavigationinformation)andPDSdatavolumesneedtobesentacross the WAN from the PDS Engineering Node at JPL to the European Space Agency (ESA) and from JPL across the LAN to the (local) PDS navigation (NAIF)node. Thedatamustbedeliveredsecurely(usingport-based,firewall passthrough)toover100variousengineeringanalysts,andprojectmanagers, totaling ten different types of users. The consensus amongst the users is that they would like the data delivered in no more than 1000 delivery intervals, and due to the underlying network capacities of the dedicated LAN network to the NAIF and WAN network to ESA, the data must be delivered in 1MB to 1 GB chunks during each delivery interval. The data must reach its destination quickly in both cases (due to the sensitivity of the information, aswellasthetimeurgencyinwhichitisneeded),soscalabilityandefficiency are the most desired properties of the transfer. S10 Ten scientists from Arizona State University and Washington State Univer- sity, both home to PDS nodes, would like to receive all PDS data from the MarsPathfindermission,totaling1TBofdataandmetadata. Thedatamust 44 be delivered securely (using port-based, firewall pass through) in exactly 10 delivery intervals, because the scientists would like to receive daily deliver- ies of the data in 100 GB chunks. The data is to be delivered from the PDS Engineering Node, at JPL, emphasizing consistency and dependability, ensuring that the data is delivered as sent. S12 The Mars Reconnaissance Orbiter (MRO) mission has just made its delivery of the first quarter of data and metadata from its high resolution imaging camera. The total amount of information produced, its exact size unknown at present time, is between 10 TB and 50 TB of information. The data needs to be delivered using as many as 100 delivery intervals (to help break up the total volume into more manageable chunks), delivering as much data as possible per interval (the dedicated WAN between the instrument team and the science team at JPL can support up to 500 GB per data delivery). There are at least ten types of different users from the science team who need to analyze the data, however, all agree that the data’s consistency is of prime importance, and should be maximized (i.e., there should be no data loss). S16 A day’s worth of raw Mars Exploration Rover (MER) images totaling 250 MB needs to be distributed from the MER science data processing system to the instrument team for calibration of the wheel motors on the MER robot. The data needs to be sent in small chunks (.25 MB) as the computer it is being sent to is running several other network-intensive applications (navigation, and control), and needs to keep resources devoted to them. The data needs to be sent over a LAN to exactly 10 users (four calibration engineers, two calibration team leads, and four developers) on the MER 45 calibration team, who will analyze the images. Dependability of the transfer needstobemaximizedasthescienceprocessingsystematJPLthatissending thedatahaslimitedre-transmissioncapabilities. Thesciencedataprocessing system needs to access the raw image data using a linux mounted filesystem on the host computer. S18 Over 1000 local JPL scientists are all simultaneously downloading all MER image data for 2005 as it was recently announced that water was discovered in a remote crater on Mars. The scientists want to scan over all data taken near the region to determine if there are any other scientific phenomena that would bolster this exciting discovery. The MER image data for 2005 totals 100 GB of data, that is sent across the internal JPL LAN. Users are required toauthenticateusingtheirJPLpasswordbeforedownloadingthedata. Most users will have to download the data in exactly three intervals as there are three different internal websites that must be accessed to receive all the images. Efficiency, consistency, and dependability should all be maximized. S19 All MRO data and metadata collected to date (nearly 150 TB) needs to be disseminated to 1000s of international and national planetary scientists, across the public WAN. The PDS-D system is sitting behind a capacious network link, that can distribute data in 10 MB intervals, even across the WAN. The MRO data will be sent by PDS-D using anywhere from 1 to 100 delivery intervals. Each user that receives the MRO data will have to log onto the PDS-D website, which requires a username and password. Because ofthelargeamountofusersthatrequirethedata,scalabilityandconsistency should be maximized. 46 S30 ThePDSphotojournal,anexternalimagevisualizationapplicationthatcom- municateswithPDS-D,needstodisseminateallplanetaryimagestakenfrom 1970 to 1995, totaling 20 TB of data, to 10s of thousands of data replica- tion computers across at least ten different organizations over the course of a six-month time-span, in order to increase availability of the photojournal system. The data is to be sent across a private LAN to each of the data replication nodes. The data will be delivered using 100 delivery intervals, at .1 GB per interval so as not to saturate the underlying internal network. Scalability of the data transfers should be maximized. 3.1.2 The Early Detection Research Network OODT is also supporting the National Cancer Institute’s (NCI) Early Detection Research Network (EDRN). EDRN is a distributed research program that unites researchers from over thirty institutions across the United States. Tens of thou- sands of scientists participate in the EDRN. Each institution is focused on the discovery of cancer biomarkers as indicators for disease [Sri05]. A critical need for the EDRN is an electronic infrastructure to support discovery and validation of these markers. We have used OODT [CKM + 06] to spearhead the development of an infras- tructure that supports the aforementioned needs of the EDRN. The infrastructure is a virtual knowledge environment called the EDRN Knowledge Environment, or EKE. EKE is a distributed architecture that contains four major components, shown in Fig. 3.2: • A distributed specimen management system, called the EDRN Resource Network Exchange, or ERNE, shown in the upper right portion of Fig. 3.2. 47 Figure 3.2: Overview of the EDRN Knowledge Environment. • AdistributedsciencedatawarehousingsystemcalledtheeDRNCatalogand Archive Service, or eCAS, shown in the upper left portion of Fig. 3.2. • A centralized biomarker database that manages biomarker metadata, shown in the lower left portion of Fig. 3.2. • A study management tool called VSIMS that manages information about clinical trials, shown in the lower right portion of Fig. 3.2. Since 2001 we have worked with the EDRN program to develop its electronic biomarker infrastructure described above. One of the major goals of the infras- tructure was to provide real-time access to bio-specimen information across the institutions of the EDRN. Bio-specimen information typically consisted of giga- bytes of specimen images, and location and contact metadata for obtaining the 48 specimen from its origin study institution. The previous method of obtaining bio-specimen information was very human-intensive: it involved phone calls and some forms of electronic communication such as email. Specimen information was not searchable across institutions participating in the EDRN. The bio-specimen catalogs were largely out-of-date, and out-of-synch with current holdings at each participating institution. We used OODT product and profile servers to construct ERNE in a similar manner to that of the PDS-D data distribution system described in the previous section. In addition to the bio-specimen information, another major goal of the EKE was the construction a distributed science data warehousing system called eCASforuseatlocallaboratories,andattheNCI.eCASwasdesignedtostorelarge volumes of raw science data products, generated by biomedical instruments (such as SELDI, and other types of mass spectrometry). We used the OODT Catalog and Archive Service (CAS) to construct the initial version of the eCAS. After eCAS, we were tasked with building a centralized data management system for storingbiomarkermetadatainformation, includinginformationabouttheirrelated organs, patients, clinical studies, and sensitivity and specificity. We employed an OODT profile server to serve back the biomarker metadata information stored in theBiomarkerDatabase. Finally, forstudymanagement, wehaveworkedwiththe Fred Hutchinson Cancer Research center to develop a centralized database system storing protocol information, patient demographics, and study information about the clinical trails performed at the sites across the EDRN. There are several data distribution scenarios within the EKE. We express eight of them that we have found particularly representative. S4 In EKE both the Biomarker database and ERNE need to efficiently deliver their 10s of GB of metadata to the public portal periodically (between 1 to 49 10 times daily, between 1 and 10 GB per interval), updating the portal so that it can provide custom views of the data, and disseminate it throughout the rest of the EDRN. This allows scientists, educators and some privileged members of the general public access to (de-identified) cancer research data. The portal data receiving service is protected by SSL/HTTP 1.0. S13 A lung cancer researcher at the University of Colorado would like to deliver a lung image dataset consisting of bronchoscopys, x-ray imaging, and floures- centscans,alongwiththeirassociatedmetadatatoacolleagueatDartmouth University. Due to the sensitive nature of the information, it must be trans- ferred across a dedicated, highly secure and reliable, but ultimately very slow network line over a WAN. The line can support a maximum of .3MB per transfer, allowing between 1 and 100 transfers a day. One the recipient end of the line is a firewall protected, port-based service waiting to accept and store the information. S14 FivebiostatisticiansatFredHutchinsonCancerResearchCenterwouldliketo distributebetween600MBand1GBofSELDImassspectrometrydataover 10 delivery intervals to six of their colleagues at USC Medical Center, UCLA Medical Center and University of Pittsburg. All their colleagues would like to receive the data in 10 MB chunks, and would like to ensure that the data is securely sent through web services running on protected ports at their respective institutions. Transfer rate is important, but more important is that the data be sent without error as the computers sending it need to be restarted very soon. S23 A cancer researcher at Fred Hutchinson Cancer Research Center has just finished producing 50 GB of SELDI mass spectrometry data investigating a 50 promisingbiomarkerforcoloncancer. Theresearcherwouldliketodistribute the SELDI dataset to colleagues at his institution, across a GB-speed LAN, using at most 50 delivery intervals. While the researcher is mostly agnostic to performance requirements, she would like the data delivered reliably since there could be many delivery intervals. S24 The same cancer researcher from the preceding scenario at Fred Hutchinson Cancer Research Center would now like to distribute her 50 GB of mass spectrometry data to colleagues at various external institutions, with more emphasis on dependability, scalability and efficiency of the transfer. S26 AlungcancerresearcheratNYUmedicalcenteriscapturingCTscandatafor patients participating in a clinical trial. The researcher would like to share 1 GB of the data with one of his colleagues at Roswell Park cancer research center in Buffalo, New York, for further analysis that will be included in a grant report. The data should be sent once over a single interval. Since their grant report is not due for some time, the researcher is performance requirement agnostic. S28 VSIMS,thestudymanagementsystemwithinEKEdeliversclinicalsummary dataandepidemiologicaldatafrom1000sofpatientsto50cancerresearchers, studymanagementpersonnelandothersatFredHutchinsonCancerResearch Center by dumping its study management information to the EDRN public portal. Thisdumpingoccursdailyoverthecourseof5intervals,with1GBof datadumpedperinterval. Thetransferishighlydependable, andconsistent, involving the dissemination of both patient epidemiological data, along with patient metadata. 51 S29 ThesamedatasentintheprecedingscenariofromVSIMSatFredHutchinson Cancer Research Center now needs to be distributed to 50 individuals, all of different types, incl. NCI program management, and other NCI personnel acrossaWAN,sending.1GBofdatapertransfer, using50deliveryintervals, across a direct network connection between the NCI and Fred Hutchinson. The consistency and dependability of the transfer is most critical. 3.1.3 The Orbiting Carbon Observatory We have recently become involved using OODT in the development of a high performance, large scale scientific processing system for the NASA earth science mission, the Orbiting Carbon Observatory (OCO) [OCO07]. All earth science missionsrequireagrounddataprocessingsystemthattakesrawscienceinstrument telemetry, and processes that data to ultimately generate science data indicating the key observations, and measurements taken by the instruments. OCO is part of a new class of missions, that impose stringent requirements on their ground data processing systems, including high performance job throughput, management of complex scientific workflow models, and data staging to distributed nodes, and data cataloging and archiving. A partial representation of the OCO ground data system processing control system (PCS) architecture is depicted in Fig. 3.3. The PCS architecture leverages one of the core OODT components, the Catalog and Archive Service (CAS). For OCO,weredesignedandrefactoredtheCASintoitscanonicalfunctions: fileman- agement, workflow management, and resource management, all of which are com- ponents shaded grey in Fig. 3.3. The File Manager is a daemon component that provides cataloging of science data product metadata, and data transfer capabili- ties to processing jobs run through the system. The Workflow Manager manages 52 File Manager Resource Manager Workflow Manager PGE PGE PGE Workflow Model (XML based) manages instantiates workflow instance WPT sends workflow task job Computing Resources executes cf generates gets info for config file p m p m produces Crawler invokes crawls ingests Figure 3.3: The Orbiting Carbon Observatory Process Control System Architec- ture. science data pipeline descriptions. A science data pipeline is a series of programs (called PGEs), executed in a pipe-and-filter fashion, where the output(s) of one program are provided as input(s) to the next. Certain programs within a pipeline may be prevented from executing until they have fulfilled certain pre-conditions (e.g., all input files available, available hardware node to execute on, etc.). The workflowmanagerisresponsibleforcreatinginstancesofthesesciencedatapipeline models, and then managing their execution (using Workflow Processor Threads, labeled as WPT within Fig. 3.3). To execute the programs within a pipeline, the workflow manager relies on the Resource Manager component. The resource manager is responsible for job execution, and monitoring of a set of underlying 53 computational resources, such as hardware nodes, network availability, and disk space. Each PGE job that is run on a particular hardware node by the resource manager is run through a PCS Task Wrapper. The task wrapper provides the nec- essary input information to the PGE, including file locations, necessary metadata, etc., and may communicate with any of the CAS manager components to obtain the necessary information to run the PGE. Once the PGE job has executed, it produces product files (labeled as p within Fig. 3.3), and metadata files (labeled asmwithinFig. 3.3). TheproductandmetadatafilesarecrawledusingaCrawler component, that ingests the science data product files and metadata into the file manager. There are several data distribution scenarios that we have encountered within OCO, six of which we have found particularly representative of the domain. We describe the six scenarios below. S2 One year of ground-based Fourier Transform Spectrometer (FTS) instrument data taken at a location in the U.S. mid-west needs to be transferred using the File Manager from an internal archive system to a data analysis machine across a LAN in preparation for space-based instrument testing. The total volume of data to transfer is 50 GB. Because of the importance of the instru- ment tests, consistency and dependability of the transfer are of prime impor- tance. S5 All FTS data collected over the course of the past two years, a total volume between 100 GB and 2 TB depending upon the inclusion of ancillary infor- mation, has to be moved (for backup purposes) to another machine available on the OCO LAN. The data is set to be backed up incrementally, using anywhere from 1 to 300 delivery intervals. Because of the capacious under- lying LAN, the network can support 50 GB per delivery interval. Due to 54 backup purposes, and the superior LAN, consistency and efficiency are to be maximized. S8 Thefirst3monthsofspace-basedinstrumentdatafromOCOneedstobedeliv- ered to 10 members of the OCO science team. The total volume is expected to be between 2TB and 4TB of information, depending upon the amount of downlinks available within the first 10 orbit cycles of the spacecraft. 1 The dataistobedeliveredto3usertypes, calibrationengineers, atmosphericsci- entists, and their graduate students. The data, due to its large volume, is to be delivered using between 1 to 1000 delivery intervals, sending 1 GB of data per interval to maximize the use of the underlying network resources. The data needs to be sent to the calibration engineers using the internal OCO LAN, in contrast to the atmospheric scientists and graduate students, who must receive the data over a dedicated WAN. Consistency and dependability are the most important elements of the transfer. S11 Developer home directories on the OCO cluster need to be backed from a world-readable linux file system mount to an archive machine. The total volume of the user home directories is between 10 GB and 50 GB, depending upon the number of system simulations run that day. The data needs to be backed up incrementally, using between 1 to 100 delivery intervals, where each interval consists of 100 MB of data. The developer home directory data is backed up over a LAN by a backup system that needs to store a small amount of metadata for each backup activity, allowing the backup system to restart (in case of interruption) a backup activity. Because this is a backup, consistency and dependability need to be maximized. 1 Weather, atmospheric conditions, and computer malfunction can all prevent downlink of spacecraft data to a ground station. 55 S17 All 150 TB of OCO data and metadata (space and ground-based FTS) from the production archive needs to be disseminated to local science processing centers, to weather forecasting centers, and to government agencies, totaling more than 20 different sites in all. The data is to be sent over the course of at least 10s of thousands of delivery interval, ranging from 1GB of data to 100 GB of data per interval, across both internal LANs and public WANs across the internet to 1000s of different users. The data needs to be accessed off of the local production archive through a local linux file system, requiring properlinuxgrouppermissionstoaccesstheinformation. Dependabilityand scalability of the transfers are most important, as the delivery of information is a very resource-expensive activity, that should not be restarted. S20 The source code for the OCO ground data system (including the PCS) needs to be duplicated and delivered to a test environment for system testers and Q/Apersonneltoevaluate. Thetotalvolumeofdatatodeliveris50GB.The data must be delivered over an internal LAN for the OCO CM repository to the testing machine. There are 50 different testers and Q/A personnel, each with a different interest in the data. The data needs to be delivered in 50 intervals, using 1 GB of network bandwidth per interval, making the most efficient use of the underlying LAN. Scalability and dependability of the transfer should be maximized. 3.2 GLIDE One of the most exciting and promising technologies in modern computing is the grid [FKT01]. Grid computing connects dynamic collections of individuals, institutions, and resources to create virtual organizations, which support sharing, 56 discovery, transformation, and distribution of data and computational resources. Distributed workflow, massive parallel computation, and knowledge discovery are only some of the applications of the grid. Grid applications involve large numbers of distributed devices executing large numbers of computational and data com- ponents. As such, they require techniques and tools for supporting their design, implementation, and dynamic evolution. Current grid technologies provide extensive support for describing, modelling, discovering, and retrieving data and computational resources. Unfortunately, they are predominantly implemented using middleware infrastructures that leverage both heavyweight and computationally intensive protocols and objects [DFS04]. As such, current grid software systems are not readily applicable to the domain of decentralized, resource constrained, embedded, autonomic, and mobile (DREAM) environments. Existing grid technologies also lack native support for system- atic application design, implementation, and evolution. Finally, the development, deployment, and runtime adaptation support for the grid is ad-hoc: shell scripts abound, makefiles are the common construction and deployment tool, and adap- tation is usually handled by restarting the entire system. Inanattempttoaddresstheaforementionedshortcomingsofcurrentgridtech- nologies (e.g., OODT, Globus [FKT01]) at the time, we developed the GLIDE (grid-basedlightweightinfrastructurefordata-intensiveenvironments)systemthat married the OODT framework, and the PRISM-MW middleware for resource- constrained, distributed, mobile devices [MMRM05]. GLIDE is a hybrid grid middleware which combines the salient properties of Prism-MW (architecture-based abstractions for development, event-based asyn- chronous communication, low footprint, and support for disconnected operation) 57 Figure 3.4: Architecture diagram of GLIDE showing its PRISM and OODT foun- dation. and core services of the grid, (data distribution, resource discovery, data collec- tion, distributed workflow) with the goal of extending the reach of the grid beyond super-computinganddesktop-or-betterplatformstotherealmofDREAMenviron- ments. To this end, the myriad of heterogeneous data (music files, images, science data,accountingdocuments,andsoon)andcomputational(webservices,scientific computingtestbeds,andsoon)resourcesmadeavailablebyheavy-weightgridscan also be made available on their mobile counterparts. Thus, mobile grids enabled by GLIDE have the potential to be both data-intensive, requiring the system to providerichmetadatadescribingtheabundantresources(andsubsequentlydeliver and retrieve representatively large amounts of them), as well as computationally- intensive, focused on discovering and utilizing data, systems, authorization, and access privileges to enable complex, distributed processing and workflow. 58 Existing grid solutions such as Globus and OODT take a completely agnostic approach to the amount of hardware, memory, and network resources available for deploying, executing, and evolving a grid-based software system. These tech- nologies consider the system’s architectural design to be outside their scope. In addition, they also fail to provide sufficient development-level support for build- ing,deploying,andevolvingsoftwareapplications. Asolutionthatovercomesthese limitations is needed to realize the widely stated vision of “data and computation everywhere”. By implementing the core grid components of OODT using Prism- MW, we believe to have created an effective prototype platform for investigating and addressing these limitations. WespecializedPrism-MWtoimplementthecorecomponentsofGLIDEshown in Fig. 3.4. Our first objective was to retain the key properties and services of Prism-MWandprovidebasicgridservices(suchasresourcediscoveryanddescrip- tion, search and retrieval) across dynamic and mobile virtual organizations. Addi- tionally, we desired GLIDE to support architecture-based design, implementation, deployment, and evolution of data-intensive grid applications in DREAM envi- ronments. Finally, we desired that GLIDE at least partially interoperate with a heavy-weight grid counterpart: because of our prior experience with the OODT middleware, it seemed the most appropriate choice; indeed, OODT directly influ- encedourdesignofthekeygridservicesprovidedbyGLIDE.GLIDE’sarchitecture in large part mirrors directly the OODT architectural components and connectors described above. 2 2 There is one lone exception. A careful examination of the GLIDE architecture shows the handlerconnectorsas components. Thisisnotanoversightandmoresoduetoourunderstanding of the OODT architecture at the time, which has since evolved. The handler components in GLIDE indeed directly correspond to their connector counterparts in OODT. 59 Each GLIDE processing component was implemented by subclassing Prism- MW’s ExtensibleComponent class, using the asynchronous mode of operation. Asynchronous interaction directly resulted in lower coupling among GLIDE’s pro- cessing components. For example the dependency relationships between GLIDE’s Client and Server processing components, which existed in OODT, are removed. GLIDE’scomponentsusePrism-MW’sEventstoexchangemessages. GLIDEdata components are sent between processing components by encapsulating them as parameters in Prism-MW Events. Leveraging Prism-MW’s Events to send and receive different types of data enables homogenous interaction among the process- ing components. We found OODT’s connectors not to be suitable for DREAM environments because of their heavy-weight (they are implemented using middleware such as RMI and CORBA). Furthermore, they only support synchronous interaction, which is difficult to effect in highly decentralized and mobile systems characterized by unreliable network links. To this end, we have leveraged Prism-MW’s asyn- chronous connectors to implement the messaging layer class in GLIDE. GLIDE’s connector leverages Prism-MW’s Port objects that allow easy addition or removal of TCP/ IP connections, thus allowing the system’s topology to be adapted at runtime. GLIDE’s connector also implements event filtering such that only the requesting client receives responses from the server. The high degree of decoupling among GLIDE messaging layer components directly aids easy dynamic adaptation, the lack of which is a key limitation in current grid systems. Ability to easily adapt a system’s software architecture is an important property missing in OODT that can be leveraged to improve the sys- tem’s functionality, scalability, availability, latency, and so on. Our recent studies 60 [MMRM05] have shown that the availability and latency of software systems in DREAM environments can be improved significantly via dynamic adaptation. Finally, to support interoperability of GLIDE with OODT, we provide two additional utility classes: XMLProfileReader and XMLQueryObjReader parse an XML representation of a resource profile and query object data structure, respec- tively. Each string is parsed into a GLIDE data object. Similarly, resource profiles and query objects can be serialized into XML. Thus, the level of interoperabil- ity with OODT is at the level of resource description and retrieval, and currently resource profiles and data can be exchanged between GLIDE and OODT. We also investigated the Web Services Resource Framework (WS-RF) as a means of enabling interoperability between GLIDE and Globus, which uses WS-RF in its latest release. 3.2.1 Mobile Media Sharing System Data distribution and dissemination has become an important part of everyday life, and is not entirely specialized to the representative science data systems dis- cussed within the context of the OODT project above. Examples of representative data distribution activities in everyday life include music file (mp3) sharing, online streaming movie distribution, and movie sharing sites, such as YouTube [You07]. Each of these data dissemination and sharing systems has a comparable architec- ture to those of the three OODT systems described above. As an example, consider the architecture of a Mobile Media Sharing system (MMS)builtusingGLIDE,showninFig. 3.5. MMSallowsausertoquery, search, locate, and retrieve MP3 resources across a set of mobile, distributed, resource- constrained devices. Users query mobile media servers for MP3 files by specifying 61 Figure 3.5: Mobile Media Application Architecture. values for genre and quality of the MP3 (described below), and if found, the MP3s are streamed asychronously to the requesting mobile media client. Fig. 3.5 shows the overall distributed architecture of the MMS application. A mobile device can act as a server, a client, or both. Mobile Media Server and Mobile Media Client correspond to the parts of the application that are running on the server and the client devices. MobileMediaClient contains a single component called MediaQueryGUI, which provides a GUI for creating MP3 queries. MP3 queries use two query parameters, 62 MP3.Genre (e.g., rock) and MP3.Quality (e.g., 192 kb/s, 128 kb/s). MediaQue- ryGUI is attached to a QueryConn, which is an instance of GLIDE’s messaging layer connector that forwards the queries to remote servers and responses back to the clients. MobileMediaServer is composed of three component types: MediaQueryServer, MediaProductServer, and MediaProfileServer. MediaQueryServer parses the query received from the client, retrieves the resource profiles that match the query from MediaProfileServer, retrieves the mp3 file(s) in which the user was interested from the MediaProductServer, and sends the MP3 file(s) back to the client. Thereareseveralrepresentativescenariosindicativeofthedistributionofmod- ern day media. We will discuss seven of them that we have found within our work on the MMS, and additionally other scenarios that are indicative of those that one would encounter in everyday, media-enabled life. S6 A user of the MMS system above would like to share (with the outside world) an entire library of MP3 files totaling between 600 MB and 1 GB of data, stored locally on the user’s PDA. The data needs to be distributed to 1000s of users, across a WAN, using anywhere between 1 to 100 delivery intervals of .3 MB per interval, to minimize resource consumption on the mobile host, and to take advantage of the MMS application’s capability for incremental data transmission. The data is delivered to other MMS-enabled systems, using a port-based, firewall pass through security mechanism. Scalability of the transfer must be maximized. S7 Awi-fienabledmusicserverrunningatamajorphonecompanyneedstodeliver different versions of a single MP3 file from a centralized file store to 1000s of visitors who have wi-fi enabled PDA phones, that are running a version of theMMS,distributedacrosstheUnitedStatesaspartofapromotionaloffer. 63 The different versions of the MP3 file correspond to different qualities (e.g., 128kbps,256kbps,etc.),andthefilerangesinsizefrom5MB(lowestquality) to10MB(highestquality). TheMP3filecanbestreamedovermanydelivery intervals(e.g., 1to100), witheachPDAphoneabletoreceiveatmost.3MB of data per streaming data delivery. Users can specify their preference for the quality of the MP3 file through the media query GUI running on their phones. The MP3 files are delivered using a simple port-based firewall pass- through capability, running on each phone. Understandably, because the data is being delivered over unreliable networks, consistency, dependability, and scalability may be sacrificed to increase efficiency. S15 A movie distribution site running a variant of the MMS targeted at movies ratherthanmusic,andindustrialqualitycomputeservicesratherthanPDAs, needs to distribute all eight seasons (5 4.7 GB of data per disc, and four disc per season) of a popular tv series to its users. There are at least five different types of the 1000s of users who would like the series delivered to them, and such user types are identified by the genre of movies that they have listed in their preferences for the site. The data can be delivered to their desktop computers periodically over 1 to 100 delivery intervals, across the public WAN.Thedeliveryofthedataisbasedonauser’sauthenticationcredentials with the site. Dependability, scalability and efficiency of the transfer should be maximized. S21 The same movie distribution site needs a one-time backup and replication of its greater than 10 TBs of data and metadata to be delivered to 1000s of storagenodesthatcompriseaninternallong-termarchivemachine. Thetotal volumeofdatashouldbedeliveredinasingleinterval,acrossaLANfromthe 64 online production movie catalog to each node. Consistency, dependability, and scalability should maximized. S22 A major record company needs to stream a 50 GB rap music video file to 50 users previewing the video after its initial production. The data should be sent across a dedicated, private WAN in 50 intervals of size 1 GB to each user. The music video is tagged with production notes, and other metadata added by its director. Efficiency of the transfer should be maximized as the preview of the music video, and subsequent discussions about its post- production should not take more than a few hours. S25 A major software company needs to deliver an operating system patch fix totaling50GBtoall20000employeesinanorganization. Thepatchconsists of mostly registry bug fixes. The patch is sent in 5 highly consistent, 10 GB intervals, across an internal LAN. Consistency, dependability, scalability and efficiency are all important and should be maximized. S27 A local traffic website needs to dynamically load and subset 50 GB of map data describing the Southern California Inland Empire for use by 20 users using the online trip planning capability of the web site. The data is to be deliveredin5intervalsof10GB,acrossthepublicWANtothedesktopcom- puters of each user. Consistency of the data transfer should be maximized as the website is praised for its high accuracy in predicting the optimal routes for its users. 65 Chapter 4 Understanding Distribution Connectors Acriticalpre-requisiteforappropriatelyidentifying, selecting, andevaluatingdata distribution connectors is developing a detailed understanding of the connectors’ underlying characteristics. Mehta et al.’s taxonomy of software connectors, dis- cussed in Chapter 2, provides us with unique facilities for accomplishing this task. Employing our experience in deploying, implementing, and evaluating data distri- bution technologies in the context of our work at NASA’s Jet Propulsion Labo- ratory, and by using this taxonomy, we have identified four major classes of dis- tribution connectors that can be used in data distribution scenarios: event-based, grid-based, P2P-based and client/server-based. Several implementations of these connector classes are widely deployed and pervasive in large-scale data distribu- tion tasks in research, industry, and government projects. Using Mehta et al.’s taxonomy we have identified the basic sequence of actions that define each distribution connector. Distribution connectors are different com- binations of six of the eight connector types defined in the taxonomy. The exact combination of the six varies across the different distribution connector classes, but in general, each distribution connector performs some form of Data Access, involving a Stream-based reading (and packaging) of data, and Distribution of the data to end users. Some connector classes are invoked via Procedure calls (e.g., client/server-based) while others are invoked via Events (e.g., Event-based) 66 Invocation Arbitrator Event Procedure Call Data Access Stream Distributor Legend Atomic Connector Type engages in packages data into distributed by 1 * * 1 * * relationship generalization Figure 4.1: General Model for Distribution Connectors or Arbitration (e.g., P2P). This understanding is depicted graphically in Fig. 4.1. Cardinality is indicated accordingly on the relationship arrows, where * refers to a cardinality of many while 1 indicates a cardinality of one. In this chapter, we provide the results of our research on classifying the event, grid, P2P, and client/server-based connector classes below, describing examples of each, and highlighting the dimensions from Mehta et al.’s taxonomy leveraged in the classification. 4.1 Event-based Distribution Connectors The event-based family of distribution connectors sends and receives data through asynchronous notifications called events. Events may arrive according to some fixedperiodicscheduleoratdiscreteaperiodicintervals. Therearemanyproducers and consumers of data in event-based distribution connectors (e.g., science data servers (producers) that alert scientists (consumers) of the availability of new data sets). Event-based distribution connectors often employ an asynchronous best 67 Distributor Stream Data Access Event No Yes Done Receiving Events? Done Retrieving Data? Yes No Figure 4.2: Event-based distribution connector activity diagram effort delivery method for data, making no guarantees as to the completion or time of data deliveries. Data delivery events sent through the connectors can be prioritized, tailored to different use cases or deployment environments, user specified, or administratively controlled. Events can be delivered using prioritized threaded queues, locally or remotely managed, or delivered using user registered preferences and the publish-subscribe delivery mechanism. Event-based distribution connectors can access and mutate data at both the producer and the consumer end. Event-based distribution connectors themselves access transient and persistent data, both public and private, communicating with transient session-based stores such as shared memory stores (e.g., Apache’s Derby project [Der07]), and also persistent stores such as repositories, databases, file systems, and Web pages. After data access by event-based distribution connectors, data is typically packaged into streams, both structured and raw, using a best effort approach, with bounded packet size and data buffering. Streams can be identified via named uniform resource identifiers (URIs) [BLFM98], and constructed using the 68 asynchronous mode of operation. Streams may be constructed either locally or remotely, depending upon how the data was accessed. Stream based data is distributed from data producers to data consumers using a naming registry, via either a hierarchical or flat-based naming model. Data streams are delivered as events using best effort delivery, and any combination of the unicast, broadcast, or multicast delivery mechanisms. Routing of events is typically determined by the specific network layer of the connector’s deployment environment. A graphical summary of this description of event-based distribution connectors is shown in Fig. 4.2. Example instances of the event-based distribution connector class include the Siena Publish Subscribe middleware [CRW01], the PRISM-MW middleware [MMRM05] and its subsequent extensions allowing grid resource location and dis- covery called GLIDE [MMB + 05], developed as part of our prior work. 4.2 Grid-based Distribution Connectors The grid-based family of distribution connectors moves and delivers large amounts of data between software components deployed in a grid environment: a virtual network of shared computing and data resources. Grid-based distribution connec- tors are invoked via a named, synchronous procedure call, often as a web service call sent using SOAP. User authentication credentials are provided to the connec- tor for integration with the Grid Security Infrastructure (GSI), a highly secure toolkit based on keypairs and certificate authorities. URLs sent via the connector invocation describe where the data is, and where it is to be sent (the consumers). Parameters are passed by value using “keyword equals value” semantics, with con- trol messages being logged to the Grid API log layer. 69 Distributor Stream Data Access Procedure Call Done Retrieving Data? Yes No Figure 4.3: Grid-based distribution connector activity diagram Grid-based distribution connectors access and mutate transient and persistent data, both private and public, as long as there is some type of standard API (e.g., dynamic data exchange over XML, a repository access, or file I/O) for accessing it. Data access through a grid connector is typically packaged as a stream of bytes (or blocks, configured via a parameter) with exactly once delivery, and (config- urable) bounded buffering provided by the TCP/IP level. Data can be structured or raw. Buffering and throughput of data are parallelized via multiple concurrent time-out synchronous TCP/IP streams. Streams are named stateful URLs and URIs, and can exist both locally or remotely, depending upon where the data was accessed. A stream can be delivered to many consumers. Grid-based distribution connectors distribute data using the grid environment for naming and location, including the Grid Metadata Catalog Service (MCS) and ReplicaLocationService(RLS)components, andnamedURLsandURIs. Naming can be hierarchical, with resources and locations sharing parent child relationships via the MCS. Alternatively, naming and discovery can be flat, structure-based. 70 Data is delivered via parallelized unicast exactly once using TCP/IP concurrent streams, flooding the underlying network to its saturation, making the most effi- cientuseoftheavailablebandwidthpossible. Reliabilitymeasuresallowredelivery of lost packets if necessary as well as partial data transfers, both sequential and out of order. Routing of data is handled by the underlying TCP/IP network layer. A graphical summary of this description of grid-based distribution connectors is shown in Fig. 4.3. Example instances of grid-based distribution connectors include the GridFTP software, bundled along with the Globus Grid Toolkit [Glo07], and the large files transfer protocol, or bbFTP for short [bbF07]. 4.3 Client/Server Distribution Connectors The client/server-based family of distribution connectors allows seamless distribu- tion of data between distributed systems using RPC. Client/server-based distribu- tion connectors are invoked via a synchronous remote procedure call that appears (to the consumers of data) as if it were a local method call. Keyword (named) parameters are passed to the method call to specify the distribution constraints, with parameters being passed by value or reference (depending upon the under- lying RPC implementation). Methods can have return values, which typically must conform to a standard set of primitive data types that are supported on all machines involved in the data distribution. Methods are interactions between one sender and one receiver, typically the producer of the data and its consumer, respectively. Upon invocation, the client/server-based distribution connector engages in a data access stage, accessing and mutating persistent and transient data. 71 Distributor Stream Data Access Procedure Call Done Retrieving Data? Yes No Figure 4.4: Client/server-based distribution connector activity diagram After data is accessed, it is packaged into streams that are delivered using exactlyoncedeliveryandboundedpacketsizemanagedbytheunderlyingTCP/IP protocol. Datacanberaworstructuredinthestream. Streamthroughputismea- sured in atomic units (bits per second), and streams are identified using stateful, named URNs. Streams can be local or remote, depending on where the data was accessed or packaged. Data is distributed from client/server-based distribution connectors using a naming registry to locate the requesting consumer of data. Data is delivered through method (or interface) parameters and return values. It is sent exactly once, using the unicast delivery mechanism. Routing is performed by the underly- ing TCP/IP network using static membership identified via a naming registry. A graphical summary of this description of client/server-based distribution connec- tors is shown in Fig. 4.4. Example instances of client/server-based distribution connectors include HTTP/REST, RMI, CORBA, SCP, FTP, UFTP, SOAP, and many commercial UDP technologies. 72 4.4 P2P Distribution Connectors TheP2Pfamilyofdistributionconnectorsisquiteunlikeanyoftheotheridentified classes. Rather than providing some sort of initial procedural or event-based invo- cation, P2P-based distribution connectors typically rely on arbitration as a means of synchronization and invocation. Arbitration involves control flow redirection between distributed resources, or peers, operating in a networked environment. Arbitrators can negotiate protocols and scheduling and timing issues. Fault han- dling and parameter passing is handled in an egalitaran fashion via voting and point-to-point or point-to-many communication between peers or groups of peers. P2P-based distribution connectors use rendezvous as a mechanism to achieve con- currency and scheduling. Transaction support is often available, and invocation can be rolled back if necessary. Upon invocation, peers in a P2P-based distribution connector engage in a data access stage, accessing transient (both process and thread specific) and persistent (e.g., repository, file I/O, etc.) data. Since peers can come and go in a P2P scenario, for the most part data access is transient in nature: a peer need not stick around for the entire distribution. Data is accessed and packaged via streams, using a best-effort bounded mech- anism. Data can be structured or raw, as long as it can be organized into identifi- able chunks that can be retransmitted, and shared between peers handled by the connector. Streams are named URNs typically identified using some hashing algo- rithm (e.g., SHA-1 or MD5 [SHA07]), and are available both locally and remotely depending upon where the data was accessed or packaged. Data is distributed in the P2P-based distribution connectors by locating other peers that have pieces of data that a user would like to distribute or obtain. Loca- tionofotherpeersisbaseduponattributessuchasresourcetype,SHA-1,andother 73 Distributor Stream Data Access Arbitrator Done Retrieving Data? Yes No Figure 4.5: P2P-based distribution connector activity diagram domain-specific attribute metadata (e.g., for movie files “product company” might be used, or for music “Artist Name”). Delivery occurs in the form of chunks, sent to and from other peers using best effort or at least once semantics over unicast and multicast transmission channels. Routing, while influenced by the underlying network layer, is handled by some sort of tracking mechanism, sometimes called trackers or super peers. Tracking mechanisms allow location of chunks of data streams that need to be distributed or obtained in a P2P connector. A graphical summary of this description of P2P-based distribution connectors is shown in Fig. 4.5. Example instances of the P2P distribution connector class include Bittorrent [Coh03] and JXTA based data distributors (such as the Peerdata project [OOD05] at JPL). 74 Chapter 5 The Distribution Connector Selection Problem As alluded to in Chapter 1, and as illustrated in Chapter 3, there is a wealth of different scenarios in which data is distributed. Recall: • a large volume of data (e.g., on the order of a terabyte) of a single type may need to be delivered over the WAN to a small number of users, possibly across multiple delivery intervals; • a similar volume of data may need to be delivered to a different set of users over the LAN in a single delivery interval; • a medium volume of data (e.g., on the order of 10 gigabytes) of different types may need to be delivered to many users and user types; • a smaller volume of data (e.g., on the order of a 100 megabytes) may need to be delivered across a WAN to a very large number of users of the same type, but with stringent performance requirements; and so forth. Such scenarios are called data distribution scenarios: scenarios that describe the distribution of data from producers of data to consumers of it. The underlined keywords in the above distribution scenarios are not incidental; they are meant to highlight the observation that common vocabulary to phrase distribution sce- narios does exist. While important descriptors such as total volume (of data to be 75 Data Distribution Delivery Schedule Performance Requirements Number of Intervals Volume Per Interval Timing of Interval Consistency Scalability Dependability Efficiency Access Policies Geographic Distribution Number of Data Types Total Volume WAN LAN Number of Users Number of User Types Producers Consumers Automatic Initiated Automatic Initiated Types of Data Data Metadata Figure 5.1: Distribution scenario problem space transferred), and number of user types (to deliver data to) are repeatedly found in descriptions of data distribution scenarios, both in literature and in practice, to our knowledge, the vocabulary has never been appropriately documented nor discussed within the context of the selection of software connectors for data dis- tribution. This implied vocabulary forms the basis for a language for describing distribution scenarios. In this chapter, we define such a language. We follow by defining formally the problem of selecting the appropriate distribution connectors that satisfy data distribution scenarios, phrased using our scenario language, and specified by a user or by a program. This chapter provides the rigorous underpin- nings of distribution connector selection, highlighting its difficulties, exploring the (sometimes conflicting) relationships between distribution scenario vocabulary (or dimensions), and the available connectors for large scale data distribution. 76 5.1 Problem Dimensions To capture the salient aspects of the data distribution problem space, we have identified eight key dimensions of data distribution (shown as the first column of terms in Fig. 5.1). The eight dimensions are based on a thorough review of existing data distribution literature incl. [CHK01, FZ97, YGM95, cFG00, NDI04, BMP + 04, Bal97] and on our own experience in the context of building systems that distribute data within the domains of planetary science and cancer research at JPL. A set of constraints (discussed below) on the combinations of these eight dimensions identifies a data distribution scenario. Although we acknowledge the existence of other dimensions of data distribution (e.g., unicast/multicast, aperi- odic/periodic delivery, see Franklin and Zdonik [FZ97] for more information), we will demonstrate in this dissertation that our eight dimensions are effective and accurate at modeling the data distribution problem space. A data distribution scenario is expressed as a constraint query against the dimensions shown in Fig. 5.1. As an example, consider the following constraint queries (i.e., distribution scenarios) that (roughly) correspond to the four postu- lated scenarios indicated above: TotalVolume = 1 TB ∧NumDataTypes = 1 ∧NumUsers≤ 10∧ GeographicDistribution = WAN∧ DeliverySchedule.NumIntervals≥ 1 (5.1) TotalVolume = 1 TB ∧NumDataTypes = 1∧ GeographicDistribution = LAN ∧NumUsers≤ 10∧ DeliverySchedule.NumIntervals = 1∧NumUserTypes≥ 1 (5.2) 77 TotalVolume = 10 GB ∧NumUsers≥ 10000 ∧NumUserTypes≥ 1000∧ NumDataTypes> 1(5.3) TotalVolume = 100 MB ∧NumUsers≥ 10000 ∧NumUserTypes≥ 1000∧GeographicDistribution = WAN∧ Performance.Consistency = 10∧Performance.Efficiencey = 10 (5.4) As can be surmized from the above examples, distribution scenarios can be expressedusingrangedanddiscretevalues,dependinguponwhetherthedimension is a number (e.g., TotalVolume), or a string (e.g., GeographicDistribution). In our dissertation work, we have found this representation to be sufficiently expressive. It was derived from direct contact with scientists in different domains at JPL, and from a review of the available literature on data distribution. The eight dimensions of data distribution and our understanding of them are expounded upon below. We explain each dimension, its relationship to the others, and briefly comment on how each dimension influences connector selection. 5.1.1 Total Volume Total volume refers to the total amount of data that needs to be transferred from providers (or sources) of data to consumers of data. It ranges from small volume transfers (typically CD size — 700 megabytes to DVD size — 4.7 gigabytes) to medium volume transfers (typically storage brick size, hundreds of gigabytes) to large volume transfers (on the order of terabytes of data or more). Each class of distribution connectors is inherently designed to handle a given total volume of data. For example, client/server-based distribution connectors, 78 relying on the client/server interaction mechanism, are often more suited for low- to-medium volumes of data because their communication mechanism induces a lot ofoverheadontheactualdatatransfer. Ontheotherhand,P2P-baseddistribution connectorsregularlydealwithlargevolumesofdatabecausetheyemployadvanced caching and partial transfer mechanisms that allow the connection overhead to be distributed across nodes in the system in an egalitarian fashion. Grid-based connectors have proven to be very capable of transferring the larger volumes of data (terabytes to petabytes) as they carefully exploit the available underlying bandwidth normally underutilized by client/server connectors. 5.1.2 Delivery Schedule Delivery schedule refers to the number, size, and frequency (timing) of intervals within which the total volume of data should be delivered. For example, content providers such as on-line movie distribution sites may deliver data to interested registered users in two daily intervals (the number of intervals), once at 6:00 AM, and once again at 11:00 PM (one component of the timing constraints on the interval), to take advantage of the potential ebb in bandwidth use on the internet at those times. Each interval may deliver two new DVD-sized movies for the monthinsuccession(anothercomponentofthetimingconstraintsontheinterval), each consisting of around 4.7 gigabtyes of data per scheduled delivery (the size of each delivery interval). On the other hand, a delivery schedule may be initiated by a consumer of data, and it may involve downloading an mp3 file (around 5 megabytes of data) from an online music distribution site. The delivery would consist of one interval, initiated at 3:00 PM and completed at 3:05 PM, when the online distribution server finishes sending a single mp3 file to the consumer’s web client. 79 Judging from the results of our research of distribution connectors, only cer- tain classes of connectors are particularly well suited to deal with different types of delivery intervals. For example, event-based and P2P distribution connectors handle periodic low-to-medium volume intervals set over an un-bounded range of time very well, because they are able to handle the uncertainty of when data will be “pushed” from providers to consumers of data. Additionally, event-based and P2Pconnectorsareabletosenddatainmanydeliveryintervalstocutdownonthe amount of bandwidth used at one time and increase the efficiency of the transfer by spreading the workload. Client/server and grid-based distribution connectors aremoreideallytailoredforaperiodic, consumerinitiateddeliveryscheduleswhere consumers of data come and go without a known fixed interval, time, or amount of data. 5.1.3 Performance Requirements Performance requirements represent any constraints on the scalability, efficiency, consistency, and dependability of the distribution scenario. Typically performance requirements are used to quantifiably evaluate connectors for use in distribution scenarios and ensure that data is delivered on time, using as much bandwidth as possible, that data delivered is data sent, and so forth. We have tried to capture the most desirable (and useful) performance requirements for data distribution scenarios in this dimension. Scalabilityreferstothenumberofnodesinadatadistributionscenario,embod- ied in the number of users (consumers and providers). Scalability also refers to the total volume predictions for the distribution scenario, and the ability of the employed distribution connectors to handle increasing data volumes. 80 Efficiency captures information such as the rate at which the data is being sent (e.g., throughput, measured in bits per second, or some other higher order unit), as well as the memory usage of the distribution connector itself. Consistency captures whether data sent in a distribution scenario is the same as the data delivered. Different distribution connectors may provide different con- sistency capabilities. For example, any distribution connector that is a member of the event-based connector family may involve asynchronous communication where data packets are sent without any guarantee that they will reach their destination. Dependability captures the number of faults per given interval that occur as induced by the connector’s behavior (e.g., under its nominal operation) during data distribution. Dependability also refers to the uptime of the connector during data distribution. Data distribution connector classes offer trade-offs among the performance dimensions. Some connector classes, such as event-based and P2P-based, are highlyscalableandefficientbutneglectdependabilityandconsistency. Conversely, connector classes such as client/server are highly dependable and consistent, but perhaps at the expense of efficiency and scalability. 5.1.4 Number of Users The number of users in a data distribution scenario refers to the unique users that the data volume needs to be delivered to (the consumers), and the unique users who are delivering the data (the producers). The number of users frames different elements of the distribution scenario’s scalability. It influences the number of user types in a distribution scenario, along with the geographic distribution of the user population, the total volume of data involved in the distribution (more users will requiremoredata),andthedeliveryscheduleusedtotransferthedatatotheusers. 81 Distributionconnectorclassesparticularlysensitivetoscalabilityinthenumber of users include the client/server-based family. In general, the remaining three distribution connector families are well suited to scaling up in the number of users in a distribution scenario. 5.1.5 Number of User Types The number of user types refers to the unique types of users, such as scientists, students, educators, or project managers to whom the data volume needs to be delivered (i.e., the consumers), along with the unique types of data producers, such as scientific instruments, content publishers (e.g., news media, content man- agement systems), movie distributors, and distribution systems. In a data distribution scenario, types of producers and consumers of data can be further classified into automatic or initiated. Automatic producers of data are distribution systems that need no formal invocation in order to initiate a data delivery. An online movie provider website or a scientific instrument that gen- erates science data at regular observation intervals could be classified into this category. Initiated producers of data are explicitly invoked to begin a data distri- bution. Examples of data producer types that fall into this category are content management systems, news media providers, even scientific instruments that sup- port manual intervention or control by an operator. Ontheotherhand,automaticconsumersofdatarequirenomanualnotification of data delivery — they go out and obtain the data themselves, or they provide a standard interface to enable data to be delivered to them without intervention (e.g., rendezvous). Examples of automatic data consumers include RSS feed lis- tenersthatcanbeautomaticallynotifiedofupdatestonewsstories, aswellasRSS feed aggregators that automatically pull new news feeds and channels. Initiated 82 consumers of data require an invocation to begin receiving data from a data dis- tribution. A wide spread example of an initiated data consumer is a Web client (e.g., a browser). All of the distribution connector class families are sensitive to the number of user types in a distribution scenario. Event-based and P2P connectors are particularly well suited to deal with automatic data consumers and initiated data producers (e.g., making a file available in a P2P network, or sending an event such as an email), while client/server-based connectors are not well suited for automatic data production and consumption, but excel at initiated production and consumption. Grid-based connectors are scriptable and support standard APIs, and thus have a wealth of capability for both automatic and initiated data production and consumption. 5.1.6 Data Types The data types in a distribution scenario refer to the number of different types of data that are part of the total volume to be delivered. Data types can be repre- sentedconceptuallyasobjectorienteddatamodels,richtaxonomiesandontologies, and carefully crafted object models. Data types define an object’s data elements and facets, and their syntactic and semantic interrelationships. The amount and types of data transferred in a data distribution can have a significant impact on the overall scenario. Data types can dictate delivery schedule, influence the total volume (different types may represent different portions of the total data volume), induce different access policies (e.g., certain data may require secure access while otherdatamaynot), andrequiredifferentperformancerequirements(e.g., because some data types may be decidedly smaller than others, it may benefit the distri- bution if they are transferred sooner rather than later), and so on. 83 Client/server-based and grid-based distribution connectors are somewhat inde- pendent of the types of data that they distribute, while P2P and event-based distribution connectors used in content-based network systems (such as publish- subscribe) are heavily influenced by the data types present in the network. 5.1.7 Geographic Distribution The geographic distribution of the data providers and consumers can generally be classified into two major types: WAN-based and LAN-based. WAN-based distri- butioncanfurtherbebrokendownintonationalorinternational,whileLAN-based distribution is restricted to hosts in a local area, within the same organization. Geographicdistributionhasanenormousimpactontheactualdatadistribution scenario. It influences the efficiency and scalability of the transfer, as the network latencies are heavily dependent on the actual physical distribution between the hosts, and whether the distribution spans countries and continents, or must cross a satellite link along the way. It may influence the delivery schedule (as consumers and producers of data that span countries may be in different time zones), as well as the users and user types. Highly efficient distribution connector classes are less affected by geographic distribution than others: P2P and grid-based distribution connectors typically perform with little noticeable effect (other than transfer time) of the geographic distribution; on the other hand, some event-based and most client/server-based connectors are typically affected by the underlying geographic distribution of the transfer. 84 5.1.8 Access Policies Access policies refer to the number and types of such policies in place at each producer and consumer of data. The access policy can have a large impact on a datadistributionscenario: itcanblockcertaindatachannelpathsandopenothers; itcanalsoexcludecertainusertypeswhileallowingothers. Accesspoliciesliterally entail methods of accessing data, such as a secure HTTP-based Web service or an unauthenticatedDBMSquery. Accesspoliciesarepresentattheactualinstitutions where the consumers and producers of data reside, and may also be part of the distribution connectors used in the scenario. All four data distribution connector families can support arbitrary access poli- cies. The ability to support different policies may vary from one specific connector technology to another, even within the same category. 5.2 Problem Definition We utilize the scenario dimensions defined in sections 5.1.1-5.1.8 and constraint queries against them to represent a distribution scenario, s, from the space S of possible distribution scenarios. We define the distribution connector selection problem formally as: Given: D, set of distribution connectors; s, distribution scenario; s∈S a function f :D x S→Q, s.t. f(c,s) = suitability of a connector c∈D to s∈S Find: max ∀c∈D f(c,s), the connector c with maximum suitability to s 85 It should be noted that the connector c with maximum suitability to s need not be the only best connector for the scenario. Because the function f computes a suitability value between 0 and 1, for each connector known, we can sort the set of connectors D, based on f(c,s), and produce a ranking of all the connectors in D. This ranking allows a user to iteratively discover the appropriate connector(s) for a scenario s in a principled fashion, rather than selecting connectors randomly. In other words, even if c 1 is initially chosen based on f, due to reasons outside of the scope of the encoded information about the connector and the scenario (e.g., organizational politics, “pet technologies”, etc.), c 2 may be chosen, or c 3 , and so on. In practice, we have observed that there are in fact a number of connector technologies that are appropriate for a given scenario s, and a number of them that are not. Such a division makes the connector ranking process a valuable tool in deciding the appropriate connector(s) for s. Anotherimportantobservationisthatifwecouldeasilycomputef,theproblem will trivially reduce to sorting D by f(c,s), and then taking c 1 from that sorted set. The most difficult aspect of solving the problem is computing f. As we will show in this dissertation, f must produce highly accurate connector ranking lists, in order to be truly useful. f must also effectively encode the mapping betweenc∈D andthedifferentrequirementsofthedistributionscenario, encoded by s∈ S. Deducing such a mapping is a non-trivial problem, with a number of different conflicting goals that must be carefully negotiated (e.g., maximize performance, ensure connector can access data from a flat file system, maximize consistency, etc.). We will see, though, that accurate and flexible representation of the mapping affords an organization the ability to capture an expert’s (i.e. organizational guru’s as can be recalled from Chapter 1) domain knowledge, and update it as the understanding of the connectors, and the scenarios themselves 86 evolves. Ultimately, this is a core component required to alleviate the problems described in the introductory chapter. 87 Chapter 6 DISCO: A Framework for Software Connector Classification and Selection The principal component of our dissertation research has been the development of a software framework for automatic connector classification, selection, and evalua- tion. The framework, dubbed DISCO (for Data-Intensive Software COnnectors), was designed to provide automatic decision making facilities for the distribution connector selection problem that is the motivation for this work. In this chapter, we discuss DISCO in its entirety. We begin with a motiva- tional example of manually selecting the appropriate distribution connector for a large-scale movie data dissemination system. The motivational example aids in highlighting that such a manual process is unwieldy and that there is a need for a framework such as DISCO. We follow our example with the presentation of DISCO’s six principal objectives. The objectives are used to discuss the DISCO framework. Each major component in the framework is described: (1) the con- structionofadistributionconnectorknowledgebase, containingmetadatadescrip- tionsofthekeypropertiesofeachconnector,(2)themotivationanddevelopmentof three different connector selection approaches, and their benefits and tradeoffs, (3) themotivationanddevelopmentofanextensibleanalysisapproachfordetermining howpreciseandaccuratetheconnectorselectionalgorithmsare,andfinally(4)the 88 Digital Movie Archive Digital Movie Repository Movie Data Access API Digital Movie Catalog Movie Metadata API Interested Users Backup Site Digital Movie Repository/ Catalog Legend Data Distribution Connector Data Distribution Connector Data Distribution Connector Decision Point data flow Repository/ Catalog Data Producer/ Consumer Figure 6.1: High level architecture of a movie distribution system. use of performance data in the areas of efficiency, consistency, dependability and scalability to further validate connector selections made by the three algorithms that we developed. Throughout the chapter, we discuss how the conglomerate of DISCO’s four components fulfills the six key objectives of the framework. 6.1 Motivating Example To motivate DISCO’s automated connector selection capability, we will examine a postulated scenario involving the construction of a movie distribution system. This example will help to illustrate the fact that the consequences for improperly selecting a connector are substantial. The representative architecture of the movie distribution system is shown in Fig. 6.1. The digital movie archive consists of a DigitalMovieRepository that stores the physical movie files in AVI or MPEG format, a DigitalMovieCatalog that stores metadata information about a movie, including information about its genre, director, and cast. The repository and 89 catalog are fronted by APIs that allow access to the both the movie data and its metadata. A BackupSite periodically connects to the DigitalMovieRepository to backupitsmoviedataandmetadata. InterestedUsersaretheobservablecustomers of the system: they expect the ability to both download movies at their own request, and have movies delivered to them by subscription mechanisms using the movie metadata as criteria for inclusion. The primary advertised benefit of the movie distribution service is a feature that affords a user the ability to subscribe to genres of movies she is interested in, and have the movie delivered to her desktop incrementally whether she is physically present or not. Envisage that the architect building the system has decided by examining various trade articles, documentation, and mailing lists, that four data distribu- tion connectors are appropriate to consider given the requirements of the sys- tem: GridFTP, Bittorrent, HTTP/REST, and FTP. There are at least three distinct distribution scenarios that can be extrapolated from the architecture of the system. The first scenario (representing the movie backup activity) can be expressed in human readable fashion as periodic delivery of terabytes of data across the WAN to 1 user. In turn, this can be translated into DISCO using the constraint query, DeliverySchedule.NumIntervals = 1∧ TotalVolume > 1 TB∧GeographicDistribution = WAN∧NumUsers = 1. The other two sce- narios represent the aperiodic and periodic movie deliveries to InterestedUsers. Even with just three scenarios, manual connector selection becomes unweildy. A reasonable idea would be for the architect to choose the connector that allows her to handle all three distribution scenarios: a connector supporting (a)periodic delivery, terabytes of data, and thousands of users. In that case, documentation suggeststhatGridFTPsupportsallthreescenarios, exceptforaperiodicdeliveries. 90 Aperiodic deliveries are the cornerstone of the business model for the movie distri- bution company though, hence they must be supported. A quick examination of the actual scenario constraints leads the architect to realize that, without a clear waytoruleouttheotherthreeconnectorsbesidesGridFTP,foreachconnectorshe must, examine between 4 to 6 scenario constraints (up to eight depending upon the level of specificity), and determine how well (if at all) each connector handles the scenario as a whole. In the worst case, the architect for the movie distribution system will have to evaluate 32 scenario dimensions for the four connectors (eight scenario dimen- sions times four connectors), a seemingly small number, though there are several subtle complexities: (1) constructing a connector evaluation matrix comprising 32 requirements to check is a large enough decision matrix in its own right, and the addition of each connector into the decision problem introduces up to eight more requirements to check; (2) the satisfaction of the scenario requirements denoted by the columns in the matrix may be difficult to discern without extensive empirical evaluation; and (3) even if the architect is able to choose the right connector(s), the critical information of why and how the connector was deemed to satisfy the requirement remains inside the architect’s head, yielding yet another guru. 6.2 Objectives Motivational scenarios like the movie distribution system discussed above helped us to realize that any principled, automatic solution to the distribution connector decision process (described in its manual form above) must support six key objec- tives, whichwedescribebelow. TheseobjectivesarethecornerstoneoftheDISCO framework. 91 Objective 1 Theframeworkshouldallowforausertoperformconnectorselection for data distribution in a principled fashion, rapidly and accurately. Objective 2 The framework should provide a means for describing the critical properties of data distribution connectors — as we saw in Chapter 4, con- nector families and properties greatly influence their appropriateness in the face of different distribution scenarios. Objective 3 Theframeworkshouldallowanalysisofdifferentconnectorselection approaches, providing a means for comparing how accurate, and precise each selection approach is. Objective 4 Theframeworkshouldbeextensible,allowingtheusertodevelopher own connector selection algorithms, and analysis approaches. This is highly important since, to our knowledge, there have been no selection approaches developed explicitly for connector selection. Therefore it is imperative that the framework allow a user to evaluate several different selection approaches and examine their benefits and tradeoffs. Objective 5 The framework should provide a means for capturing and reproduc- ing the relevant domain knowledge typically held in the minds of organiza- tional data distribution “gurus”. Objective 6 The framework should not produce a single choice of “best” con- nector for a scenario. Rather, it should produce a set of appropriate con- nectors, and a set of inappropriate connectors. As we alluded to in Chapter 5, since there are often a number of different distribution connectors that are appropriate/inappropriate for a particular distribution scenario, rather 92 S S S DISCO Connector Performance Data Perforamnce Data KB Connector KB Selectors Connector Profiles Analysis S S R Figure 6.2: DISCO’s overall architecture. than a single best/worst choice, the framework should allow for comparison of connector appropriateness for distribution scenarios. In light of these objectives, we have developed the Data-Intensive Software COnnectors framework, or DISCO for short. DISCO is a highly accurate, extensi- ble and efficient software technology that allows evaluation and exploration of the distribution connector trade-off space. In the remainder of this chapter, we describe DISCO, focusing on the soft- ware components that comprise DISCO and on the critical data components that DISCO software components exchange and interpret. DISCO’s architecture is shown in Fig. 6.2. S indicates a distribution scenario, and R indicates an out- put precision/recall analysis result, generated by the analysis component. DISCO contains two separate knowledge bases, one for capturing architectural metadata 93 information described in Chapter 4, and another for capturing connector perfor- mance data. As we will see in the remainder of this chapter, the need for two separate knowledge bases is due to the distinction between two types of connector experts, and the variation between the information about the connectors that each expert provides. 6.3 Building the Connector Knowledge Base OurinitialstepinbuildingDISCOwastodevelopaknowledgebaseofdistribution connectordescriptions(shownastheConnectorKBcomponentintheuppermiddle portionofFig. 6.2). Thisaffordedusameansforhousingourunderstandingofthe key properties of each distribution connector that we studied, helping to address objectives 1 and 2 above. To build this knowledge base, we needed to determine theimportantmetadataforeachconnector. WeleveragedMehtaetal.’sconnector taxonomy [MMP00], and our understanding of distribution connectors detailed in Chapter 4, in support of this task. Mehta et al.’s taxonomy defines core dimensions of importance for atomic con- nectors: e.g., data access transient availability, whose definition is “whether data being accessed is transiently available, and if so, where you go to find it: the Heap, the system Stack, in a Register, etc”. According to Mehta et al., each of these dimensions of interest comprises a part of the description for one of the eight atomic connector types (recall Chapter 4). As explained in Chapter 4, recall that distributionconnectorsbelongtooneoffouridentifiedfamilies—grid,peer-to-peer, or P2P, client/server and event—and each of the four families involves some com- bination of six of the eight Mehta et al. atomic connector types. In addition, each 94 of these connector families induces its own set of properties. It is those properties that our connector descriptions needed to accurately model. Thefirststepinbuildingtheconnectorknowledgebasewastoutilizetheatomic connector dimensions as a guideline for describing the distribution technologies we studied. Theselectionofdimensionsfromthetaxonomyforeachatomicconnector type was not exhaustive however, and each dimension was carefully selected (or omitted) based upon two criteria: 1. its importance in describing the elements of data distribution, and 2. its understandability with respect to a connector expert being able to popu- late metadata for the dimension. As an example, consider the Mehta et al.’s dimension data access locality, which is a part of the Data Access atomic connector. For distribution connectors, accessing data is one of the key steps in distributing data, and data access locality defines the scope in which data can be accessed through a connector: its valid values are Process, Global, and Thread-specific. Such a dimension is wholly appro- priate for inclusion in our connector metadata, as distribution connectors require understanding of the locality in which their data accesors can get at the data. In addition, it is also quite reasonable to consider that a software connector expert would be able to annotate a distribution connector with one of the above three valid values. On the other hand, consider the Mehta et al. dimensiondataaccess lifecycle, indicating the initialization of the data access, occurring in either the ConstructorortheDestructoroftheconnector. Judgingthisdimensionagainstour criteria, we see that this dimension does not fit criterion 1 (importance); based on ourknowledgeofdistributionconnectors,thetraceabilityofdataconceptionwithin 95 Grid DCP Event DCP Client/ Server DCP P2P DCP bbFTP GridFTP HTTP/REST Bittorrent JXTA Siena describes describes DCP Schemas describes describes describes describes Figure 6.3: DCPs and what they describe. the connector has little effect on the connector’s applicability to a distribution sce- nario. 1 Furthermore, the dimension does not fit criteria 2 (understandability), in thesensethatitwouldbehighlydifficultforasoftwarearchitect, withoutintimate knowledge of the actual connector code, to provide values for it. Below, we provide the complete set of Mehta et al.’s dimensions selected to describe each connector family. We distinguish situations in which there can be manyvalues(1...∗)capturedforthedimensionversusasinglevalue(1). Weexplain the definition of each dimension, and its value range. We briefly point to reasons why the Mehta et al. dimensions (if any) were excluded. The selected dimensions form the crux of defining our model for instances of the distribution connector metadata, called Distribution Connector Profiles, described in the next section. 1 In other words, regardless of whether the data is created within the constructor of the data connectororthedestructorweassertthatthedimensionasawholedoesnotaffecttheconnector’s observable applicability to a distribution scenario 96 6.3.1 Distribution Connector Profiles Distributionconnectorprofiles(DCPs)aremetadataschemathatdescribeadistri- butionconnector. DCPsarecategorizedbyconnectorfamily,causingthemetadata that they store for a connector to vary, depending upon what family the connector described originates from, as illustrated in Fig. 6.3. As described above, the DCP metadata is derived from Mehta et al.’s atomic connector type dimensions. While it may appear from Fig. 6.3 that there is no metadata overlap amongst the classes thisisnotthecase. Onthecontrary, aswewilldescribebelow(andasdescribedin Chapter 4), all distribution connectors share the commonality of the data access, stream, and distributor atomic connector types, and all distribution connectors are invoked using one of three atomic connector invocation types: procedure call, event, or arbitration. The dimensions selected to describe the data access metadata for a distribu- tion connector are given in Table 6.1. The dimension data access lifecycle was omitted as it failed to meet either criterion 1 or 2 from above. One missing area is whether or not the data access mechanism is threaded, such as in the parallel/streaming grid connectors (e.g., bbFTP and GridFTP). However, we claim that in fact, there is no omission of such information as it is natively handled by the framework’s classification of connectors into different families (because bbFTP and GridFTP are grid connectors, inherently, we know that their data access capability is parallelized). The dimensions selected to describe the stream metadata for a distribution connectoraregiveninTable6.2. TheMehtaetal. dimensionstreamthroughput was not included as which failed to meet criterion 1. Stream throughput defines the unit of measurement employed to identify throughput in a connector (e.g., lower-level measurements such as bits per second, or higher-level measurements, 97 Table 6.1: Data Access DCP metadata. Dimension Description Values Locality (1) Refers to the scope of data access that is available within a distribution connector. Selecting “Process” indicates that the connector only has access to data available from within its process instance while “Global” indicates that the connector can access data externally as well. Process, Global Persistance (1...*) The data access persistence denotes the available persistence mechanisms from which a distribution connector can access data. Dynamic Data Exchange, Database Access, Repository Access, File I/O Transient Availability (1) Denotes whether data being accessed is transiently available, and if so, where you go to find it: the Heap, the system Stack, in a Register, etc Session-based, Peer-based, Cache Receiver Cardinality (1) Denotes the number of receivers of data accessed by this connector. One, Many Accesses (1...*) Denotes whether or not the connector has read-only permission (accessor), or write permission (mutator) on the data it accesses. Accessor, Mutator Sender Cardinality (1) Denotes the number of senders of data accessed by this connector. One, Many such as transactions per hour/etc.). All other dimensions fit criteria 1 and 2 from above, and thus were included. The dimensions selected to describe the distributor metadata for a distribution connector are given in Table 6.3. The distributor metadata in general describes characteristics of the underlying path used to distribute the data. All Mehta et al. dimensions fit criteria 1 and 2 from above, and thus were included. The metadata given in Tables 6.1, 6.2, and 6.3 are common to all distribution connectors. Depending upon the connector family and invocation method, how- ever, there is also domain specific connector metadata describing the invocation for the distribution connector (recall Chapter 4), as we will detail below. Such metadata is based on Mehta et al. ’s atomic connector types procedure call, event, and arbitration. 98 Table 6.2: Stream DCP metadata. Dimension Description Values Formats (1...*) Denotes the organization of data sent through, and accessed by, a distribution connector. Raw data has no principled organization other than a sequential set of bytes. Structured data may be organized, delineated, and packaged into formal data units. Raw, Struc- tured Sender Cardinality (1) Denotes the number of senders of data streams packaged by this connector for distribution. One, Many Locality (1...*) Denotes whether or not data streams sent by this connector are remote or local. Local indicates that the stream data was packaged locally by the connector; remotely indicates packaging on-the-fly, typically performed by both the sender and receiver of data dynamically. Remote, Local Deliveries (1...*) Indicates the delivery method for the data streams sent by the distribution connector. Exactly Once, At Least Once, Best Effort Receiver Cardinality (1) The number of receivers of data streams packaged by this connector.). One, Many State (1) Denotes whether or not streams sent by the distribution connector are stateful, or transient (stateless). Stateful, State- less Bounds (1) Denotes whether or not there is a bounds on the size of a data stream sent by the distribution connector (Bounded), or there are no bounds (Un-Bounded). Bounded, Un- Bounded Synchronicity (1) Denotes whether or not streams sent by this connector can be accessed synchronously, or asynchronously. Synchronous, Time-out Synchronous, Asynchronous Buffering (1) Denotes whether or not the data stream supports buffered access. Buffered, Un- Buffered Bothgrid-based,andclient/serverconnectorsutilizeaprocedurecallinvocation medium. The metadata attributes captured for procedure calls are given in Table 6.4. AllMehtaetal. dimensionswereincludedfortheprocedurecallmetadata,with the exception ofentrypoint,invocation, andaccessibility. In our research, we have found entrypoint to be highly difficult for an architect to discern, because it varies greatly from connector to connector rather than originating from a fixed discrete-valued domain. In addition, according to our research to date, its impor- tance in describing the connector is minimal, excluding it from inclusion based on 99 Table 6.3: Distributor DCP metadata. Dimension Description Values Routing Membership (1) Denotes the way that data is routed while being distributed by the connector. Ad-hoc, Bounded Naming Type (1) Denotes the way that data being distributed by this connector is identified: Registry-based indicates that the data is distributed and controlled by software registries, and Attributed-based indicates the routing of the data is affected by attribute metadata. Registry-based, Attribute- based Naming Structure (1...*) Identifies how named data is distributed through this connector: hierarchical indicates that content is distributed by a hierarchical topology: flat indicates that data is distributed point-to-point. Hierarchical, Flat Routing Type (1) Identifies how data is routed by this connector, enforcing a topology of data flow. tcp/ip, content-based, architecture configuration, udp, tracker Delivery Semantics (1...*) Identifies the connectors data delivery policy. Exactly Once, Best-Effort, At Most Once Routing Path (1...*) Denotes the path used by the connector to route data. Static, Dynamic, Cached Delivery Mechanisms (1...*) Denotes the delivery method employed by this connector indicating whether data is transferred point-to-point (unicast), point-to-many (multicast) or many-to-many (broadcast). Unicast, Multi- cast, Broadcast criterion 1. For invocation, we found all the distribution connectors we stud- ied to have the same value (Method Call), suggesting this the dimension is not a point of high variance, causing it to be excluded on the basis of criterion 1. For accessibility, we assume all invocation of data distribution connectors within our framework to be publicly accessible, excluding it based on criterion 1. Event-based connectors are invoked via Events. The metadata captured for events is listed in Table 6.5. All Mehta et al. attributes besides mode causality and hardware mode were included. Mode causality was excluded on the basis of criterion 2, as we assertthatitwouldbedifficultforanyarchitecttoannotatedistributionconnectors 100 Table 6.4: Procedure Call DCP metadata. Dimension Description Values Parameter Data Trans- fer Method (1...*) Denotes the data transfer method for the parameters sent via a (remote) procedure call. Value, Refer- ence Parameter Semantics (1) Denotes the method by which parameters are identified when they are sent by the procedure call. Keyword ParameterReturnValue (1) Denotes the return value data type/methodology from the pro- cedure call. HTTP response, RMI response message, etc. Parameter Invocation Record (1) Denotestherecordformatandlocationforaprocedurecallinvo- cation. Globus Log Layer, bbFTP Message, HTTP Server log, etc. Synchronicity (1) Identifies if the procedure call is blocking, or non-blocking. Synchronous, Asynchronous, Time-out Synchronous Sender Cardinality (1) Denotes the number of senders of this procedure call One, Many Receiver Cardinality (1) Denotes the number of receivers of this procedure call One, or Many with a selected value for it (its definition as defined by Mehta et al. is unclear), while hardware mode was excluded on the basis of criterion 1 (its importance in describing a distribution connector was minute). As discussed in Chapter 4, P2P distribution connectors utilize a process of arbitration to begin their data distribution activities. The connector metadata captured for arbitrator connectors is given in Table 6.6. Excluded from the set of attributes in Table 6.6 are concurrency mech- anism, transactions isolation, and scheduling time. We excluded concur- rencymechanismbasedoncriterion2(understandability),astheinformationfor the attribute would be difficult to populate by an architect (it would require deep understandingofaconnector’simplementation). Fortransactionsisolation, the attribute was excluded based on both criteria 1 and 2. We excluded scheduling weight on the basis of criteria 1 and 2, as we were not able to clearly understand the attribute’s meaning within the context of the Mehta et al.’s taxonomy. 101 Table 6.5: Event DCP metadata. Dimension Description Values Producer Cardinality(1) Denotes the number of unique event producers sending events to this distribution connector. One, Many ObserverCardinality(1) Denotes the amount of event observers listening to events sent from this connector. Keyword Producers (1) Denotes the producers of events sent to this connector. Access Points, Components, etc. Observers (1) Denotes the observers of events sent out from this connector. Clients, Threads, etc. Event Patterns (1...*) Denotes the different event delivery mechanisms employed by this connector. Pub-Sub, Broadcast, Unicast, Multicast, Asynchronous, Best Effort Delivery (1) Denotes the behavior employed by the connector to deliver events. BestEffort,etc. Outgoing Priority (1) Identifies who sets the outgoing event priority, the Producers or the Consumers. Producers, Consumers Incoming Priority (1) Identifies who sets the incoming event priority, the Producers or the Consumers. Producers, Consumers Synchronicity (1) Denotes whether events are sent in a blocking, or non-blocking fashion. Asynchronous, Synchronous, Time-out Synchronous Synchronicity (1) Denotes the routing behavior of the connector when sending events. Pub-sub, Queued Dis- patch, Polled, Central Update Mode Softwares (1...*) Denotes the software employed to deliver events. Signals, etc. 6.3.2 Distribution Scenarios As described in Chapter 5, a distribution scenario captures the intrinsic require- ments of a data distribution activity. Each distribution scenario S is a (set of) constraint(s) on one or more of the eight identified dimensions of data distribution described in Chapter 5. The DISCO data component representing a distribution scenario can be thought of as a two-column table of constraints, where each row in the table represents the entire constraint. The first column represents the scenario dimension constrained, and the second column represents the actual value domain 102 Table 6.6: Arbitrator DCP metadata. Dimension Description Values Fault Handling (1) Identifies the approach for dealing with faults such as lost con- nection. Voting, etc. Concurrency Weight (1) Denotes the size and complexity of needed resources for manag- ing concurrency within the connector. Light-weight, Heavyweight Transaction Nesting (1) Denotes whether or not single or multiple transactions can be nested within the connector. Single, Multi- ple Transaction Awareness (1) Denotes whether or not the connector is capable of transactions. None, Sup- ported, Required, New Security Authorization Type (1) Identifies how authorization is handled within the connector. Capabilities, Access Control Lists Security Authentication (1) Identifies how authentication is handled within the connector. Kerberos, Grid Security Infrastructure, HTTPS, etc. Privacy (1...*) Identifies how privacy is dealt with by the connector. Encryption, Information padding, screening/fire- walls Integrity (1) Identifies the manner in which the connector deals with data consistency. Redundancy check, certifi- cates Durability (1...*) Denotes whether or not the connector maintains resource alloca- tions and information across sessions. Single Session, Multi-session Scheduling (1) Denotes the strategy employed by the connector to deal with communication, and invocation. LRU, etc. restriction, as illustrated in Table 6.7. The scenario captured within this table should be familiar: it is one of the scenarios from the movie distribution system from Section 6.1. This example will be used throughout the rest of the chapter as a means for illustrating the connector selection algorithms. 6.4 Developing Connector Selection Algorithms In this section, we describe three connector selection algorithms that we developed as part of DISCO. This set of algorithms was not intended to be exhaustive, but 103 Table 6.7: Example distribution scenario. Dimension Constraint TotalVolume > 1 TB GeographicDistribution WAN DeliverySchedule.NumIntervals 1 NumUsers 1 more so representative of the two canonical approaches for selecting data distri- bution connectors that we will describe below. To address objective 4, DISCO’s connector selection is extensible, allowing the development of several different con- nectorselectionalgorithms, ratherthanmandatingasingle“onesizefitsall”selec- tion algorithm. A critical distinction between the three approaches we developed (and more generally, distribution connector selection as a whole) is illustrated in Fig. 6.4. Black boxapproachestoconnectorselectionfocusonmappingtheobservableprop- erties (e.g., performance requirements, etc.) of the distribution connectors to the constraints of the distribution scenario, neglecting to consider the internal proper- tiesofthe distributionconnector(i.e., treatingitlikea“blackbox”). Ontheother hand, a white box connector selection approach leverages the intrinsic connector properties (e.g., described using metadata such as that above), and maps those properties to the key properties of the distribution scenarios (in many ways akin to “opening up the hood” on the connector). In our research, we have observed that there are distinct advantages to each approach. These tradeoffs will become more apparent in the discussion below, but in general black box approaches require as input simple cursory knowledge of the distribution connector, typically in the form of behavioral knowledge of the con- nector’s observable properties in the face of representative distribution scenarios. Because of this, a black box approach requires high level domain knowledge, e.g., 104 Distribution Connector Selection black box white box Bayesian Optimizing Score-based Figure 6.4: Approach to developing connector selection algorithms. understanding the relationship between a scenario dimension (such as Number of Users), and the observable property (such as efficiency). 2 In contrast to black box approaches, white box approaches require an archi- tect’s domain knowledge in the form of detailed architectural understanding of the internals of distribution connector, and their applicability to different data distri- bution scenarios. The white box domain knowledge is more fine-grained, which makes it more difficult to capture and understand. However, as we will see in the forthcomingBayesianalgorithm,makingthearchitecturalknowledgeexplicitleads to a more explicit model and better mapping of distribution scenario to distribu- tion connector, aiding in validating Hypothesis 2, described in the introductory chapter. In addition, the ability of our selection algorithms to capture the afore- mentioned domain knowledge aids DISCO in addressing objective 5 described in the beginning of this chapter. Below, we first discuss the data structure, a connector rank list, that will be used to represent the output of connector selection. We follow by discussing and formally describing our connector selection algorithms: (1) a white-box, Bayesian selection algorithm, (2), a black-box, Score-based algorithm that uses addition to 2 This type of relationship will become more concrete in the discussion of the Score-based algorithm below. 105 Table 6.8: Partial DCPs of two postulated connectors. Procedure Call Data Access Stream Distributor Connector Data Transfer Method Transient Availability Formats Delivery Mechanisms c 1 Value Cache Raw Broadcast,Unicast c 2 Reference Session Structured Unicast,Multicast compute a connector score, and (3) a black-box non-linear optimization connector selection algorithm. Each algorithm is discussed in the context of a representative distribution scenario (Table 6.7) and a small knowledge base of connectors (Table 6.8). 6.4.1 Connector Rank Lists Each connector selection algorithm that we developed adhered to the following general interface: List<ConnectorRank> select(Scenario S, List<DCP> conns) The returned value from each selection algorithm is a connector rank list, a sorted (based on a rank r) list of (connector,r) mapping pairs. This list serves as a critical interface in DISCO, providing an ordered ranking of connectors. Each rank r for a connector c represents c’s suitability to a distribution scenario. As discussed in Chapter 5, such a list affords a user the ability to iteratively decide betweendistributionconnectorsinaprincipledfashion. Anexampleofaconnector rank list is shown below. (bbFTP, 0.15789473684210525) (FTP,0.15789473684210525) (GridFTP,0.15789473684210525) 106 (HTTP/REST, 0.15789473684210525) (SCP, 0.15789473684210525) (UFTP, 0.15789473684210525) (Bittorrent, 0.02105263157894737) (CORBA, 0.005263157894736843) (Commercial UDP Technology, 0.005263157894736843) (GLIDE, 0.005263157894736843) (RMI, 0.005263157894736843) (Sienna, 0.005263157894736843) (SOAP, 0.005263157894736843) Theconnectorranklist,andselectionalgorithminterfacedescribedaboveallow DISCO to address objective 1 identified in Section 6.2. Below, we will discuss the three connector selection algorithms that implement this interface in the context of the representation data distribution scenario shown in Table 6.7. We will also considerthefollowingtwopostulatedconnectors(onefromtheclient/serverfamily and the other a grid based connector) and their (partial) distribution connector profiles depicted in Table 6.8. Since the connectors c 1 and c 2 are grid and clien- t/serverconnectors,theyhaveprocedurecallmetadata,inadditiontothestandard data access, stream and distributor metadata. For brevity, we have only included a single DCP attribute from each of the four atomic connector types of the DCP. 6.4.2 Bayesian Algorithm TheBayesian[Win03]selectionalgorithmtakesawhite-boxapproachtoconnector selection. Its goal is to use a connector’s architectural metadata (provided by its DCP)asameansofrelatingthescenariodimensionstoaconnector. TheBayesian algorithmtakesasinputadomainprofile(theequivalentofaconditionalprobability 107 Table 6.9: Example domain profile knowledge for the Bayesian algorithm. Data access transient availability Total Volume > 1 TB Geographic Distribution =WAN Delivery Sched- ule.Num Intervals = 1 Num Users = 1 Cache 0.80 0.80 0.60 0.20 Peer 0.30 0.95 0.95 0.80 Session 0.10 0.10 0.10 0.80 table [Win03]) that captures an architect’s knowledge of the relationship between the DCP metadata and the values for scenario dimensions. As an example, consider a distribution scenario depicted in Table 6.7. Addi- tionally,assumethatwearegiventheDCPsfortwoconnectors,c 1 andc 2 ,shownin Table6.8. Considerthedata access transient availabilitymetadataattribute, spec- ifying whether a connector has Session-based, Cache-based, or Peer-based access to data. TheBayesianalgorithmallowsanarchitecttofavorCache-baseddataaccess, which is faster and more scalable than the other two alternatives. An architect may encode this knowledge by assigning the probability value 0.80 to any connec- tor that has Cache-based data access, 0.30 to connectors that exhibit Peer-based access, and finally 0.10 to connectors that have Session-based access. Assigning the canonical “correct” probability is not a requirement as long as there is suffi- cient differentiation between the conditional probabilities recorded for each DCP attribute value. The probability information would be recorded as shown in Table 1. 3 The complete listing for the Bayesian algorithm is provided in Algorithm 1. For each attribute of each DCP (indicated by the a∈ A for loop), the algorithm computes the value range (e.g., Cache, Peer, Session for data access transient 3 It may seem strange that the three probabilities do not sum up to 1; however, according to Bayes theorem for discrete probabilities the conditional probabilities need not sum up to 1. 108 availability) for the attribute, and assumes equal probability that a distribution connector could take on any of the attribute’s values. The algorithm then applies the discrete version of Bayes theorem [Win03], shown in its general form below. P( ˜ θ =θ j |˜ y =y k ) = P( ˜ θ =θ j )P(˜ y =y k | ˜ θ =θ j ) X i P( ˜ θ =θ i )P(˜ y =y k | ˜ θ =θ i ) In Bayes theorem, θ represents the variable of interest (in our case the DCP attribute, e.g., data access transient availability), θ j represents the jth value (of i possible values)thatthe DCP attribute can take on (e.g.,Cache), ˜ y represents the observed variable (in our case the scenario dimension, e.g., Total Volume), and y k represents the scenario dimension constraint (e.g.,≥ 10 MB) . Algorithm 1: Bayesian Connector Selection input : distribution scenario S, set D of distribution connector profiles, conditional probabilities P output: Ordered list of connectors that best suit the given scenario S Data: prior c ←− all connectors equally probable Data: set O of observed vars ←−transformScenario(S) Data: posterior c ←−∅ foreach d∈D do foreach a∈A do Data: valRange←−getValRange(a) Data: prior a ←− all v∈valRange equally probable foreach o∈O do posterior a ←−prior a ∗bayes(prior a ,o,P) prior a ←−posterior a posterior c [d]←−posterior c [d]∗posterior a Data: rankList←−makeRanksFromProbDist(posterior c ) Sort(rankList) Normalize(rankList) return rankList 109 The distribution scenario and its constraints are used as a set of observed “evi- dence” (the o∈O loop). For example, in the distribution scenario shown in Table 6.7, there would be four variables of observed evidence in O: {Total Volume > 1 TB, Geographic Distribution = WAN, DeliverySchedule.NumIntervals = 1, NumUsers = 1}. Each application of Bayes theorem revises the attribute’s original (equally probable) probability distribution across all of its potential val- ues. After Bayes theorem is applied for each o∈O, the algorithm arrives at a final probability distribution (posterior a ) for each value that the attribute a can take on. An example of such a probability distribution for the data access transient availability attribute would be: Cache 0.95, Session 0.02, Peer 0.03. The proba- bility corresponding to the attribute value for the connector’s DCP is aggregated through multiplication (the posterior c [d]∗posterior a step) to formulate a set of final probabilities for each DCP. These are, in turn, used to rank and compare the connectors of interest. Relating to our example shown in Tables 6.7 and 6.8, a domain expert would encode her knowledge as shown in Table 6.9. Given the conditional probabilities recorded for the data access transient availability attribute, and the initial prob- ability distribution Cache : .333,Peer : .333,Session : .333, the algorithm would revise the initial probability distribution four times based on the observed scenario variables as shown below (where TV represents Total Volume): P(Cache|TV > 1 TB) = P(Cache)×P(TV>1TB|Cache) P(Cache)P(TV >1TB|Cache)+P(Session)P(TV>1TB|Session)+P(Peer)P(TV>1TB|Peer) 110 = 0.333×0.80 0.80×0.333+0.10×0.333+0.30×0.333 = 0.666 P(Session|TV > 1 TB) = P(Session)×P(TV>1TB|Session) P(Cache)P(TV >1TB|Cache)+P(Session)P(TV>1TB|Session)+P(Peer)P(TV>1TB|Peer) = 0.333×0.10 0.80×0.333+0.10×0.333+0.30×0.333 = 0.083 P(Peer|TV > 1 TB) = P(Peer)×P(TV>1TB|Peer) P(Cache)P(TV >1TB|Cache)+P(Session)P(TV>1TB|Session)+P(Peer)P(TV>1TB|Peer) = 0.333×0.30 0.80×0.333+0.10×0.333+0.30×0.333 = 0.250 The above three equations yield a new probability distribution, gen- erated after the observation of Total Volume > 1 TB: Cache : 0.666, Session : 0.083, Peer : 0.250. The application of the other three observed variables{Geographic Distribution = WAN, DeliverySchedule.Num Intervals = 1, NumUsers = 1} is performed in a similar fashion. After the final distribution 111 is generated, then the resultant probabilities corresponding to the connector’s attributevalue(e.g.,inthecaseofc 1 ,P(Cache)wouldbeused,andinthecaseofc 2 , P(Session) would be used) are aggregated through multiplication. As an example, the score for data access transient availability for connector c 1 would be computed as: P(Cache|Total Volume > 1 TB)× P(Cache|Geographic Distribution = WAN)×P(Cache|Delivery Schedule.Num Intervals = 1)×P(Cache|Num Users = 1). Similarly, the score for data access transient availability for connector c 2 would be computed as: P(Session|Total Volume > 1 TB) × P(Session|Geographic Distribution = WAN) × P(Session|Delivery Schedule.Num Intervals = 1)×P(Session|Num Users = 1). This process is repeated for each attribute specified in the connector’s DCP, and aggregated through multiplication in the same fashion to arrive at the connector’s final score. After the application of all observed variables for each attribute of the DCP provided, the final aggregated probability values are used as the rank values for each connector, and normalized so that the ranks are all between 0 and 1, and so that the sum of the ranks of each connector evaluated add up to 1. These final values are then returned as the output of the Bayesian algorithm. 6.4.3 Score-based Algorithm TheScore-basedconnectorselectionalgorithmtakesablack-boxapproachtoselect- ing connectors. To use the algorithm data distribution experts with experience using a particular connector technology, but without experience of the technol- ogy’s architecture, develop score functions comprised of discrete, known points relating a particular distribution scenario dimension to the performance require- ments on the scenario (recall Chapter 5). One of the limitations of this approach 112 Figure 6.5: Sample score functions relating NumUsers and efficiency for 6 of the 13 connectors in the knowledge base. is its basis on a data distribution expert’s observable knowledge of the distribution connector, which may be incomplete or localized to a particular realm of expe- rience (i.e., experience only with large volume, or low volume scenarios, or only with certain connectors). On the other hand, if the expert knew the distribution connector technology intimately (which is not the case), then she could encode the score function in its entirety. The benefit of the score-based algorithm is that the entire score function definition is not needed, and can be interpolated from the provided discrete key points. This allows the capture of a guru’s domain knowl- edge, even without extensive internal knowledge of the distribution connector, and even without extensive empirical experience using the connector, and observing its behavior. As an example, an expert may define a score function relating the Number of Users dimension with the efficiency performance requirement using a few empiri- cally determined performance points for a given connector. A sum of least squares method is used to interpolate scenario values that have not been explicitly defined within the score function. Functions derived (using past experience on similar systems that employed the same distribution connectors) from these points for a 113 Table 6.10: Data from a score function relating num users and efficiency for six distribution connectors. Num Users bbFTP Bittorrent CORBA CUDP FTP HTTP/REST 0 0 0 10 10 0 0 1 100 10 75 75 80 100 10 100 20 75 55 75 75 100 100 30 75 55 70 50 1000 90 50 50 30 70 50 10000 85 75 20 20 65 35 100000 75 90 20 20 65 25 1000000 50 95 10 15 60 20 10000000 30 97 5 10 30 15 100000000 20 100 0 0 25 10 subset of the connectors in our knowledge base are shown in Fig. 6.5 and Table 6.10. 4 WithinTable6.10,theinitialcolumnrepresentsdiscreteperformancepoints (determined by the expert with experience using the connector) for the amount of users that a connector can support. The ensuing columns in the table repre- sent each connector technology. Each cell (identified by a chosen value for Num Users and by a chosen connector) represents a score that the user assigns as the Efficiency of the connector in the face of the particular number of users chosen. Thevaluerangeofthescoresarearbitraryinthesensethattheironlyrequirement is that they reflect the variation between the different connector technologies. In other words, a connector that is assigned a 25 value for efficiency versus another assigned a value of 75 (for the same Number of Users) should in fact be two times less efficient than its counterpart. ThescorebasedselectionalgorithmisprovidedinAlgorithm2. Inall,therecan be 28 score functions for each connector (seven dimensions, excluding dimension 4 CUDP represents a commercial UDP technology that is discussed in Chapter 8. 114 Algorithm 2: Score-based Connector Selection input : distribution scenario S, set D of distribution connector profiles, score functions SF, objective function O output: Ordered list of connectors that best suit the given scenario S Data: scores←−∅ foreach d∈D do Data: score d ,score e ,score s ,score c ←− 0 foreach s∈S do score e ←−score e +f e d (s) score s ←−score s +f s d (s) score c ←−score c +f c d (s) score d ←−score d +f d d (s) scores[d]←−O(score e ,score s ,score c ,score d ) Data: rankList←−scores Sort(rankList) Normalize(rankList) return rankList 3 5 , times the four performance requirements). To compute a rank for each con- nector, the algorithm iterates over all the connectors. For each connector, sums are computed corresponding to the performance requirements of interest. For each sum, the algorithm iterates over the seven scenario dimensions (shown as the s∈S for loop). If there are score functions defined for the connector for the given performance requirement and scenario dimension, then the function is evaluated using the distribution scenario value for the function’s dimension. For example f e GridFTP (TotalVolume = 10 MB) would use the score function for GridFTP that relatesefficiencyandTotal Volume,providingasinputthevalue10MB.Theresul- tant value obtained from each function’s evaluation is added to the corresponding sum. Finally, foreachoftheconnectors, aweightingfunctionO (providedasinput to the algorithm) is used to influence each of the sums, allowing a user to define 5 Dimension 3 is excluded because it represents the four performance requirements which we relate the other seven dimensions to. 115 Table 6.11: Data from four score functions for two postulated connectors. e s c d c 1 c 2 c 1 c 2 c 1 c 2 c 1 c 2 Total Volume 100 GB 20 70 30 100 70 70 95 95 1 TB 15 90 20 120 70 70 80 80 Num Users 1 50 80 40 90 70 70 95 95 10 50 80 40 90 70 70 95 95 Geographic Distribution WAN 70 90 70 90 35 70 95 95 LAN 80 75 80 78 75 75 70 85 DeliverySchedule.Num Intervals 1 70 90 20 120 70 70 80 80 10 70 90 5 90 40 70 70 85 her interest in the performance requirements. As an example, the user may choose an objective function that maximizes consistency while neglecting efficiency (e.g., 1×score c +.2×score e ). Relating to our example shown in Tables 6.7 and 6.8, a domain expert with experience evaluating the performance of connectorsc 1 andc 2 may construct score functions for each connector, the relevant subset of which is shown in Table. 6.11. Since the given scenario from Table 6.7 did not explicitly supply performance requirements, then the default objective function (1×score e +1×score s +1× score c + 1× score d ) would be applied. Scores computed for each of the four performance dimensions for each of the two connectors are computed as follows. 116 score c 1 = f e +f s +f c +f d score c 1 = (15+50+70+70)+(20+40+70+20)+ (70+70+35+70)+(80+95+95+80) = 205+150+245+350 = 950 score c 2 = f e +f s +f c +f d score c 2 = (90+80+90+90)+(120+90+90+120)+ (70+70+70+70)+(80+95+95+80) = 350+420+280+350 = 1400 After computation, scores are sorted, and then normalized (e.g.,), yield- ing: normalized score c 1 = scorec 1 scorec 1 +scorec 2 = 950 950+1400 = 950 2350 = .404, and normalized score c 2 = scorec 2 scorec 1 +scorec 2 = 1400 950+1400 = 1400 2350 =.596. 6.4.4 Optimizing Algorithm Oneofthethelimitationsofbothselectionalgorithmsdescribedaboveisthateach of them takes an agnostic approach to under-specified distribution scenarios, i.e., scenariosinwhichsomeofthedimensionsarenotspecified(werefertothesekinds of scenarios as relaxed). In these situations, the un-specified scenario dimension constraints were simply not considered in the overall applicability computation. 117 Todealwithrelaxedscenarios inamoreprincipledfashion, weinvestigatedthe development of a linear programming-based selection algorithm. Linear program- ming is an optimization method that attempts to maximize a particular objective function (in our case, a connector score) based upon constraint variables, similar to the above described score-based algorithm. However, linear programming does notallowintegervariablesinthesystem, excludingtheempiricallygatheredpoints in the score functions. Algorithm 3: Non-linear Optimizing Connector Selection input : distribution scenario S, set D of distribution connector profiles, competency functions CF, objective function O output: Ordered list of connectors that best suit the given scenario S Data: scores←−∅ Data: S’ set of scenario dimensions / ∈ S foreach d∈D do Data: score d ,score e ,score s ,score c ←− 1 foreach s∈S do score e ←−score e ∗f e d (s) score s ←−score s ∗f s d (s) score c ←−score c ∗f c d (s) score d ←−score d ∗f d d (s) if|S 0 |> 0 then scores[d]←−hook jeeves(O,score e ,score s ,score c ,score d ,S 0 ,D,CF) else scores[d]←−O(score e ,score s ,score c ,score d ) Data: rankList←−scores Sort(rankList) Normalize(rankList) return rankList Because of these limitations, we had to look for more general forms of opti- mization solutions, such as non-linear programming. Non-linear programming is similar to linear programming; however, it is able to deal with variables from the realsetofnumbers. Aparticularlyusefulandefficientalgorithmtosolvenon-linear 118 programming models is the algorithm produced by Hooke and Jeeves [HJ61]. This algorithm does not bother with modeling the scenario constraints and only works based on the objective function, embedding the constraints within its calculation. BasedontheHookeandJeevesalgorithm,wedevisedourthirdconnectorselection algorithm, which we refer to as the Optimizing algorithm. The domain knowledge provided to the optimizing algorithm is multiplication based competency functions. Competency functions are similar to score functions with the lone exception that competency functions produce fractions (competency indexes) associated to dimensions for each connector, yielding values between 0 and 1 (as opposed to the score functions in which a domain expert records integer values) . In our research, we have observed that multiplying by fractions (to aggregate the connector score for each performance dimension) is more accurate than by whole numbers 6 . The algorithm works with the scenario values for each dimension by first converting them to the logarithmic scale, which lets the Hooke andJeevesalgorithmconvergebetter,sincethekeypointsofthedomainknowledge for each function relating performance requirements to scenario dimensions are frequently based on powers of 10 (as seen for the Num Users to Efficiency relation in Table 6.10). The complete non-linear optimizing selection algorithm is provided in Algo- rithm 3. When provided a scenario with constraints for all possible dimensions (shown as theelse condition in Algorithm 3), then no optimization work is needed and the optimizing algorithm will sort the connectors based on their computed performance index. However, in contrast to the above two selection algorithms, the optimizing algorithm is capable of choosing connectors when one or more of 6 This will be shown in the evaluation chapter, Chapter 8. 119 the scenario dimensions have been relaxed (i.e., when|S 0 | > 0). The more vari- ables that are relaxed, the harder the algorithm tries to optimize the objective function for all the connectors. This optimization is performed using Hooke and Jeeves’s algorithm, which devises optimum values for the relaxed scenario dimen- sions, focusing on the provided objective function. The benefit of the algorithm is providing connector selection to users that have flexible requirements for some dimensions or alternatively providing such users a means to compute optimum dimension values. 6.5 Performing Connector Selection Analysis The aforementioned connector selection algorithms and connector descriptions afford users of DISCO a means for ranking and understanding the applicability of data distribution connectors to distribution scenarios. Any sound decision mak- ing framework, however, requires a means for analyzing how precisely and how accurately the framework is performing. In the case of DISCO, we were inter- ested in how precise and accurate the selection algorithms within DISCO were at deciding the most appropriate connector for a distribution scenario. One of the critical observations made within our research is that, in contrast to having a single, “best” connector for a distribution scenario, there are often a number of different connectors appropriate/inappropriate for it. This observation lent itself naturally to the following data structure, that we describe below, the appropriate and inappropriate bins. These bins, and a provided expert answer key, allow DISCO to perform precision-recall analysis, a quantitative means for evaluating the strength of a decision making process, such as the solution that we have developed for the distribution connector selection problem. In addition, 120 the bins aid DISCO as a framework to address our objective 6 described in the introductory paragraph of this chapter. 6.5.1 Appropriate and Inappropriate Bins Appropriate and inappropriate bins are defined formally as two unordered sets that contain ConnectorRank objects. As stated earlier, each connector rank is a mapping of connector name, to its suitability value returned from the selection algorithm. Appropriate indicates that the connector (identified by the Connec- torRank) is appropriate for the scenario, inappropriate indicates the opposite. An example of a an appropriate bin and its corresponding inappropriate bin is given in upper right portion of Fig. 6.6. 6.5.2 Answer Key For each distribution scenario, for the purpose of evaluation and refinement of our selection algorithms, we consider that there is an answer key, indicating the canonical set of connectors appropriate for a scenario, and those inappropriate. An answer key is a special type of appropriate/inappropriate bin data structure, devised in a number of different ways including: • Consulting so-called data distribution gurus within an organization. • Empirically determining the connectors applicability based on some evalua- tion domain, such as performance. • Gathering statistics and surveys of practitioners in the field of data distribu- tion. The relationship between the answer key, and the appropriate and inappro- priate bins is illustrated graphically in Fig. 6.6. The connector rank list is run 121 Inappropriate Appropriate (bbFTP, 0.15789473684210525) (FTP,0.15789473684210525) (GridFTP,0.15789473684210525) (HTTP/REST, 0.15789473684210525) (SCP, 0.15789473684210525) (UFTP, 0.15789473684210525) (Bittorrent, 0.02105263157894737) (CORBA, 0.005263157894736843) (Commercial UDP Technology, 0.005263157894736843) (GLIDE, 0.005263157894736843) (RMI, 0.005263157894736843) (Sienna, 0.005263157894736843) (SOAP, 0.005263157894736843) Clustering Process connector rank list Precision/ Recall Analysis expert answer key accuracy evaluation report Figure 6.6: An overview of connector selection analysis within DISCO. throughaclusteringprocessthatoutputsanappropriate/inappropriatebincombi- nation. Thisoutputappropriate/inappropriatebinsetcanbecomparedagainstan expertanswerkeyusingprecision-recallanalysis. TheabilityofDISCOtoperform precision-recall analysis allows it to address framework objective 3. Becauseourconnectorselectionalgorithmsdonotnativelyproduceappropriate and inappropriate sets, we were motivated by the desire to automatically cluster theconnectorranklist,theoutputoftheselectionalgorithms,intotwosets,appro- priate and inappropriate. To this end, we studied the k-means algorithm [Mac67] as amethodfor clusteringaconnectorranklist. K-means is astatistical clustering process that focuses on an initial data set, and a user’s desire to partition that set into k subsets. We will describe k-means in more detail below as part of the discussion of a connector clustering algorithm that we implemented based on it. 122 6.5.3 Exhaustive K-Means Connector Clustering Algo- rithm To perform the desired connector clustering of a connector rank list into appro- priate and inappropriate sets of connectors, we employed a variant of the k-means clusteringalgorithm. K-meansisaniterativedataclusteringapproachthattakesa setofdatapoints,andthenusesmeanvaluescalledcentroidsasareasaroundwhich to cluster other data points. Initially k-means randomly or heuristically divides a setofdatapointsintok sets(forthepurposesofconnectorclustering, weareinter- ested in two sets, appropriate and inappropriate, so k = 2). Then, the algorithm computes the mean point (or centroid) of each of the two clusters. The algorithm then associates each of the data points in the rank list (e.g., (bbFTP,0.54)) into the respective cluster with the closest centroid (to the data point), and the process is repeated until: • The points do not shift from either cluster; or • The centroids do not change The k-means connector clustering algorithm is provided in Algorithm 4. For the purposes of connector clustering, we are considering each point to be a ConnectorRank, of the form(conn name ,r), where r∈Q. Connectors are clustered around the centroid computed from their ranks. Because the set of connectors being clustered is not variable, and small in size, an exhaustive implementation of the k-means algorithm was employed (the alternative, vanilla k-means, is a heuris- ticapproachnotguaranteedtofindtheoptimalcluster). Tofacilitateeaseofunder- standing and implementation, we devised a clever means for computing a priori all possible combinations of the original connectors within 2 clusters. We leveraged base 2 numbers as a specification for whether or not a connector c belongs in one 123 of the two partitioning groups. To begin, the algorithm creates a map (connMap in Algorithm 4) associating each ConnectorRank with a number, 0 to|R|. The algorithm then iterates over all numbers starting from 0 to 2 |R| . Algorithm 4: Exhaustive K-means Connector Clustering (k=2) input : connector rank list R, distribution scenario S output: answer key K, containing a set of appropriate connectors, and inappropriate connectors Data: c n ←− 2 |R| Data: min d ←− maxQ Data: connMap←−∅ for i=0 to |R| do add{i⇒ R[i]} to connMap Data: low G ←−∅ for i=0 to c n do Data: cSetSpec←−toBinaryString(i) Data: cg 1 ,cg 2 ←−∅ for j=0 to |cSetSpec| do if cSetSpec{j} = 1 then add connMap{j}to cg 1 else if cSetSpec{j} = 0 then add connMap{j}to cg 2 Data: ad 1 ←−avg d (cg 1 ) Data: ad 2 ←−avg d (cg 2 ) if ad 1 +ad 2 <min d then low G ←− (cg 1 ,cg 2 ) min d =ad 1 +ad 2 Data: K←− (S,low G ) return K Each number, 0 to 2 |R| , is converted into its binary representation (e.g., 1 becomes 0000000000001), of size|R|. For each number, the algorithm iterates over each index in the binary representation, and places a connMap[j] into one of two connector groups based on the following method. If the value at index j (0≤ j <|R|) in the binary representation is a zero, the connector is placed 124 into group 1 (cg 1 ): if the value is a one, it is placed into group 2 (cg 2 ). This forms the basis for two connector groups, which are then evaluated by computing the average distance (avg d ) of each connector rank in the group from the group’s centroid (recall, the centroid is the mean rank value within the connector group). The two average distances for each group (avg d (cg 1 ) and avg d (cg 2 )) are then summed and the minimum sum is computed as the algorithm iterates over all pos- sible connector groups defined by each binary number representation. Given the two groups with the minimum sum, the assignment of appropriate and inappro- priate occurs as follows: the appropriate group is the group that is home to the connector with the highest rank: the inappropriate group is the other. 6.5.4 Precision-Recall Analysis AsdepictedinFig. 6.6,theoutputoftheconnectorclusteringprocess(oranalysis) are appropriate/inappropriate bins. Provided with an expert answer key, we can utilize these two pieces of data to perform a precision-recall analysis. Precision- recall is a measure of accuracy, and is based on four key pieces of information: True Positivies (TP) Theconnectorslabeledintheexpertanswerkeyasappro- priate that were also labeled as appropriate in the output of the clustering process. True Negatives (TN) The connectors labeled in the expert answer key as inap- propriate that were also labeled as inappropriate in the output of the clus- tering process. False Positives (FP) The connectors labeled in the expert answer key as inap- propriate that were labeled as appropriate in the output of the clustering process. 125 False Negatives (FN) Theconnectorslabeledintheexpertanswerkeyasappro- priate that were labeled as inappropriate in the output of the clustering pro- cess. Using these pieces of information, we can compute the following four measures of accuracy: Precision The fraction of connectors correctly identified as appropriate for the scenario. This measure is computed as TP TP+FP . Recall The fraction of connectors identified as appropriate for the scenario com- pared to all existing appropriate connectors. The measure is computed as TP TP+FN . Accuracy The fraction of connectors correctly identified as either appropriate or inappropriate for the scenario compared to all labeled connectors. The measure is computed as TP+TN TP+TN+FP+FN . Error Rate Thefractionofconnectorsincorrectlyidentifiedaseitherappropriate or inappropriate for the scenario compared to all labeled connectors. This measure can be computed as either (1−accuracy) or as FP+FN TN+TP+FP+FN . These four fractional measures provide us with an understanding of how well our connector selection framework is performing in lieu of expert gathered answer keys for distribution scenarios. In the following section we will discuss how we can further validate our connector selections against actual connector performance data gathered empirically. 126 6.6 Validating selections against performance dimensions A key component in our research has been to empirically evaluate connector per- formance as a means of validating if the selected connectors in fact adhered to the as-stated performance requirements captured in a distribution scenario (recall, performance requirements is one of the eight key dimensions of data distribution fromChapter5). Specifically,weconsiderinourresearchfourperformancerequire- ments: efficiency, consistency, dependability, and scalability. We will describe our approach to empirically measuring these performance properties in detail in the evaluation chapter, Chapter 8. The results of these measurements are captured in the data structure described below, the distribution connector performance pro- files. 6.6.1 Distribution Connector Performance Profiles Each distribution connector profile stores four computed scores ranging in value from 1 to 10, where a score of 1 indicates poorest performance and a score of 10 indicates the highest possible performance value, for each distribution connector. Thefourscoresarecomputedforthefourperformancedimensions(andnormalized to a value between 1 and 10) according to the following formulas. S below indi- cates a representative space of 30 distribution scenarios (the same 30 distribution scenarios described in Chapter 3) used to capture performance profiles. 127 efficiency Where bps s indicates bits per second transferred by the connector for scenario s, and avg mem s indicates average memory consumption (in megabytes, MB) by the connector for scenario s, we score efficiency as: e(scenario) = bps scenario P s∈S bps s + 1− avg mem scenario P s∈S avg mem s ∗10 consistency Where bytes d s indicates the amount of bytes of data that the con- nector delivered as-sent for scenario s, we score consistency as: c(scenario) = bytes d scenario P s∈S bytes d s ∗10 dependability Where n f s indicates number of faults by a connector (observable downtime instances) for scenario s, we score dependability as: d(scenario) = 1− n f scenario P s∈S n f s ∗10 scalability Where total vs indicates the total data volume (in bytes) delivered by the connector for a scenario s, and num userss indicates the number of users that the connector delivered data to for a scenario s, and num types s indicates the number of unique user types that the connector delivered data to for a scenario s, we score scalability as: s(scenario) = num usersscenario P s∈S num userss + num types scenario P s∈S num types s + total vscenario P s∈S total vs ∗10 Each computed performance dimension score is normalized to a 0 to 10 scale by taking score+ 10−max s∈S score dimension . 128 Usingtheseperformanceprofiles,wecanrefinetheconnectorselectionsdecided by the connector selectors in the DISCO framework. Since each distribution sce- nario may specify user-requested performance requirements on a scenario (on the 0 to 10 scale), we can leverage the profiles to further validate the accuracy of the selection algorithm’s decision making process in the following manner: • If the performance profile value for the dimension is≥ the user specified requested value and the connector was originally selected by the clustering process as appropriate, then the connector remains appropriate. • If the performance profile value for the dimension is≥ the user specified requested value, and the connector was originally selected as inappropriate (because of performance requirements), then the connector becomes appro- priate. This is a manual process, though, because it is somewhat difficult to point the provenance of the connector selection to any one singular dimen- sion, such as performance requirements. • If the performance profile value for the dimension is < the user specified requested value, and the connector was originally selected by the clustering process as appropriate, then the connector becomes inappropriate. • If the performance profile value for the dimension is < the user specified requested value, and the connector was originally selected by the clustering process as inappropriate, then the connector remains inappropriate. This validation is performed against the output appropriate/inappropriate sets of connectors returned from the clustering step of DISCO. Provided a scenario and the appropriate/inappropriate bins (generated from the k-means clustering algorithm), and the performance profiles for DISCO, we can employ the above process to further validate/revise the connector selections. 129 Chapter 7 Tool Support In this chapter, we discuss DISCO’s implementation as a Java-based API and tool suite. We begin by describing DISCO’s extension point interfaces, the “hooks” allowing developers to customize DISCO, or alternatively to adapt its existing functionality. We follow with a discussion of DISCO’s architecture, including its core architecture and the architecture of a graphical environment for executing DISCO. We discuss standard XML data formats to represent DISCO data compo- nents, includingdistributionscenarios, distributionconnectorprofiles, anddomain knowledge. We conclude the section discussing DISCO’s tools package, contain- ing command line interface tools that can be used to interact with DISCO using different operating systems and programming languages. 7.1 Extension Points Within DISCO, there are two important modular interfaces, called extension points. An extension point was bourne out of recent technological patterns employedinseveralmajoropensource, javasoftwareprojects, mostnotablyIBM’s Eclipse project [Ecl07], as well as other projects, e.g., Apache Nutch [Nut07]. An extension point is most similar to a java interface, or more generally an object oriented interface, describing the external behavior/contract of a component with the outside world. Extension points allow pluggable back end implementations so long as those implementations behave according to the extension point contract, 130 and define the appropriate methods, with the appropriate return types, and so forth. Typically the extension point behavior/contract is defined using an par- ticular schemata and implemented using an XML file, though it is not explicitly required. DISCO’s extension points are described below. Selector Theselectorextensionpointdefinesasinglemethod selectConnectors that takes as input a java.util.List<DistributionConnectorProfile>, representing the connector knowledge base, and a DistributionScenario, representing the data distribution scenario instance. The method returns a java.util.List<ConnectorRank>, representing the ordered ranked list of connectors returned for the scenario. Analyzer The analyzer extension point defines a single method, analyzeScenarioResult that takes as input a DistributionScenario, representing the data distribution scenario instance, and a List<ConnectorRank> representing the results of connector selection (from a Selector). The method returns a AnswerKey, containing a java.util.Set<ConnectorRank> representing the appropriate connectors for the distribution scenario, and a java.util.Set<ConnectorRank> representing the inappropriate connectors for the distribution scenario. 7.2 Framework Architecture The design of DISCO’s implementation is shown in Fig. 7.1 in standard UML notation. Gray shading indicates a UML interface; otherwise no shading indicates a standard UML class. The classes shown in gray and light shading in the diagram are not exhaustive, and meant to give the reader an appropriate understanding of DISCO’scoreimplementationclasses. Inthissectionforbrevityweomitadetailed 131 Analyzer <<interface>> StatsGenerator disco.analysis ExhaustiveK Means disco.analysis.kmeans disco.connector.atomic Event Connector DataAccess Connector disco.connector Distributor Connector Stream Connector Procedure Call Connector Arbitrator Connector disco.connector.dcp Distribution Connector Profile P2P Distribution Connector Profile Event Distribution Connector Profile Grid Distribution Connector Profile Client/Server Distribution Connector Profile Selector <<interface>> disco.selection Bayesian Selector disco.selection.bayesian Score Based Selector disco.selection.score Optimizing Selector disco.selection.optimizing Connector Rank disco.servlet Selector Servlet disco.structs Distribution Scenario Answer Key Distribution Scenario Result Precision Recall Data Scenario Value Ranged Value disco.tools Selector Test Runner DCP Stats Generator Random DCP Generator Scenario Runner Scenario Analyzer Runner Scenario Analyzer disco.util Answer Key Reader Bayesian Domain Profile Reader Competency Function Reader DCP Reader DCP Writer Distribution Scenario Reader Objective Function Reader Bayesian Domain Profile Reader XML Utility Performance Profile Launcher disco.gui disco.gui.util XML Filter Properties Filter disco.gui.model Scenario Results Model disco.gui.controller ScenarioEditor Delegate ScenarioRunner Delegate disco.gui.view ScenarioEditor View ScenarioResults View ScenarioRunner View Figure 7.1: Implementation-level architecture for the DISCO framework. description of each and every class, focusing instead on the critical packages and classes within the DISCO infrastructure. Whenever possible, we strove to ensure that classes were organized diligently into existing class packages. Those packages that were not existing, that warranted creation, were created, otherwise the class was placed into the disco.util utility package. In Fig. 7.1, we highlight 8 critical of DISCO’s 14 packages. The disco.selection package contains the generic Selector extension point, defining the behavior that all connector selection algorithms must imple- ment. The package contains the core data structure, ConnectorRank that stores 132 the connector name and an associated score, assigned by the selection algo- rithm. There are three sub-packages within disco.selection, corresponding to each selection algorithm and any necessary classes required to implement it. BayesianSelector implements the aforementioned Bayesian connector selection algorithm, ScoreBasedSelector implements the score-based selection algorithm andOptimizingSelectorimplementsthenon-linearoptimizationconnectorselec- tion algorithm. The disco.analysis package contains the generic Analyzer extension point, that defines the connector grouping method, resulting in the generation of an AnswerKey, containing appropriate and inappropriate sets of connectors. PrecisionRecallData is a data structure that stores the number of true posi- tives, true negatives, false positives and false negatives returned from a precision recall analysis. StatsGenerator uses this data to compute precision, recall, accu- racy and error rate. The ExhaustiveKMeans class implements the aforementioned k-means connector clustering algorithm. The disco.servletpackagecontainsasingleconcreteclass SelectorServlet that exposes the DISCO connector selection algorithms over an HTTP/REST interface. The class returns an XML representation of a java.util.List<ConnectorRank>. This class is the main interface method to the external COTS interoperability evaluation framework that we have integrated DISCO with, and that we will discuss further in Chapter 8. The disco.connector package contains the concrete class that rep- resent the DistributionConnectorProfiles that describe connectors. There are four sub-classes of DistributionConnectorProfile including GridDistributionConnectorProfile, P2PDistributionConnectorProfile, ClientServerDistributionConnectorProfile, and 133 EventDistributionConnectorProfile. Each one of these classes rep- resents the DCP metadata for one of the four connector families iden- tified in Chapter 4. This metadata is represented by one of the six atomic connector metadata containers in the disco.connector.atomic package, EventConnector, DistributorConnector, DataAccessConnector, StreamConnector, ArbitratorConnector, and ProcedureCallConnector. The core DISCO data structures are housed in the disco.structs package. This package contains the DistributionScenario class, and the DistributionScenarioResult class, which is an aggregation of a DistributionScenario and a java.util.List<ConnectorRank>, resultant from connector selection. ScenarioValue is a meta class representing discrete and RangedValuesusedindistributionscenarioconstraints. ThePerformanceProfile class stores the DISCO performance profile information about a connetor. The disco.util package contains utility classes including readers and writers for the core DISCO data structs. For example, AnswerKeyReader reads in an XML representation of the AnswerKey data structure used in connector selection analysis. The package also contains a generic XMLUtility factory/helper class aiding in the reading and writing of DISCO XML data structures. The disco.tools package contains command line tools that expose DISCO’s APIs for selection and analysis of connectors. We will describe this package in detail in Section 7.5 below. 7.3 GUI Architecture The DISCO GUI was implemented as a Java SWING-based extension to the core DISCOarchitecturedescribedabove. TheimplementationarchitectureoftheGUI 134 is shown in upper right portion of Fig. 7.1. The architecture follows the tradi- tional Model View Controller (MVC) [KP98] paradigm for building graphical user interfaces. In MVC, the GUI is centered around a model, which stores the tangible runtimeinformationthatisbeingmanipulatedbyauser. Theviewprovidesapar- ticularview(typicallyspecifiedbytheuser)ofthemodel, andthecontrollerallows the user to interact with the models and views by forwarding control and delega- tion from the user along to the model and the view. MVC was chosen because of its native support for extensibility, and its clear separation of concerns. Using MVC,wewereabletocleanlyseparatetheDISCOGUI’suseofDISCO’scoredata structures, from the different ways in which users wanted to manipulate them. The class package disco.gui is organized in an MVC fashion. The Launcher classisresponsibleforcreatingtheGUIframe,andinstantiatingthemodels,views, and controllers for the system. Within disco.gui.util, there are two classes that extend java.io.FileFilter. PropertiesFilter is a filter responsible for identifying DISCO property files, specifying runtime configuration options for the DISCO framework, and for the GUI (e.g., locations of the connector knowledge base,locationsofthedomainprofileinformation,userprovideddefaultsfornumber of connectors per rank list, and so forth). XMLFilter is responsible for identify- ing DISCO xml data structure files, such as distribution scenarios, and domain knowledge xml files for the selectors. The class package disco.gui.model is home to a single class, ScenarioResultsModel, that is responsible for serving as the interaction data structure between a user of the DISCO GUI, and the DISCO framework’s connectorselection results. Theclassis awrapperdatastructurearoundaDISCO DistributionScenario, mapped to a java.util.List<ConnectorRank>, serving to mediate manipulation of scenario results by the user and the view for the GUI. 135 The class package disco.gui.view stores the views of the ScenarioResultsModel portion of the DISCO GUI. ScenarioEditorView is a view class that presents a view of a distribution scenario, allowing a user to specify the different constraints of a distribution scenario. ScenarioRunnerView is the view screen for tailoring and execution of DISCO’s connector selection algorithms. ScenarioResultsView is a view screen that allows a user to explore the results of a connector selection, a java.util.List<ConnectorRank>. Classes within the disco.gui.controller package are responsible for bridg- ing the gap between the user interaction with the GUI, and the GUI views and backend model. ScenarioEditorDelegate allows the user to interactively specify a distribution scenario by bringing up the appropriate ScenarioEditorView, and by setting up the appropriate distribution scenario for use as input to connector selection. ScenarioRunnerDelegate allows a user to interactively run connector selection on existing and newly defined (by the user) distribution scenarios, and to visualize the results using the appropriate ScenarioResultsView, and to model the results using the appropriate ScenarioResultsModel. 7.4 XML Data Formats Several of DISCO’s core data structures can be (de-)serialized to and from XML, aiding in their manipulation via external systems and users. We have developed XML document type definitions (DTDs) for each data structure, allowing valida- tion and syntax checking for the DISCO implementation framework. DTDs are essentially schema definitions, defining the valid elements, the order in which they should appear within an XML document, and any attributes that the elements 136 require. As an example, consider the following XML DTD below, defining the XML structure for the DISCO distribution scenario. <!ELEMENT distributionScenario ( totalVolume?, deliverySchedule?, numUsers?, userTypes?, geographicDistribution?, performancereqs?, accessPolicies?, dataTypes? ) > <!ELEMENT totalVolume ANY> <!ATTLIST totalVolume valtype CDATA "discrete"> <!ELEMENT accessPolicies ( policy+ ) > <!ELEMENT policy ( #PCDATA ) > <!ELEMENT dataTypes ( type+ ) > <!ELEMENT type ( #PCDATA ) > <!ELEMENT deliverySchedule ( volumePerInterval, numIntervals ) > <!ELEMENT volumePerInterval ANY > <!ATTLIST volumePerInterval valtype CDATA "discrete"> <!ELEMENT numIntervals ANY > <!ATTLIST numIntervals valtype CDATA "discrete"> <!ELEMENT performancereqs ( consistency, dependability, scalability, efficiency ) > <!ELEMENT scalability ( #PCDATA ) > <!ELEMENT dependability ( #PCDATA ) > 137 <!ELEMENT consistency ( #PCDATA ) > <!ELEMENT efficiency ( #PCDATA ) > <!ELEMENT geographicDistribution ( type+ ) > <!ELEMENT numUsers ANY > <!ATTLIST numUsers valtype CDATA "discrete"> <!ELEMENT userTypes ANY > <!ATTLIST userTypes valtype CDATA "discrete"> The above structure indicates that a distribution scenario can have either zero or one (the ? symbol) of all of the scenario constraints, allowing for any or none of the dimensions to be specified in a scenario. The keyword ANY indicates that a particular element may have a ranged value inside of it, or a discrete value. The DTD also indicates elements that should have one or more values, such as dataTypes, which should have one or more (denoted by +) type XML tags inside of it. WeconstructedXMLDTDsforourdistributionscenariodatastructure,forour distribution connector profiles, for our answer keys, and for the domain knowledge passedalongtotheconnectorselectionalgorithms(e.g.,theconditionalprobability tablepassedtotheBayesianalgorithm,orthescorefunctionsdefinedforthescore- based algorithm). A full listing of all the DISCO DTDs is provided at [DIS07a]. 138 7.5 Command Line Tools The disco.tools package contains several command line tools that allow a user to interact with DISCO from a command line interface. The tools (shown in the bottom left periphery of Fig. 7.1) include: Selector Test Runner A tool to measure the amount of selection time (in miliseconds) of a series of connector selections using a particular DISCO selector. DCP Stats Generator A tool to examine a connector knowledge base and gen- erate statistics about a set of DCPs, including counts for each of their attribute values, allowing analysis of different connector families. Random DCP Generator A tool to randomly generate (using information about valid metadata values) distribution connector profiles that belong to differentfamilies,foruseinevaluation/benchmarkingofselectionalgorithms. Scenario Runner A tool to run connector selection algorithms on a set of one or more distribution scenarios, and output the results. Scenario Analyzer Atooltorunadistributionscenariothroughconnectorselec- tion and analysis, outputting precision, recall, accuracy, and error-rate. Scenario Analyzer Runner A tool to run connector selection/analysis on one ormoredistributionscenarios,individuallyprintingoutprecisionrecalldata, and printing out the aggregate precision recall data at the end of execution. Each DISCO command line tool exposes (at least) one element of the core DISCO API and framework. Initially these tools served as a means for evaluating DISCOandtestingit,however,thesecommandlinetoolshaveevolvedintouseable 139 interfaces for the system, allowing DISCO to be integrated into other applications developed in heterogeneous programming languages and environments 1 . 1 In essence, any system developed in any programming language that can execute command line applications can invoke DISCO through these command line tools. 140 Chapter 8 Evaluation To evaluate our work, we have conducted a series of focused experiments, per- formed theoretical algorithmic evaluation, and engaged in quantitative and qual- itative analysis of the different parts of the DISCO framework. In this chapter, we report on our evaluation strategy, and present the results of each evaluation, demonstrating DISCO’s prowess in each of the evaluation areas, and tracing our evaluation of DISCO to our original four hypotheses. Section 8.1 begins the evaluation through the presentation of results from two separate connector surveys that we conducted, asking experts, practitioners and studentsinthefieldsofdatadistributionandsoftwarearchitecturetomakeconnec- tor selections for our thirty canonical distribution scenarios described in Chapter 3. The results are used to further motivate DISCO’s utility and benefit to the state-of-the-art in this area. In Section 8.2 we evaluate DISCO’s connector model (the distribution connector profile) and its expressiveness in capturing the varia- tions amongst the thirteen connectors that we studied. In Section 8.3 we evaluate DISCO’s accuracy and precision in capturing an expert’s domain knowledge for connector selections. To this end, we utilize thirty real world distribution scenar- ios across the fields of planetary and earth sciences, and cancer research, identified in the context of our experience in each of these domains at NASA’s Jet Propul- sion Laboratory. In Section 8.4 we evaluate DISCO’s scalability both theoretically, using formal mathematical analysis, and empirically, by benchmarking DISCO’s performance in making large numbers of connector selections. Section 8.5 is home 141 toanevaluationoftheexpressivenessofDISCO,exploringtheabilityofourdistri- bution scenario language (described in Chapter 5) to capture the aforementioned canonical thirty distribution scenarios. In Section 8.6 we describe how we used performance data in the areas of efficiency, consistency, scalability, and depend- ability to refine our connector selections and validate their accuracy. In Section 8.7 we describe the integration of our DISCO software as a plugin web service to an external COTS interoperability assessment framework [BMMB07] and round out the chapter. 8.1 Connector Survey To ascertain the current state-of-the-practice in selecting connectors for highly distributed and voluminous data-intensive systems, we constructed a connector survey that included two key parts. The first part was a set of general questions used to calibrate the survey taker’s understanding of data distribution and distri- bution connectors. The second part of the survey consisted of the presentation of the same 30 representative distribution scenarios described in Chapter 3 and in Appendix II. For each scenario, we asked survey takers to respond with their first and second choice appropriate connector, and the rationale behind their decisions (The complete survey is provided in Appendix II). The survey was sent to 33 ‘experts’ in representative fields requiring data dis- tribution. Amodifiedversionofthesurveywasprovidedtostudentsinagraduate- level software architecture class at the University of Southern California (USC). Thestudentsconsistedofbothpractitionersinthefield(so-called“distanceeduca- tionnetwork”students, whoareprofessionalstakingtheclass)aswellasgraduate- level USC students. Below, we describe the results of both surveys, using them 142 Figure 8.1: Results of expert connector survey. to illustrate the apparent lack of any principled methodology for connector selec- tion in this domain, and to illustrate the need for a principled framework such as DISCO to improve overall understanding of this decision making progress within the domain. 8.1.1 Expert Survey The results of the expert survey are shown in Fig. 8.1, and are quite telling. The upper right quadrant of the figure shows the demographic breakdown of data distribution experts to whom the survey was sent. The experts included those from the fields of cancer research, planetary science, and earth science, from which thedistributionscenariosinthesurveywerederived. Thosesurveyedalsoincluded 143 experts from other large-scale data dissemination fields, including grid computing (6%, 2 surveyees), web technologies (6%) and open source and industry (12% each). Included in the experts surveyed were developers of at least three of the major connector technologies that we studied. The most prevalent answer (67%) received from these experts was No Response, which refers to experts who declined to respond to our survey, despite repeated attempts at contacting them. Of those who responded to our survey, 15% of the experts declined to fill out our survey because they were Not Com- fortable providing responses to our questions, some of them even suggesting that it was impossible to derive meaning out of what is currently in most cases an art form. Another 15% of experts declined our survey due to No Time, or lack of time to fill it out. Finally, 3% of those surveyed provided a (partial) response to our survey, filling out Part I in its entirety, but neglecting to answer Part II (which wouldrequire“muchfineranalysis”intheexpert’sownwords). The3%amounted to a single complete response. The implications of this are very important — only one person whom we surveyed was willing (and perhaps able) to even start our connector survey (though not finish it), and moreover, none of those surveyed pro- vided her decisions on what the most appropriate connectors to select were in the face of what should have been very familiar data distribution scenarios, similar in nature to what the expert architect would have faced during the course of her work. The results of the expert survey are highly indicative of what is an extremely difficult, and largely undocumented process. Many data distribution “experts” prefer not to be labeled as such (as indicated by our survey), and the remaining experts often are unwilling to discuss the rationale behind their decisions, or more disturbingly cannot point to rationale, instead referring to prior systems built and 144 experiencewithaparticular“pet”technology. Withoutaclearmeansofcapturing the knowledge from these experts, the connector selection decision making process remains ad-hoc, unrepeatable, and costly for any organization to engage in. This is especially the case within system evolution scenarios, and as data volumes grow each day. Furthermore, answers such as those provided by the experts surveyed are what our DISCO framework was designed to address. Throughtheuseofthetechniquesandarchitectureoutlinedinthisdissertation, we assert that we have developed an efficient, accurate and expressive framework foralleviatingtheproblemoforganizationaldatadistributiongurus,takingaqual- itative and quantitative step towards encoding their knowledge, and making their decisions repeatable. Our framework is also wholly applicable to the increasing trade-off space of software connectors for data distribution, and is extensible as new connectors (and families) are identified and evaluated for inclusion into data distribution systems. The data used to train DISCO’s selection algorithms, the input domain knowledge, was bred from discussions and elicited from data dis- tribution experts, not through our survey, but through face-to-face meetings, and through data structures that were more familiar to the experts (e.g., the Bayesian domain profile, or the score functions used as input to our selection algorithms). In the next section we will discuss our results from a similar connector sur- vey adapted for a graduate level software architecture class at the University of Southern California. 8.1.2 Software Architecture Class Survey In Spring 2007 we asked a graduate level software architecture class at the Univer- sity of Southern California to participate in our connector survey. The students’ input was elicited for the following reasons: 145 Figure 8.2: Results of software architecture class connector survey. • The graduate students enrolled in the class included not only full-time stu- dents, but professionals in industry taking the course remotely. 1 • Some (or all) of the professionals currently face or will face, the construction of data dissemination systems, which will involve selecting the appropriate connector(s). • The students in the course were being trained to make architectural design decisions (including connector selection). Thestudents’inputwasavaluabledatapointforidentifyingthecurrentindus- try state-of-the-practice in connector selection and for identifying the thought pro- cess behind it. The students are also representative of practitioners who must make connector selections in their day-to-day professions. 1 These students are referred to as “DEN” students, for Distance Education Network. 146 Each student and DEN student were provided with a modified version of the connector survey described in the previous section (and provided in full in Appendix II). The survey was divided up into 3 sets of 10 distribution scenar- ios, and 3 sets of 4, 4 and 5 connectors, respectively. Each student was given a modified version of the original survey comprising one distribution scenario group, and one connector group, ensuring that the aggregation of all the students who participated in the homework would comprise full coverage of all 30 scenarios and 13 connectors. The summary results of the survey are provided in Fig. 8.2. As shown in the upper right quadrant of the figure, 26% of those surveyed were practitioners in the field of software engineering, whereas 74% of those surveyed were on-campus students. As described in Section 8.1, the connector survey included a set of calibration questions to determine the survey taker’s level of understanding with regardstodatadistributionandwithregardstolargescaledistributionconnectors. The results of those calibration questions for the students surveyed is shown in the upper left quadrant of Fig. 8.2. All questions had negligible error rates except for one (Q3), which asked the students if dependability is the most important issue in a WAN-based transfer. As can be gleaned from the discussion throughout this dissertation,theanswertothatquestionistrueinsomecases,andfalseinothers, making the general answer false. However, nearly 60% of the students surveyed believed dependability to be the most important issue in all cases, highlighting a commonpitfallinconnectorselection, expertandnovicealike: focusingonasingle performance requirement or distribution scenario requirement and neglecting to thoughtfully consider the rest. The lower left quadrant of Fig. 8.2 shows the students’ connector selections broken down by scenario type: high (> 100 GB), medium (10 GB-100 GB) and low volume (< 10 GB). The majority of those 147 Figure 8.3: Appropriate and inappropriate connectors identified in software archi- tecture class connector survey. surveyed were agnostic as to the appropriateness/inappropriateness of connectors within a particular scenario type, however, overall those surveyed believed that the connectors that they were deciding amongst were most appropriate in the medium volume distribution scenarios and most inappropriate in the high volume scenarios. Thisindicatesageneralindecisivestancetowardsconnectorapplicability regardless of the underlying scenario data volume requirements. Finally, in the 148 lower right quadrant of Fig. 8.2, connector appropriateness by family is shown. By far, the most appropriate (and inappropriate) connector family as selected by those surveyed was the grid family of connectors. Interestingly enough, the grid family has the second-lowest number of connectors (2) amongst the 13 connectors present in the survey. Again, a general agnostic disposition towards connector appropriateness/inappropriateness within a family is present, showing a distinct indecisive nature for those surveyed overall when it comes to picking a particular connector family as most appropriate (or inappropriate). This is exacerbated by the overall percentage of connectors selected as appropriate and inappropriate across all scenarios, shown in Fig. 8.3. These results drive home the indecisive nature of those surveyed, students and practitioners alike. Nearly every connector is equally appropriate and inappropriate, modulo the FTP connector. As will be shown in the next section, FTP is the most pervasive form of file transfer, and likely the most familiar connector presented to those surveyed. A telling data point is the fact that the students labeled FTP as equivocally appropriate (24%) and inappropriate (20%) across all the scenarios. This suggests that the students listed FTP because of their familiarity with it, which is another pitfall (“pet technologies”) in connector selection and was also a running theme within the expert survey analyzed in the previous section. In the following section, we will describe each of the 13 connector technologies thatwestudiedingreaterdetail,anddemonstrateDISCO’sabilitytocapturetheir important properties. 149 Distribution Connectors P2P Client/Server Event Grid Bittorrent FTP UFTP Commercial UDP Technology HTTP/REST SOAP RMI CORBA GLIDE/Prism- MW Siena bbFTP GridFTP Figure 8.4: Distribution Connectors Studied. 8.2 Connectors Studied and their Variations We studied 13 distribution connectors across all four families that we identified (recall, P2P, grid, event-based and client/server), as shown in Fig. 8.4. We constructed distribution connector profiles (DCPs) for each of the 13 connectors. Below, we briefly discuss each connector studied. We follow by describing the relationship of each of the four connector families to the attributes and values cap- tured across all DCPs. The discussion aids in validating that our connector model is expressive, capturing the majority of the key properties required to differenti- ate connectors within the four identified families, helping to validate Hypothesis 2. Recall, Hypothesis 2 states that Data movement technologies can be modeled expressively and accurately using an explicit software connector model. The connectors that we evaluated were: FTP The classic File Transfer Protocol (FTP) defines a capability for clients to send data to servers using the underlying TCP/IP protocol of the public 150 Internet. FTP is an open standard and widely deployed in almost every operating system, is well documented, and is broadly accepted as the de facto data movement mechanism available in modern software. FTP is a client/server distribution connector. SCP The Secure Copy Protocol (SCP) [SCP07] allows for the secure transfer of files between two machines using the SSH protocol for authentication and encryption over a network. Akin to FTP, SCP is built on top of the underly- ing TCP/IP protocol and is a public, open standard. SCP is a client/server distribution connector. GridFTP GridFTP [FKT01] extends the FTP protocol with new features required for large volume, fast data transfer, such as striping, partial file access, and highly parallel stream-based usage of TCP/IP. GridFTP is the basic data movement mechanism provided by the Globus Toolkit, the widely adopted software package for implementing grid-based software applications. GridFTPisagrid-baseddistributionconnectorbecauseitnativelyintegrates into a grid services environment (i.e., has the ability to authenticate against grid security infrastructure, communicate using grid protocols, etc.). bbFTP bbFTP [bbF07] is an open-source parallel TCP/IP data movement tech- nology. bbFTP’s main capabilities are the ability to use SSH and certificate- based authentication, on-the-fly data compression, and customizable time- outs. bbFTP is a grid-based distribution connector for the same reasons as those of GridFTP above. UFTP UFTP [Bus05], or UDP-based file transfer protocol with multicast, is an open-source data movement mechanism designed for efficient and reliable transferoflargeamountsofdatatomultiplereceiverssimultaneously. UFTP 151 is particular effective over a satellite link (with two way communication), or over high-delay Wide Area Networks (WANs) where the reliability mech- anisms in TCP/IP severely under-utilize the available network throughput capabilities. UFTP is a client/server distribution connector. CUDP CUDP 2 is a commercially available product built on top of a proprietary protocol which fully utilizes the capabilities of UDP to send bursts of data from one place to another. CUDP builds on top of the SCP protocol to pro- vide secure, reliable, and most importantly fast transfer of voluminous data setsindependentofnetworklatencyandpacketloss. CUDPisaclient/server distribution connector. HTTP/REST HTTP/REST [Fie00] is a simple, light-weight protocol built on top of TCP/IP in which a client and server communicate by transferring control messages in the hyper text transfer protocol (HTTP) format, and streaming data back synchronously to the client. HTTP/REST is the de facto protocol used by the public Internet to browse web pages, transfer data and engage in distributed communication. HTTP/REST is a client/server distribution connector. CORBA CORBA [Vin97], or the Common Object Request Broker Architecture, is a middleware platform for distributed components. CORBA provides an underlying messaging substrate that is based on TCP/IP, and involves com- ponent registries and object brokers. CORBA is a client/server distribution connector. 2 Due to licensing restrictions, we are unable to reveal the commercial UDP technology we tested. Throughout this chapter we refer to the technology as “CUDP”. 152 SOAP SOAP [Ere04] is a simple protocol built on top of HTTP, in which mes- sages exchanged between servers and clients are XML formatted documents conforming to the SOAP envelope schema. SOAP is the communication pro- tocol and mechanism for the recently popular Web Services platform that encouragesthedevelopmentofhighlymodularserviceorientedarchitectures. SOAP is a client/server distribution connector. RMI RMI [Dow98] is Java’s implementation of remote procedure call, in which dataistransferredbetweencomponentsinthesystemthroughvaluedparam- eters which are streamed over TCP/IP. RMI is a client/server distribution connector. Bittorrent Bittorrent [Coh03] is a peer-to-peer based file sharing protocol, in which peers share pieces of a unique content item, and use geographic loca- tion, transfer rate, and reputation to transfer data amongst large numbers of users and systems. Bittorrent is a P2P distribution connector. GLIDE GLIDE [MMB + 05] is an event-based data sharing middleware built to asynchronously and synchronously transfer data between clients and servers in a distributed system. GLIDE provides grid services on top of the PRISM- MW[MMRM05]middlewareplatform. GLIDEisanevent-baseddistribution connector. Sienna Siena [CRW01] is an implementation of a publish-subscribe data sharing middlewareforlarge-scaledistributedsystems. Sienasupportscontent-based data dissemination in a scalable, efficient manner. Siena is an event-based distribution connector. As stated in Chapter 4, all distribution connectors initially engage in one or moredataaccesses. Fig. 8.5presentstheoverallattributedistributionforthedata 153 Figure 8.5: Data Access DCP attributes within connectors studied. access DCP metadata in all connectors studied. Nearly all connectors (12 of 13) support accessing data from file I/O as well as session-based access of transient data (11 of 13). Many (7 of 13) connectors support process-based data access. On the other hand, few (1 of 13) of the distribution connectors studied are able to accesstransientdatabycaching,orbypeer-basedaccess. Allconnectorssupported accessing data, while 9 of 13 connectors supported mutating it as well. 5 of 13 connectors were able to take the data they accessed and package it into streams using many senders, while 8 of 13 connectors used one sender to package the accessed data into streams. Finally, 6 of 13 connectors were able to access data using more than one receiver, while 7 of the 13 distribution connectors studied were limited to sequential data access via one receiver. Fig. 8.6 shows the breakdown of connector attributes for the Stream DCP metadata, present in all distribution connectors. All distribution connectors stud- ied support the packaging of data into both raw and structured streams, as well as the construction of local and remote data streams. All connectors allow stream 154 Figure 8.6: Stream DCP attributes within connectors studied. throughput in bits per second, and construct named, bounded streams. There are several variation points, however. For example, 8 of 13 connectors studied sup- port exactly once stream delivery, while only 2 of 13 connectors studied support at least once, and 1 of 13 support best effort. Another variation point is stream statefulness, where 6 of the 13 distribution connectors we studied support stateful streams, while 7 of 13 distribution connectors support stateless streams. Finally 4 of the 13 connectors were studied deliver streams asynchronously, whereas 9 of the 13 distribution connectors deliver their data streams in a synchronous fashion. All distribution connectors employ distributor connectors to disseminate their datastreams, asnotedinChapter4. Fig. 8.7highlightsthelargeamountofdiffer- ences between the Distributor DCP metadata captured for each connector. 3 of 13 connectors studied have the ability to use ad-hoc routing membership; the remain- ing 10 must use bounded routing membership for the delivery of their streams across the network. Delivery type is a stark dimension of contrast. At most, 2 of the13connectorsdeliveranHTTPmessage,andanother2deliverdatastreamsvia 155 Figure 8.7: Distributor DCP attributes within connectors studied. events. The 9 remaining connectors all utilize different deliver types, ranging from FTP messages, to bbFTP messages, to SOAP XML-formatted messages, to peer- pieces. 11 of the 13 distribution connectors studied used registry- based naming to identifyrecipientsofdatadeliveries,whiletheother2connectorsrequiredattribute basednamingmechanisms. 9distributionconnectorsemployedhierarchicalrouting structures, while the remaining 4 employed flat routing. The routing type utilized by each distribution connector was another high variation point. 2 of 13 con- nectors used UDP-based routing, one connector employed content-based routing systems, whereas 8 relied on the routing ability provided by TCP/IP alone. One distribution connector (GLIDE) utilized architecture configuration-based routing, another(Bittorrent)usedtrackersforrouting. Alargenumber(8of13)connectors were able to deliver their data using exactly once semantics, whereas 3 connectors supported at least once, and 5 of 13 connectors supported best effort delivery. 3 3 The number of connectors does not add up to 13 for this attribute because the distributor delivery semantics DCP attribute is a multi-valued property, so a distribution connector can support more than one type of semantics. 156 Figure 8.8: Event DCP attributes within connectors studied. 5 of 13 connectors were able to use a dynamic data routing path, 3 connectors were able to use a cached routing path, and 11 of 13 connectors were able to use a static routing path. All connectors supported unicast data delivery, 3 connectors supported multicast, and 5 of 13 connectors supported broadcast data delivery. Two(GLIDEandSiena)ofthethirteendistributionconnectorsthatwestudied usedanevent-basedinvocationmechanism. Bothconnectorsbelongedtotheevent- based family of distribution connectors. As explained in Chapter 4, this family employs an event-based primitive connector for initial invocation and data access anddeliverypurposes. Themetadatathatwecapturedforthesetwoconnectorsfor their event-based DCP properties is shown in Fig. 8.8. Each connector supported a different form of event notification, one publish-subcribe (Siena) and the other (GLIDE)queueddispatch. Bothconnectorssupportedunicast,aysnchronous,mul- ticast, broadcastandbestefforteventdeliverypaths, whileonlyoneoftheconnec- tors (Siena) supported publish-subscribe 4 . One event connector employed clients 4 Though GLIDE does support publish-subcribe, it does not support it natively. 157 Figure 8.9: Procedure Call DCP attributes within connectors studied. for event cardinality observation, while the other connector utilized threads for the same. Both connectors allow consumers to set the incoming event priority, and producers to set the outgoing event priority. One connector (Siena) leveraged access points for producers, whereas the other connector (GLIDE) leveraged com- ponents to produce events. Both connectors support many observers of events. Grid-based (e.g., GridFTP) and client/server connectors (e.g., FTP) are invoked through procedure calls. The relevant DCP metadata for procedure calls is shown in Fig. 8.9. As seen in the figure, the metadata captured to describe procedure calls is a high variation point in all the connectors. For example, 2 of 10 connectors have procedure call return values of UNIX process, while the other 8 all had different types of return values, e.g., HTTP Response messages, bbFTP messages, CORBA messages, etc. All 10 connectors support explicit invo- cation through a method call, whereas the location of the invocation record varies amongst nearly all of the 10 connectors. 3 of 10 connectors store their invocation 158 record in the operating system’s process table, whereas the other 7 all store their invocation record in various locations, such as the HTTP server log, the RMI reg- istry, or the web server log. 8 of the 10 connectors supports data transfer by value, whereas 6 of the 10 support data transfer by reference. All 10 connectors allow public invocation, while 8 of 10 allow protected, and even fewer, 5 of 10, allow private invocation. All 10 connectors have exactly one receiver of the procedure call invocation, indicating their high level of synchronicity. The P2P family of distribution connectors leverages arbitration as a means of invocation, as detailed in Chapter 4. Despite the small sampling of connectors from this family, there is still a large amount of variation captured by the DCP metadata for this connector. As an example, Bittorrent is multiple transaction- aware, allowing roll-back of an invocation or data transfer request if there is an anomaly or failure during the invocation. Point-to-point concurrency within the connector is heavy-weight requiring a wealth of resources to support communica- tion and synchronization between the peers involved in the transfer. Scheduling of invocation is based on fairness, and on peer capability. Invocation security is han- dled through the SHA algorithm, and dealt with using rendezvous. Fault handling is dealt with using voting. 8.3 Accuracy OuraccuracyexperimentsarefocusedondemonstratingthatDISCOiswellsuited to capturing a guru’s domain knowledge, accurately and precisely, alleviating the problem of anecdotal evidence with regards to distribution connector selections. To this end, we devised an experiment that will properly highlight that DISCO is highly accurate, and under certain conditions, is able to capture as much as 80.6% 159 of a guru’s domain knowledge using our connector selection algorithms. This will validate the second major element of ourHypothesis 3, which states Given a set of≥ 1 OTS distribution connectors, and distribution scenarios and performance requirements that must be satisfied, an efficient algorithm can calculate within 20% precision of a guru’s domain knowledge as to what connectors to select to meet the requirements. Our accuracy experiments involve the application of the k-means connector clustering algorithm, that generates “appropriate” and “inappropriate” sets of connectors, described in detail in Chapter 6. The experiments applied this algo- rithmtoresultsofconnectorselectionfromthe30real-worlddistributionscenarios described in Chapter 3. Using our collective domain knowledge, we devised an “answer key” ahead of time in which we classified each connector analyzed into one of the above two connector sets: appropriate for the scenario or inappropriate. Our collective domain knowledge included first-hand experience by this disserta- tion’s author in the construction of data distribution systems, as well as interview information from admitted architects of the data distribution systems in planetary science, cancer research, and earth science, along with information obtained from the literature. The answer key was used in tandem with the results of the con- nector clustering to create a summary confusion matrix for all 30 scenarios. We variedthefollowingamongtheDISCOselectionalgorithmsinordertomeasurethe ability of each algorithm to capture different amounts of guru domain knowledge given as input: 1. number of conditional probabilities in the Bayesian algorithm 2. number of score functions in the Score-based algorithm 3. number of competency functions in the Optimizing algorithm 160 Table 8.1: Confusion matrix used in precision-recall analysis of three connector selection algorithms with varying amounts of knowledge. TP TN FP FN Low Knowledge Bayesian 35 247 19 89 Score 74 165 101 50 Optimizing Medium Knowledge Bayesian 40 246 20 84 Score 79 164 102 45 Optimizing High Knowledge Bayesian 100 242 24 24 Score 73 169 97 51 Optimizing 63 129 137 61 We computed precision/recall and error rate from the confusion matrix apply- ing the techniques described in Chapter 6, and recorded the results. For each of the above three domain knowledge inputs to each selection algorithm, we con- structed low, medium and high knowledge bases for use as input to each algorithm. In all cases, low indicated low-quality, “dumbed down” versions of the domain knowledge used as input to the selection algorithm. The low sets consisted of 10 randomly-chosen conditional probabilities, 8 score functions and 8 competency functions used as input to the Bayesian, Score-based and Optimizing selection algorithms, respectively. The medium quality domain knowledge bases contained more decision making information. This knowledge based provided 50 randomly- chosen conditional probabilities, 16 score functions, and 16 competency functions for the Bayesian, Score-based and Optimizing selection algorithms. Finally, high quality knowledge bases included all as-developed domain knowledge that we have constructed so far, including 100 conditional probabilities, and 24 score and com- petency functions (these probabilities and functions are provided at [DIS07b]). 161 The confusion matrix generated as the result of our experiments is shown in Table 8.1. Due to the nature of the Optimizing selection algorithm, we were only able to generate data for a high quality domain knowledge base. As shown in Chapter 6, this algorithm tries to optimize the provided objective function using a number of means, including Hooke-Jeeves [HJ61]. To perform this optimization, the algorithm requires at least a full set of competency functions for all scenario dimensionsprovidedinthescenariobeingevaluated,makingresultsonlysignificant inthecaseofahighqualitydomainknowledgebasewithallsuchfunctionsdefined. As shown in Table 8.1, lower quality domain knowledge for the Score-based algorithm was able to identify more true positives and fewer false negatives than the Bayesian algorithm with the same quality input. This trend continued for the medium quality knowledge profile. However, for the high knowledge profile the Bayesian algorithm was the clear winner, identifying 100 out of 124 true positives and and 242 out of 266 true negatives among the connectors and scenarios. For the same knowledge profile, the Score-based algorithm was able to identify more true positives and true negatives than the Optimizing algorithm, which performed the worst among the three. Figure 8.10 summarizes the results of the precision-recall analysis that we per- formed for all three selection algorithms. For low-medium quality knowledge pro- files,theBayesianandScore-basedselectionalgorithmsarerelativelycloseinpreci- sion rate (72.3-73.3% compared to 61.2%-62.3% for the Bayesian and Score-based, respectively); however, the Bayesian algorithm with a high quality knowledge pro- file is the most precise of the three algorithms, with 80.6% precision. All algo- rithms are at least 49.2% (Optimizing algorithm) and at most 87.7% (Bayesian algorithm) accurate, which makes all three of them clearly applicable for the con- nector selection decision making process. For low and medium quality domain 162 Figure 8.10: Precision-recall analysis of the three connector selection algorithms demonstrating low, medium, and high knowledge for each selection algorithm. knowledge, the Score-based algorithm demonstrates higher recall (59.6%-63.7%) than the Bayesian (28.2%-32.2%), though for a high quality knowledge profile the Bayesian shows the highest recall (80.6%) compared to 58.6% and 50.6% for the Score-based and Optimizing algorithms respectively. For error-rate across all low-high quality knowledge profiles the Bayesian algorithm demonstrates the low- est, while the Score-based algorithm maintains a fairly consistent error rate inde- pendent of the domain knowledge given. The Optimizing algorithm’s error rate (50.7%) is by far the highest of all three algorithms across any profile. The high degree of accuracy and precision exhibited by our Bayesian algorithm can be attributed to the fact that the Bayesian algorithm is a white-box approach, requiringthearchitecttounderstandhowtheinternalworkingsofaconnectorsuit a given distribution scenario. The Score-based and Optimizing algorithms, while both more efficient than the Bayesian (as will be shown in the next section), both 163 showlowerprecisionandaccuracy(andhighererrorrate)duetothefactthatthey require the user to have an understanding of how performance requirements and distributionscenariosinfluenceeachotherdependingontheconnector,ignoringits internalarchitecture. Thisleavesadisconnect, inthatthearchitectmustmaintain an implicit mental model of the connector, as opposed to the Bayesian algorithm, where this model is made explicit. In addition, we believe that the compact yet powerful nature of the Bayesian algorithm’s required input domain knowledge (recall a set of conditional proba- bilities, akin to Table 6.9) is a contributing factor to its superior nature in all evaluated dimensions. Each conditional probability is a simple statement, relating a property (e.g., transient availability) of the connector to the probability that value (e.g., Cache) for that property is applicable in the face of a particu- lar scenario constraint (e.g., Total Volume = 10 GB). This simple construct is much easier to understand than having to perform the arduous task of profiling 100s of performance functions for all connectors in the knowledge base, as in the case of the Score-based and Optimizing algorithms. These functions relate sce- nario constraints (e.g., Total Volume = 10 GB) to a observed property (e.g., high scalability) of a connector. This currently time consuming task could be made simpler through the use of approximation techniques (or empirical evaluation) for identifying functions relating distribution scenario requirements and performance; however, those techniques do not exist at the present time. 5 Black-boxapproachesliketheScore-basedandOptimizingselectionalgorithms do have their place though. In our experience, there will be gurus within an organization who are only able to encode their domain knowledge in the form of performance relationships (and other observed properties) of the connectors, and 5 We do suggest such techniques as part of our future work, in Chapter 9. 164 the different scenarios in which they have used them. In these situations, it is better to provide at least some means of encoding the guru’s domain knowledge, and we believe, despite their limitations as shown in the precision-recall analysis, that our black box algorithms provide complimentary methods of accomplishing thistask. Infact,thisease-of-usefactorforcertainorganizationaldatadistribution gurus suggests itself as the cause for the Score-based algorithm’s higher recall and similarprecisionrate(tothatofitswhite-boxBayesiancounterpart)withthelower and medium quality knowledge profiles. One of the important aspects of developing connector selection algorithms cur- rentlyandwithaneyetowardsthefutureishavinganappropriateunderstandingof thequalityoftheunderlyinginputdomainknowledge. Forexample,thelargejump (nearly 50 percentage points) in the recall rate of the Bayesian algorithm between low, medium, and high-precision input is a testament to the fact that some of the 50 additional conditional probabilities added into the high quality knowledge profile were more applicable to the distribution scenarios evaluated and their asso- ciated constraints than that of the initial 10 and 50 probabilities used for the low andmediumqualityknowledgeprofiles. Qualitymayalsohavebeenacontributing factor in the Score-based algorithm’s relative precision-recall independence from underlying domain knowledge at the low and medium knowledge profile levels: if the additional score functions present in the medium and high-precision inputs did not provide sufficient variation between the connector technologies, then the resultant connector ranks would not change from those of the low-precision inputs. As detailed in Chapter 9, part of our future work is developing strong quality met- rics that assess the actual quality of domain knowledge profiles, allowing greater flexibility and accuracy in tailoring and capturing of a guru’s domain knowledge. 165 8.3.1 Sensitivity The accuracy of each of the connector selection algorithms is driven in large part by the domain knowledge used as input to them. This is corroborated by the accuracy results (e.g., for the Bayesian selection algorithm) reported above. The domain knowledge used as input to a connector selection algorithm is typically incomplete, and representative of the sampling of experts from which the knowl- edge was collected. This is largely a “hit and miss” process. The selection algo- rithm may contain knowledge (e.g., a function) mapping a particular distribution scenario constraint to a performance dimension (in the case of the Score-based and Optimizing algorithms), or it may contain knowledge mapping a connector property to a scenario constraint (e.g., in the case of the Bayesian algorithm). If this knowledge is present, it can be considered a “hit”, meaning that the selection algorithmwilltakeintoconsiderationtheconnectorpropertyorscenarioconstraint when ranking the connectors provided as input to it. On the other hand, if any such knowledge (in either case) is omitted, then it can be considered a “miss”, meaning that the scenario dimension or connector property will be ignored, and will not influence a connector’s rank. This “hit and miss” process can cause parity within connector rankings in the case that many of the connectors being evaluated by the selection algorithm have similarDCPproperties(forawhite-boxselectionalgorithm), orscore/competency functions (for a black-box selection algorithm). Because of the nature of certain DCP properties, without appropriate variation between the conditional probabil- ities mapping a particular scenario constraint (e.g., Total Volume = 10 GB) to different values for the connector property (e.g., data access locality = Cache, or data access locality = Session), many connectors will be similarly, in some cases equally, ranked. The same goes for kindredly defined connector score functions. 166 Many black-box connector experts form an equivalence between certain kinds of connectors, i.e., those from the same family (e.g., GridFTP and bbFTP) or those built using a similar technology (e.g., HTTP/REST and SOAP), which leads to comparable,ifnotequivalent,scorefunction(andcompetencyfunction)definitions that can also cause parity amongst the connector rankings. InthecaseoftheconnectorDCPproperties,theissuelieswithinthesometimes coarse-grained property-value space used to classify a particular connector, and those from its family. These properties could be made finer-grained by identifying more values for them, if they exist. The problem with this strategy, however, is that within the distribution connectors that we studied, much of the connector classification beyond the canonical properties and values that we have identified are anecdotal. The same is true for the score functions that capture the observable connector properties. Take for example the score function relating Number of Users to efficiency. A wealthofempiricalevidencewouldberequiredtocompletelyandaccuratelydeter- mine all of the canonical values for this function, and even then those values may end up providing incomplete evidence (e.g., a user may experience high efficiency up to 500000 users, but any value beyond that causes low efficiency). Because of the inherent incomplete nature of domain knowledge, in our experience the best strategy is to provide a means for the experts to approximate their experience using a connector (black box), or understanding its internal architecture (white box), and then use those approximations to drive the process of connector selec- tion. As shown in the accuracy results above, our strategy can be highly accurate in replicating the same decision made by a distribution connector domain expert, and providing the explicit rationale behind the decision as well. 167 Additionally,becauseofthenatureofclusteringconnectorranklistsintoappro- priate/inappropriateconnectors, finer-grainedquantizationofDCPpropertiesand score functions in most cases has minimal effect. As an example, consider the fol- lowing two connector rank lists (capped at three connectors for brevity). The first connector rank list contains 2 equally ranked connectors, and is illustrative of the connectorparityresultantfromourcoarse-grainedDCPpropertiesandscore/com- petency functions. The second connector rank list contains 3 differently ranked connectors, and is illustrative of using a more finer-grained quantization: Connector Rank List 1 (A, .20991) (B, .20991) (C, .15001) Connector Rank List 2 (A, .25991) (B, .22991) (C, .15001) The results of providing each of these lists as input to our k-means clustering algorithm described in Chapter 6 are provided in Table 8.2. In the table RL 1 refers to Connector Rank List 1, and RL 2 refers to Connector Rank List 2. App indicates the set of connectors being considered as appropriate, and Inapp indicatesthesetofconnectorsbeingconsideredasinappropriate. avg d (RL 1 )isthe average distance of each connector from the centroid (or mean value) of the con- nectorgroupfor Connector Rank List 1;avg d (RL 2 )isdefinedsimilarly. Finally, avg d (RL 1 )+avg d (RL 2 ) is the sum of both average distances — as indicated in the k-means algorithm listing within Chapter 6, this is the value used to compute the appropriate/inappropriate connector groups during clustering. 168 Table 8.2: Fine-grained versus coarse-grained quantization of DCP properties and score functions and its implication on connector clustering. App avg d (RL 1 ) Inapp avg d (RL 2 ) avg d (RL 1 )+avg d (RL 2 ) RL 1 (A,B,C) 0.079866667 () 0 0.079866667 (A,B) 0 (C) 0 0 (A,C) 0.0599 (B) 0 0.0599 (B,C) 0.0599 (A) 0 0.0599 (A) 0 (B,C) 0.0599 0.0599 (B) 0 (A,C) 0.0599 0.0599 (C) 0 (A,B) 0 0 () 0 (A,B,C) 0.079866667 0.079866667 RL 2 (A,B,C) 0.126533333 () 0 0.126533333 (A,B) 0.03 (C) 0 0.03 (A,C) 0.1099 (B) 0 0.1099 (B,C) 0.0799 (A) 0 0.0799 (A) 0 (B,C) 0.0799 0.0799 (B) 0 (A,C) 0.1099 0.1099 (C) 0 (A,B) 0.03 0.03 AscanbeseenfromtheboldedvalueswithinTable8.2,forbothconnectorrank listsRL 1 andRL 2 , thegroupingof(A,B)asappropriateand(C)asinappropriate are the results obtained from the k-means clustering algorithm. Recall that the algorithm looks for the two connector groups with the lowest summed average distance. If two groupings (of appropriate/inappropriate connectors) have the same lowest summed average distance, then the connector grouping that correctly contains the highest ranked connector in the appropriate set is chosen. As evidenced by the results of this postulated scenario, if there is not sufficient differentiationbetweentheconditionalprobabilitiesorscore/competencyfunctions provided as input domain knowledge to the selection algorithms, then the impli- cations of fine versus coarse grained quantization are of little importance because each strategy produces the same connector groupings. 169 8.3.2 Relaxing Scenario Requirements As discussed above, each of the connector selection algorithms is driven by three main inputs: 1. The connectors to rank 2. The input data distribution scenario 3. The domain knowledge that the selection algorithm is provided Clearly, we have shown that both families, white box and black box, of con- nector selection algorithms are highly sensitive to inputs 1 and 3. As part our evaluation, we consider the sensitivity of each selection algorithm to its input data distribution scenario, specifically with regards to relaxing one or more of the sce- nario constraints, and then determining its effect on a selection algorithm’s output connector rank list, and its accuracy. For example, consider a data distribution scenario that originally specifies 1,000,000 users, and causes all connectors that it evaluates to be ranked equivalently. Would a clear winner emerge if the selection algorithm considered the data distribution scenario to have 500,000 users? It would appear that the answer to the above question is driven almost entirely bythedomainknowledgeprovidedasinputtotheselectionalgorithm. Thedomain knowledge provides the critical mapping between the distribution scenario con- straint, and a connector’s architectural (white box) or observable (black box) nature. Put concretely, if the domain knowledge provides the same or no dif- ferentiation for 1,000,000 users as it does for 500,000 users, then the results will be the same as well: the connectors will still be ranked equivalently. Within the context of the 13 connectors that we studied, however, it became apparent that domain knowledge is only one influential element in understand- ing how relaxing scenario constraints affects the output of a selection algorithm, 170 Table 8.3: Relationship between connector families and distribution scenario dimensions. Family Dimensions P2P Number of Users, Total Volume, Number of Delivery Intervals, Geographic Distribution Grid Total Volume, Number of Delivery Intervals, Geographic Distribution, Access Policies Event Number of Delivery Intervals, Volume Per Interval, Number of User Types Client/Server Total Volume, Number of Users, Number of Delivery Intervals, Geographic Distribution specifically with regards to the value assigned to the scenario dimension (e.g., 10 MB assigned to Total Volume) in the constraint. In our experience, a connector’s family is also highly influential in driving the domain knowledge captured for it, and ultimately how influential the distribution scenario dimensions are within a connector’s ranking. For instance, the data distribution system architects whom we interviewed and solicited domain knowledge from were prone to provide similar black and white box knowledge for connectors from the same family, or as-stated in the previous section, constructed using the same technology. As an example, an architect highly familiar with scaling a data distribution system to large numbers of users may provide similar favorable score/competency functions or conditional probabilities for all connectors within the P2P family. This trend suggests that there may be a particular set of distribution scenario requirements that causes a family of connectors to emerge as a highly ranked candidate, and that causes connectors outside of the family to be less relevant. Table 8.3 suggests our experience with each family and the important scenario dimensions by which it is influenced. The table is not meant to be exhaustive but to be more of a guideline that can be used to influence decision making amongst similarly ranked connectors for classes of distribution scenarios. 171 P2P distribution connectors, as described in Chapter 4, are heavily influenced by the amount of users on the network, the amount of (allowed) delivery intervals between two peers, whether or not the distribution connector is run over a Local Area Network (LAN) versus a Wide Area Network (WAN), and the total volume of data to be delivered. Grid distribution connectors are also influenced by total volume and the number of delivery intervals as well as the underlying network; wheretheydifferisintheirsupportforcertaindomain-specificaccesspolicies(e.g., Grid Security Infrastructure, or linux file system permissions). Event connectors are almost entirely influenced by the number of user types to deliver data to (and their preferences), the volume per delivery interval, and the amount of delivery intervals. Finally, client/server distribution connectors are influenced by total volume, number of users, number of delivery intervals, and by the underlying geographic distribution of the network. Thisrelationofconnectorfamiliestorequirementspresentsitselfasinitialguid- ance towards weeding out the “best ranked” connector amongst a group of those thataresimilarlyranked. Inturn, Table8.3certainlysuggeststheimportanceand influence of distribution scenario dimensions to a distribution connector family, and ultimately to a distribution connector’s, overall applicability to a distribution scenario. 8.4 Scalability In order to validate that our selection algorithms performed efficiently, we for- mally evaluated the time complexity of our three algorithms. Ensuring that our algorithms make decisions efficiently increases their utility in making real-time software connector selections for users and software systems, especially as new 172 connectors are identified. With respect to our specific research, these evaluations aidinvalidatingthefirstmajorelementofHypothesis3,whichstatesGivenaset of≥ 1 OTS distribution connectors, and distribution scenarios and performance requirements that must be satisfied, an efficient algorithm can calculate within 20% precision of a guru’s domain knowledge as to what connectors to select to meet the requirements. In addition to the theoretical analysis, we empirically mea- suredrunningtimesofthealgorithms, varyingthesizeoftheunderlyingconnector knowledge bases from 100 to 1000 to 10000 DCPs, and the number of distribu- tion scenarios evaluated from 1 to 10000. Our formal arguments and empirical evaluation demonstrate that each algorithm runs in less than polynomial time, the canonical benchmark for efficiency in theoretical computer science. 8.4.1 Time Complexity of Bayesian algorithm The Bayesian algorithm uses the set A of DCP attributes defined in Chapter 6. In addition, the set D is the set of distribution connector profiles that the algorithm will iterate over and rank. We will assume that getValRange takes O(1) time as the function is responsible for returning all valid values for an attribute a, making it easily implemented using a hash structure with O(1) lookup time. For the bayes function we will assume a O(n) running time, consistent with the published literature on the running time of Bayes theorem [Win03]. Given this, the main part of the algorithm (the two for loops) takes Θ(|D||A||O|n). The sort and normalize, using an efficient sort algorithm, takes Θ(nlogn) in the number of rank elements,|D|. So, the complete running time is Θ(|D|log|D|+|D||A||O|n). However, we know the max size of each of the three sets,|D|,|A|, and|O|, allowing us to reduce the running time to Θ(|D|log|D|+n) and to reduce the sort time as well to Θ(|D|) time. This reduction eliminates the 173 Figure 8.11: Empirical evaluation showing selection time within Bayesian algo- rithm using knowledge bases of 100, 1000, and 10000 DCPs. first term altogether and results in the actual running time, O(n), where n is the number of conditional probabilities input into the algorithm (in our case, the size of the input domain profile). Thus, the Bayesian algorithm is linear in the running time. Our empirical evaluation of the Bayesian algorithm is shown in Fig. 8.11. The amountoftimeforconnectorselectionandrankingrangedfrom1secto5.56hours 174 with the 100 DCP knowledge base, and from 17 seconds to 55.56 hours (approx- imately 2 days 7 hours) with the 10000 DCP knowledge base. To generate these knowledge bases, we used the Random DCP Generator tool described in Chapter 7 to generate random DCP profiles with random attribute values drawn from the appropriate value space defined in our DCP model. 8.4.2 Time Complexity of Score-based algorithm TheScore-basedconnectorselectionalgorithmrequirestwosets,theS distribution scenario constraints and the D knowledge base of DCPs (let their sizes be|S| and |D|, respectively). In addition, it requires four score functions, f e , f d , f c and f s , one for each performance attribute. Assuming that it takes 1 time step to execute each f function (reasonable in the sense that each score function is a mapping between a set of empirically deter- mined “important” points and their associated impact values on the performance attributes, making it ideal for an O(1) hash lookup), the first portion of the algo- rithm (the two for loops) takes Θ(|S||D|) time. The sort and normalize portion of the algorithm (again, using an efficient sort algorithm) takes Θ(|D|log|D|) time. So overall, the algorithm takes Θ(|S||D|+|D|log|D|) time steps to execute. Akin to the Bayesian algorithm, since we know the max size of the sets|D| and|S| a priori, we can sort it in Θ(|D|) time, removing the 2nd term altogether, and resulting in a running time of Θ(|S||D|). Again, because we know the size of both of these sets, they too can be reduced to constant time, and thus the running time of the algorithm is Θ(1). Our empirical evaluation of the Score-based algorithm corroborates our theo- retical analysis, and is shown in Fig. 8.12. The amount of time for connection selection and ranking ranged from 2 miliseconds to 1.5 seconds in the 100 DCP 175 Figure 8.12: Empirical evaluation showing selection time within Score-based algo- rithm using knowledge bases of 100, 1000, and 10000 DCPs. knowledge base and from 40 miliseconds to 1 minute, 4 seconds in the 10000 DCP knowledge base. 8.4.3 Time Complexity of Optimizing Algorithm The running time of the Optimizing Algorithm can be compared to that of the Score-based algorithm, i.e., Θ(1), with additional time introduced by the running 176 Figure 8.13: Empirical evaluation showing selection time within Optimizing algo- rithm using knowledge bases of 100, 1000, and 10000 DCPs. time of the Hooke-Jeeves [HJ61] generalized pattern search algorithm (the else case of the if|S 0 |> 1 condition in the middle portion of Algorithm 3 in Chapter 6). Hooke-Jeeves is an exploratory optimization technique with a running time that is linear in the amount of “states” and “links” between those states (call the set of states S and the set of links L). 177 States, in the case of the Optimizing algorithm, can be thought of as the dif- ferent potential values for the relaxed distribution scenario constraints. In the most general case (e.g., the case of continuous functions), this value domain is an infinite set. However, for the purposes of our work, we have reduced this set trivially to a set of empirically determined ”important” points, P, whose size |P| is fixed. Consider, too, that from any previous explored point (e.g., Total Volume = 10 MB), any next point (e.g., Total Volume = 100 GB) is a valid one to explore for optimization. So, the number of links|L|, between each state is equivalent to|P|× (|P|− 1). Thus, the total time for Hooke-Jeeves in our case is bounded by Θ(|P| 2 (|P|−1)), which, because we know the size of|P|, can be reduced, making the running time for the Optimizing algorithm constant time Θ(1). The empirically-gathered results of our scalability experiments shown in Fig. 8.13 corroborate our theoretical analysis. The amount of time for connector selec- tion and ranking ranged from 15 miliseconds to 35 seconds in the 100 DCP knowl- edge base and from 446 miliseconds to 1 hour and six minutes in the 10000 DCP knowledge base. 8.4.4 Discussion The Score-based algorithm by far is the most scalable of the three algorithms. Clearly this has to do with its reliance on simple addition and fast lookup func- tions to tally a score for the connectors in the knowledge base. Similarly, the Optimizing algorithm is driven by its own fast lookup functions, and uses multi- plication, rather than addition, to tally up a connector’s score. In the cases where the Optimizing algorithm uses Hooke-Jeeves to approximate the best value for 178 an unspecified scenario dimension, as shown in the prior section, the Optimizing algorithm still runs in constant time. The Bayesian algorithm, on the other hand, is bounded entirely by the time complexityofBayestheorem,O(n),andisbyfartheleastscalableofthethreealgo- rithms. However, even though it is the least scalable, the algorithm is still highly efficient. Considering that there are, to date, at most less than a hundred distri- bution connectors identified, the Bayesian algorithm still provides highly accurate results in a reasonable amount of time (1 second for 1 connector rank list, and around 14 minutes for 1000 connector rank lists). Because of this, the Bayesian algorithm is most appropriate in situations where there is a human in the deci- sion making process, and where the decisions made are design-based, and not real time. The Score-based algorithm, because of its high efficiency (2 milliseconds for 1 connector rank list, and 182 milliseconds for 1000 connector rank lists), and reasonable accuracy, is most appropriate for real-time connector selections, such a self-optimizing data distribution system where decisions must be made extremely quickly, without a human-in-the-loop. 8.5 Expressiveness As explained in Chapter 5, we have developed a distribution scenario language for use in capturing the key requirements of a distribution scenario. The language is built from eight canonical dimensions, that we will reiterate and summarize here for convenience: 1. Total Volume — the total amount of data that needs to be transferred from providers of data to consumers of data. 179 2. Delivery Intervals — the number, size, and frequency (timing) of intervals wherein which the volume of data should be delivered. 3. Performance Requirements — any constraints and requirements on the consistency, efficiency, scalability, and dependability of the distribution sce- nario. 4. Number of Users — the number of unique users to whom data volume needs to be delivered. 5. NumberofUserTypes—thenumberofuniqueusertypes(e.g.,scientists, students) to whom the data volume needs to be delivered. 6. Data Types — The number of different data types (e.g., data, metadata) that are part of the total volume to be delivered. 7. Geographic Distribution — The geographic distribution of the data providers and consumers. 8. AccessPolicies—Thenumberandtypesofaccesspoliciesinplaceateach producer and consumer of data. Our scenario language is based on combinations of constraints assigned to each of these dimensions. To evaluate the expressiveness of our language, we used it to construct a set of thirty representative distribution scenarios derived from our experience in developing real-world data distribution systems at JPL in three scientific domains. In addition to the three domains, we also tried to extrapolate some representative data distribution scenarios from everyday life, such as music sharing activities, and movie distribution and downloading. The results of using the scenario language to describe the distribution scenarios is provided within the 180 secondportionoftheconnectorsurveyinAppendixII.Thethirtyscenariospresent in the second part of the connector survey are the same thirty scenarios described in plain English language within Chapter 3 in the context of data distribution system domains. Theabilityofourdistributionscenariolanguagetocapturedistributionscenar- iosfromthesecommondomainsaidsinvalidatingourHypothesis1,whichstates Eight key dimensions of data distribution can describe the wealth of data distribu- tion scenarios that exist. Each of the eight dimensions are interrelated, and have different levels of influence on the types of connectors that suit their constraints. The distribution scenario dimensions, the available classes of distribution scenar- ios, and performance requirements on those scenarios are highly interdependent. One of the two forms of input to DISCO’s connector selection algorithms is a data distribution scenario, expressed using our language (recall the other required input is a set of connectors to rank). The connector selection algorithms use the two given inputs, and expert domain knowledge (e.g., a conditional probability table for the Bayesian algorithm) to rank the given input connector list, allowing a user to select the appropriate highly ranked connector(s) from the connector rank list. As shown in Section 8.3, the selection algorithms that we have developed are highly accurate in capturing the same decision that an expert would make. 6 Part of the reason for this is the novel representation of the domain knowledge, but part of the reason for this is also the novel representation of a data distribution scenario, using our language. Within the context of over thirty data distribu- tion scenarios, initially expressed to us in plain English by several different archi- tects on data distribution systems within planetary science, earth science, cancer 6 More generally, the algorithms can capture the decision of any human from which domain knowledge has been captured. 181 research, the scenario language has provided us with a common medium for rep- resenting the explicit desirements of data distribution architects for the scenarios that their systems must support. Clearly, with a less expressive language missing the appropriate dimensions (e.g., Total Volume) and vocabulary, more of the data distributionsystemarchitects’desirementswouldbelosttransitioningfromaplain English description to a data representation format that DISCO can understand. As shown with the Optimizing selection algorithm, suggesting optimal values or guessing values for relaxed (or unspecified) scenario dimension constraints yields a significant hit on the accuracy of a connector selection. On the other hand, there are a wealth of different data distribution scenarios beyond the thirty that we investigated. These other scenarios involve other value constraints on the scenario dimensions (e.g., Number of Delivery Intervals = 46) than those present in the thirty scenarios investigated. We would argue; however, that the thirty scenarios that we investigated were representative of the potential variations in distribution scenarios across any domain. Clearly, the four domains thatweinvestigatedarehighlyheterogeneousscientificdomains(planetaryscience on one end of the spectrum to cancer research on the other) — yet still, the same scenario language was used to accurately represent the distribution scenarios con- veyedtousinplainEnglishbyeachrespectivedatasystemarchitect. Thissuggests that the distribution scenario language itself captures the canonical requirements of a distribution scenario independent of the underlying system domain. More generally, the scenario language can be used as a starting point to explore distri- bution scenarios outside the context of the thirty that we investigated. This would require representing the scenario(s) using our language, and then eliciting domain knowledge from expert users with experience in similar data distribution scenarios to determine their connector selections. 182 Figure 8.14: Precision-recall analysis of the three connector selection algorithms using performance data to revise selections. 8.6 Revising Connector Selections using Perfor- mance Data ToevaluateDISCO’sabilitytoleveragerealconnectorperformancedatatovalidate connector selections made by its selection algorithms, we devised an experiment that utilizes performance data (captured using DISCO’s PerformanceProfile structure described in Chapters 6 and 7) to revise connector groupings and to perform a new precision-recall analysis based on these revisions. This aids in vali- dating our Hypothesis 4 which states that We can accurately capture empirical performance measurements in the areas of consistency, dependability, efficiency, and scalability for each connector, and use such measurements to validate connec- tor selections. 183 We built a PerformanceAnalyzer component that implemented the DISCO Analyzerextensionpoint. ThePerformanceAnalyzerimplementedthealgorithm described in Section 6.6 that re-groups connectors into the inappropriate group if empirically gathered performance data suggests that the connector in fact did not satisfy the scenario performance requirements. Data was obtained empirically for alloftheconnectorsintheknowledgebaseforasubsetofthethirtydistributionsce- narios that were possible to simulate (given disk space constraints, and availability of system resources, i.e., network bandwidth, or presence of the particular con- nector technology, etc.). We gathered data for all four performance requirements, efficiency, consistency, dependability and scalability, using the following measures, described in Section 6.6, reiterated and summarized here for convenience: Efficiency The data transfer rate and average memory footprint of a connector for a particular distribution scenario. Consistency The actual number of bytes delivered as-sent for a particular distri- bution scenario. Scalability The total data volume, and total number of users and users type that a connector was able to handle for a particular distribution scenario. Dependability Thenumberofconnectorfaults(periodsofdowntime)thatoccur during a particular distribution scenario. Recall that DISCO’s PerformanceProfile data structure provides a means for transforming the collected performance data above into a numerical rating between 1 and 10. Using the PerformanceProfiles and the re-grouping algo- rithm, we revised our initial accuracy experiments for all three connector selection algorithms, taking the output from the k-means clustering algorithm and running 184 it through our PerformanceAnalyzer clustering process. We varied the amount of performance requirements that the PerformanceAnalyzer validated from 1 (any of the four performance requirements needed to be satisfied) to 4 (all four perfor- mancerequirementsmustbesatisfied)andthenmeasuredtherevisedprecision/re- call statistics. The resultant diagram is shown in Fig. 8.14. Theupperleftquadrantofthediagramshowstheprecisionrecallanalysisforall three connector selection algorithms where the PerformanceAnalyzer was config- ured to check satisfaction of all four performance requirements. Even in this most constrainedcase,theBayesianalgorithmstillexhibited71%accuracyand79%pre- cision, still making it reasonably accurate in the face of actual performance data. The Score-based algorithm showed reasonable accuracy (61%) as did the Optimiz- ingalgorithm(60%,anearly20%jumpfromtheresultsobtainedwithoutusingthe PerformanceAnalyzer). ErrorratefortheScore-basedandOptimizingalgorithms remained relatively constant with or without the PerformanceAnalyzer however for the Bayesian algorithm error rate increased by nearly 20%. All three selection algorithms exhibited low (~10%) recall. This may suggest that the expert answer keys that we employed during the initial accuracy experiments within Section 8.4 were not particularly sensitive to the performance requirements of the distribution scenarios. More likely though is the reality that, like the other seven distribution sce- nario dimensions, performance requirements is not a scenario requirement that must be fulfilled entirely all of the time. In contrast, similar to its counterparts, performance requirements must also be traded off, even between the four perfor- mance dimensions. This is suggested by the experimental results present in the upper right (3/4 requirements), lower left (2/4 requirements) and lower right (1/4 requirements) portions of Fig. 8.14. In each of these experiments, the precision, 185 recall, accuracy, and error-rate for all three selection algorithms was unaffected by the PerformanceAnalzyer’s regroupings. This indicates that the expert domain knowledge used as input to the selection algorithms (and the answer keys) was in fact sensitive to the performance requirements, and their satisfaction, however, akin to the rest of the scenario dimensions, even the experts explicitly (or implic- itly) realized that performance requirements must be traded off as well. 8.7 Integration with COTS assessment frame- work Software systems today are composed from prefabricated commercial components and connectors that provide complex functionality and engage in complex interac- tions. Unfortunately, because of the distinct assumptions made by developers of theseproducts,successfullyintegratingthemintoasoftwaresystemcanbecompli- cated, often causing budget and schedule overruns. A number of integration risks can be resolved by selecting the right set of COTS components and connectors that can be integrated with minimal effort. One of the collaborative efforts that we have recently been involved in is the development of a framework for selecting COTS software components and con- nectors, ensuring their interoperability in software-intensive systems. One of the key requirements of this framework is an external connector selection component called the Quality of Service Connector Selection Framework. ThearchitectureoftheCOTSinteroperabilityframeworkisshowninFig. 8.15. The figure is illustrative of the overall complexity of the COTS interoperability framework; however, for the purposes of our discussions, we will restrict our focus to the the Quality of Service Connector Selection component shown in the lower 186 COTS Definition Generator COTS Connector Selector Architecting User Interface Component Definitions of COTS used in the architecture Deployment Architecture COTS Definitions Connector Query/Response Integration Analysis Component Connector Options Interoperability Analyzer COTS Selector Quality of Service Connector Selection Framework COTS Interoperability Analysis Report COTS Definition Repository Integration Rules Repository Integration Rules Connector Options Project Analyst Define Architecture & COTS combinations Figure 8.15: Integration of DISCO with externally developed COTS interoperabil- ity evaluation framework. left section of the diagram. In this section, we will describe our experience using DISCO to implement this component. The Quality of Service Connector Selection Framework is an extensible compo- nentbuiltforidentifyingqualityofservicespecificconnectors. Ourparticularfocus was to aid in the selection of highly distributed and voluminous data connectors, though the system was built such that other quality of service extensions may be plugged in as well. These other extensions may include selection of connectors for mobile-computing environments that require low memory footprint, or connectors for highly reliable, fault-tolerant systems. Tocreateaqualityofserviceextension,adeveloperfirstidentifiesneededCOTS attribute information and ensures the information is captured in the COTS def- inition repository (shown in the top left portion of Fig. 8.15). This information 187 will typically describe the scenario requirements for COTS connector selection for the particular level of service (e.g., our extension built for data-intensive systems includestheTotalVolume,NumberofDeliveryIntervals,andpossiblytheNumber of Users present in the data transfer). The quality of service extension developer then can construct a simple web-based service that accepts the COTS connector definition information, and any other needed data, and then returns the appropri- ate COTS connectors to select to satisfy the desired level of service scenario. We leveraged DISCO’s architecture and software implementation described in Chapter 7 to build a REST-based web service interface that accepts HTTP GET requests defining a data distribution scenario. The interface is of the form: http://my.host/disco?<scenario_dimension_1> <op> <value 1>& <scenario_dimension_2> <op> <value 2>& ... <scenario_dimension_n> <op> <value n> The return value of our DISCO web service is an XML-based java.util.List<ConnectorRank>, described in Chapter 7. An example of such a list showing the return of one ranked connector (for brevity) is shown below: <?xml version="1.0"?> <Response> <Status>OK</Status> <Message>Connector Rank List</Message> <Value> <ConnectorList> <Connector> 188 <Name>bbFTP</Name> <Score>0.4459999</Score> </Connector> </ConnectorList> </Reponse> Our integration of DISCO with the COTS interoperability evaluation frame- work has provided system analysts dealing with data distribution system architec- tures a means for incorporating the ability of DISCO to rank COTS data distri- bution connectors. This capability is critical because, in our experience, to only consider COTS components in these type of architectures is a serious mistake, often requiring re-engineering of major portions of the system. This is due to the fact that the relationship between COTS distribution connectors and the architec- tures that they plug-in to is one that needs to be considered up-front, during the architectural analysis portion of the system development effort. COTS distribution connectors may impose constraints, such as operating sys- tem requirements and external libraries, which may conflict with other such oper- ating systems and libraries required by the COTS components in the data distri- bution system. Other constraints, such as the interaction nature of the connector with its external data producers and consumers is also of high importance (e.g., it may be a P2P distribution connector, which will require accompanying P2P data producers and consumers). The requirements of such COTS components (e.g., database, data archives, content management systems) need to be carefully mit- igated amongst the requirements of the COTS connectors, and we believe that our integration of DISCO into the aforementioned COTS interoperability frame- work has provided a qualitative “step in the right direction” for addressing this important issue. The integration that we have performed allows an analyst who 189 is trying to understand the right COTS components to select, to also consider the right COTS connectors. Given the set of COTS connectors to choose amongst, the analyst can compare the COTS components being considered and come to a sharedunderstandingoftherightsetoftechnologiestoemploywhenimplementing the data distribution system being considered. 190 Chapter 9 Conclusion and Future Work As hundred gigabyte hard disks and commodity storage become pervasive, the rise in the ability to warehouse and consume large volumes of data will only continue. In parallel to this, content providers continue to rapidly increase the volume of data that they produce. As new software systems are constructed in these volumi- nous data-intensive environments, a comprehensive understanding and capability for modeling, selecting, and analyzing software connectors for large-volume data distribution will become an invaluable tool for the requirements, architecture, and design/implementation phases of the software engineering lifecycle. The work conducted in this dissertation is, to our knowledge, the first step in developing a systematic, end-to-end framework to support decision making with respecttoselectingsoftwareconnectorsforthosehighlyvoluminousanddistributed data-intensive systems described above. To that end, in the rest of this chapter, we summarize our key contributions, and point to future avenues of research to extend and improve our framework. 9.1 Contributions Overall, this dissertation has provided a set of models, algorithms, techniques and tools to represent data distribution scenarios, classify and select data distribution connectors, and explore the trade-off space when architecting large scale data dis- tribution systems. We have conducted experiments using 13 real-world connectors 191 Theoretical (Guru Domain Knowledge) Actual Performance Data Capture Domain Knowledge Select Connectors based on Domain Knowledge Measure Accuracy of Capturing Domain Knowledge w/ Statistics Distribution Scenarios Capture Connector Performance Select Connectors based on Empirical Performance Measure Accuracy of Selecting Connectors based on Performance Revise Accuracy Measurement based on actual performance data Accurate and reliable connector selections Figure 9.1: Thesis high level summary diagram. across four connector families to validate the utility of our framework and ground it within real-world problems. In summary, our key contributions, highlighted in Fig. 9.1, include: • The definition and implementation of the architecture of two data dissemi- nationtechnologies, OODT,adatagridtechnologydevelopedatNASA’sJet Propulsion Laboratory (JPL), and GLIDE, a light-weight mobile data grid infrastructure developed at USC and JPL. • Definitionofadatadistributionscenariolanguage,forexpressingdistribution scenariorequirements. Thelanguageisbasedoneightkeydimensionsofdata distribution that we identified during the course of this research. 192 • Classification model for data distribution connectors, and detailed classifica- tions of 13 major OTS data distribution connectors. • A dynamic software technology for connector classification and selection, accompanied by tool support for exploring the trade-off space of connectors and distribution scenarios. • Empirical connector quality-of-service (QoS) performance evaluation in the areas of consistency, dependability, efficiency, and scalability. • Integration of our software technology with a real-world COTS interoper- ability evaluation framework, demonstrating our technology’s utility in a systems-of-systems environment, and furthermore, its applicability in the area of COTS interoperability evaluation. 9.2 Future Work There are several fruitful areas of work that we will pursue within DISCO. Firstly, we plan on investigating the role that architectural mismatch plays when inte- grated selected connectors into the architectures of the data distribution systems thatwillemploythemtotransferdata. Anotherareaoffutureworkinvolvesinves- tigation of the ability to dynamically redeploy software connectors to existing data distribution systems, allowing better suited connectors to be “swapped out” on- the-fly in a distribution system, without bringing the system down. We also plan on researching the incorporation of other QoS attributes (e.g., latency) beyond the four performance attributes that we have considered for the distribution con- nectors studied. Our future work will additionally involve developing techniques and methodologies for evaluating the quality of an expert’s domain knowledge 193 that our framework captures, including automatic expert knowledge generation approaches that yield highly accurate connector selections. We will investigate the development of new connector selection approaches based on other decision making techniques (e.g., reinforcement learning [SB98]). Finally, we plan on inves- tigating the ability to dynamically combine connectors and reap the benefits of the unification of their key properties, yielding new, superior connectors for data distribution. Each of these areas of future work is elaborated in more detail below. 9.2.1 Investigation of Architectural Mismatch The detection of architectural mismatches for selected connectors and their corre- sponding data distribution system architecture is a challenging problem, observed more generally by Garlan et al. [GAO95]. This is an important avenue of future research, because of the necessity to ensure that a selected connector is in fact compatible with a data producer system, and its technology. Consider that a data producer system (e.g., an archive) is implemented using an HTTP/REST based architecture, requiringaccessandretrievalofdatausingtheHTTPprotocol. This will certainly bear influence as to a connector to select because not all connectors (e.g., UFTP) have the ability to access data made available via a HTTP/REST interface. On the other hand, some of the other connectors (e.g., GLIDE, SOAP) we studied did have such a capability. Initially,weplanoninvestigatingasimplepair-wisemismatchalgorithm(based onthefoundationalworkbyGacek[Gac98])thatcomparesadistributionconnector and its corresponding data producer system along a set of quality attributes we will identify (discussed in Section 9.2.3 below), and that compares the two using 194 our DCP metadata. Recall that, e.g., data access persistence mechanism 1 is a DCP attribute stored for all distribution connectors modeled in DISCO. In the example above, we could compare whether or not a distribution connector allows for HTTP-based data access from a data producer system. In general, for each attribute, the algorithm will detect potential mismatch areas and decide whether the mismatches identified were severe enough to reduce a connector’s appropriateness (reified in the form of its rank) for a data producer system. Otherwise, the algorithm will identify the connector as appropriate. The detected mismatches will be labeled using a simple, but adaptable set of mismatch levels,suchassevereorallowable. Aseveremismatchforinstancemaypreventuse, or lower the rank, of an otherwise highly ranked connector selected to distribute data from a data producer system to its consumers. An allowable mismatch, on the other hand, may not severely affect a selected connector’s ranking, making it still appropriate for selection. The exact policy and sensitivity of this approach to mismatch detection would need to be solidified, and would be part of future work. 9.2.2 Ability to Dynamically Redeploy Connectors DISCO would certainly benefit from the ability to dynamically incorporate its connectorselectiondecisionmakingprocessintoadataproducingsystem,allowing the system to dynamically swap out the connector technology it currently employs with the “better” connector selections produced by DISCO. Such a capability has implications on the data producer system, as it would require an abstraction layer to be constructed on top of the connector tech- nology employed. The abstraction layer would need to support operations 1 Recall also that the data access persistence DCP attribute defines the available persistence mechanisms that a distribution connector can access data from, e.g. Dynamic Data Access, Web-based Access, etc. 195 such as stopConnector(Connector c), startConnector(Connector c), and replaceConnector(Connector c), allowing an existing connector to be stopped, a new connector to be deployed, and the deployed connector to be restarted, so that it may service data distribution requests again. Such operations have further implications on the system, such as: • What does the connector do during downtime? • Should it employ caching strategies for data waiting to be sent? • Shouldtheconnectorbetransactionbased, consideringthat(re-)deployment of the connector may fail? • How does the system restart the connector? This is just a sampling of the wealth of quality research questions that would need to be addressed in any type of general solution to the above problem, and we plan on pursuing it as part of our future work. 9.2.3 Incorporation of other QoS attributes Currently, DISCO focuses on a particular set of QoS attributes, geared towards distribution connector performance (i.e., scalability, efficiency, dependability, and consistency). This set is not exhaustive, and was based on discussions with data distribution experts, architects, and implementers of such systems, as well as a thorough review of existing literature. The results of the discussions and literature review indicated that these four performance attributes were the most important observable performance properties to consider in data distribution activities. One area of future work would involve investigating the utility of including other QoS attributes, including attributes such as: 196 Latency would measure the mean time between data inception and its recep- tion, observed between data deliveries from a data producer to one or more consumers. This attribute would shed light into the characteristics of the underlyingnetworkbetweentheproduceranditsconsumers,providingricher detail into other attributes, such as efficiency (e.g., transfer rate). Availability would be a measurement of the amount of actively responding data producersandconsumersinadatadistributionactivity. Activelyresponding is often defined as the degree to which the system is operational and acces- sible when required for use [IEE90]. This attribute will help us to better understand distribution connector consistency, and dependability. Ofcourse, thesearejustasmallcross-sectionofthespectrumofQoSattributes that may have implications on connector selection for data-intensive systems, and more generally, connector selection as a whole. We plan on investigating these and other attributes to determine their applicability in data-intensive systems, and in other domains (e.g., mobile computing). 9.2.4 Evaluation of Expert Domain Knowledge Quality One of the most important properties of DISCO is its ability to accurately cap- ture an organizational data distribution expert’s domain knowledge, i.e., the how and the why behind her connector selections. DISCO’s current ability to capture this knowledge, however, does not speak to the quality of the knowledge that it captures. 2 To this end, we propose to develop quality metrics as part of our future work that will evaluate the overall utility of an expert’s domain knowledge. Recall, we 2 The “garbage-in, garbage-out” philosophy preaches that if your system accepts low-quality input, then its output will also be low quality. 197 havedevelopedtwocanonicalformsofexpertdomainknowledge: black boxdomain knowledge (used as input to the score-based and optimizing selection algorithms) captures a guru’s knowledge of performance attributes in relation to scenario con- straints, anddoesnotconsidertheinternalpropertiesoftheconnector, whilewhite box domain knowledge (used as input to the Bayesian selection algorithm) focuses on mapping a connector’s constituent properties to those of the distribution sce- nario constraints that they must satisfy. We believe that metrics can be developed for each of these two types of domain knowledge in a similar fashion. The metrics would be functions that take as input the domain knowledge, and then look for “red-flags”, such as: • overuse of a particular score or probability given to a relation (such as one between TotalVolume and efficiency or one between TotalVolume and data access transient availability); • low mean distance between assigned scores to relations. This is a problem becausewithoutamplescoreorprobabilitydifferentiationbetweenconnector properties and their scenarios, all connectors will seem equally applicable; • low coverage of possible relations, i.e., low number of relations within the domain knowledge, or low number of relations that are a ”hit” and match a corresponding distribution scenario constraint. This is problematic, because with a small number of applicable mappings and scores between connector propertiesandscenarioconstraints, thereismoreparitybetweentheconnec- tors as they are being ranked, and “clear winners” become harder to discern. The functions would sum up a count of all such red flags, and then output a score based on the number and severity of the red flags found, providing a way to evaluate the guru’s domain knowledge captured. Of course, the proposed above 198 metrics would serve as our initial starting point into this area of future work, and are not meant to be exhaustive, but rather representative of the investigation space. 9.2.5 AutomaticGenerationofExpertDomainKnowledge Using the connector surveys described in Chapter 8, we plan on providing approaches for automatically generating expert domain knowledge, based on the information provided to us by experts and practitioners in the field of data dis- tribution. Automatically generating domain knowledge (and the assessment of its quality as described in the previous sub-section) has the ability to increase the exploration space when evaluating connectors and distribution scenarios. Often the domain knowledge received from data distribution experts is centered around their particular experiences using a connector technology. These experiences may neglect to sufficiently aid in selecting connectors for scenariosthatthedomainexperthasnotencountered. Giventheabilitytousethe expert’sknowledge(andpotentiallythecombinedknowledgeofmanyexperts,asin the case of our survey described in Section 8.1) as “training data”, we believe that automatic learning approaches, such as reinforcement learning [SB98] can provide an excellent means of training DISCO to automatically generate such knowledge, leveraging the quality metric functions that we develop. 9.2.6 Development of New Connector Selection Approaches Our dissertation research on DISCO has contributed the identification of the two canonicalformsofdistributionconnectorselectionapproaches,black-boxselection, 199 and white-box selection. Within these forms, we have constructed two black- box algorithms, one based on additive scoring, and another based on non-linear optimization, and multiplicative scoring. We have also constructed one white box selection algorithm based on Bayesian inference and decision making. There are other decision making approaches that could be cast as connector selection approaches, .e.g, reinforcement learning as mentioned above. We are tar- geting the development of a reinforcement learning based selection algorithm, that tries to learn the relationships between connector properties, and distribution sce- narioconstraints, akintotheautomaticgenerationofgurudomainknowledgethat we have discussed above. In addition, within black box and white box connector selection, we will investigate the development of a Bayesian black box selection algorithm, and score-based and optimizing white box approaches, to provide addi- tional data points to aid in evaluation of white box versus black box selection approaches. 9.2.7 Connector Integration Our final area of future work will focus on the ability to dynamically combine soft- ware connectors selected by DISCO’s selection algorithms, inducing the aggrega- tionofdesiredpropertiesandbenefitsfromhighrankingconnectorsforaparticular distribution scenario. To this end, we will need to augment DISCO with the capa- bility to deploy, adapt, combine and replace different software connectors. The crux of the framework will be an Integrator component, supporting four major types of connector integration, that we will discuss below. We have identified the four connector integration types from our survey into connector combination described in Chapter 2: 200 Wrapper Take≥ 2 distribution connectors and provide a wrapper in between them to allow them to communicate. This approach does not modify the existing two connectors being combined, and simply provides a mediation layer between them, allowing either of their constituent modules (e.g., data access, stream, etc.) to be utilized. Switch/Alternator Take ≥ 2 distribution connectors and determine how to divide the data up between them to distribute data over a time interval. This approach does not modify the connectors, and focuses on methods of utilizing capabilities from both existing connectors by modifying the data to be distributed, splitting it up appropriately. Extractor - Integrate x% of distribution connector A and y% of distribution con- nector B to arrive at a new distribution connector C. This tactic may modify the connectors, by introducing new functionality, code, external libraries, that leverage the capabilities of each of the existing original connectors, to arrive at a new connector with their combined important properties. Combiner Completelyintegrateandcombinethefunctionalityoftwodistribution connectors. Similar to the Extractor, this approach may modify the connec- tors, by integrating the entire set of capabilities from both of the original existing two connectors. Although the goal of the proposed Integrator component would be to support all four connector integration methods, initially, we will focus on Wrapping and Switching, leaving the Extractor and Combiner methods to be investigated last. By necessity Extractors and Combiners are ad-hoc, and existing literature (e.g., [DMT99]) has shown these two integration methods to be difficult to realize. We 201 hypothesize that at least the Extractor method can be synthesized using combina- tionsofWrappersandSwitchers,evidencedbythearrayofresearchintocombining wrapper technologies (e.g., Spitznagel and Garlan [SG01, SG03], Arbab [Arb04], Lauetal. [LEW05],Lopesetal. [LWF03]),bothconnectorsandcomponentsalike. Component wrapping is a well-known technique provided by many out-of-the-box middleware services. Our task will be to evaluate how well wrapping works when it comes to software connectors, and what the challenges and lessons learned from this experience are, and which of the four integration methods can be synthe- sized from Wrappers and Switchers. The answer to this question will be a unique contribution in its own right, and will advance the state of the understanding of connectors. Ultimately, the output from DISCO’s Integrator component will be the actual distribution connector code (or modifications to it). This could entail UNIX patch files carrying additional “glue” or “wrapper” code, it could entail completely new source code files, and it may also entail new a binary distribution of the connector code. We see each of these as potential outputs from the integration component. 202 Bibliography [AA96] A.Abd-Allah. ComposingHeterogeneousSoftwareArchitectures. PhD thesis, Univ. Southern California, 1996. Chair-Barry Boehm. [AAB + 99] Mehmet Altinel, Demet Aksoy, Thomas Baby, Michael J. Franklin, William Shapiro, and Stanley B. Zdonik. Dbis-toolkit: Adaptable middleware for large scale data delivery. In SIGMOD Conference, pages 544–546, 1999. [AAFZ95] Swarup Acharya, Rafael Alonso, Michael J. Franklin, and Stanley B. Zdonik. Broadcast disks: Data management for asymmetric com- munications environments. In SIGMOD Conference, pages 199–210, 1995. [AFCF05] Carina Alves, Xavier Franch, Juan Pablo Carvallo, and Anthony Finkelstein. Using goals and quality models to support the match- ing analysis during cots selection. In ICCBSS, pages 146–156, 2005. [AG97] Robert Allen and David Garlan. A formal basis for architectural connection. ACM Trans. Softw. Eng. Methodol., 6(3):213–249, 1997. [Arb04] Farhad Arbab. Reo: a channel-based coordination model for com- ponent composition. Mathematical Structures in Computer Science, 14(3):329–366, 2004. [Bal97] M. S. Baltuch. Unidata’s internet data distribution (idd) system: Two years of data delivery. In 13th Intl’ Conference on Interactive Information and Processing Systems for Meteorology, Oceanography , and Hydrology, 1997. [bbF07] bbftp - large files transfer protocol, http://doc.in2p3.fr/bbftp/, 2007. [BLFM98] T. Berners-Lee, R. Fielding, and L. Masinter. Uniform resource iden- tifiers (uri): Generic syntax. Technical Report RFC 2396, 1998. 203 [BMMB07] Jesal Bhuta, Chris A. Mattmann, Nenad Medvidovic, and Barry Boehm. A framework for the assessment and selection of software components and connectors in cots-based architectures. In WICSA, 2007. [BMMG05] Somo Banerjee, Chris A. Mattmann, Nenad Medvidovic, and Leana Golubchik. Leveraging architectural models to inject trust into soft- ware systems. In SESS ’05: Proceedings of the 2005 workshop on Software engineering for secure systemsbuilding trustworthy applica- tions, pages 1–7, New York, NY, USA, 2005. ACM Press. [BMP + 04] Jonathan Beaver, Nicholas Morsillo, Kirk Pruhs, Panos K. Chrysan- this, and Vincenzo Liberatore. Scalable dissemination: What’s hot and what’s not. In WebDB, pages 31–36, 2004. [BP00] D. B´ alek and F. Pl´ a˘ sil. Software connectors: A hierarchical model. Technical Report 2000/2, Charles University, 2000. [Bus05] Dennis Bush. Uftp - udp based ftp with multicast. http://www.tcnj.edu/ bush/uftp.html, 2005. [cFG00] UgurC ¸etintemel, MichaelJ.Franklin, andC.LeeGiles. Self-adaptive user profiles for large-scale data delivery. In ICDE, pages 622–633, 2000. [CFK + 00] Ann Chervenak, Ian Foster, Carl Kesselman, Charles Salisbury, and Steven Tuecke. The data grid: Towards an architecture for the dis- tributed management and analysis of large scientific data sets. J. Network and Computer Applications, 23(3):187–200, 2000. [Cha07] B. Chafin. Oco ops pipeline peer review, jpl internal document, 2007. [CHK01] Dan Crichton, Steve Hughes, and Sean Kelly. A distributed data architecture for the 2001 mars odyssey data distribution. In Space Mission Challenges for Information Technology (SMCIT), 2001. [CKM + 06] Daniel Crichton, Sean Kelly, Chris Mattmann, Qing Xiao, J. Steven Hughes, Jane Oh, Mark Thornquist, Donald Johnsey, Sudhir Srivas- tava, Laura Essermann, and William Bigbee. A distributed infor- mation services architecture to support biomarker discovery in early detection of cancer. In E-SCIENCE ’06: Proceedings of the Second IEEE International Conference on e-Science and Grid Computing, page 44, Washington, DC, USA, 2006. IEEE Computer Society. 204 [Coh03] Bram Cohen. Incentives build robustness in bit torrent. In Workshop on Economics of Peer-to-Peer Systems, 2003. [CRW01] Antonio Carzaniga, David S. Rosenblum, and Alexander L. Wolf. Designandevaluationofawide-areanotificationservice. ACM Trans. Computer Systems., 19(3):332–383, 2001. [DCM99] Dublin core metadata element set, 1999. [Der07] Apache derby project, http://derby.apache.org, 2007. [DFS04] Nigel Davies, Adrian Friday, and Oliver Storz. Exploring the grid’s potential for ubiquitous computing. IEEE Pervasive Computing, 03(2):74–75, 2004. [DIS07a] Disco document type definitions, http://www- scf.usc.edu/~mattmann/disco/dtd/, 2007. [DIS07b] Disco evaluation data, http://www- scf.usc.edu/~mattmann/disco/evaldata/, 2007. [DMT99] Eric M. Dashofy, Nenad Medvidovic, and Richard N. Taylor. Using off-the-shelf middleware to implement connectors in distributed soft- ware architectures. In ICSE, pages 3–12, 1999. [Dow98] T. B. Downing. Java RMI: Remote Method Invocation. IDG Books Worldwide, Inc., 1998. [DPE04] Giovanni Denaro, Andrea Polini, and Wolfgang Emmerich. Early performance testing of distributed software applications. In WOSP, pages 94–103, 2004. [Ecl07] Eclipse.org home, http://eclipse.org, 2007. [Emm00a] Wolfgang Emmerich. Engineering Distributed Objects. John Wiley and Sons, Chichester, UK, 2000. [Emm00b] Wolfgang Emmerich. Software engineering and middleware: a roadmap. In ICSE - Future of SE Track, pages 117–129, 2000. [Ere04] Justin R. Erenkrantz. Web services: Soap, uddi, and semantic web. TechnicalReportUCI-ISR-04-3,InstituteforSoftwareResearch, 2004. [Fie00] Roy Thomas Fielding. Architectural styles and the design of network- based software architectures. PhD thesis, UC Irvine, 2000. Chair- Richard N. Taylor. 205 [FKT01] Ian Foster, Carl Kesselman, and Steven Tuecke. The anatomy of the grid: Enabling scalable virtual organizations. J. Supercomputing Applications., pages 1–25, 2001. [FZ97] Michael J. Franklin and Stanley B. Zdonik. A framework for scalable dissemination-based systems. In OOPSLA, 1997. [Gac98] Christina Gacek. Detecting Architectural Mismatch During Systems Composition. PhD thesis, Univ. Southern California, 1998. Chair- Barry Boehm. [GAO95] David Garlan, Robert Allen, and John Ockerbloom. Architectural mismatch or why it’s hard to build systems out of existing parts. In ICSE, pages 179–185, 1995. [Glo07] The globus alliance, http://www.globus.org, 2007. [HCK + 04] J. S. Hughes, D. Crichton, S. Kelly, C. Mattmann, R. Joyner, J. Wilf, and J. Crichton. A planetary data system for the 2006 mars recon- naisance orbiter era and beyond. In 2nd Symposium on Ensuring Long-term Preservation and Adding Value to Scientific and Technical Data, 2004. [HCK + 05] J.StevenHughes,DanielJ.Crichton,SeanC.Kelly,ChrisMattmann, Jerry Crichton, and Thuy Tran. Intelligent resource discovery using ontology-based resource profiles. Data Science Journal, 4:171–188, 2005. [HCKM05] J. S. Hughes, Daniel Crichton, Sean Kelly, and Chris Mattmann. The semanticplanetarydatasystem. In3rdSymposiumonEnsuringLong- term Preservation and Adding Value to Scientific and Technical Data, 2005. [HJ61] Robert Hooke and T. A. Jeeves. Direct search solution of numerical and statistical problems. Journal of the ACM, 8:212–229, April 1961. [HM98] J.StevenHughesandSusanK.McMahon.Theplanetarydatasystem. a case study in the development and management of meta-data for a scientific digital library. In ECDL, pages 335–350, 1998. [IEE90] Ieee standard computer dictionary: A compilation of ieee standard computer glossaries, 1990. 206 [KKB + 98] R. Kazman, M. Klein, M. Barbacci, T. Longstaff, H. Lipson, and J. Carriere. The architecture tradeoff analysis method. In 4th Inter- national Conference on Engineering of Complex Computer Systems, 1998. [KP98] G. E. Krasner and S. T. Pope. A cookbook for using the model-view- controller user interface paradigm in smalltalk-80. J. Object-Oriented Programming, 1(3):26–49, 1998. [KS01] M. M. Kand´ e and A. Strohmeier. Modeling crosscutting concerns using software connectors. In OOPSLA Workshop on Advanced Sep- aration of Concerns, 2001. [Lee07] University of leeds mass spectrometry facility page, http://www.astbury.leeds.ac.uk/facil/mass.htm, 2007. [LEW05] Kung-Kiu Lau, Perla Velasco Elizondo, and Zheng Wang. Exogenous connectors for software components. In CBSE, pages 90–106, 2005. [LWF03] Antonia Lopes, Michel Wermelinger, and Jose Luiz Fiadeiro. Higher- order architectural connectors. ACM Trans. Softw. Eng. Methodol., 12(1):64–104, 2003. [MA05] Ed Mancebo and Anneliese Amschler Andrews. A strategy for select- ing multiple components. In ACM SAC, pages 1505–1510, 2005. [Mac67] J. B. MacQueen. Some methods for classification and analysis of mul- tivariate observations. In 5th Berkeley Symposium on Mathematical Statistics and Probability, volume 1, pages 281–297, 1967. [Mat07] Chris Mattmann. Software connectors for highly dis- tributed and voluminous data-intensive systems survey, http://softarch.usc.edu/disco/survey.htm, 2007. [MCH + 04] Chris Mattmann, Daniel J. Crichton, J. Steven Hughes, Sean C. Kelly, and Paul M. Ramirez. Software architecture for large-scale, distributed, data-intensive systems. In WICSA, pages 255–276, 2004. [MCMH06] Chris Mattmann, Daniel J. Crichton, Nenad Medvidovic, and Steve Hughes. A software architecture-based framework for highly dis- tributed and data intensive scientific applications. In ICSE, pages 721–730, 2006. 207 [MDT03] NenadMedvidovic, EricM.Dashofy, andRichardN.Taylor. Therole of middleware in architecture-based software development. Interna- tional Journal of Software Engineering and Knowledge Engineering, 13(4):367–393, 2003. [MFC03] Chris Mattmann, Dana Freeborn, and Daniel J. Crichton. Towards a distributedinformationarchitectureforavionicsdata. InICWI,pages 829–832, 2003. [MHM03] Sanda Mandutianu, Sean Hardman, and Chris Mattmann. Dirac- a framework for coordination and cooperation. Poster Session, Proc. of ACM/IFIP/USENIX International Middleware Conference, 2003. [MHO96] Mark J. Maybee, Dennis Heimbigner, and Leon J. Osterweil. Mul- tilanguage interoperability in distributed systems. In ICSE, pages 451–463, 1996. [MM07] Chris Mattmann and Nenad Medvidovic. The gridlite dream: Bring- ing the grid to your pocket. In Reliable Systems on Unreliable Net- worked Platforms, volume 4322 of LNCS, pages 70–87. Springer Ver- lag, 2007. [MMB + 05] Chris Mattmann, Sam Malek, Nels Beckman, Marija Mikic-Rakic, Nenad Medvidovic, and Dan Crichton. Glide: A grid-based, lightweight, infrastructure for data-intensive environments. In EGC, pages 68–77, 2005. [MMP00] Nikunj R. Mehta, Nenad Medvidovic, and Sandeep Phadke. Towards a taxonomy of software connectors. In ICSE, pages 178–187, 2000. [MMRJ05] ChrisMattmann, NenadMedvidovic, PaulM.Ramirez, andVladimir Jakobac. Unlocking the grid. In CBSE, pages 322–336, 2005. [MMRM05] Sam Malek, Marija Mikic-Rakic, and Nenad Medvidovic. Prism-mw: A style-aware architectural middleware for resource constrainted, dis- tributed systems. IEEE Trans. Software Eng.., 31(3):256–272, 2005. [MRCH04] Chris Mattmann, Paul Ramirez, Daniel Crichton, and J. S. Hughes. Packaging data products using data grid middleware for deep space missionsystems.In8thInternationalConferenceonSpaceOperations, 2004. [MS03] Chris Mattmann and Bilal Shaw. Adding meta-architectural under- standingtoresourceawaresoftwarearchitecturesrequiringdevicesyn- chronization. In ICWI, pages 1265–1268, 2003. 208 [MT00] Nenad Medvidovic and Richard N. Taylor. A classification and com- parison framework for software architecture description languages. IEEE Trans. Software Eng., 26(1):70–93, 2000. [NDI04] Amol Nayate, Michael Dahlin, and Arun Iyengar. Transparent infor- mation dissemination. In Middleware, pages 212–231, 2004. [Nut07] Nutch project website, http://lucene.apache.org/nutch/, 2007. [OCO07] Orbiting carbon observatory, http://oco.jpl.nasa.gov, 2007. [OOD05] Jxta company spotlight, http://www.jxta.org/companies/comp- anyarchive.html#jpl, 2005. [Ope05] Open channel foundation, http://openchannelsoftware.com/order- s/index.php?group id=332, 2005. [ORT98] Peyman Oreizy, David S. Rosenblum, and Richard N. Taylor. On the role of connectors in modeling and implementing software architec- tures. Technical Report ICS-TR-98-04, UC Irvine, 1998. [Poi07] Pointcast, http://en.wikipedia.org/wiki/pointcast %28dotcom%29, 2007. [PW92] Dewayne E. Perry and Alexander L. Wolf. Foundations for the study ofsoftwarearchitecture. ACMSIGSOFTSoftwareEngineeringNotes, 17(4):40–52, 1992. [RM04] Paul M. Ramirez and Chris Mattmann. Ace: Improving search engines via automatic concept extraction. In IRI, pages 229–234, 2004. [Ros98] D. Rosenblum. Challenges in exploiting architectural models in soft- ware testing. In NSF/CNR Workshop on the Role of Software Archi- tecture in Testing and Analysis, pages 49–53, 1998. [RPM04] Robert Raskin, Michael J. Pan, and Chris Mattmann. Enabling semantic interoperability for earth science data. In 4th NASA Earth Science Technology Conference, 2004. [SB98] R. S. Sutton and A.G. Bartolo. Reinforcement Learning: An Intro- duction. MIT Press, 1998. [SCP07] Secure copy, http://en.wikipedia.org/wiki/secure copy, 2007. [SEL07] Seldi proteinchip technology proteomics equipment page, http://dir.niehs.nih.gov/proteomics/seldi.htm, 2007. 209 [SG96] M. Shaw and D. Garlan. Software Architecture - Perspectives on an emerging discipline. Prentice Hall, 1996. [SG01] Bridget Spitznagel and David Garlan. A compositional approach for constructing connectors. In WICSA, pages 148–157, 2001. [SG03] Bridget Spitznagel and David Garlan. A compositional formalization of connector wrappers. In ICSE, pages 374–384, 2003. [Sha93] Mary Shaw. Procedure calls are the assembly language of software interconnection: Connectorsdeservefirst-classstatus. InICSE Work- shop on Studies of Software Design, pages 17–32, 1993. [SHA07] Sha hash functions, http://en.wikipedia.org/wiki/sha-1, 2007. [Sri05] Sudhir Srivastava. Informatics in proteomics. Taylor and Fran- cis/CRC Press, 2005. [TMA + 96] Richard N. Taylor, Nenad Medvidovic, Kenneth M. Anderson, E. James Whitehead Jr., Jason E. Robbins, Kari A. Nies, Peyman Oreizy, and Deborah L. Dubrow. A component- and message-based architectural style for gui software. IEEE Trans. Software Eng., 22(6):390–406, 1996. [TTC95] Richard N. Taylor, Will Tracz, and Lou Coglianese. Software devel- opment using domain-specific software architectures: Cdrl a011a curriculum module in the sei style. SIGSOFT Softw. Eng. Notes, 20(5):27–38, 1995. [VDGD05] TomVerdickt,BartDhoedt,FrankGielen,andPietDemeester. Auto- matic inclusion of middleware performance attributes into architec- tural uml software models. IEEE Trans. Software Eng., 31(8):695– 711, 2005. [Vin97] S. Vinoski. Corba: integrating diverse applications within dis- tributed heterogeneous environments. Communications Magazine, IEEE, (2):46–55, February 1997. [WD01] Eric Wohlstadter and Prem Devanbu. A lazy approach to separating architectural concerns. In ICSE Workshop on Advanced Separation of Concerns in Software Engineering, 2001. [Win03] R. L. Winkler. An Introduction to Bayesian Inference and Decision. Probabilistic Press, 2003. 210 [YGM95] T. Yan and H. Garcia-Molina. Sift - a tool for wide-area information dissemination. In USENIX Technical Conference, 1995. [You07] Youtube - broadcast yourself, http://www.youtube.com, 2007. [Zar04] ApostolosZarras. Acomparisonframeworkformiddlewareinfrastruc- tures. Journal of Object Technology, 3(5):103–123, 2004. 211 Appendices 212 Appendix A Distribution Connector Profiles In this section we provide DCPs of the 13 distribution connectors that we studied. A.1 Client Server Distribution Connectors A.1.1 File Transfer Protocol (FTP) <?xml v e r s i o n="1.0" encoding="UTF-8"?> <disco:connector xmlns:disco="http://softarch.usc.edu/1.0/disco" name="FTP" class="Client/Server"> <procedurecall> <parameters> <datatransfer> <method>value</method> <method>reference</method> </datatransfer> <semantics>keyword</semantics> <returnvalue>FTP Control Message</returnvalue> <invocationrecord>FTP Server Log</invocationrecord> </parameters> <entrypoint> <single>The entry point is an invocation to an FTP client program that transfers a file (or set of files ) by reference from the client filesystem to the server filesystem . 213 </single> </entrypoint> <invocation> <explicit>Method Call</explicit> </invocation> <synchronicity>Synchronous</synchronicity> <cardinality sender="1" receiver="1"/> <accessibility private="false" public="true" protected="true"/> </procedurecall> <dataaccess> <locality>Global</locality> <accesses> <access>Accessor</access> <access>Mutator</access> </accesses> <availabilities> <transient>Session−Based</transient> <persistents> <persistent>File I/O</persistent> </persistents> </availabilities> <accessibility private="false" public="true"/> <lifecycle/> <cardinality senders="*" receivers="1"/> </dataaccess> <stream> <deliveries> 214 <delivery>Exactly Once</delivery> </deliveries> <bounds>Bounded</bounds> <buffering>Buffered</buffering> <throughput>bps</throughput> <state>Stateful</state> <identity>Named</identity> <localities> <locality>Remote</locality> <locality>Local</locality> </localities> <synchronicity>Time Out Synchronous</synchronicity> <formats> <format>Structured</format> <format>Raw</format> </formats> <cardinality senders="1" receivers="1"/> </stream> <distributor> <naming type="registry-based"> <structure>Flat</structure> </naming> <delivery type="FTP Message"> <semantics> <semantic>Exactly Once</semantic> </semantics> <mechanisms> <mechanism>Unicast</mechanism> 215 </mechanisms> </delivery> <routing type="tcp/ip"> <membership>bounded</membership> <path static="true" dynamic="false" cached="false"/> </routing> </distributor> </disco:connector> A.1.2 Secure Copy Protocol (SCP) <?xml v e r s i o n="1.0" encoding="UTF-8"?> <disco:connector xmlns:disco="http://softarch.usc.edu/1.0/disco" name="SCP" class="Client/Server"> <procedurecall> <parameters> <datatransfer> <method>value</method> <method>reference</method> </datatransfer> <semantics>keyword</semantics> <returnvalue>UNIX Process Status</returnvalue> <invocationrecord>OS Process Table</invocationrecord> </parameters> <entrypoint> <single>The entry point is a command line program , scp , that transfers a file (or set of files ) from a client host filesystem to a server filesystem . </single> 216 </entrypoint> <invocation> <explicit>Method Call</explicit> </invocation> <synchronicity>Synchronous</synchronicity> <cardinality sender="1" receiver="1"/> <accessibility private="false" public="true" protected="true"/> </procedurecall> <dataaccess> <locality>Global</locality> <accesses> <access>Accessor</access> </accesses> <availabilities> <transient>Session−Based</transient> <persistents> <persistent>File I/O</persistent> </persistents> </availabilities> <accessibility private="false" public="true"/> <lifecycle/> <cardinality senders="1" receivers="1"/> </dataaccess> <stream> <deliveries> <delivery>Exactly Once</delivery> </deliveries> 217 <bounds>Bounded</bounds> <buffering>Buffered</buffering> <throughput>bps</throughput> <state>Stateful</state> <identity>Named</identity> <localities> <locality>Remote</locality> <locality>Local</locality> </localities> <synchronicity>Time Out Synchronous</synchronicity> <formats> <format>Structured</format> <format>Raw</format> </formats> <cardinality senders="1" receivers="1"/> </stream> <distributor> <naming type="registry-based"> <structure>Flat</structure> </naming> <delivery type="SCP Encrypted Data Message"> <semantics> <semantic>Exactly Once</semantic> </semantics> <mechanisms> <mechanism>Unicast</mechanism> </mechanisms> </delivery> 218 <routing type="tcp/ip"> <membership>bounded</membership> <path static="true" dynamic="false" cached="false"/> </routing> </distributor> </disco:connector> A.1.3 Common Request Broker Architecture: CORBA <?xml v e r s i o n="1.0" encoding="UTF-8"?> <disco:connector xmlns:disco="http://softarch.usc.edu/1.0/disco" name="CORBA" class="Client/Server"> <procedurecall> <parameters> <datatransfer> <method>value</method> <method>reference</method> </datatransfer> <semantics>keyword</semantics> <returnvalue>CORBA Message</returnvalue> <invocationrecord>CORBA Name Registry</invocationrecord> </parameters> <entrypoint> <single>The entry point is delivered by the corresponding CORBA Name Server Registry . The entry point is in the form of a specific port that the CORBA Name Server component runs on. </single> </entrypoint> 219 <invocation> <explicit>Method Call</explicit> </invocation> <synchronicity>Synchronous</synchronicity> <cardinality sender="1" receiver="1"/> <accessibility private="true" public="true" protected="true"/> </procedurecall> <dataaccess> <locality>Process</locality> <accesses> <access>Accessor</access> <access>Mutator</access> </accesses> <availabilities> <transient>Session−Based</transient> <persistents> <persistent>Repository Access</persistent> <persistent>File I/O</persistent> <persistent>Dynamic Data Exchange</persistent> <persistent>Database Access</persistent> </persistents> </availabilities> <accessibility private="true" public="true"/> <lifecycle/> <cardinality senders="1" receivers="1"/> </dataaccess> <stream> 220 <deliveries> <delivery>Exactly Once</delivery> </deliveries> <bounds>Bounded</bounds> <buffering>Buffered</buffering> <throughput>bps</throughput> <state>Stateful</state> <identity>Named</identity> <localities> <locality>Remote</locality> </localities> <synchronicity>Time Out Synchronous</synchronicity> <formats> <format>Structured</format> <format>Raw</format> </formats> <cardinality senders="1" receivers="1"/> </stream> <distributor> <naming type="registry-based"> <structure>Flat</structure> </naming> <delivery type="HTTP Message"> <semantics> <semantic>Exactly Once</semantic> </semantics> <mechanisms> <mechanism>Unicast</mechanism> 221 </mechanisms> </delivery> <routing type="tcp/ip"> <membership>bounded</membership> <path static="true" dynamic="false" cached="false"/> </routing> </distributor> </disco:connector> A.1.4 REST <?xml v e r s i o n="1.0" encoding="UTF-8"?> <disco:connector xmlns:disco="http://softarch.usc.edu/1.0/disco" name="HTTP/REST" class="Client/Server"> <procedurecall> <parameters> <datatransfer> <method>value</method> </datatransfer> <semantics>keyword</semantics> <returnvalue>HTTP Response</returnvalue> <invocationrecord>HTTP Server log</invocationrecord> </parameters> <entrypoint> <single>Socket Listener , Port 80 (Server) HTTP formatted request sent through socket to port 80 (Client ). </single> </entrypoint> <invocation> 222 <explicit>Method Call</explicit> </invocation> <synchronicity>Synchronous</synchronicity> <cardinality sender="1" receiver="1"/> <accessibility private="false" public="true" protected="false"/> </procedurecall> <dataaccess> <locality>Global</locality> <accesses> <access>Accessor</access> <access>Mutator</access> </accesses> <availabilities> <transient>Cache</transient> <persistents> <persistent>Repository Access</persistent> <persistent>File I/O</persistent> <persistent>Dynamic Data Exchange</persistent> <persistent>Database Access</persistent> </persistents> </availabilities> <accessibility private="true" public="true"/> <lifecycle/> <cardinality senders="1" receivers="1"/> </dataaccess> <stream> <deliveries> 223 <delivery>Exactly Once</delivery> </deliveries> <bounds>Bounded</bounds> <buffering>Buffered</buffering> <throughput>bps</throughput> <state>Stateless</state> <identity>Named</identity> <localities> <locality>Local</locality> <locality>Remote</locality> </localities> <synchronicity>Time Out Synchronous</synchronicity> <formats> <format>Structured</format> <format>Raw</format> </formats> <cardinality senders="1" receivers="1"/> </stream> <distributor> <naming type="registry-based"> <structure>Heirarchical</structure> </naming> <delivery type="HTTP Message"> <semantics> <semantic>Exactly Once</semantic> </semantics> <mechanisms> <mechanism>Unicast</mechanism> 224 </mechanisms> </delivery> <routing type="tcp/ip"> <membership>ad−hoc</membership> <path static="true" dynamic="false" cached="false"/> </routing> </distributor> </disco:connector> A.1.5 Java Remote Method Invocation (RMI) <?xml v e r s i o n="1.0" encoding="UTF-8"?> <disco:connector xmlns:disco="http://softarch.usc.edu/1.0/disco" name="RMI" class="Client/Server"> <procedurecall> <parameters> <datatransfer> <method>value</method> <method>reference</method> </datatransfer> <semantics>keyword</semantics> <returnvalue>RMI message</returnvalue> <invocationrecord>RMI registry</invocationrecord> </parameters> <entrypoint> <single>The entry point is delivered by the corresponding RMI Registry . The entry point is in the form of a specific port that the RMI Server component runs on. </single> 225 </entrypoint> <invocation> <explicit>Method Call</explicit> </invocation> <synchronicity>Synchronous</synchronicity> <cardinality sender="1" receiver="1"/> <accessibility private="true" public="true" protected="true"/> </procedurecall> <dataaccess> <locality>Process</locality> <accesses> <access>Accessor</access> <access>Mutator</access> </accesses> <availabilities> <transient>Session−Based</transient> <persistents> <persistent>Repository Access</persistent> <persistent>File I/O</persistent> <persistent>Dynamic Data Exchange</persistent> <persistent>Database Access</persistent> </persistents> </availabilities> <accessibility private="true" public="true"/> <lifecycle/> <cardinality senders="1" receivers="1"/> </dataaccess> 226 <stream> <deliveries> <delivery>Exactly Once</delivery> </deliveries> <bounds>Bounded</bounds> <buffering>Buffered</buffering> <throughput>bps</throughput> <state>Stateful</state> <identity>Named</identity> <localities> <locality>Remote</locality> </localities> <synchronicity>Time Out Synchronous</synchronicity> <formats> <format>Structured</format> <format>Raw</format> </formats> <cardinality senders="1" receivers="1"/> </stream> <distributor> <naming type="registry-based"> <structure>Flat</structure> </naming> <delivery type="RMI Message"> <semantics> <semantic>Exactly Once</semantic> </semantics> <mechanisms> 227 <mechanism>Unicast</mechanism> </mechanisms> </delivery> <routing type="tcp/ip"> <membership>bounded</membership> <path static="true" dynamic="false" cached="false"/> </routing> </distributor> </disco:connector> A.1.6 Simple Object Access Protocol (SOAP) <?xml v e r s i o n="1.0" encoding="UTF-8"?> <disco:connector xmlns:disco="http://softarch.usc.edu/1.0/disco" name="SOAP" class="Client/Server"> <procedurecall> <parameters> <datatransfer> <method>value</method> </datatransfer> <semantics>keyword</semantics> <returnvalue>SOAP Message</returnvalue> <invocationrecord>Web Server</invocationrecord> </parameters> <entrypoint> <single>The entry point is delivered by the service WSDL document, which can be discovered from an associated UDDI registry , or from the service URL by appending "?wsdl" 228 </single> </entrypoint> <invocation> <explicit>Method Call</explicit> </invocation> <synchronicity>Synchronous</synchronicity> <cardinality sender="1" receiver="1"/> <accessibility private="true" public="true" protected="true"/> </procedurecall> <dataaccess> <locality>Process</locality> <accesses> <access>Accessor</access> <access>Mutator</access> </accesses> <availabilities> <transient>Session−Based</transient> <persistents> <persistent>Repository Access</persistent> <persistent>File I/O</persistent> <persistent>Dynamic Data Exchange</persistent> <persistent>Database Access</persistent> </persistents> </availabilities> <accessibility private="true" public="true"/> <lifecycle/> <cardinality senders="1" receivers="1"/> 229 </dataaccess> <stream> <deliveries> <delivery>Exactly Once</delivery> </deliveries> <bounds>Bounded</bounds> <buffering>Buffered</buffering> <throughput>bps</throughput> <state>Stateless</state> <identity>Named</identity> <localities> <locality>Remote</locality> </localities> <synchronicity>Time Out Synchronous</synchronicity> <formats> <format>Structured</format> <format>Raw</format> </formats> <cardinality senders="1" receivers="1"/> </stream> <distributor> <naming type="registry-based"> <structure>Flat</structure> </naming> <delivery type="SOAP Message"> <semantics> <semantic>Exactly Once</semantic> </semantics> 230 <mechanisms> <mechanism>Unicast</mechanism> </mechanisms> </delivery> <routing type="tcp/ip"> <membership>bounded</membership> <path static="true" dynamic="true" cached="true"/> </routing> </distributor> </disco:connector> A.1.7 Commercial UDP Technology <?xml v e r s i o n="1.0" encoding="UTF-8"?> <disco:connector xmlns:disco="http://softarch.usc.edu/1.0/disco" name="Commerical UDP Technology" class="Client/Server"> <procedurecall> <parameters> <datatransfer> <method>reference</method> </datatransfer> <semantics>keyword</semantics> <returnvalue>Command−line messges</returnvalue> <invocationrecord>OS Process Table</invocationrecord> </parameters> <entrypoint> <single>The entry point for this commercial UDP technology is a command−line program that accepts paths to files that should be transferred from a client program to a server . 231 </single> </entrypoint> <invocation> <explicit>Method Call</explicit> </invocation> <synchronicity>Synchronous</synchronicity> <cardinality sender="1" receiver="1"/> <accessibility private="true" public="true" protected="true"/> </procedurecall> <dataaccess> <locality>Process</locality> <accesses> <access>Accessor</access> </accesses> <availabilities> <transient>Session−Based</transient> <persistents> <persistent>File I/O</persistent> </persistents> </availabilities> <accessibility private="false" public="true"/> <lifecycle/> <cardinality senders="1" receivers="1"/> </dataaccess> <stream> <deliveries> <delivery>Best Effort</delivery> 232 </deliveries> <bounds>Bounded</bounds> <buffering>Buffered</buffering> <throughput>bps</throughput> <state>Stateless</state> <identity>Named</identity> <localities> <locality>Remote</locality> <locality>Local</locality> </localities> <synchronicity>Asynchronous</synchronicity> <formats> <format>Structured</format> <format>Raw</format> </formats> <cardinality senders="1" receivers="1"/> </stream> <distributor> <naming type="registry-based"> <structure>Heirarchical</structure> </naming> <delivery type="CUDP Data Message"> <semantics> <semantic>Best Effort</semantic> <semantic>At Least Once</semantic> </semantics> <mechanisms> <mechanism>Unicast</mechanism> 233 <mechanism>Broadcast</mechanism> </mechanisms> </delivery> <routing type="udp"> <membership>bounded</membership> <path static="true" dynamic="false" cached="false"/> </routing> </distributor> </disco:connector> A.1.8 UDP-based FTP with Multicast (UFTP) <?xml v e r s i o n="1.0" encoding="UTF-8"?> <disco:connector xmlns:disco="http://softarch.usc.edu/1.0/disco" name="UFTP" class="Client/Server"> <procedurecall> <parameters> <datatransfer> <method>reference</method> </datatransfer> <semantics>keyword</semantics> <returnvalue>UNIX Process Status</returnvalue> <invocationrecord>OS Process Table</invocationrecord> </parameters> <entrypoint> <single>The entry point is a command line program , uftp , that transfers data from a client program to potential set of uftp server programs. </single> 234 </entrypoint> <invocation> <explicit>Method Call</explicit> </invocation> <synchronicity>Asynchronous</synchronicity> <cardinality sender="1" receiver="1"/> <accessibility private="false" public="true" protected="true"/> </procedurecall> <dataaccess> <locality>Global</locality> <accesses> <access>Accessor</access> </accesses> <availabilities> <transient>Session−Based</transient> <persistents> <persistent>File I/O</persistent> </persistents> </availabilities> <accessibility private="false" public="true"/> <lifecycle/> <cardinality senders="1" receivers="*"/> </dataaccess> <stream> <deliveries> <delivery>Best Effort</delivery> <delivery>At Least Once</delivery> 235 </deliveries> <bounds>Bounded</bounds> <buffering>Buffered</buffering> <throughput>bps</throughput> <state>Stateful</state> <identity>Named</identity> <localities> <locality>Remote</locality> <locality>Local</locality> </localities> <synchronicity>Asynchronous</synchronicity> <formats> <format>Structured</format> <format>Raw</format> </formats> <cardinality senders="1" receivers="*"/> </stream> <distributor> <naming type="registry-based"> <structure>Flat</structure> </naming> <delivery type="UFTP Message"> <semantics> <semantic>At Least Once</semantic> <semantic>Best Effort</semantic> </semantics> <mechanisms> <mechanism>Unicast</mechanism> 236 <mechanism>Multicast</mechanism> <mechanism>Broadcast</mechanism> </mechanisms> </delivery> <routing type="udp"> <membership>bounded</membership> <path static="true" dynamic="false" cached="false"/> </routing> </distributor> </disco:connector> A.2 Grid Distribution Connectors A.2.1 bbFTP <?xml v e r s i o n="1.0" encoding="UTF-8"?> <disco:connector xmlns:disco="http://softarch.usc.edu/1.0/disco" name="bbFTP" class="grid"> <procedurecall> <parameters> <datatransfer> <method>value</method> </datatransfer> <semantics>keyword</semantics> <returnvalue>bbFTP Message</returnvalue> <invocationrecord>None</invocationrecord> </parameters> <entrypoint> 237 <single>The entry point is a method invocation to call the bbFTP data transfer client program. </single> </entrypoint> <invocation> <explicit>Method Call</explicit> </invocation> <synchronicity>Synchronous</synchronicity> <cardinality sender="1" receiver="1"/> <accessibility private="true" public="true" protected="true"/> </procedurecall> <dataaccess> <locality>Global</locality> <accesses> <access>Accessor</access> <access>Mutator</access> </accesses> <availabilities> <transient>Session−Based</transient> <persistents> <persistent>Repository Access</persistent> <persistent>File I/O</persistent> <persistent>Dynamic Data Exchange</persistent> <persistent>Database Access</persistent> </persistents> </availabilities> <accessibility private="true" public="true"/> 238 <lifecycle/> <cardinality senders="*" receivers="*"/> </dataaccess> <stream> <deliveries> <delivery>Exactly Once</delivery> </deliveries> <bounds>Bounded</bounds> <buffering>Buffered</buffering> <throughput>bps</throughput> <state>Stateless</state> <identity>Named</identity> <localities> <locality>Remote</locality> </localities> <synchronicity>Time Out Synchronous</synchronicity> <formats> <format>Structured</format> <format>Raw</format> </formats> <cardinality senders="*" receivers="*"/> </stream> <distributor> <naming type="registry-based"> <structure>Heirarchical</structure> </naming> <delivery type="bbFTP Message"> <semantics> 239 <semantic>Exactly Once</semantic> </semantics> <mechanisms> <mechanism>Unicast</mechanism> </mechanisms> </delivery> <routing type="tcp/ip"> <membership>bounded</membership> <path static="true" dynamic="false" cached="false"/> </routing> </distributor> </disco:connector> A.2.2 GridFTP <?xml v e r s i o n="1.0" encoding="UTF-8"?> <disco:connector xmlns:disco="http://softarch.usc.edu/1.0/disco" name="GridFTP" class="grid"> <procedurecall> <parameters> <datatransfer> <method>value</method> </datatransfer> <semantics>keyword</semantics> <returnvalue>GridFTP Message</returnvalue> <invocationrecord>Globus Log Layer</invocationrecord> </parameters> <entrypoint> <single>The standard single entry point for 240 a GridFTP connector is the GridFTP client API, defined in the GridFTP RFC. The API is provided by the underlying Globus Toolkit . </single> </entrypoint> <invocation> <explicit>Method Call</explicit> </invocation> <synchronicity>Synchronous</synchronicity> <cardinality sender="1" receiver="1"/> <accessibility private="false" public="true" protected="false"/> </procedurecall> <dataaccess> <locality>Global</locality> <accesses> <access>Accessor</access> <access>Mutator</access> </accesses> <availabilities> <transient>Session−Based</transient> <persistents> <persistent>Repository Access</persistent> <persistent>File I/O</persistent> <persistent>Dynamic Data Exchange</persistent> <persistent>Database Access</persistent> </persistents> </availabilities> 241 <accessibility private="true" public="true"/> <lifecycle/> <cardinality senders="1" receivers="*"/> </dataaccess> <stream> <deliveries> <delivery>Exactly Once</delivery> </deliveries> <bounds>Bounded</bounds> <buffering>Buffered</buffering> <throughput>bps</throughput> <state>Stateful</state> <identity>Named</identity> <localities> <locality>Remote</locality> <locality>Local</locality> </localities> <synchronicity>Time Out Synchronous</synchronicity> <formats> <format>Structured</format> <format>Raw</format> </formats> <cardinality senders="1" receivers="*"/> </stream> <distributor> <naming type="registry-based"> <structure>Heirarchical</structure> </naming> 242 <delivery type="GridFTP Message"> <semantics> <semantic>Exactly Once</semantic> </semantics> <mechanisms> <mechanism>Unicast</mechanism> </mechanisms> </delivery> <routing type="tcp/ip"> <membership>ad−hoc</membership> <path static="true" dynamic="true" cached="true"/> </routing> </distributor> </disco:connector> A.3 Event Distribution Connectors A.3.1 GLIDE <?xml v e r s i o n="1.0" encoding="UTF-8"?> <disco:connector xmlns:disco="http://softarch.usc.edu/1.0/disco" name="GLIDE" class="Event"> <event> <cardinality producers="*" observers="*"> <producers>Components</producers> <observers>Threads</observers> <eventpatterns> <pattern>Broadcast</pattern> 243 <pattern>Unicast</pattern> <pattern>Multicast</pattern> <pattern>Asychronous</pattern> <pattern>Best Effort</pattern> </eventpatterns> </cardinality> <delivery>Best Effort</delivery> <priority> <outgoing setby="producers"/> <incoming setby="consumers"/> </priority> <synchronicity>Asychronous</synchronicity> <notification>Queued Dispatch</notification> <causality/> <mode type="Event"> <software>Signals</software> </mode> </event> <dataaccess> <locality>Process</locality> <accesses> <access>Accessor</access> <access>Mutator</access> </accesses> <availabilities> <transient>Session−Based</transient> <persistents> <persistent>Repository Access</persistent> 244 <persistent>File I/O</persistent> <persistent>Dynamic Data Exchange</persistent> <persistent>Database Access</persistent> </persistents> </availabilities> <accessibility private="true" public="true"/> <lifecycle/> <cardinality senders="*" receivers="*"/> </dataaccess> <stream> <deliveries> <delivery>Best Effort</delivery> </deliveries> <bounds>Bounded</bounds> <buffering>Buffered</buffering> <throughput>bps</throughput> <state>Stateful</state> <identity>Named</identity> <localities> <locality>Local</locality> <locality>Remote</locality> </localities> <synchronicity>Asynchronous</synchronicity> <formats> <format>Structured</format> <format>Raw</format> </formats> <cardinality senders="*" receivers="*"/> 245 </stream> <distributor> <naming type="registry-based"> <structure>Flat</structure> </naming> <delivery type="Event"> <semantics> <semantic>Best Effort</semantic> </semantics> <mechanisms> <mechanism>Unicast</mechanism> <mechanism>Broadcast</mechanism> <mechanism>Multicast</mechanism> </mechanisms> </delivery> <routing type="architecture configuration"> <membership>bounded</membership> <path dynamic="true" static="false" cached="false"/> </routing> </distributor> </disco:connector> A.3.2 Sienna <?xml v e r s i o n="1.0" encoding="UTF-8"?> <disco:connector xmlns:disco="http://softarch.usc.edu/1.0/disco" name="Sienna" class="Event"> <event> <cardinality producers="*" observers="*"> 246 <producers>Access Points</producers> <observers>Clients</observers> <eventpatterns> <pattern>Pub−Sub</pattern> <pattern>Broadcast</pattern> <pattern>Unicast</pattern> <pattern>Multicast</pattern> <pattern>Asychronous</pattern> <pattern>Best Effort</pattern> </eventpatterns> </cardinality> <delivery>Best Effort</delivery> <priority> <outgoing setby="producers"/> <incoming setby="consumers"/> </priority> <synchronicity>Asychronous</synchronicity> <notification>Pub−Sub</notification> <causality/> <mode type="Event"> <software>Signals</software> </mode> </event> <dataaccess> <locality>Process</locality> <accesses> <access>Accessor</access> <access>Mutator</access> 247 </accesses> <availabilities> <transient>Session−Based</transient> <persistents> <persistent>Repository Access</persistent> <persistent>File I/O</persistent> <persistent>Dynamic Data Exchange</persistent> <persistent>Database Access</persistent> </persistents> </availabilities> <accessibility private="true" public="true"/> <lifecycle/> <cardinality senders="*" receivers="*"/> </dataaccess> <stream> <deliveries> <delivery>Best Effort</delivery> </deliveries> <bounds>Bounded</bounds> <buffering>Buffered</buffering> <throughput>bps</throughput> <state>Stateful</state> <identity>Named</identity> <localities> <locality>Local</locality> <locality>Remote</locality> </localities> <synchronicity>Asynchronous</synchronicity> 248 <formats> <format>Structured</format> <format>Raw</format> </formats> <cardinality senders="*" receivers="*"/> </stream> <distributor> <naming type="attribute-based"/> <delivery type="Event"> <semantics> <semantic>Best Effort</semantic> </semantics> <mechanisms> <mechanism>Unicast</mechanism> <mechanism>Broadcast</mechanism> <mechanism>Multicast</mechanism> </mechanisms> </delivery> <routing type="content-based"> <membership>bounded</membership> <path dynamic="true" static="false" cached="false"/> </routing> </distributor> </disco:connector> A.4 P2P Distribution Connectors A.4.1 Bittorrent 249 <?xml v e r s i o n="1.0" encoding="UTF-8"?> <disco:connector xmlns:disco="http://softarch.usc.edu/1.0/disco" name="Bittorrent" class="P2P"> <dataaccess> <locality>Process</locality> <accesses> <access>Accessor</access> </accesses> <availabilities> <transient>Peer−Based</transient> <persistents/> </availabilities> <accessibility private="true" public="true"/> <lifecycle/> <cardinality senders="*" receivers="*"/> </dataaccess> <stream> <deliveries> <delivery>Best Effort</delivery> <delivery>At Least Once</delivery> </deliveries> <bounds>Bounded</bounds> <buffering>Buffered</buffering> <throughput>bps</throughput> <state>Stateful</state> <identity>Named</identity> <localities> <locality>Remote</locality> 250 <locality>Local</locality> </localities> <synchronicity>Time Out Synchronous</synchronicity> <formats> <format>Structured</format> <format>Raw</format> </formats> <cardinality senders="*" receivers="*"/> </stream> <arbitrator> <faulthandling>Voting</faulthandling> <concurrency type="Point-to-Point"> <mechanism>Rendezvous</mechanism> <weight>Heavy</weight> </concurrency> <transactions> <nesting>Multiple</nesting> <awareness>Supported</awareness> <isolation>Read/Write</isolation> </transactions> <security required="false"> <authentication/> <authorization> <authType>capability</authType> <authType>fairness</authType> </authorization> <privacies> <privacy>Anonymous</privacy> 251 <privacy>Tracked</privacy> </privacies> <integrity>SHA</integrity> <durability>Multi−Session</durability> </security> <scheduling>Ensure Reassmbly</scheduling> </arbitrator> <distributor> <naming type="attribute-based"/> <delivery type="Peer Pieces"> <semantics> <semantic>Best Effort</semantic> <semantic>At least once</semantic> </semantics> <mechanisms> <mechanism>Unicast</mechanism> <mechanism>Broadcast</mechanism> </mechanisms> </delivery> <routing type="tracker"> <membership>ad−hoc</membership> <path static="true" dynamic="true" cached="true"/> </routing> </distributor> </disco:connector> 252 Appendix B Software Connector Survey The following survey has 2 major parts to it: 1. A set of general questions about data distribution technologies and scenarios and their properties 2. A survey of 30 real world distribution scenarios that major universities and labs are currently facing and your opinion as to the best connector to select for each scenario. B.1 General Questions 1. The scalability of a distribution scenario is influenced by the Total Volume of data to be transferred. • TRUE • FALSE 2. For a distribution scenario in which there are gigabytes of data that need to be distributed to thousands of Users and hundreds of User Types, in which consistencyisnotparticularlydesired,infavorofincreasingscalability,which family of connectors would be most appropriate for the scenario? • P2P • Grid 253 • Event • Client/Server 3. For a WAN based Geographic Distribution between producers of data and consumers of data, dependability of the transfer is the most important issue. • TRUE • FALSE 4. Which connectors support the SSL access policy (select all that apply)? • GridFTP • Bittorrent • HTTP/REST • RMI 5. What types of connectors are appropriate for publish-subscribe protocols (select all that apply)? • P2P • Grid • Event • Client/Server 6. What types of connectors are appropriate for point-to-point transfers (select all that apply)? • P2P • Grid 254 • Event • Client/Server 7. The Number of Intervals and Volume Per Interval on a distribution scenario have no effect on the connector (or connector type) to choose. • TRUE • FALSE 8. GeographicDistribution(WANversusLAN)haslittleeffectonadistribution scenario. • TRUE • FALSE 9. All data distribution scenarios have been identified and are known. • Strongly Agree • Agree • Disagree • Strongly Disagree 10. The architecture of a data distribution system has little influence on which connector to select. • Strongly Agree • Agree • Disagree • Strongly 255 11. If both bbFTP and UFTP support the same transfer rate, and get data from point A to point B in the same amount of time, across the same network, then they are equally applicable to the data distribution scenario. • TRUE • FALSE 12. UDP based technologies sacrifice dependability while bolstering efficiency in a data distribution scenario. • TRUE • FALSE B.2 Distribution Scenarios For the following 30 distribution scenarios below, please provide your first and secondchoiceforthedistributionconnectortouse. Ifyouhavetime,pleaseprovide rationale behind your decision. This will help to provide meaningful evidence behind our results. You can select from the connectors that are described in Section B.3 at the end of this survey. Additionally, for the Performance Requirements scenario dimen- sions, note that each of the 4 performance requirements are on a numerical scale from 1-10. For example, a “2” for scalability would be considered “significantly less scalable” than a “10” for scalability. More detailed definitions of these as well as other terms are given within section B.3 at the end of the survey. 256 1. TYPE: Medium Volume, Few Intervals, WAN (point-to-point) Total Volume: 100s of GB Number of User Types: 1 Number of Delivery Intervals: 10 Volume Per Interval: 10 GB Geographic Distribution: WAN Performance Requirements: • Consistency: 8 • Dependability: 8 • Scalability: 2 • Efficiency: 3 1st Choice Connector: 2nd Choice Connector: Rationale: 2. TYPE: Medium Volume, Single Interval, LAN (point-to-point) Total Volume: 50 GB Number of User Types: 1 Number of Delivery Intervals: 1 Volume Per Interval: 50 GB Geographic Distribution: LAN Performance Requirements: • Consistency: 10 • Dependability: 8 • Scalability: 1 • Efficiency: 2 Number of Users: 1 Access Policies: linux file system 1st Choice Connector: 2nd Choice Connector: Rationale: 257 3. TYPE: High Volume, Few Intervals, Many users and types, metadata, WAN (point-to-many) Total Volume: 100s of GB Number of User Types: 3, scientists, educators, public Number of Delivery Intervals: > 1 AND < 10 Volume Per Interval: < 1 TB AND > 10 GB Geographic Distribution: WAN Performance Requirements: • Consistency: 8 • Dependability: 4 • Scalability: 10 • Efficiency: 5 Number of Users: > 10000 Access Policies: SSL/HTTP 1.0 Data Types: metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 4. TYPE:LowVolume,FewIntervals,Manyusersandtypes,data, WAN (point-to-many) Total Volume: 10s of GB Number of User Types: 3, scientists, educators, public Number of Delivery Intervals: > 1 AND < 10 Volume Per Interval: > 1 GB AND < 10 GB Geographic Distribution: WAN Performance Requirements: • Consistency: 2 • Dependability: 2 • Scalability: 8 • Efficiency: 5 Number of Users: > 10000 Access Policies: SSL/HTTP 1.0 Data Types: data 1st Choice Connector: 2nd Choice Connector: Rationale: 258 5. TYPE: High Volume, Many Intervals, Single User Type LAN (point-to-point) Total Volume: > 100 GB AND < 2 TB Number of User Types: 1 Number of Delivery Intervals: > 1 AND < 300 Volume Per Interval: 50 GB Geographic Distribution: LAN Performance Requirements: • Consistency: 9 • Dependability: 5 • Scalability: 2 • Efficiency: 9 Number of Users: 1 Access Policies: linux file system per- missions Data Types: metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 6. TYPE: Low Volume, Many Intervals, Many Users, Single User Type, WAN (many-to-many) Total Volume: > 600 MB AND < 1 GB Number of User Types: 1 Number of Delivery Intervals: > 1 AND < 100 Volume Per Interval: .3 MB Geographic Distribution: WAN Performance Requirements: • Consistency: 2 • Dependability: 2 • Scalability: 8 • Efficiency: 2 Number of Users: 1000s Access Policies: port-based, firewall pass through Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 259 7. TYPE: Low Volume, Few Intervals, Many Users, Single User Type, WAN (many-to-many) Total Volume: > 5 MB AND < 10 MB Number of User Types: 1 Number of Delivery Intervals: > 1 AND < 100 Volume Per Interval: .3 MB Geographic Distribution: WAN Performance Requirements: • Consistency: 1 • Dependability: 1 • Scalability: 1 • Efficiency: 4 Number of Users: 1000s Access Policies: port-based, firewall pass through Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 260 8. TYPE: High Volume, Many Intervals, Few Users, Many User Types, WAN and LAN (point-to-many) Total Volume: > 2 TB AND < 4 TB Number of User Types: 3, calibration engineers, atmosphericscientists, grad students Number of Delivery Intervals: > 1 AND < 1000 Volume Per Interval: 1 GB Geographic Distribution: WAN and LAN Performance Requirements: • Consistency: 8 • Dependability: 8 • Scalability: 2 • Efficiency: 4 Number of Users: 10s Access Policies: port-based, firewall passthrough,linuxfilesystempermis- sions Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 261 9. TYPE: Medium Volume, Many Intervals, Many Users, Many User Types, WAN and LAN Total Volume: > 100 GB Number of User Types: > 10 Number of Delivery Intervals: > 1 AND < 1000 Volume Per Interval: > 1 MB AND < 1 GB Geographic Distribution: WAN and LAN Performance Requirements: • Consistency: 4 • Dependability: 2 • Scalability: 8 • Efficiency: 8 Number of Users: 100s Access Policies: port-based, firewall pass through Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 10. TYPE: High Volume, Single Interval, Few Users, Single User Type, WAN Total Volume: 1 TB Number of User Types: 1 Number of Delivery Intervals: 10 Volume Per Interval: 100 GB Geographic Distribution: WAN Performance Requirements: • Consistency: 8 • Dependability: 8 • Scalability: 5 • Efficiency: 5 Number of Users: 10 Access Policies: firewall pass through Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 262 11. TYPE: Medium Volume, Single INterval, Single User, Single User Type, LAN Total Volume: > 10 GB AND < 50 GB Number of User Types: 1 Number of Delivery Intervals: > 1 AND < 100 Volume Per Interval: 100 MB Geographic Distribution: LAN Performance Requirements: • Consistency: 8 • Dependability: 8 • Scalability: 2 • Efficiency: 3 Number of Users: 1 Access Policies: completely open Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 12. TYPE: High Volume, Many Users, Many User Types, WAN, data and metadata Total Volume: > 10 TB AND < 50 TB Number of User Types: > 10 Number of Delivery Intervals: > 1 AND < 100 Volume Per Interval: 500 GB Geographic Distribution: WAN Performance Requirements: • Consistency: 8 • Dependability: 3 • Scalability: 9 • Efficiency: 4 Number of Users: 10000s Access Policies: completely open Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 263 13. TYPE: Low Volume, Single User, Single User Type, Many Delivery Intervals, WAN Total Volume: > 100 MB Number of User Types: 1 Number of Delivery Intervals: > 1 AND < 100 Volume Per Interval: .3 MB Geographic Distribution: WAN Performance Requirements: • Consistency: 2 • Dependability: 8 • Scalability: 2 • Efficiency: 2 Number of Users: 1 Access Policies: port-based, firewall pass through Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 14. TYPE: Low Volume, Few Users, 10 Intervals, WAN, data Total Volume: > 600 MB AND < 1 GB Number of User Types: 1 Number of Delivery Intervals: 10 Volume Per Interval: 10 MB Geographic Distribution: WAN Performance Requirements: • Consistency: 2 • Dependability: 8 • Scalability: 2 • Efficiency: 5 Number of Users: 10s Access Policies: port-based Data Types: data 1st Choice Connector: 2nd Choice Connector: Rationale: 264 15. TYPE: Medium Volume, Many Users, 5 User Types, Many Delivery Intervals, WAN authenticated Total Volume: 200 GB Number of User Types: > 5 Number of Delivery Intervals: > 1 AND < 100 Geographic Distribution: WAN Performance Requirements: • Consistency: 4 • Dependability: 8 • Scalability: 9 • Efficiency: 7 Number of Users: 1000s Access Policies: user authentication Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 16. TYPE: Low Volume, 10 Users, 3 User Types, Between 1 and 10 Intervals, LAN Total Volume: 250 MB Number of User Types: 3 Number of Delivery Intervals: > 1 AND < 10 Volume Per Interval: .25 MB Geographic Distribution: LAN Performance Requirements: • Consistency: 2 • Dependability: 8 • Scalability: 3 • Efficiency: 1 Number of Users: 10 Access Policies: linux file system per- missions Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 265 17. TYPE: High Volume, 10s of Thousands of Users, Many User Types, WAN and LAN Total Volume: > 150 TB Number of User Types: > 20 Number of Delivery Intervals: > 10000 Volume Per Interval: > 1 GB AND < 100 GB Geographic Distribution: WAN and LAN Performance Requirements: • Consistency: 4 • Dependability: 8 • Scalability: 7 • Efficiency: 4 Number of Users: 1000s Access Policies: linux file system per- missions, user authentication Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 18. TYPE: Medium Volume, Many Users, Single User Type, LAN based data Total Volume: 100 GB Number of User Types: 1 Number of Delivery Intervals: 3 Data Types: data Geographic Distribution: LAN Performance Requirements: • Consistency: 8 • Dependability: 8 • Scalability: 5 • Efficiency: 9 Number of Users: 1000s Access Policies: OS authentication 1st Choice Connector: 2nd Choice Connector: Rationale: 266 19. TYPE: High Volume, Many Users, SIngle User Type, WAN Total Volume: 150 TB Number of User Types: 1 Number of Delivery Intervals: > 1 AND < 100 Volume Per Interval: 10 MB Geographic Distribution: WAN Performance Requirements: • Consistency: 9 • Dependability: 5 • Scalability: 9 • Efficiency: 3 Number of Users: 1000s Access Policies: user authentication Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 20. TYPE: Medium Volume, 50 Users, 50 User Types, LAN Total Volume: 50 GB Number of User Types: 50 Number of Delivery Intervals: 50 Volume Per Interval: 1 GB Geographic Distribution: LAN Performance Requirements: • Consistency: 5 • Dependability: 8 • Scalability: 10 • Efficiency: 5 Number of Users: 50 Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 267 21. TYPE: Large Volume, 1 User, 1 User Type, LAN Total Volume: > 10 TB Number of User Types: 1 Number of Delivery Intervals: 1 Volume Per Interval: 10 TB Geographic Distribution: LAN Performance Requirements: • Consistency: 8 • Dependability: 8 • Scalability: 7 • Efficiency: 4 Number of Users: 1000s Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 22. TYPE: Medium Volume, 50 Users, 50 User Types, WAN Total Volume: 50 GB Number of User Types: 50 Number of Delivery Intervals: 50 Volume Per Interval: 1 GB Geographic Distribution: WAN Performance Requirements: • Consistency: 4 • Dependability: 4 • Scalability: 3 • Efficiency: 9 Number of Users: 50 Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 268 23. TYPE: Medium Volume, 10 Users, 5 User Types, LAN Total Volume: 50 GB Number of User Types: 5 Number of Delivery Intervals: 50 Volume Per Interval: 1 GB Geographic Distribution: LAN Performance Requirements: • Consistency: 3 • Dependability: 4 • Scalability: 3 • Efficiency: 3 Number of Users: 10 Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 24. TYPE: Medium Volume, 10 Users, 5 User Types, WAN Total Volume: 50 GB Number of User Types: 5 Number of Delivery Intervals: 50 Volume Per Interval: 1 GB Geographic Distribution: WAN Performance Requirements: • Consistency: 3 • Dependability: 8 • Scalability: 7 • Efficiency: 5 Number of Users: 10 Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 269 25. TYPE: Medium Volume, Many Users, 50 User Types, WAN, metadata Total Volume: 50 GB Number of User Types: 10 Number of Delivery Intervals: 5 Volume Per Interval: 10 GB Geographic Distribution: LAN Performance Requirements: • Consistency: 8 • Dependability: 8 • Scalability: 7 • Efficiency: 6 Number of Users: > 20000 Data Types: metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 26. TYPE: Low Volume, 2 Users, 1 User Types, WAN Total Volume: 1 GB Number of User Types: 1 Number of Delivery Intervals: 1 Volume Per Interval: 1 GB Geographic Distribution: WAN Performance Requirements: • Consistency: 3 • Dependability: 3 • Scalability: 2 • Efficiency: 3 Number of Users: 2 Data Types: data 1st Choice Connector: 2nd Choice Connector: Rationale: 270 27. TYPE: Medium Volume, Few Users, Many User Types, WAN Total Volume: 50 GB Number of User Types: 50 Number of Delivery Intervals: 5 Volume Per Interval: 10 GB Geographic Distribution: WAN Performance Requirements: • Consistency: 9 • Dependability: 5 • Scalability: 2 • Efficiency: 3 Number of Users: 20 Data Types: data 1st Choice Connector: 2nd Choice Connector: Rationale: 28. TYPE: Low Volume, 50 Users, 50 User Types, LAN Total Volume: 5 GB Number of User Types: 50 Number of Delivery Intervals: 5 Volume Per Interval: 1 GB Geographic Distribution: LAN Performance Requirements: • Consistency: 9 • Dependability: 10 • Scalability: 4 • Efficiency: 3 Number of Users: 50 Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 271 29. TYPE: Low Volume, 50 Users, 50 User Types, WAN Total Volume: 5 GB Number of User Types: 50 Number of Delivery Intervals: 50 Volume Per Interval: .1 GB Geographic Distribution: WAN Performance Requirements: • Consistency: 8 • Dependability: 8 • Scalability: 2 • Efficiency: 3 Number of Users: 50 Data Types: data and metadata 1st Choice Connector: 2nd Choice Connector: Rationale: 30. TYPE: High Volume, Many Users, Few User Types, LAN Total Volume: 20 TB Number of User Types: > 10 Number of Delivery Intervals: > 100 Volume Per Interval: .1 GB Geographic Distribution: LAN Performance Requirements: • Consistency: 4 • Dependability: 2 • Scalability: 7 • Efficiency: 5 Number of Users: > 10000 Data Types: data 1st Choice Connector: 2nd Choice Connector: Rationale: 272 B.3 Definitions 1. Total Volume — the total amount of data that needs to be transferred from providers of data to consumers of data. 2. Delivery Intervals — the number, size and frequency (timing) of intervals wherein which the volume of data should be delivered. 3. Performance Requirements — any constraints and requirements on the consistency, efficiency, scalability and dependability of the distribution sce- nario. Each of these dimensions is measured using a numeric scale, with 1 indicating the lowest possible value, and 10 indicating the highest possible value. Forexample,a7forscalabilityismorescalablethana2forscalability. • Scalability refers to the number of nodes in a data distribution sce- nario, embodied in the number of users (consumers and providers) present in the scenario. Scalability also refers to the total volume pre- dictions of the distribution scenario, and the ability for the distribution connectors used to handle increasing amounts of volume. • Efficiency in a distribution scenario captures information such as the rate in which the data is being sent (e.g., throughput, measured in bits per second, or some other higher order unit), as well as the actual pure efficiency of the distribution connector itself, in terms of its memory usage and footprint. • Consistency deals with the simple desire that data sent in a distribu- tion scenario is data delivered, i.e., that whatever is sent from providers of data arrives at consumers of data, in its full form, without any data loss, or error. Different distribution connectors may provide different 273 consistency capabilities. For example, any distribution connector that is a member of the event- based connector family may involve asyn- chronouscommunication,wheredatapacketsaresentwithoutanyguar- antee that they will reach their destination. • Dependability captures the number of faults per interval (e.g., sec- onds) that occur as induced by the connectors behavior (e.g., under its nominal operation) during a data distribution. Dependability also refers to the uptime of the actual connector during a data distribution. 4. Number of Users — the amount of unique users to whom data volume needs to be delivered. 5. NumberofUserTypes—theamountofuniqueusertypes(e.g.,scientists, students) to whom the data volume needs to be delivered. 6. Data Types — The number of different data types (e.g., data, metadata) that are part of the total volume to be delivered. 7. Geographic Distribution — The geographic distribution of the data providers and consumers. 8. AccessPolicies—Thenumberandtypesofaccesspoliciesinplaceateach producer and consumer of data. B.3.1 Acronyms TB terabyte GB gigabyte MB megabyte 274 B.3.2 Connector Descriptions GridFTP extends the FTP protocol with new features required for large volume, fast data transfer, such as striping, partial file access and highly parallel stream-based usage of TCP/IP. GridFTP is the basic data movement mech- anism provided by the Globus Toolkit, the widely adopted software packages for implementing grid-based software applications. HTTP/REST isasimple,light-weightprotocolbuiltontopofTCP/IPinwhich a client and server communicate by transferring control messages in the hyper text transfer protocol (HTTP) format, and streaming data back syn- chronously to the client. CORBA , or the Common Object Request Broker Architecture, is a middleware platform for distributed components. CORBA provides an underlying mes- saging substrate that is based on TCP/IP, and involves component registries and object brokers. SOAP is a simple protocol built on top of HTTP, in which messages exchanged between servers and clients are XML formatted documents conforming to the SOAP envelope schema. RMI is Java’s implementation of remote procedure call, in which data is trans- ferred between components in the system through valued parameters which are streamed over TCP/IP. Bittorrent is a peer-to-peer based file sharing protocol, in which peers share pieces of a unique content item, and use geographic location, transfer rate, and reputation to transfer data amongst large amounts of users and systems. 275 GLIDE is an event-based data sharing middleware built to asynchronously and synchronously transfer data between clients and servers in a distributed sys- tem. Sienna is an implementation of a publish-subscribe data sharing middleware for large-scale distributed systems. FTP The classic File Transfer Protocol (FTP) defines a capability for clients to send data to servers using the underlying TCP/IP protocol of the public Internet. FTP is an open standard and widely deployed in almost every operating system, is well documented and is broadly accepted as the de facto data movement mechanism available in modern software. SCP The Secure Copy Protocol (SCP) allows for the secure transfer of files betweentwomachinesusingtheSSHprotocolforauthenticationandencryp- tion over a network. Akin to FTP, SCP is built on top of the underlying TCP/IP protocol and is a public, open standard. UFTP UFTP or UDP-based file transfer protocol with multicast is an open- source data movement mechanism designed for efficient and reliable transfer of large amounts of data to multiple receivers simultaneously. UFTP is par- ticular effective over a satellite link (with two way communication), or over high-delay Wide Area Networks (WANs) where the reliability mechanisms in TCP/IP severely under- utilize the available network throughput capabil- ities. bbFTP is an open-source parallel TCP/IP data movement technology. bbFTP’s main capabilities are the ability to use SSH and certificate based authenti- cation, on-the-fly data compression and customizable time-outs. 276 CUDP CUDP is a commercially available product built on top of a proprietary protocolthatfullyutilizesthecapabilitiesofUDPtosendburstsofdatafrom one place to another. CUDP builds on top of the SCP protocol to provide secure, reliable, and most importantly fast transfer of voluminous data sets independent of network latency and packet loss. 277 Appendix C DISCO GUI in Action In this appendix, we describe the DISCO GUI interface in greater detail. We describe the original goals of the GUI. We then describe the requirements derived from the original goals. The appendix concludes with a series of screen shots illustrating the DISCO GUI in action, relating the GUI views and functionalities to the goals and requirements described. C.1 Goals We envisioned the DISCO GUI being an environment for manipulating, browsing and tailoring the connector selection process, including adapting the connector definitions(ifneeded), editingdomainknowledgefortheselectionalgorithms(e.g., recall the score functions for the Score-based algorithm, as an example), running selection algorithms, and so forth. This thought process led us to the three main goals for the DISCO GUI, pre- sented below: Goal 1 A user should be able to classify new connectors and edit classifications of existing connectors in the knowledge base. The DISCO GUI should sup- port adaptation of the existing distribution connector profiles, as well as the addition of new profiles and removal of existing ones. Goal 2 A user should be able to tailor and assign domain knowledge for each selection algorithm so long as the domain knowledge conforms to a standard 278 representation format, e.g., XML. This allows the user to adapt domain knowledge and discern how it affects a connector’s ranking. Goal 3 The system should allow a user to interactively execute DISCO using the available knowledge bases (of connectors, and of performance requirements as described in Chapter 6). Each goal was a guiding principle as we developed the DISCO GUI and ulti- mately each goal helped us to target our work and focus our effort. Below, we describe the system requirements inferred from the above goals, and from the above discussion regarding the vision for the DISCO GUI. C.2 Requirements The requirements for the DISCO GUI were driven by the over-arching objective of having a graphical environment for data distribution system designers (e.g., software architects) to take distribution scenarios (elicited from the users of their data distribution systems) and investigate the benefits and tradeoffs of particular distribution connectors in light of the scenarios. The DISCO GUI requirements (shown in Table C.1) were driven by the three goals described above. C.3 GUI in Action The main window for the DISCO GUI is shown in Fig. C.1. The main window allows a user to seed the GUI environment with the appropriate distribution con- nectorprofiles(shownastheDCPDirectoryinput). ThishelpstoaddressGoal 1, because it allows users to select and adapt the DCP knowledge base used in the ensuing connector selections. 279 Table C.1: DISCO GUI requirements. Requirement Description R1 Basic Graphic User Inter- face Ausershouldhaveabasegraphicaluser interface (GUI) for manipulating and invoking DISCO. R2 “Pluggable” Connector Selection A user should have the capability to “plug in” different connector selection algorithms. R3 Selection Execution Envi- ronment Ausershouldhavethecapabilitytoexe- cutedifferentselectionalgorithmsavail- able via DISCO easily. R4 Result Visualization A user should have the capability to graphically visualize the results of the connector selection algorithms and to compare them. Our original goal was to allow graphical editing of the connector profiles via the DISCO GUI, but time constraints prevented its completion. The Properties File parameter in the main window of the DISCO GUI allows a user to select different DISCO properties files that configure the location and types of domain knowledge passed to the connector selection algorithms, helping to address Goal 2 and requirement R2. The Selector input in the main window allows a user to specifywhichconnectorselectionalgorithmtoexecutefromDISCO.Theconnector selectionalgorithmsaredynamicallypopulatedinthislistusingJavaintrospection. TheGUIidentifiesanyclasswithintheDISCOAPIthatimplementstheSelector extension point and populates the list with those classes. As an example, Fig. C.2 shows the three available selectors currently within DISCO, the ScoreBasedSelector, the BayesianSelector and the OptimizingSelector. In this fashion, the DISCO GUI addresses requirements R2 (the user need only build a java class that implements the Selector extension 280 Figure C.1: Overview screenshot of the DISCO GUI. Figure C.2: Screenshot of choosing selectors within the DISCO GUI. 281 Figure C.3: Screenshot of adding scenarios to run connector selection. pointandmakeitavailabletotheGUIatruntime), andR3, allowingtheselection of different connector selection algorithms. TheScenariosinputparameterallowsausertospecifyfullfilepathstodistri- butionscenarioXMLfilesonthefilesystemusingthe XMLFilter, asshowninFig. C.3. Alternatively, a user can build a new distribution scenario dynamically (and save it) using the window shown in Fig. C.4. Selected scenarios can be removed using the corresponding button in the main window for the DISCO GUI. Once the aforementioned information has been specified, a user can execute DISCO interactively through the DISCO GUI, addressing requirement R3 and Goal 3. The user input parameters on the bottom left portion of Fig. C.5 control the resultant display of connector selection. Display all results and Display only N results are inputs that control the top N amount of connectors displayed connector rank lists returned from the connector selection. Enable the display 282 Figure C.4: Screenshot of specifying a custom distribution scenario. of scores can be turned on or off, and it specifies whether or not the GUI should display the associated rank for a connector in the list (on), or simply print the connector name (off). TheresultsofconnectorselectionareshowninFig. C.6. AfterinvokingDISCO, the connector rank list is shown as a table mapping distribution scenario, to its resultant connector rank list, addressing requirementR4. The connector rank list 283 Figure C.5: Screenshot of connector selection for a single scenario. Figure C.6: Screenshot of connector selection results. view shown in Fig. C.6 allows the rank list to be sorted based on connector rank, name or associated scenario. While the early experience with the DISCO GUI has been quite positive, and thoughwewereabletoaddressallofourgoalsandrequirements, thereareavenues of future work to extend the GUI. We note that, because of our utilization of the 284 MVC architecture within the GUI, the process of adding new models, new views, and new controllers to expose more of DISCO’s underlying functionality, and to provide more of a unified environment, will be inherently simpler, and require less adaptation of our existing GUI implementation. 285

Abstract (if available)

Abstract Data-intensive systems and applications transfer large volumes of data and metadata to highly distributed users separated by geographic distance and organizational boundaries. A dominating factor in these large volume data transfers is the selection of the appropriate software connector that satisfies user constraints on the required data distribution scenarios. This task is typically accomplished by consulting "gurus'' who rely on their intuitions, at best backed by anecdotal evidence.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

A user-centric approach for improving a distributed software system's deployment architecture

PDF

Design-time software quality modeling and analysis of distributed software-intensive systems

PDF

Architecture and application of an autonomous robotic software engineering technology testbed (SETT)

PDF

Domain-based effort distribution model for software cost estimation

PDF

Architectural evolution and decay in software systems

PDF

Automated synthesis of domain-specific model interpreters

PDF

Policy based data placement in distributed systems

PDF

Constraint-based program analysis for concurrent software

PDF

Assessing software maintainability in systems by leveraging fuzzy methods and linguistic analysis

PDF

Designing an optimal software intensive system acquisition: a game theoretic approach

PDF

Prediction of energy consumption behavior in component-based distributed systems

PDF

Using metrics of scattering to assess software quality

PDF

Proactive detection of higher-order software design conflicts

PDF

Techniques for methodically exploring software development alternatives

PDF

Software architecture recovery using text classification -- recover and RELAX

PDF

A reference architecture for integrated self‐adaptive software environments

PDF

Quantitative and qualitative analyses of requirements elaboration for early software size estimation

PDF

The incremental commitment spiral model process patterns for rapid-fielding projects

PDF

Software quality understanding by analysis of abundant data (SQUAAD): towards better understanding of life cycle software qualities

PDF

Security functional requirements analysis for developing secure software

Asset Metadata

Creator Mattmann, Chris Alan (author)

Core Title Software connectors for highly distributed and voluminous data-intensive systems

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Computer Science

Publication Date 09/10/2007

Defense Date 07/20/2007

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag data distribution,data grid,grid,OAI-PMH Harvest,Software Architecture,software connector

Language English

Advisor Medvidovic, Nenad (committee chair), Boehm, Barry W. (committee member), Gupta, Sandeep K. (committee member), Horowitz, Ellis (committee member)

Creator Email mattmann@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-m811

Unique identifier UC1138699

Identifier etd-Mattmann-20070910 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-550712 (legacy record id),usctheses-m811 (legacy record id)

Legacy Identifier etd-Mattmann-20070910.pdf

Dmrecord 550712

Document Type Dissertation

Rights Mattmann, Chris Alan

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Repository Name Libraries, University of Southern California

Repository Location Los Angeles, California

Repository Email cisadmin@lib.usc.edu