University of Southern California Dissertations and Theses

Detecting anomalies in event-based systems through static analysis

(USC Thesis Other)

Detecting anomalies in event-based systems through static analysis

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Detecting Anomalies in Event-Based Systems Through Static Analysis by Gholamreza Sa A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulllment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (COMPUTER SCIENCE) December 2016 Copyright 2016 Gholamreza Sa Dedication To My Parents with Love Mahmoud Sa and Maryam Alibabaei My Two Best Teachers in Life ii Acknowledgments The completion of this dissertation marks an end to some of the most important years of my life. The past six years have had many up and downs. There were so many people who supported me during bad times and were beside me during the good times, and I would like to take the opportunity to thank all of them in this section. First, I want to thank my advisor, Nenad Medvidovic, for his guidance, support, and patience. I learned a lot from Neno during the past six years; he was a truly role model for me as an honest and hard-working researcher. His teaching prepared me to tackle the diculties with my research, and his openness to exploring new ideas helped me to dene my research topic. Beyond helping me to become a better researcher and software engineer, he also taught me how to become a better presenter and communicator of my ideas, a skill that is useful even in non-academic environments. I also had the opportunity, during my years as a graduate student, to work with two other incredibly helpful committee members, Professor William GJ Halfond and Professor Sandeep Gupta. GJ was a truly wonderful mentor for me during the past six years, and I am thankful for his comments and kind advice. The rst course that I took with GJ at my rst semester at USC was one of the best courses that I have ever taken. That course left me certain that I would work with GJ throughout my PhD. I would also like iii to thank Professor Sandeep Gupta for his advice and comments, which were crucial in helping me steer the direction of my research. I genuinely appreciated working with members of the SoftArch research lab. Daniel Popescue's research helped me understand problems in event-based systems. Ivo Krka, Joshua Garcia, and Farshad Tajali helped me by providing me with valuable feedback. Arman Shahbazian worked with me through the evaluation of my proposed techniques and was consistently helpful. Other members of the lab that I would like to thank include Jae Young Bang, Chanin Laohaphan, Youn Kyu Lee, Duc Minh Le, Pooyan Behnamghader, Daniel Link, Rohullah Shemirani, Yixue Zhao. I also would like to thank Supannika Koolmanojwong, Pong Tip Aroonvatanaporn, and Ramin Moazzeni for sharing their experiences about graduate school with me and being good friends. Thank you to Lizsl De Leon for all of her help during the past six years, which always made my life as a PhD student much easier. I also want to thank my friends at USC and Los Angeles who were always there for me, especially Safar Hatami, Mohammadreza Chitgarha, Taha Bahadori, Mahdy Khayyamian, Mehdi Kamal, and Javad Dousti. My family was always supportive of me during my PhD. My sister Leila, who had a great impact on my academic achievements by helping me with my school work when I was a kid; my brother-in-law Reza, who was always like a good friend to me and a role model; my brother Majid, who was always a source of hope for me; my brother Hamid, who was always a great source of inspiration and motivation for me, and I owe him a lot because of all of his support. Hamid was my inspiration to continue my studies in computer science and was always there for me whenever I needed him. My sister-in-law iv Mahta, who was always kind and supportive. I want to thank Shima, Mahan, Arman, and Darwin, my kind, smart, and beautiful nephews and nieces, who were sources of happiness for me. I have missed them a lot in the past six years. My special thanks will go to my parents, my mom Maryam and my father Mahmoud. These two lovely people were my true teachers throughout my life. I cannot thank them by using words. I know that I am the luckiest person in the whole universe because I have them. They always tried hard to open the way for me, my brothers, and my sister to achieve our goals, while at the same time not having any expectations. It was hard to be away from them during the years of my PhD, but every time that I talked with them they gave me lots of support and made me forget about the diculty of my work. Last, but not least, I want to thank the love of my life and the light of my eyes, my wife, Nura Hossainzadeh, for all of her patience, love, kindness and support during the past three years. I could not imagine how my life would have looked like without her being with me during the hardships of my PhD. Without Nura's endless love, help, and support during the moments of frustration and hopelessness, I would not be able to nish this dissertation. I cannot describe how thankful I am. v Table of Contents Dedication ii Acknowledgments iii List Of Tables viii List of Figures x Abstract xii Chapter 1 Introduction 1 1.1 Event Anomalies 3 1.2 Synchronization Anomalies 7 1.3 Thesis Statement and Supporting Hypotheses 11 1.4 Evaluation Strategy 13 1.5 Contributions 13 1.6 Structure of This Dissertation 14 Chapter 2 Foundations 15 2.1 Running Example 15 2.2 Consumed Events 16 2.3 Ambiguous Interfaces 16 2.4 Callback Methods 17 2.5 Eos: A Technique to Extract Event Types 18 2.6 Program Analysis Concepts 19 2.7 Denition of Event Anomalies 21 2.8 Denition of Synchronization Anomalies 23 Chapter 3 Approach 26 3.1 Detecting Event Anomalies 26 3.2 Detecting Synchronization Anomalies 42 Chapter 4 Evaluation 55 4.1 Evaluation of DEvA 55 4.2 Evaluation of DSA 72 4.3 Limitations and Threats to Validity 102 vi Chapter 5 Related work 105 5.1 Related Work to DEvA 105 5.2 Related Work to DSA 110 Chapter 6 Conclusion 115 6.1 Future Work 119 References 121 vii List Of Tables 3.1 LoadBalancer component information 30 3.2 LoadBalancer component's ArgChanged and ArgShared 50 4.1 Systems used in the evaluation of DEvA 57 4.2 Results of applying DEvA on the subject systems 61 4.3 Number of highly ranked anomalies after ranking phase 65 4.4 Results of applying Chord on the subject systems 68 4.5 Results of UF analysis by CAFA, and DEvA after Ranking 70 4.6 Systems used in the evaluation of DSA 73 4.7 Summary of the application of DSA on the Planner system 77 4.8 Summary of the application of DSA on the KLAX system 77 4.9 Summary of the application of DSA on the DRADEL system 77 4.10 Summary of the application of DSA on the ERS system 78 4.11 Summary of the application of DSA on the Troops system 78 4.12 Summary of the application of DSA on the Stoxx system 78 4.13 Summary of the application of DSA on the JMSChat system 78 4.14 Summary of the application of DSA on the JMS2009-PS system 78 4.15 Summary of the application of DSA on the JFreeChart system 79 viii 4.16 Summary of the application of DSA on the Spark system 80 4.17 Summary of the application of DSA on the JavaGame system 80 4.18 Summary of the application of DSA on the WebSocket system 81 4.19 Summary of the application of DSA on the Chronicle system 81 4.20 DSA's Analysis time for each system 83 4.21 The results of removing synchronization anomaly (SA)s without consider- ing the longest sequence metric 101 4.22 The results of removing SAs by considering the longest sequence metric 102 4.23 The results of removing SAs by considering the loop metric 102 ix List of Figures 1.1 Excerpt from the MyTracks Android application 4 1.2 Excerpt from the Spark application 8 2.1 Event-based load-balancing component 20 2.2 ICFG of LoadBalancer 21 2.3 PDG of LoadBalancer 22 4.1 Portion of the MyTracks Conguration used by DEvA 58 4.2 A view of DEvA's architecture 60 4.3 Excerpt from the KLAX system 64 4.4 Excerpt from the ToDoWidget Android application 67 4.5 Excerpt from the Planner system 75 4.6 Excerpt from the JMS2009-ps system 76 4.7 Excerpt from the WebSocket system 76 4.8 Queue length for synchronized methods 90 4.9 Queue length for concurrent methods 91 4.10 Markov model for the system with two dierent arrival rates and two dif- ferent service times 93 4.11 Comparing number of requests in the system when the parameters have a large dierence 96 x 4.12 An illustration of how to remove a SA 100 xi Abstract The event-based paradigm allows developers to design and build systems that are highly exible and can be easily adapted. There are two main complications that can occur in the systems that are based on this paradigm. The rst complication concerns inter- component interactions. Events that are used by components for this purpose are sent, received, and processed nondeterministically, due to the systems' reliance on implicit invocation and implicit concurrency. This nondeterminism can lead to event anomalies, which occur when an event-based system receives multiple events that lead to the write of a shared eld or memory location. Event anomalies can lead to unreliable, error-prone, and hard-to-debug behavior in an event-based system. The second complication concerns intra-component interactions that usually occur through method calls. Each sequence of method calls introduces an execution path to the system. It is possible that there exist multiple execution paths that are not accessing the same memory locations or sharing data but are unnecessarily synchronized with each other despite the fact that they can be executed concurrently. I call this situation synchronization anomalies. To detect event anomalies, this dissertation presents a new static analysis technique, DEvA, for automatically detecting event anomalies. DEvA has been evaluated on a set of open-source event-based systems against a state-of-the-art technique for detecting data xii races in multi-threaded systems and a recent technique for solving a similar problem with event processing in Android applications. DEvA exhibited high precision with respect to manually constructed ground truths and was able to locate event anomalies that had not been detected by the existing solutions. Also, this dissertation presents a new static analysis technique, DSA, for automati- cally detecting synchronization anomalies. I have evaluated DSA both empirically and analytically. My empirical evaluation shows that synchronization anomalies are a com- mon problem and can occur in any randomly chosen system. Also, DSA is ecient and scalable while exhibiting high precisions after being applied to fourteen subject systems. The analytical evaluation of DSA provides guidelines for the situations where removing a synchronization anomaly can be more benecial. By removing just one synchronization anomaly from nine of our subject systems based on the provided guidelines, on average there was an enhancement of 10% in the performance of those systems. xiii Chapter 1 Introduction The event-based paradigm allows developers to design and build systems that are highly exible and can be easily adapted [36, 42, 54, 97]. These advantages have made event- based systems (EBSs) popular in a range of domains [27, 28, 35]. However, the event- based paradigm can also introduce complications due to the nature of the underlying event processing. In event-based systems, components make use of events for inter- component interactions. Components in event-based systems also use method calls to provide modularization for intra-component interactions. In inter-component interactions, events can be delayed, damaged, or lost because of environment problems, such as network errors or hardware deciencies. More gener- ally, events can be consumed nondeterministically, independent of their order of occur- rence [59]. While directly enabling EBS exibility, this nondeterminism can also lead to unpredictable system behavior. This dissertation will rst address one specic type of problem related to event handling in inter-component interactions: event anomaly (EA). An EBS has an EA when the processing of two or more events results in accesses to the same memory location (e.g., a variable containing state or data) and at least one of those 1 is a write access. The impact of an EA can vary based on the role the aected variable plays in the EBS but could include undesirable behaviors ranging from inconsistent state to data corruption. There could be some problems in intra-component interactions as well. As mentioned earlier, intra-component interactions usually happen through the use of method calls. Each sequence of method calls introduces an execution path to the system. It is possible that there exist multiple execution paths that are not accessing similar memory locations or sharing any data but are executing sequentially in a way that one of those paths is unnecessarily waiting for the other path to nish. This means that, an unnecessary order is enforced here between independent execution paths. I will call this situation SA. When there exists a SA, there is an unnecessary determinism in the order of method calls. The methods could have been called concurrently without any need to be synchronized with each other or to wait for each other to nish. This unnecessary wait wastes computation resources and will cause a reduction in the performance of the system. In the last part of my dissertation, I will focus on SAs. The goal of this dissertation is to help improve the quality of event-based systems by, rst, detecting harmful problems that can be caused by non-determinism, and second, detecting problems that can be caused by the unnecessary determinism in the execution order of independent execution paths that are introduced by using synchronous method calls. In other words, I will explore event-based systems to nd problems that may occur both in the presence of non-determinism or in its absence to help developers improve the quality of such systems. 2 In the remainder of this chapter, I will rst provide a motivating example for EAs, and then I will discuss the challenges of detecting EAs and also will talk about existing work on detecting EAs. After that, I will explain my proposed approach for detecting EAs. Following this, I will also discuss SAs in more detail by providing a motivating example, discussing the challenges of detecting them, providing an overview of related work, and introducing my approach to detect SAs. After explaining EAs and SAs in more detail, I will provide my insights and hypotheses. 1.1 Event Anomalies Motivating Example About EAs: MyTracks Android Application As a motivating example for EAs, consider MyTracks, an Android application developed by Google. MyTracks records the location, path, distance, and elevation of a Google Maps user. Figure 1.1 shows a portion of this application; specically, the handlers for two events: onLocationChangedAsync (line 4) processes LocationChangedAsync events and onDestroy (line 9) processes Destroy events. Destroy is a common event used for memory management that, when it occurs, frees the memory assigned to an activity or service of a given Android application. 1 In the case of MyTracks, the EA occurs when bothDestroy andLocationChangedAsync events occur but Destroy is processed rst. The onDestroy handler will free memory by 1 Note that invoking onDestroy is not equivalent to performing garbage collection in Android. onDe- stroy only provides an opportunity to clean things up before an activity or service is destroyed: it can change the memory locations referenced by the activity or service instance or free them by setting them to null, but it does not release the instance. 3 1 public class TrackRecordingService extends Service 2 implements LocationListener { 3 ... 4 private void onLocationChangedAsync( Location location){ 5 ... 6 Location lastRecordedLocation = providerUtils.getLastLocation(); 7 ... 8 } 9 public void onDestroy() { 10 Log.d(TAG, "TrackRecordingService.onDestroy"); 11 ... 12 providerUtils = null; 13 ... 14 } 15 ... 16 } Figure 1.1: Excerpt from the MyTracks Android application setting the private class variable providerUtils to null. At some point shortly there- after, onLocationChangedAsync will attempt to access providerUtils, thus generating a null pointer exception. The cause of this exception is an EA, since each of the two events results in an access to the same memory location and one of them is a write access. The example in Figure 1.1 is an instance of an EA known as Use-After-Free (UF). My proposed approach, DEvA, was able to detect this EA, but existing techniques, such as CAFA [56], were unable to do so. These types of EAs also occur in other real-world systems, such as Firefox for Android [14,15]. Challenges of Detecting EAs Detecting EAs is dicult for several reasons: Nondeterminism: When a tester suspects that an EBS contains an EA, nondeter- minism can make it dicult to execute or reproduce the EA. This reduces the 4 eciency and eectiveness of standard test-based detection techniques and popular spectrum-based fault localization techniques. Implicit Invocation: EBSs rely on callbacks, which are methods registered with an event-notication facility and called when the notier receives an appropriate event. The resulting implicit invocation makes it dicult to identify the control- ow of the code when an event is received and processed. Ambiguous Interfaces: Event callback methods often accept a generic Event type. They must examine the event's attributes to determine its actual type [47] and to dispatch the event appropriately [61]. Ambiguous interfaces make it dicult to determine the event type that is responsible for an EA. Implicit Concurrency: Dierent types of received events may result in dierent method invocations in an EBS, thereby introducing dierent execution paths. Each of these paths is independent of the others and may be executed in any order, depending on when the events are consumed [31]. The resulting implicit concurrency makes it dicult to identify when a variable may be accessed by two dierent execution paths. Implicit invocation, implicit concurrency, and ambiguous interfaces are useful mech- anisms for EBSs. Together they allow EBSs to be loosely coupled and to scale easily. At the same time, these very mechanisms make it harder to determine execution order and memory-access patterns in an EBS, whose interplay results in EAs. As a consequence, it can be very challenging for developers to detect EAs in EBSs. 5 Existing Approaches for Detecting EAs Researchers have recognized the need for automatically discovering EAs. For instance, CAFA [56] identies Use-After-Free (UF), a common type of EA in Android applications, whileWebRacer [85] andEventRacer [90] focus on detecting certain types of EAs in client-side web applications. However, these techniques are based on dynamic analysis; therefore, they oer no guarantees of completeness and can only identify EAs that have been exposed during an execution. Other approaches try to address the problem of EAs by introducing new programming language constructs [39, 65]. However, these approaches are not applicable to existing systems that are written in general-purpose languages, such as Java. Furthermore, they tend to consider callback handlers with explicitly typed events as the potential source of anomalies, when for many EBSs the cause of EAs is the processing of generically typed events inside the handlers, followed by dispatching to dierent component methods. Approach for Detecting EAs and Its Contributions To address these limitations of existing techniques, this dissertation contributes a new static analysis technique that specically targets EBSs and can handle the semantics of implicit invocation, ambiguous interfaces, and implicit concurrency. This technique, called DEvA (for Detecting Event Anomalies), builds on some recent work on static analysis to identify ambiguous interfaces in EBSs [47] and adds further analyses to identify variables that may be modied as a result of receiving an event|a My approach for detecting EAs has two main contributions: 6 1. Design and evaluation of DEvA{ a static analysis technique that achieves high pre- cision and recall (i.e., a measure of completeness) when extracting event anomalies from a broad range of existing EBSs. 2. Design and evaluation of a static analysis approach toward understanding the harm- fulness of anomalies. 1.2 Synchronization Anomalies Motivating Example About SAs As a motivating example for SAs, consider Spark, a free open source cross-platform mes- saging client that is optimized for businesses and organizations. Figure 1.2 shows a portion of this system, specically a method in Spark's PacketReader component. PacketReader processes the incoming network packets. Receiving a packet can be considered an event that PacketReader will handle. The method parsePackets in this component acts as a handler for dierent kinds of packets that can be received by PacketReader. As can be seen in gure 1.2, the packet name, which is the main indicator of the type of packet, is stored in the Name attribute of the packet. The PacketReader component can handle several packets, and in response to each of them, it will execute an independent execution path (Lines 3-14). These paths, which are executed through method calls are not sharing any data or information, but they must still wait for each other to nish. parsePackets cannot process a new packet before nishing up the consumption of the previous packet. This is an unnecessary wait, since 7 1 class PacketReader { 2 private void parsePackets(...) { 3 if (parser.getName().equals("message")) { 4 processPacket(PacketParserUtils.parseMessage(parser)); 5 } 6 ... 7 if (!"1.0".equals(parser.getAttributeValue("", "version"))) { 8 releaseConnectionIDLock(); 9 } 10 ... 11 else if (parser.getName().equals("features")) { 12 parseFeatures(parser); 13 } 14 else if (parser.getName().equals("proceed")) { 15 resetParser(); 16 } 17 } 18 ... 19 } Figure 1.2: Excerpt from the Spark application the consumptions of the dierent packets are completely independent of each other and there is no need for a packet's processing to wait for another packet's consumption. This unnecessary wait can cause an increase in the system's response time and a reduction in its performance. A preferred way of performing this would have been to de- sign a pool of threads and assign each packet's consumptions to a thread from that pool, so that each packet consumes concurrently without making other packets' consumptions wait. Using a thread pool can introduce some overhead to the system. Having too many unused threads in the thread pool will waste resources. Because of this, determining an optimal size for the thread pool is the key in the improvement of the system's perfor- mance. In the evaluation chapter of this dissertation, I will perform an analysis to provide guidelines on the conditions under which the highest gain in performance can be achieved through the removal of SAs. 8 Challenges of Detecting SAs The main challenge in detecting SAs through static analysis is to reason about the same memory locations that can be accessed in dierent method calls. For example, a memory location's address can be passed as an argument to several methods, and depending on whether that memory location is changed inside those methods, independent execution paths may be introduced into the system. Figuring out whether an argument to a method is changed inside that method and whether it is shared with other methods is dicult. This reasoning process can also suer from scalability issues and any proposed approach should take this into account as well. There are also challenges that involve diculties with deciding where in the code an enhancement to the performance can be made. Software components may have several methods, and each of those methods can contain several independent execution paths. It is challenging to decide at what level in the sequence of method calls independent execution paths should be executed concurrently. This is the main reason that compiler optimization techniques cannot automatically parallelize a sequential code. Existing Approaches for Detecting SAs Compiler optimizations techniques try to generate optimized code by detecting and re- moving redundant code or optimizing loops [23, 58, 66, 98]. A compiler optimization technique needs to get some feedback from the user if they want to do further optimiza- tions, and this kind of feedback is not available at compile time. Without user's feedback, sometimes an optimization may cause an unwanted behavior [33,52,89]. 9 There is a class of bugs, called performance bugs or PBs, that have been studied before by other research groups. PBs are similar to SAs, since they also will cause a reduction in the performance of the system. Existing techniques to detect PBs are either based on analyzing the source code [81] of a system or monitoring the runtime characteristics of that system and reproducing the potential bugs [40,106]. Source code analysis techniques usually try to nd a common pattern in the code that will cause a PB [57,78]. This pattern can be determined by either analyzing the history of bugs or reasoning about data ow and code redundancy [34, 57]. There is a study [78] that shows code reasoning reveals more PBs compared to bug reproducing techniques. None of the existing techniques that are based on code analysis consider independent sequences of method calls as the source of performance bugs. Also, they either need feedback from developers, or they make assumptions to simplify their analyses. Approach to Detecting SAs and Its Contributions To extract SAs that are based on the existence of independent sequences of method calls, this dissertation contributes a new static analysis technique. This technique, called DSA, detects independent sequences of method calls by performing an analysis over the call graph of the system along with data and control ow analysis at the level of each compo- nent of the system. DSA is designed to scale by using caching and dynamic programing at the level of each method of each component. DSA also provides visualization of the locations in the code that are suering from SAs and most likely should be considered for change. 10 My approach for detecting SAs has one primary contribution. This contribution is the design and evaluation of a static analysis technique that can provide useful information to the users about possible points of performance improvement in the code. 1.3 Thesis Statement and Supporting Hypotheses The overall goal of the dissertation is to extract event anomalies that can occur in an event-based systems and also reduce the eort required in studying the harmfulness of those anomalies. The challenges of analyzing event-based systems and also the short- comings of existing approaches motivate the following hypothesis, which this dissertation tests. Hypothesis 1 Insight1A: The order of consumption of events in event-based systems is non-deterministic, but the place that an event consumption occurs in the code is deterministic. Insight 1B: An event anomaly occurs when two independent execution paths that start from the locations of two dierent event consumptions access the same memory location, and at least one of those accesses is a write access. Hypothesis 1: By focusing on the source code of a system that uses event-based mecha- nisms, a static analysis approach can report all the possible event anomalies in the system with high precision and recall 11 Hypothesis 2 Insight 2A: Event anomalies are similar to data races in multi-threaded systems; how- ever, because of implicit invocation and implicit concurrency, in event-based systems it is hard to determine the execution order and trace memory-access that is caused by event consumptions. Race detection techniques in multi-threaded systems just consider the overlapping of executions between dierent threads and do not take implicit concurrency into account. Hypothesis 2: A race detection technique that is applied over an event-based system cannot report event anomalies that my approach will report. Hypothesis 3 Insight 3A: The eects of an anomaly on the system can be determined by analyzing the ow of data after a memory access that participates in the event anomaly. Hypothesis 3: A static analysis can determine potential harmful anomalies and rank anomalies. The ranking can be dened based on the eect of the anomaly on the state of the system Hypothesis 4 Insight 4A: Independent execution paths can be executed concurrently. Insight 4B: Sequences of method call can introduce reusable independent execution paths to the system. Hypothesis 4: A static analysis technique can be proposed to extract all the synchro- nization anomalies from the source code of a Java-based system with high precision 12 1.4 Evaluation Strategy I evaluated DEvA on 20 open-source event-based applications and libraries from dier- ent domains. I compared DEvA's performance to two state-of-the-art techniques, one targeting traditional data races in multi-threaded systems and the other targeting UF anomalies in Android. The evaluation results show that DEvA has high precision and was able to nd EAs that the other techniques were unable to detect. DEvA was also fast, averaging one minute per application analysis, which is signicantly faster than the previously proposed techniques. I also have evaluated DSA both empirically and analytically. I used 14 subject systems for the evaluation of DSA. The evaluation results of DSA showed that SAs are a common problem across dierent systems. Using probability and queuing theory concepts, I also provided an analytical evaluation of DSA. This analytical evaluation provided useful guidelines for the situation in which removing SAs is more benecial. Using the guidelines from the analytical evaluation, I applied simple changes to nine of the subject systems and got on average an increase of 10% in the performance of those systems. 1.5 Contributions This dissertation makes four primary contributions: 1. Design and evaluation of DEvA{ a static analysis technique that achieves high pre- cision and recall (i.e., a measure of completeness) when extracting event anomalies from a broad range of existing EBSs. 13 2. Design and evaluation of a ranking approach to help developers rank event anomalies based on their harmfulness. 3. Design and evaluation of DSA{ a static analysis technique that can provide useful information to users concerning possible points of performance improvement in the code. 4. Creation of an analytical evaluation to determine situations where removing syn- chronization anomalies is most benecial 1.6 Structure of This Dissertation The remainder of this dissertation is structured as follows: Chapter 2 denes the concepts and denitions on which this dissertation is based. The proposed solutions and approaches are discussed in Chapter 3, and Chapter 4 describes the evaluation. Chapter 5 summarizes the related work. Finally, Chapter 6 concludes the dissertation. 14 Chapter 2 Foundations This chapter introduces the underlying concepts and terminology that are used through- out this dissertation. In EB systems, components interact with each other through sending and receiving events. An event usually considered as something that can be perceived by a component. This can be a change in the state of the component or its environment or an event that is received from another component. To illustrate the concepts related to EB systems, I will rst describe a running example in section 2.1. Then 2.1 Running Example To illustrate my denitions, I will use the implementation of a LoadBalancer component shown in Figure 2.1. LoadBalancer monitors the state of a server. Whenever the load on the server rises above a specied limit, it will stop responding to new requests. A new request comes to LoadBalancer through a NewRequest event. At lines 6 and 7 of Figure 2.1, LoadBalancer consumes this event and decides whether to process it on the server or send a notication event to inform other parts of the system that the server is 15 overloaded. The limit for the load on the server is set whenever LoadBalancer receives a SetLimit event. At lines 9 - 13, LoadBalancer consumes this event, sets the limit, and checks if the new limit is less than the previous one. If so, it informs other parts of the system about the limit reduction. 2.2 Consumed Events In an EBS, the process of a component receiving an event and accessing the information dened in the event's attributes is called event consumption. At lines 9 - 11 of Figure 2.1, the event SetLimit is consumed, since LoadBalancer receives the event and processes the event's contents by checking its \Name" and \Limit" attributes. An EBS component's entry point for event consumption is called a sink, or alternatively an event handler. In Figure 2.1, the handle method dened at line 5 is a sink. 2.3 Ambiguous Interfaces To reason about event-based systems, we need to reason about the types of the events that a component consumed or publish. An event's type refers to the kind of information that can be stored in the event. In some EB systems, those event types are explicitly captured (e.g., the two event types MyTracks from Figure 1.1 is able to receive). In other EB systems, however, a component's event handler do not explicitly reveal the event types. Instead, event handlers receive an event of a generic type as their argument and the handler will decide about the consumption process by checking the information that is stored in the event. We call these kind of handlers ambiguous interfaces [46]. For 16 example the LoadBalancer in Figure 2.1 has an ambiguous interface since it receives all events via the same sink (Line 5). Identifying the types of events consumed by an EBS in the presence of ambiguous interfaces is challenging because all events handled by a system arrive as a single, generic Event type. The actual type of each event is determined at runtime by checking the attributes contained within the event. A recent technique in whose development I par- ticipated [47] resulted in a static analysis technique, called Eos, that analyzes the imple- mentation of an EBS to infer the set of event types it can consume at runtime. I discuss Eos in more detail in the section 2.5. 2.4 Callback Methods In event-based systems, usually an interface exists for each event that can be perceived by components. These interfaces are either dened for explicitly typed events, or they are ambiguous interfaces as dened in Section 2.3. A component that is interested in an event will implement the event's corresponding interface and will register that imple- mentation with a notier. Whenever the notier perceives an event, it will callback all the implementations that were registered with it. Those registered implementations are called callback methods. Because callback methods decrease the coupling between components, they are widely used in event-based systems. In the systems that are based on message oriented mid- dleware (MOM-based systems) or frameworks such as Android the list of callbacks are available through documentation of the provided API. The user-dened callbacks can 17 also be dened and any approach that wants to be applicable over event-based systems should consider both system-provided and user-dened callbacks. 2.5 Eos: A Technique to Extract Event Types As I mentioned earlier, Eos is a static analysis technique that extracts the event types from the source code of an EB system in the presence of ambiguous interfaces. Eos leverages consumed event revealing (CER) statements to infer information about the consumed event types (CETs) of a component. A CER statement is a method call that retrieves information stored in an event without modifying the event's attributes. A CET is dened by a set of attributes and their values, which can be inferred from CER statements. Methods called by a CER statement are of the following forms: 1. getAttr(a) represents methods retrieving the value of the attribute named a of an event. As an example, consider getAttr in Figure 2.1 at line 11. 2. retrieveSpecicEventInfo() represents methods whose names indicate the informa- tion the methods directly obtain about an event; this is in contrast to getAttr(a), which relies on parameter a to specify the information of interest. For example, in Figure 2.1 at line 6e:getName() is used to retrieve the name parameter of event e, while at line 7 e:getRequest() is used to retrieve the request parameter contained in e. 3. hasAttribute(a) represents methods that check whether an event has a specic at- tribute with the name a. 18 4. m(e) indicates methods that accept an event e as an argument. The explicitly declared type of the argument, required by the programming language, reveals information about the type of the event e, which may have been cast elsewhere in the system before m() was invoked. Eos identies CER statements in a component and aggregates their information along the paths in the CFGs of the component's methods. The type information revealed by the CER statements along each path is used to dene a CET of the component. I refer to the set of all event types that are consumed in a component as its CET set. Eos also returns all of the nodes in the ICFG that contain a CER statement that contributed to the denition of each CET. To illustrate, consider the LoadBalancer component from Figure 2.1. Its CET set consists of two event types. The rst event type has two attributes, one with the name \Name" (whose value is NewRequest) and one with the name \Request". The second event type also has two attributes, \Name" (with the value SetLimit) and \Limit". 2.6 Program Analysis Concepts My proposed approaches in this dissertation rely on control- ow information in the form of a control- ow graph (CFG). A CFG is a directed graph in which each node represents a basic block of code and each edge represents the relationship between two blocks that at runtime may be executed after one another. I also use the inter-procedural control- ow graph (ICFG), which is a directed graph that contains the CFGs of a component's dierent methods. If one method calls another, then there is an edge in the ICFG between the method invocation point and the entry node of the target method's CFG. 19 1 public class LoadBalancer extends Component{ 2 Long curLoad; 3 Long lmt; 4 Long preLmt; 5 public void handle(Event e){ 6 if(e.getName().equals("NewRequest")){ 7 manageCurrentLoad(e.getRequest()); 8 } 9 if(e.getName().equals("SetLimit")){ 10 preLmt = lmt; 11 lmt = Long.parseLong(e.getAttr("Limit")); 12 if(lmt < preLmt) 13 send(new Event("LimitReduced")); 14 } 15 } 16 public void manageCurrentLoad(Request r){ 17 if(curLoad < lmt){ 18 curLoad = curLoad + 1; 19 //Start processing new request 20 } 21 else 22 send(new Event("LoadTooHigh")); 23 } 24 } Figure 2.1: Event-based load-balancing component The ICFG of LoadBalancer is shown in Figure 2.2. The nodes in the ICFG correspond to the statements in Figure 2.1. I use this graph to illustrate two more concepts. First, a CFG nodeX is control dependent on nodeY if execution ofY determines whetherX will be executed. Consider the CFG of the handle method in the LoadBalancer component, which is shown in Figure 2.2 starting at node Entry in the top middle of the graph. Nodes 7 and 8 are control dependent on 6 being true but node 9 is not, since 9 will execute for either condition of node 6. Second, a node X is data dependent on node Y if there is a path in the CFG from Y to X, X uses the value of a variable that has been dened at Y , and no other node on that path redenes that variable [53]. In the LoadBalancer 20 Entry 6 8 T 9 F Exit F 10 11 12 T T handle(Event e) Entry 17 18 21 22 T F 19 20 14 13 F manageCurrentLoad(Request r) 7. Call 7. Call 7. return 7. return Exit Figure 2.2: ICFG of LoadBalancer example, node 12 is data dependent on node 11, since at line 12 in Figure 2.1 the if statement uses the lmt variable and lmt is dened at node 11. To show all control- and data-dependency relationships between dierent nodes of an ICFG, we can use the Program Dependence Graph (PDG). In a PDG, the nodes represent basic blocks in the program and edges represent control or data dependencies between the nodes. Figure 2.3 shows the PDG for the example from Figure 2.1. 2.7 Denition of Event Anomalies An event consumption may cause an access to the state of a component, which is repre- sented by the component's elds as well as any shared, non-local variables. For example, in the LoadBalancer component, variables curLoad, lmt and preLmt are its elds. An 21 Entry 6 8 9 T 10 12 11 T Entry 17 21 18 19 22 T T handle(Event e) manageCurrentLoad(Request r) 20 T F 13 14 T 7.call 7.call 7.return 7.return T preLmt lmt Exit Exit Figure 2.3: PDG of LoadBalancer access to a eld is either a read (i.e., use) or a write (i.e., denition). When the con- sumption of two dierent CETs causes accesses to the same eld, with at least one access being a write, a problem may occur: the consumption order of the two events may be dependent on the system's environment and, therefore, the system's behavior may be nondeterministic. I refer to this potential problem as an event anomaly (EA). The goal of one of my proposed techniques, DEvA, is to identify and report such anomalies to developers. Let Components be the set of all components in a system. For a givenc2 Components, I deneF c as the set of all ofc's elds. Furthermore,cerExtract(e) is Eos's function that extracts allCER statements for an evente. There exists an EA in componentc over the eld f2F c due to event types e 1 ;e 2 2 CET c i the following conditions hold: 1. e 1 6=e 2 2. 9fX 1 ;:::;X p gNodes(ICFG)j 1i<m; X i+1 iscontrol or datadependenton X i ^ X 1 2 cerExtract(e 1 ) 22 3. 9fY 1 ;:::;Y q gNodes(ICFG)j 1j <n; Y j+1 is control or data dependent on Y j ^ Y 1 2 cerExtract(e 2 ) 4. X p is a use or denition of f 5. Y q is a denition of f The intuition behind this denition is that we can say an access to a eld has been caused by an event consumption whenever the occurrence of consumption determines that the access must happen (control dependency) or the consumption aects the value that is stored in that eld (data dependency). The above denition has three principal parts: (i) the CET set of a component (condition 1); (ii) the control or data dependency paths from CETs to elds that imply causality (conditions 2 and 3); and (iii) determining those paths that access the same eld with at least one write access (conditions 4 and 5). These three elements form the foundation of my approach for extracting event anomalies. 2.8 Denition of Synchronization Anomalies Dierent event consumptions in an EB system, or even dierent method calls in an or- dinary system can introduce dierent execution paths to the system. If these execution paths are not accessing the same memory locations, there is no need for them to be syn- chronized, and they can be executed concurrently. Having these independent execution paths synchronized will reduce the performance of the system. I call this situation a synchronization anomaly, or SA. The goal of DSA, which is my other proposed approach, is to analyze the source code of a system and extract all the possible SAs that are caused by synchronized method calls. 23 Considering the denitions in section 2.7, SAs can be dened as follows. For a given c2 Components, there exists a SA in componentc due to the synchronization of methods m 1 and m 2 inside method m i the following conditions hold: 1. 9fX 1 ;:::;X p g Nodes(ICFG)j 1 i < m; X i+1 is connected to X i ^ X 1 contains a call to method m 1 2. 9fY 1 ;:::;Y q gNodes(ICFG)j 1j <n; Y j+1 isconnectedtoY j ^ Y 1 contains a call to method m 2 3. X 1 ;Y 1 ;X p ;Y q belongs to method m 4. X 1 and Y 1 are not control or data dependent on each other 5. X p is the return node for the call to method m 1 6. Y q is the return node for the call to method m 2 7. If X i is a denition of f2F c then69k2fY 1 ;:::;Y q g;Y k is a use or denition of f 8. If Y i is a denition of f2F c then69l2fX 1 ;:::;X p g;X l is a use or denition of f 9. If one of m 1 0 s arguments; which doesn 0 t have a primitive type; is changing inside the body of m 1 ; that argument is not used by m 2 10. If one of m 2 0 s arguments; which doesn 0 t have a primitive type; is changing inside the body of m 2 ; that argument is not used by m 1 The intuition behind the denition of SAs is that if there are dierent sequences of method calls that are not accessing any shared memory locations, and those sequences 24 synchronized with each other, there exists a problem in code. The unnecessary synchro- nization will cause a reduction in the performance of the system. 25 Chapter 3 Approach In this chapter, I will present my solutions to the problems that I dened in Chapter 2. In section 3.1, I propose my solution for detecting event anomalies. In section 3.2, I propose the solution for detecting synchronization anomalies. 3.1 Detecting Event Anomalies As I discussed in Section 1.1 of this dissertation, detection of EAs involves challenges such as non-determinism in an event-based environment, implicit invocation, existence of ambiguous interfaces, and implicit concurrency. To overcome these challenges and also the shortcomings of the existing approaches, I have developed an automated static anal- ysis technique that examines the implementation of an event-based system and identies points in the code where EAs may occur. This technique is called DEvA. DEvA rst identies all possible EAs, regardless of their impact on the EBS. After extracting all the possible EAs, I rank the reported anomalies based on the criteria that I discuss in section 3.1. The remaining anomalies are the potential harmful anomalies. Determining the impact of each of the potential harmful EAs is challenging, in part because it can 26 vary signicantly. For example, prior work has found that some races are harmless and that removing them by introducing additional checks in the code can compromise the performance of the system [56, 77, 94]. On the other hand, EAs can lead to signicant reliability problems and could impact system scalability [30]. The determination of the category into which an EA falls is generally a task for a system's engineers who must evaluate the EA's impact on the EBS's functional and non-functional requirements. The main inputs to the analysis are the implementation of an EBS and a description of the framework used for processing events in the system. The description must specify: (1) the list of all methods used as event sinks (in the case of an EBS with ambiguous interfaces) or callback methods that serve as event sinks (in the case of frameworks, such as Android, that rely on explicit event interfaces); (2) the base class used to implement events in the system; and (3) the set of methods used as consumed event revealing (CER) statements. All of the information in the description can be derived from the API specication of the underlying event-based framework and only needs to be identied once per framework. DEvA's analysis can be divided into four distinct phases: 1. extraction| In this phase, the analysis identies all of the consumed event types (CETs) and elds accessed by each component of the system. 2. causation| In this phase, a path-based analysis will be performed to determine if there is a connection between the CETs and accessed elds. 27 3. joining| In this phase, the analysis identies CETs that will lead to an access of the same eld, that is, a possible EA. The CETs and elds identied in the last phase are returned to the developer for more investigation. 4. ranking| This phase ranks anomalies based on their potentially harmfulness. This will reduce the investigation eorts of developers. In the remainder of this section, I discuss each of the four phases in more detail. Extraction During this phase, DEvA identies two types of information about the EBS that will be used in the later phases to identify anomalies. The rst is the set of locations within each component where the component's elds are accessed either by a use or denition. The second is the set of CETs accessed by each of the components. DEvA identies a component's eld accesses via a static intra-procedural analysis of the component's implementation. Formally, I dene a eld access as a tuplehf;ni in whichf2F c is a eld of componentc (recall thatF c is the set of all elds of component c) and n2 N m is a node in a method m's control- ow graph (N m represents the set of all nodes in method m's CFG) that represents the location in the code where the eld is used or dened. DEvA generates two dierent sets, FUse m that contains eld uses and FDef m that contains eld denitions. FDef m =fhf;nijf2F c ^ n2N m ^ n denes fg FUse m =fhf;nijf2F c ^ n2N m ^ n uses fg 28 To compute these sets, DEvA rst builds the CFG of each component method. It then traverses the CFG and checks each node to determine if it accesses a eld. To analyze a eld an alias analysis was performed that rst used class hierarchy analysis (CHA) to identify potential targets of a eld access and then used points-to analysis based on SPARK [63] to rene the results. To illustrate this step, consider the LoadBalancer component shown in Figure 2.1 and its ICFG shown in Figure 2.2. The FDef and FUse sets are shown in Table 3.1. In the handle method, since there is a denition of eld preLmt at node 10 of the ICFG (corresponding to line 10 in LoadBalancer's implementation) and a denition of eld lmt at node 11, tupleshpreLmt; 10i andhlmt; 11i are added to FDef handle . Also since there are two uses of eld lmt at nodes 10 and 12, and a use of eld preLmt at node 12 tuples hpreLmt; 12i,hlmt; 10i andhlmt; 12i are added to FUse handle . A recent technique in whose development I participated, Eos [47], is able to identify CETs in EBSs that use ambiguous interfaces. Running Eos on the LoadBalancer compo- nent would identify the two event types discussed in Section 2.2: the rst event with the attributes, Name and Request, that are accessed using CER statements at lines 6 and 7 of Figure 2.1; and the second event with the attributes, Name and Limit, that are retrieved using the CER statements at lines 9 and 11. In addition to the CET set, I extended Eos to also output the code locations at which dierent attributes of each CET are retrieved, rather than just the name and value of each attribute. I thus extract those locations and record them along with the name of corresponding event in the CET set. Therefore, the CET set for LoadBalancer will befh\NewRequest"; 6; 7i;h\SetLimit"; 9; 11ig. 29 Table 3.1: LoadBalancer component information Methods FDef FUse handle fhpreLmt; 10i;hlmt; 11ig fhpreLmt; 12i;hlmt; 12i ;hlmt; 10ig manageCurrentLoad fhcurLoad; 18ig fhcurLoad; 17i;hcurLoad; 18i ;hlmt; 17ig Algorithm 1: Callback sink extraction Input: Components, Interfaces, CET and S (sink) sets for each component Output: Updated CET and S (sink) sets for each component 1 foreach i2Interfaces do 2 foreach c2Components do 3 if c implements i then 4 add (c;i) to ImplementedInterf 5 if9f2F c such that f and i have the same type t then 6 add (c;t;i) to Candidates 7 foreach (c,i)2ImplementedInterf do 8 if9f2F c of type t f and9(t f ;t;i)2Candidates then 9 foreach m2Methods i do 10 add (m:name,m:entry node) to CET c 11 add m to S c 12 if9n2Methods c ^ n is called by a thread defined inside m then 13 add (n:name,n:entry node) to CET c 14 add n to S c Eos targets EBSs with ambiguous interfaces and is not able to identify callback meth- ods, which serve as sinks for explicitly typed events in frameworks such as Android. Example callback sinks in Android are onLocationChangedAsync and onDestroy from Figure 1.1. Standard callbacks, such as onDestroy, can be easily identied in the code. However, custom, application-specic event sinks, such as onLocationChangedAsync, must also be identied. Algorithm 1 identies two common patterns of custom event sinks, 30 and uses these to identify sinks and CETs for Android applications. The rst pattern comprises an interfacei, a componentc 1 that implements interfacei, and a componentc 2 that denes a eld variablef 1 with the same type as interfacei; in turn,c 1 should dene a eld variable f 2 with the same type as c 2 . Two components have the same type when they are either instances of the same class or one extends or implements the other. In this situation, whenever component c 1 is instantiated, it will provide its \this" reference viaf 2 to componentc 2 , andc 2 will save the reference inf 1 . By doing this,c 2 will be able to call methods that are declared in interface i on component c 1 . The second pattern comprises a denition of a thread inside a callback method and a call to another method inside that thread to perform the required operation asynchronously. Algorithm 1 nds the rst pattern by rst searching for all components c that imple- ment a given interfacei (lines 1 - 4) and stores thehc;ii tuples in theImplementedInterf set. After that, Algorithm 1 checks if a given componentc has a eld variable of the same type t as the interface i (lines 5 - 6) and stores thehc;t;ii tuple in the Candidates set. Finally, Algorithm 1 searches through ImplementedInterf to check for all components c that implement an interface i and have a eld variable of type t f , such that t f 's class, in turn, has a eld variable of type i (lines 7 - 8). In other words, Algorithm 1 checks for tuples of the formht f ;t;ii in the Candidates set. If such a tuple is found, then the methods in interface i will be added to c's sinks and the CET set (lines 9 - 11). Algo- rithm 1 nds the second pattern by identifying methods called asynchronously inside a thread, and adding them to theS (i.e., sink) andCET sets of componentc (lines 12 - 14). onLocationChangedAsync from the MyTracks application of Figure 1.1 is an example of an asynchronous callback in Android. 31 Causation The second phase of DEvA's analysis identies whether eld accesses are dependent on the consumption of specic event types. To this end, DEvA analyzes each eld access location to determine if it is control or data dependent on CER statements that are used to dene a CET. The intuition is that if such dependencies exist, then the eld access occurs, at least in part, due to the consumption of an event type, and may be part of an EA. Intuitively, one can think of this phase as analyzing the program dependence graph (PDG) of a component. For each eld access, DEvA performs a backwards traversal of the edges in the PDG. If a CER statement is encountered during this traversal, then DEvA identies the corresponding CET and the originating eld as being connected. The phase outputs these connected CETs and elds. For larger systems, generating and traversing a PDG is not scalable. Case in point, to naively analyze the subject systems used in the evaluation (see Chapter 4), it would be necessary to generate PDGs for over 35 methods on average. This could consume hours for a typical application. To address this issue, DEvA only generates PDGs for a component's sink methods, and then uses the call graph (CG) of the system to identify methods that are reachable from each sink. A eld's denition or use in method m that is reachable from sink s may be caused by an event's consumption if the invocation in s that initiates the call to m is control or data dependent on that consumption. I now detail the algorithms that implement this approach. 32 The algorithm for this phase is shown as Algorithm 2. The inputs to this algorithm are the componentc to be analyzed, the CETs of the component (CET c ), the call graph of the component (CG c ), and the FUse and FDef sets for each method inc. Note thatCET c , FUse, and FDef are the outputs of the rst phase discussed in Section 3.1. The outputs of the algorithm are two sets, ConsumedToDef and ConsumedToUse, which contain tuples representing the elds, and CETs that are linked by a dependency relationship. Each tuple is of the formhf;n;ei where f is the eld, n is the location of the eld's access in the code, ande is the CET. The set ConsumedToDef contains tuples wheren represents a denition of f, while in ConsumedToUse n represents a use of f. Algorithm 2: Causation Input: c2Components, CG c , CET c ,8m2 Methods c : FDef m ; FUse m Output: ConsumedToDef c , ConsumedToUse c 1 Let S c = set of all c's event sinks 2 foreach m2 Methods c do 3 if (m2S c ) or (9s2S c so that m is reachable from s in CG c ) then 4 ConsumedToDef c + = eldAccessBackToConsumption(c;m; FDef m ; CET c ) 5 ConsumedToUse c + = eldAccessBackToConsumption(c;m; FUse m ; CET c ) Algorithm 2 rst accesses a set S c that contains all sink methods in c. This set is dened using method signatures for applications with ambiguous interfaces, but is aug- mented with the results of Algorithm 1 for Android applications. Then the algorithm iterates over each methodm inc (lines 2{5). Ifm is a sink or it can be reached from a sink in the call graph (line 3), then DEvA analyzes the eld accesses in m by calling eldAc- cessBackToConsumption (lines 4 and 5). The function eldAccessBackToConsumption is shown in Algorithm 3. At a high-level, eldAccessBackToConsumption iterates over each eld access in m (line 6) and each event in CET c (line 7) to determine if a dependency 33 relationship exists between them. Within this iteration, there are two cases to consider. The rst case (at line 8) is when m is a sink. In this case, DEvA simply checks the PDG of m to see if the eld access is dependent on any CER node for the current CET (line 9). If so, the tuple representing the eld, location, and event type is added to the output set (line 10). The second case (at line 11) is for any non-sink method. DEvA begins by iterating over the set of all sink methods S m that can reach m (line 12). Within each t2S m , DEvA also iterates over each node k that is in t and can reach m (line 13). This reachability relationship will be computed by determining if k can reach an invocation in t's CFG that, in turn, reaches m via the call graph. This relationship is encapsulated in the function SToM (line 5). If k is dependent on a CER node, then this relationship is added to the output set (lines 14 and 15). The intuition here is that the eld access hf;ni can be reached via a statement k that is itself dependent on a CER node. Let us now consider an example for each of the two cases. To illustrate the rst case, consider the handle method of the LoadBalancer component from Figure 2.1. Since han- dle is a sink, eldAccessBackToConsumption will use the PDG (shown in Figure 2.3) to extract those members of FDef handle , listed in Table 3.1, that are control or data depen- dent on a node that contains a CER statement. Considerhlmt; 11i in the PDG. Node 11 is connected to node 9, which contains a node with a CER statement for the SetLimit event; sohlmt; 11; \SetLimit"i will be added to the return set. Next,hpreLmt; 10i is also dependent on node 9, sohpreLmt; 10; \SetLimit"i will be added to the return set. To illustrate the second case, consider the method manageCurrentLoad. Based on the ICFG in Figure 2.2, only node 7 in the sink handle can reach manageCurrentLoad. StoM (handle,manageCurrentLoad) thus returns a singleton containing node 7. Based on 34 Algorithm 3: eldAccessBackToConsumption Input: c2Components, m2 Methods c , FieldAccesses, CET c Output: ConsumedtoAccess c 1 Let N m = set of all nodes in m's CFG 2 Let cerNodes(e;m) where e2 CET c and m2 Methods c = fnjn2N m ^n2 cerExtract(e)g 3 Let S c = set of all of c's event sinks 4 Let S m S c = set of sinks that can reach m in CG c 5 Let StoM(s;m) = set of nodes in the CFG of s2S c that can reach m through direct or indirect calls 6 foreach (f;n)2 FieldAccesses do 7 foreach e2 CET c do 8 if m2S c then 9 if9l2 cerNodes(e;m) such that n is directly or transitively control or data dependent on l then 10 add (f;n;e) to ConsumedtoAccess c 11 else 12 foreach t2S m do 13 foreach k2 StoM(t;m) do 14 if9l2 cerNodes(e;t) so that k is directly or transitively control or data dependent on l then 15 add (f;k;e) to ConsumedtoAccess c 16 return ConsumedtoAccess c the PDG of handle (Figure 2.3), node 7 is control dependent on node 6, which contains a CER statement of the NewRequest event. Therefore, any denition or use inside the manageCurrentLoad method is dependent on this event. Since FDef manageCurrentLoad containshcurLoad; 18i, the algorithm will addhcurLoad; 18; \NewRequest"i to the return set for denition accesses of manageCurrentLoad. After Algorithm 2 completes its analysis of the LoadBalancer component from Figure 2.1, its output would be: 35 ConsumedToDef LoadBalancer =f hlmt; 11; \SetLimit"i; hcurLoad; 18; \NewRequest"i; hpreLmt; 10; \SetLimit"ig ConsumedToUse LoadBalancer =f hcurLoad; 17; \NewRequest"i; hcurLoad; 18; \NewRequest"i; hlmt; 17; \NewRequest"i; hlmt; 10; \SetLimit"i; hlmt; 12; \SetLimit"i; hpreLmt; 12; \SetLimit"ig Joining DEvA's third phase analyzes the dependencies between CETs and elds to determine which ones may lead to an EA. The intuition is that if one CET{eld dependency writes to a eld and another one either writes to or reads from that eld, then this is an EA. The algorithm for this phase is shown in Algorithm 4. The inputs to the algorithm are ConsumedToDef c andConsumedToUse c , which were generated in the second phase, and whose values for LoadBalancer are shown at the end of Section 3.1. The goal of the rst step (lines 1{4) of the algorithm is to remove uses that may result in false positives. This step is analogous to identifying reaching denitions [23]: a denition of a given eld f reaches a node n in a CFG if there is a path in the CFG from the node at which f is dened to n without any other denition of f on that path. If there is a denition of a eld that dominates a use of the same eld, then that use cannot be involved in an EA condition with any other denition of the eld. Domination occurs when all paths from 36 the entry node of a CFG to the location of eld f's use include the node that denes f. In that case, f's use is removed from the ConsumedToUse set at line 4 of Algorithm 4. To illustrate this rst step, consider the denition of lmt at line 11 of Figure 2.1. This denition dominates the use of lmt at line 12. Therefore, the use of lmt at line 12 will be removed from ConsumedToUse LoadBalancer , after which we will have: Algorithm 4: Joining Input: c2 Components, ConsumedToDef c , ConsumedToUse c , ICFG Output: EventAnomalies c 1 foreach hf;n;ei2 ConsumedToDef c do 2 foreachhf 2 ;n 2 ;e 2 i2 ConsumedToUse c ^ f =f 2 ^ e =e 2 do 3 if f's denition at node n kills all other denitions that reach node n 2 then 4 removehf 2 ;n 2 ;e 2 i from ConsumedToUse c 5 foreach hf;n;ei2 ConsumedToDef c do 6 foreachhf 3 ;n 3 ;e 3 i2 ConsumedToDef c ^ f =f 3 ^ e6=e 3 do 7 addhf;e;e 3 i to EventAnomalies c 8 foreachhf 4 ;n 4 ;e 4 i2 ConsumedToUse c ^ f =f 4 ^ e6=e 4 do 9 addhf;e;e 4 i to EventAnomalies c ConsumedToUse LoadBalancer =f hcurLoad; 17; \NewRequest"i; hcurLoad; 18; \NewRequest"i; hlmt; 17; \NewRequest"i; hlmt; 10; \SetLimit"i; hpreLmt; 12; \SetLimit"ig The second step of the algorithm (lines 5{9) iterates over each CET{eld dependency where the eld access is a denition, and checks whether there are any denitions or uses of the same eld that can be triggered by the consumption of dierent CETs. Essentially, this performs a join over two inputs when the eld in each tuple is the same. If such CET{ eld dependencies exist, then an EA is detected, and a tuple containing the aected eld 37 and the two event types is added to the output set. To illustrate the joining algorithm, consider the ConsumedToDef LoadBalancer set, reported at the end of Section 3.1, and ConsumedToUse LoadBalancer after line 4 of Algorithm 4, reported earlier in this section. There is one CET{eld dependency that denes lmt. This denition happens at line 11 and is caused by the SetLimit event. There is also one CET{eld dependency that uses lmt. This use happens at line 17 and is caused by the NewRequest event. These paths access the same eld with one of them being a write access and are dependent on dierent event types, introducing an EA. The output of Algorithm 4 for LoadBalancer is, therefore,fhlmt; \SetLimit"; \NewRequest"ig. Ranking To determine which of the potential anomalies are harmful, an engineer would need to examine each reported potential harmful anomaly. To aid engineers in this task, in this phase each reported EA will be analyzed and ranked for harmfulness. The intuition behind this phase is that a harmful anomaly will cause an eect on the state of the component or the state of the system. The state of the component will change if its eld variables change, and the state of the system may change if a component sends an event to another component. To elaborate more, each anomaly can have one of the two below forms, based on the denition of EAs: 1. Anomaly consists of a read to a eld variable that is caused by an event consumption and a write to the same eld variable that is caused by another event consumption 38 2. Anomaly consists of two dierent writes to the same variable, where each of those write accesses is caused by a dierent event consumption In the rst case, if the read to the eld variable does not impact the state of the component or system, the anomaly's impact will be limited. The read to the variable can impact the state of the component or system if that read is used in the denition of another eld variable, in the creation of an event, or in branches whose outcome can either change a variable or send an event. In other words, an anomaly is harmless if all the execution paths that are control or data dependent on the read, do not contain a denition of a eld variable or publishing of an event. In the second case, if the eld that is accessed through a write is not used afterward, either in the denition of another eld variable or in the decision to send an event, the anomaly's impact on the system will be limited as well, and the order of the occurrence of the two writes to the eld variable will not matter. Using data and control dependency relationships, it is possible to examine the dierent cases that I discussed above. The idea is that a static analysis approach can be proposed that will rst identify locations in the code where an event is published or a shared memory location accessed, and then it will check for control/data dependency relations between these locations and the locations that participate in an EA. If there are such dependencies, the EA is deemed harmful; otherwise, it is harmless. In certain cases, the harmfulness of an EA will depend on the application's semantics \behind the code" and a developer's assistance will be needed to make the ultimate determination. 39 Algorithm 5: Ranking Input: c2 Components, Methods c , EventAnomalies c , ConsumedToUse c , ICFG, IgnoreList Output: HarmfulEventAnomalies c 1 foreach hf;e 1 ;e 2 i2 EventAnomalies c do 2 if 9hf 2 ;n;ei2 ConsumedToUse c ^ f =f 2 ^ (e =e 1 _ e =e 2 ) then 3 foreach node m that is transitively control or data dependent on node n do 4 if m contains a eld denition or a publication of an event and does not use IgnoreList then 5 addhf;e 1 ;e 2 i to HarmfulEventAnomalies c 6 else 7 foreach m2 Methods c do 8 if9hf;ki2 FUse m then 9 if k contains a eld denition or a publication of an eventn and does not use IgnoreList then 10 addhf;e 1 ;e 2 i to HarmfulEventAnomalies c 11 else 12 foreach node j that is transitively control or data dependent on node k do 13 if j contains a eld denition or a publication of an event and does not use IgnoreList then 14 addhf;e 1 ;e 2 i to HarmfulEventAnomalies c The algorithm that accomplishes ranking is shown in Algorithm 5. The inputs to Algorithm 5 are the list of reported event anomalies, ConsumedToUse c { which is com- puted in the causation phase { and an ignore list that indicates parts of the code that may be ignored by the analysis, because developers of the system do not care about the eects of anomalies on those parts. For example, developers may decide that the logging portion of a system may be ignored because it is based on the use of o-the-shelf products. This ignore list customizes DEvA to focus on the most sensitive parts of the system. For each anomaly, Algorithm 5 at line 2 rst checks whether the EA consists of a read access and 40 a write access to a eld variable (belongs to the rst case that was described earlier). If the anomaly is a read-write EA, then Algorithm 5 at lines 3 - 5 will consider all the nodes of ICFG that are transitively control or data dependent on the node that contains a use of the eld that is participating in the anomaly. If among those nodes there exists a node that contains either a denition of a eld or a publication of an event, the corresponding event anomaly can potentially be harmful, and it will be ranked as a potential harmful anomaly at line 5 of Algorithm 5. If the EA happens because of two write accesses to a eld variable (belongs to the second case that was described earlier), at lines 7 - 14 Algorithm 5 considers all the nodes of the ICFG that contain a use of the eld that is participating in the EA. For each of these nodes, like nodek, Algorithm 5 will rst check whetherk contains either a denition of a eld or a publication of an event. In this case, the algorithm will add the EA to the set of potential harmful anomalies (Lines 9 - 10). Ifk doesn't contain a eld denition or an event publication, at lines 11 - 14 Algorithm 5 will check whether other nodes that are control or data dependent on k contain either a eld denition or an event publication. If such nodes exist, the EA will be ranked as a potential harmful anomaly. Considering the LoadBalancer example in Figure 2.1, DEvA reported an event anomaly of the formfhlmt; \SetLimit"; \NewRequest"ig. Since this anomaly is a read-write anomaly, lines 3 - 5 of Algorithm 5 will be executed when we are examining the harmful- ness of this anomaly. In Figure 2.1, at line 22 an event is getting sent. The corresponding node of this line based on the ICFG of LoadBalancer is node 22. According to the PDG of LoandBalancer that is shown in Figure 2.3, node 22 is control dependent on node 17. Node 17 contains the use of variablelmt that is participating in the event anomaly. This 41 will cause the condition at line 4 of Algorithm 5 to become true and the reported anomaly by DEvA to be ranked a potential harmful anomaly. Algorithm 5 provides a ranking of anomalies. In this ranking process, only informa- tion that can be extracted from the code at the level of components and their interactions is considered. It is possible that a given anomaly aects the properties of the system that cannot be perceived by just looking at the code. For example, an anomaly may impact a call to a method in a way that will not change the state of the component but will cause a degradation in performance because of the method's complex functionality. By just looking at the code, without gathering information about the runtime performance or semantics of the method, we are not able to decide whether the method's functionality is complex. This may cause false positives in DEvA's ranking approach. As a poten- tial improvement, a ranking approach can be designed by using historical data of the reported issues in previous versions of a system [70]. The problem with this approach is that historical data usually just includes previously known anomalies and, by denition, ignores still-unknown anomalies. This will cause false positives in the ranking process. Also, existing techniques that are based on the use of historical data require additional eort form developers to model the system using a new notation. My goal was to provide a ranking method without putting too much eort on developers. 3.2 Detecting Synchronization Anomalies As discussed in Section 1.2 of this dissertation, it is challenging to extract SAs using static analysis because of the need to determine accesses to memory locations in the presence 42 of aliasing and in the presence of dierent calls to the same methods. To overcome this main challenge and also the shortcomings of existing approaches, I have developed an automated static analysis technique that examines the implementation of a system and identies points in the code where SAs may occur. This technique is called DSA. The main input to DSA is the implementation of a system. DSA's analysis can be divided into three distinct phases: 1. extraction| In this phase, the analysis identies all of the elds accessed by each method of each component of the system. It also identies the sequences of method calls that starts from each method of each component of the system. 2. checking| This phase will check each non-primitive argument of each method to see whether it is changed inside the method. 3. categorization| In this phase, the analysis will perform a categorization for each of the independent sequences of method calls that start from that method. The inde- pendence of sequences of method calls will be determined based on the information that was obtained in the previous phases of the analysis. The output of this phase will be the list of independent sequences of method calls that are synchronized, or in other words, the list of SAs. In the remainder of this section, I discuss each of the three phases in more detail. Extraction During this phase, DSA identies two types of information about the system under analy- sis that will be used in the later phases to identify SAs. The rst type of information that 43 DSA extracts is the set of locations within each component where the component's elds are accessed either by a use or a denition. A method may call several other methods inside its body, and each of those called methods may also call other methods, so there could be several sequences of method calls that start from a single method call. If those sequences of method calls do not share a memory location, they can be executed con- currently. Because of the importance of this information, the second type of information that DSA extracts in this phase is the set of sequences of method calls that starts from each method of each component of the system. DSA identies a component's eld accesses via a static intra-procedural analysis of the component's implementation. This is very similar to the extraction phase of DEvA that is discussed in Section 3.1. Similar to what is discussed there, DSA also computes two sets for each method: FDef and FUse. To compute these sets, DSA traverses over CFG of each method and performs an alias analysis like the one that is described in Section 3.1. DSA identies the set of sequences of method calls that start from each method m of a component c by considering the call graph of c and checking all the edges of the call graph that start from m. Formally, I will dene a set CalledBy for each method of a component as follows: CalledBy m =fnjn2Methods c ^ 9(m;n)2CG c g The intuition is that the sequences of method calls that start from each method are like a list of methods, where the head of that list exists in CalledBy set. This means we 44 can compute the sequences of method calls that start fromm by computing the sequences of method calls that start from each method like n in CalledBy m . The algorithm for this phase is shown in Algorithm 6. The inputs to this algorithm is a component c, a method of that component m, and CG c , which is the call graph of c. The outputs of the algorithm are three sets. For each method m, FDef m , FUse m , and all the sequences of method calls that start from m will be extracted. The sequences of method calls will be stored in a hash-table with the nameSeq. This hash-table accepts a method as its key and the set of sequences of method calls that start from that method as its value. Algorithm 6: Extracting sequences of method calls Input: c2 Components, m2 Methods c , CG c Output: FDef m , FUse m , Sequences of method calls that are started from m 1 callSeq(m,c) 2 if Seq(m) is not empty then 3 return Seq(m) 4 compute FDef m and FUse m 5 foreach n2CalledBy m do 6 if Seq(n) is empty then 7 Seq(n) = callSeq(n,c) 8 foreach s2Seq(n) do 9 add n to s 10 add s to Seq(m) 11 return Seq(m) Algorithm 6 will rst check at line 2 whether the hash-table already contains the information about method m. In that case, the algorithm will just return the already computed results. If this is not the case, then the algorithm rst computes the FDef m and FUse m sets, and then, at lines 5 - 10 for each method n that is called by m, it will check whetherSeq already has the sequences that starts from n. If not, then it will recursively 45 call thecallSeq method overn to compute the sequences of method calls that start from n. After that, for each sequence s that starts from n, it will add n to s to compute all the sequences that start from m and will store this new sequence in Seq(m). Considering the LoadBalancer example in Figure 2.1, two of the outputs of the Al- gorithm 6 are the sets FDef and FUse that are shown in Section 3.1. The third out- put is the Seq hash-table that looks as follows: fh\handle" :h\mangeCurrentLoad" "send"i;h\send"iih\mangeCurrentLoad" :h\send"iih\send" :;ig. The reason is that handle is calling two methods: manageCurrentLoad and send. manageCurrentLoad also calls send; however, send is not calling any other methods. Checking The second phase of DSA identies whether local variables are shared between dierent method calls that occur inside a method. It is possible that two dierent method calls use a similar subset of arguments. If an argument of a call to a method has a non- primitive type, and that method has a write access to that argument inside its body, the other methods that are using the same argument will become dependent on that method and cannot be executed concurrently since their execution may have dierent outcomes depending on the changes that the method is making to the argument. Intuitively, one can think of this phase as an escape analysis [29, 93]. The goal of an escape analysis is to determine whether an object can be accessed outside of the scope that its memory is allocated. In other words, an object is considered to have escaped from a method if its lifetime exceeds the runtime of the method [29]. Using an escape analysis in this phase of DSA, we can extract all of the objects in each method that are 46 accessible outside of the method. Then we should check if those objects are accessed by other methods. This should be checked at each location where a call to a method occurs in order to consider all the possible aliasings between local variables and arguments of a method. Because of this, escape analysis can be a heavy analysis and may not scale if the system is large in terms of SLOC. To avoid these problems, DSA proposed a summery- based modular approach. DSA is a summery-based approach since it stores computations for each method to avoid redundant computations.DSA is also modular, since it computes the escaped local variables for each method independent of their call-site and will do an on-demand mapping between the call-site's and the method's local objects. The algorithm for this phase is shown in Algorithm 7. In Algorithm 7, for each method m of component c, all methods that are called by m will be considered (Line 1). For each method n that is called by m, the algorithm will consider all the locations inside m where n is called (Line 2). I call these locations call-sites of n. For each of these call-sites, all the local variables that are passed to n are considered (Line 6). As you will see later, DSA checks whether a method's argument, which is identied by its position in the list of the method's arguments, is changed by the method. To map that argument with the local variables, Algorithm 7 denes a variable r order in Line 3 that is used to store the position of a local variable in the list of arguments of a method n in a specic call-site. Using this approach, there is no need to recompute whether a method is changing its arguments for each call-site. At Line 7, Algorithm 7 checks whether local variabler that is used as an argument in a call ton is live. A local variable is live if and only if it is used somewhere else after the current location. DSA uses data ow analysis to reason about liveness of local variables 47 Algorithm 7: Methods' arguments checking Input: c2 Components, m2 Methods c , CG c Output: ArgShared 1 foreach n2CalledBy m do 2 foreach call-site of n inside m do 3 r order = 0 4 s = Get the call statement of n in call-site 5 ret = Get the local variable containing the return value from call-site 6 foreach r2Args s do 7 if r is live^ ArgShared(r) is Empty then 8 if ArgChanged(n +r order )6=null^ArgChanged(n +r order ) =truethen 9 Add n to ArgShared(r) 10 if ArgSharedRead(r) is not empty then 11 Add all methods in ArgSharedRead(r) to ArgShared(r) 12 else if checkArgIsChanged(r order ,n,c) then 13 Add n to ArgShared(r) 14 if ArgSharedRead(r) is not empty then 15 Add all methods in ArgSharedRead(r) to ArgShared(r) 16 else if ArgShared(r) is not Empty then 17 Add n to ArgShared(r) 18 else if r is live then 19 Add n to ArgSharedRead(r) 20 r order =r order + 1 21 if ret6=null^ ret is live then 22 Add n to ArgShared(ret) as discussed in [23]. If a local variable like r is determined to be live after a call-site, it can be used as an argument to another method or it can be used somewhere else inside m's body. In that case, we should check if r is changing inside n. A method like n can be called by several methods other than m. Because of this, by storing the result of checking if an argument of n is changing inside n or not, we will avoid duplicate computation. The ArgChanged hash-table in Algorithm 7 is responsible for this. ArgChanged accepts a method, and the position of the local variable in the 48 argument list as its key and will return a boolean indicating whether the argument in that position in the argument list is changed, if this has been checked before and the result has been stored in ArgChanged. At Line 8, the algorithm rst tries to see whether the key consisting of the method and the argument position exists in the ArgChanged. If this is not the case, the check will be performed for the rst time by calling checkArgIsChanged method at line 12, and its results will be stored inArgChanged for future use. checkArgIsChanged checks whether an argument is changing inside a method through traversing the CFG of the method to see if there is a node that contains a denition of the argument. If there is no such node in the call graph of the method, checkArgIsChanged will return false, indicating that the argument and its corresponding local variable is not changed. If an argument is changing inside n, and it is also used in other places inside m, it means that the execution of n is not independent of other parts of m. This informa- tion will be saved in ArgShared, indicating that the local variable r is shared between n and other parts of m. ArgShared is also a hash-table that stores computations to avoid duplicate and unnecessary computations. There is also another hash-table with the name ArgSharedRead that stores whether a method just has a read access to r. Whenever it turns out that a method is changing r, Algorithm 7 will check whether the ArgSharedRead(r) is empty, and if not, it will add all the methods inArgSharedRead(r) to ArgShared(r). This is because those read accesses cannot concurrently occur with a write access. The variableret that is dened at line 5 of Algorithm 7 is referring to the local variable that stores the return value from a call to method n. At line 21, the algorithm checks 49 Table 3.2: LoadBalancer component's ArgChanged and ArgShared ArgChanged ArgShared fhhandle + 1;falsei;hmangeCurrentLoad + 1;falseig ; whether such local variable exists, and if it exists, whether it is live. The liveness of this local variable indicates that the return value from n has been used somewhere else in the body of m, or in other words, it has been shared with other parts of m. As a result of this, n will be added to ArgShared(ret) at line 22. In the LoadBalancer example, the methods are not changing their arguments and also don't have return values. This means that they are not sharing arguments or return values either. You can see what ArgChanged andArgShared look like for that example in Table 3.2. Categorization In the extraction phase and checking phase, DSA gures out the eld variables that each method is accessing, the sequences of method calls, and the local variables that are shared between dierent method calls. Using this information, it is possible to group sequences of method calls in independent categories. A category may contain a single sequence or multiple sequences of method calls. The sequences that are in the same category are dependent on one another and should be synchronized. However, dierent categories don't need to be synchronized with each other and can be executed concurrently. In the categorization phase, DSA will categorize into independent categories the sequences of method calls that exist in each method. 50 The algorithm for this phase is shown as Algorithm 8. This algorithm computes all the independent sequences of method calls that start from a method m in component c. It receives as its inputs all the sequences of method calls that start from m inside Seq m , which is computed in the extraction phase, along with ArgShared m , which is computed in checking phase. Other inputs are the FDef and FUse sets for all of the c's methods, and also the call graph of c. Algorithm 8 will compute the eld variables that are accessed by dierent methods and are shared between dierent sequences that exist inSeq m . The hash-tableConflicts is responsible for storing this information. Conflicts receives as its key a eld variable or local variable and will return the set of methods that have accessed that eld or local variable with at least one of those methods has a write access to that variable. As you can see in Algorithm 8, to compute the Conflicts for eld variables, for each sequence the algorithm will consider all the methods that participate in that sequence. For each method, at lines 3 - 5 it will consider the set of denitions of that method and for each eld in the set, the algorithm will add the head method of the sequence to the Conflicts table using the eld as the key. The head method of a sequence will be stored in Conflicts, since only the head method is called inside m. At lines 6 - 9, the algorithm will consider all the methods of each sequence to check if there is a use of a eld variable that belongs to a sequence's method and that eld has a denition in one of the other sequences. If there is no denition of that eld, the use will be ignored, since a use is a read access, and it is possible to have multiple concurrent reads of the same variable. 51 Algorithm 8: Categorization Input: c2 Components, m2 Methods c , Seq m , ArgShared m , 8t2Methods c : FDef t , FUse t , CG c Output: Categories of independent sequences of method calls 1 Let head s , where s is a sequence of method calls, be the head method of s 2 foreach s2Seq m do 3 foreach n2s do 4 foreach f2 FDef n do 5 Add head s to Con icts(f) 6 foreach s2Seq m do 7 foreach n2s do 8 foreach f2 FUse n do 9 if Con icts(f) is not empty then 10 Add head s to Con icts(f) 11 foreach l in locals m do 12 if ArgShared m (l) is not empty then 13 foreach n2ArgShared m (l) do 14 Add n to Con icts(l) 15 id =1 16 newId = 0 17 foreach conf2Conflicts do 18 id =1 19 foreach n2conf do 20 if Group(n)6=null then 21 id =Group(n) 22 foreach n2conf do 23 if id6=1 then 24 Group(n) =id 25 else 26 Group(n) =newId 27 newId + + 28 CategoryInd =newId 29 foreach n2CalledBy m do 30 if Group(n)6=null then 31 Add n to Categories(Group(n)) 32 else 33 Add n to Categories(CategoryInd) 34 CategoryInd + + 52 As shown in Algorithm 8, at lines 11 - 14 theConflict is computed for local variables. The algorithm will check dierent local variables of m to see if they have been shared between dierent method calls. In that case, the methods that share the same local variable will be added to the con ict set of that variable. After computation of Conflicts, we can group the dierent methods that are called by m based on the eld or local variable that they are sharing. Two methods will be grouped in dierent groups if and only if they are not sharing any eld or local variable. The computation of groups is done in Algorithm 8 at lines 15 - 27. The variable id that is dened at line 15 is used to store the ID of an existing group. The group IDs are recorded in a hash-table with the name Group that receives a method as its key and returns the ID of the group to which the method belongs. Variable newId stores the minimum unassigned ID that can be assigned to any newly discovered groups of methods. Finally, at lines 28 - 34, the algorithm will compute the actual categories. Basically it will consider the group ID of each method that was assigned to each method in the previous step. If a method doesn't have a group ID, a new category will be created, and the method will be added to that category. If the method has a group ID, it will be added to the category with the same ID as the group of the method. In the end all the methods with the same group ID will be added to the same category. Methods that are in dierent categories can be executed concurrently. Considering the LoadBalancer example and the outputs of previous phases, by ap- plying Algorithm 8 over method handle, the Conflict hash-table looks like the following after line 14 of the algorithm: 53 Conflict =fh\currentLoad" : \manageCurrentLoad"i; h\lmt" : \manageCurrentLoad"ig The Group hash-table will have just one member before line 28 and that mem- ber ish\manageCurrentLoad 00 : 0i. As a result of this, after the algorithm reaches its last line, there will be two categories in Categories: one category just consists of manageCurrentLoad method, and the other category just includes send method. This means that DSA suggesting that manageCurrentLaod and send can be executed con- currently, and there is no need for them to be synchronized. This means that managing a new request is independent of sending an event. There is no need for the handle method to wait for manageCurrentLoad to nish its execution before moving on to execute its other parts or handling another event. 54 Chapter 4 Evaluation In this chapter, I will provide the evaluation of my proposed approaches. In Section 4.1, I discuss the evaluation of DEvA and in Section 4.2 the evaluation of DSA. In Section 4.3, I discuss the threats to validity and the limitations of my evaluation. 4.1 Evaluation of DEvA I have empirically evaluated DEvA (1) to measure its accuracy in extracting EAs; (2) to compare it with a state-of-the-art race detection technique for multi-threaded systems; (3) to compare its performance with a recently published race detection technique for Android applications; and (4) to examine its execution time. DEvA's implementation and all evaluation results are available online [18,21]. Subject Systems and Implementation Table 4.1 contains information about the 20 subject systems I have used in my evaluation; 18 of them are applications and the remaining two are widely used Android libraries (e.g., these libraries are used in several of the Android applications from Table 4.1). 55 All subjects are implemented in Java, but are from dierent application domains (App Type), of dierent sizes (SLOC ), and use dierent underlying mechanisms for consuming events (Event Mechanism). As you can see not only my subject systems are from various domains, but also they are very diverse in terms of size. In selecting these subjects, rst a corpus of suitable systems was located that make use of events in their implementations. Two PhD students examined a number of open- source applications and identied likely candidates by looking for possible instances of EAs. Each system for which the preliminary examination indicated a potential presence of EAs was then carefully analyzed by the two students with the help of the Eclipse IDE to obtain the ground truth. The generation of the ground truths took slightly more than 15 person-hours per system on average. As discussed in Section 3, even seemingly harmless races may actually harm a system in subtle ways [30], hence the ground truths contained all possible races regardless of their impact on the EBSs. A notable outlier among my subject systems is Project.net, which is signicantly larger than the other systems. It partly uses event-based interactions on its server side, but mostly relies on web-based interactions. I only provided Project.net's event-based portion, totaling around 10 KSLOC, as an input to DEvA. However, to pinpoint this portion, I had to analyze the entire system. DEvA is implemented in Java and Scala, and it uses the Soot [103] program analysis library to generate call graphs, control ow graphs, and program dependency graphs. To analyze EBSs that rely on ambiguous interfaces|this includes all non-Android applica- tions from Table 4.1|I used an extension of Eos [47] for generating the required inputs 56 Table 4.1: Systems used in the evaluation of DEvA App Name App Type SLOC Event Mechanism Planner AI Planner [82] 6K c2.fw [71], Java events KLAX Arcade Game [96] 5K c2.fw [71], Java events DRADEL Software IDE [72] 11K c2.fw [71] ERS Crisis Response [68] 7K Prism-MW [69] Troops Simulator [69] 9K c2.fw [71] Stoxx Stock Ticker [74] 6K REBECA [75] JMSCHAT Chat System [10] 12K ActiveMQ, Java events Project.net Project Mgmt [12] 247K Spring [13] ToDoWidget ToDo List Recorder [5] 2K Android Events FBReader Book Feed Reader [3] 34K Android Events MyTracks Location Tracker [4] 13K Android Events ZXing Barcode Scanner [2] 16K Android Events Firefox Browser App [7] 58K Android Events ConnectBot SHH Client [1] 23K Android Events VLC Media Player [6] 101K Android Events Browser Android Browser App 22K Android Events Camera Android Camera App 13K Android Events Music Android Audio Player 8K Android Events android.support.v4 Support Library 7K Android Events android.support.v7 Support Library 11K Android Events for DEvA, as described in Section 2.2. I ran DEvA on a quad-core Intel i7 2.80GHz sys- tem with 8GBs of memory, running Windows 7 Professional. I will discuss more details about the implementation of DEvA including its architecture in the next subsection. More Details about the Implementation of DEvA I have made available a package allowing independent replication of the analysis re- sults of DEvA [18]. This package contains DEvA's implementation, the ground truths used for DEvA's evaluation, and all evaluation results. A manual describing how to use DEvA is also included. The DEvA replication package has been successfully evaluated by ESEC/FSE 2015's Replication Packages Evaluation Committee and found to meet the Committee's expectations. 57 DEvA uses conguration les to set the environment to perform its analysis. In Figure 4.1, part of the conguration le for analyzing the MyTracks Android application is shown. In this conguration le, at line 1, the maximum heap size for the JVM is set. Line 2 indicates the location of the jar le that contains the source code of the system that DEvA wants to analyze. mainComponent at line 4 indicates the component that is the entry point of DEvA's analysis. leNameForRaceResults at line 7 indicates the location of the output le for the list of event anomalies, and leNameForUFRaceResults at line 9 represents the location of the le for the list of UF anomalies in the case of Android systems. middlewareID at line 12 shows the mechanism the system under analysis uses for event-based communications. 1 jvm.xmx = 2g 2 process.dir = 3 ${user.dir}/applications/MyTracks.jar 4 mainComponent = 5 com.google.android.apps.mytracks.MyTracks 6 7 fileNameForRaceResults = 8 ${user.dir}/results/MyTracks-detected-EAs.txt 9 fileNameForUFRaceResults = 10 ${user.dir}/results/MyTracks-detected-UF.txt 11 12 middlewareID = Android Figure 4.1: Portion of the MyTracks Conguration used by DEvA Figure 4.2 shows a view of DEvA's architecture. After providing the conguration le to DEvA, the Initializer component will load the related part of the DEvA to perform the analysis. As it was discussed in the approach chapter, DEvA acts dierently when dealing with Android and non-Android systems. If the system that is under analysis is an Android-based system, the Initializer will ask the Callback Detector component to extract user-dened callback methods. After the Callback Detector nishes its job, it will provide 58 the EA Detector with the list of user-dened callbacks as well as the callbacks from the Android life-cycle. After this, the EA detector will perform its analysis to extract EAs, and will print out the detected EAs in a le that is indicated in the conguration le. It also provides these EAs to the UF Detector component to extract use-after-free bugs for Android systems. The UF anomalies will be stored in a le that is indicated in the conguration le. For non-Android systems, the Initializer will initiate Eos to extract information about event types. Eos will provide this information to the EA Detector. In turn, the EA Detec- tor will extract EAs and will report them in a le that is mentioned in the conguration le. To customize DEvA to run on a new system that uses an event-based mechanism not supported by the replication package, one needs to provide the necessary information for DEvA. To do this, one must rst create a Scala object for the new mechanism in the edu.usc.softarch.helios.middleware package that is part of the Initializer component. The Scala objects corresponding to the supported mechanisms can be found in that package, and can be used as examples for the new mechanisms. After creating the needed Scala object, one also needs to add the appropriate case line to the edu.usc.softarch.helios. middleware.Middleware object. Accuracy of EA Detection To assess DEvA's accuracy in detecting EAs, I applied it to the subject systems and compared its results to the ground truths in two dierent settings. In the rst setting, I considered DEvA without the ranking phase and compared the results with the ground 59 Initializer Eos Callback Detector EA Detector UF Detector Flow of data for Android systems Flow of data for Non-Android systems ______ ______ ______ UFs ______ ______ ______ EAs ______ ______ ______ Source Code ______ ______ ______ Configuration --- Output --- Input Figure 4.2: A view of DEvA's architecture truth that also was not ranked. In the second setting, I considered the ranking phase and compared the output with the highly ranked ground truth. In both settings, if an anomaly reported by DEvA was not in the ground truth, it was counted as a false positive; conversely, an anomaly in the ground truth that was not reported by DEvA was counted as a false negative. The results of DEvA without ranking are summarized in Table 4.2, and the results by considering the ranking are shown in Table 4.3. Considering DEvA without ranking, for all but three of the systems, DEvA was able to nd each anomaly identied 60 in the ground truth. The three exceptions were KLAX, for which DEvA yielded 2 false negatives, ToDoWidget, with 12 false negatives, and MyTracks, with 1 false negative. DEvA did not report any results that had not been found and conrmed as EAs in the ground truths, i.e., it had no false positives. Table 4.2: Results of applying DEvA on the subject systems System Anomalies Time (s) Planner 11 62 KLAX 12 62 DRADEL 447 122 ERS 37 54 Troops 30 75 Stoxx 1 66 JMSCHAT 3 29 Project.net 1 27 ToDoWidget 7 24 FBReader 74 58 MyTracks 73 65 ZXing 4 52 Firefox 39 81 ConnectBot 32 78 VLC 52 42 Browser 111 36 Camera 212 45 Music 65 33 android.support.v7 50 35 android.support.v4 109 59 In the cases of KLAX, ToDoWidget, and MyTracks, the false negatives occurred be- cause these systems rely on non-standard mechanisms to access component state and communicate state changes. For example, MyTracks dispatches an event by directly accessing the event's \what" attribute, while KLAX passes the entire current state of the game from one component to another via a single event with a single, very complex parameter. DEvA could be relatively easily modied to cover each of these exceptional 61 cases. However, I have chosen not to do so for the evaluation because it will always be possible to use other unforeseen, non-standard \hacks" when generating and processing events so that DEvA or a similar technique would not catch them without accounting for additional special cases. DEvA relies on EBS engineers to exercise relatively minimal discipline when developing their systems. To evaluate the ranking phase of DEvA, I obtained highly ranked ground truths by a manual analysis of each reported anomaly in the original ground truths. The ranking was done based on the criteria dened in section 3.1 of this dissertation, where the harmfulness of an anomaly is dened based on the type of anomaly. An anomaly is of the type read- write if it is consists of two accesses to a eld variable that are caused by two dierent events, and one of the events causes a write to the eld while the other causes a read from the eld variable. An anomaly is of the type write-write if both events cause a write to the eld variable. A read-write anomaly is harmless if the read to the participating eld variable does not impact the state of the component or system. This means that any execution path that is control or data dependent on the read does not contain a denition of a eld variable or publication of an event. A write-write anomaly is harmless if the eld that is written by consuming events is not used afterward, either in the denition of another eld variable or in the decision to send an event. This means that no execution path that is data dependent on the writes contains a denition of a eld variable or publication of an event. As an example, consider the code snippet in Figure 4.3. This gure shows an event anomaly in one of the components of the KLAX video game system. This event anomaly is 62 over eld variablecurrent num lives, involving the events NumberOfLivesSelector (con- sumed at lines 4 - 8) andClockSuspended (consumed at lines 9 - 12 ), and it is a read-write anomaly. The read occurs at line 9 and the write occurs at line 6 of Figure 4.3. The call to the method game over that occurs at line 11 is control dependent on the read to current num lives at line 9. Inside the body of game over at line 16 a method with the name chang text is called. Since this call is control dependent on the entry ofgame over, and the call togame over is control dependent on the read to current num lives, the call to chang text is also control dependent on the read to current num lives. As seen inside the body of change text at line 22, an event is published. Based on the criteria dened in this dissertation, the anomaly that is depicted in Figure 4.3 is harmful, since there is an execution path that publishes an event, and it is control dependent on the read that is participating in the anomaly. By executing this system one can see the harmfulness of this anomaly: the user can suspend the game right before the component perceives the fact that current num lives is equal to zero; once the clock becomes unsuspended, the user incorrectly gets one more life, and this is an undesirable state. Considering DEvA with the ranking phase, for all but two of the systems, DEvA was able to nd each anomaly that exists in the list of highly ranked ground truths. The KLAX system reported two false negative, and ToDoWidget reported nine false negatives. The reason for these false negatives are similar to what I discussed for experiments without ranking; systems made use of non-standard ways to access components' states. There was also a false positives for Project.net. This occurred because the anomaly that was reported by DEvA for Project.net aects the data that is stored in database, and considering other places that the database is read, the anomaly is harmful. DEvA wasn't able to detect 63 1 public class StatusArtist extends Component { 2 public synchronized void handle (Notification n){ 3 ... 4 else if (message_name.equals(KlaxSelectorTable.NumberOfLivesSelector)) 5 { 6 current_num_lives = ((Integer)n.getParameter("num_lives")).intValue(); 7 ... 8 } 9 else if (message_name.equals(KlaxSelectorTable.ClockSuspended) && (current_num_lives <= 0)){ 10 ... 11 game_over(); 12 } 13 } 14 void game_over() { 15 ... 16 change_text (num_lives_label, "Game Over"); 17 } 18 protected void change_text (Hashtable text_obj, String new_text){ 19 text_obj.put ("value", new_text); 20 Notification n = new Notification (KlaxSelectorTable.ObjectModifiedSelector); 21 n.setAllParameters (text_obj); 22 send (n); 23 } 24 25 } Figure 4.3: Excerpt from the KLAX system other places that read those data from database. Overall, the ranking approach marked, on average, 40% of anomalies as harmless anomalies across all the systems. For three of the subject systems (Planner, KLAX, DRADEL) I added the libraries that were used for logging and exception handling to their ignore lists. These ignore lists increased the number of harmless anomalies by 8%, on average, in these three systems. These results are a function of the denition of harmless anomaly adopted by this dissertation. It is possible to, due to this denition, some of these harmless anomalies may, in fact, have 64 certain negative eects on the system. The technique presented here is independent of this denition, however, and can accommodate alternative denitions. Table 4.3: Number of highly ranked anomalies after ranking phase System Anomalies Time (s) Planner 0 70 KLAX 9 68 DRADEL 34 145 ERS 31 59 Troops 20 52 Stoxx 1 57 JMSCHAT 2 53 Project.net 0 32 ToDoWidget 5 28 FBReader 37 72 MyTracks 46 81 ZXing 4 58 Firefox 27 93 ConnectBot 11 90 VLC 36 54 Browser 83 45 Camera 160 62 Music 44 51 android.support.v7 38 41 android.support.v4 38 70 DEvA reported several hundred EAs across the 20 subject systems. As discussed above, DEvA was able to identify a great majority of the anomalies present in the ground truths, and it has just one false positives after ranking in one of the systems. Consider- ing DEvA each reported anomaly did, in fact, re ect implicitly-concurrent accesses to a component eld with at least one access being a write. An EA may have one of three possible outcomes: 1. It is clearly a bug. An example is the UF anomaly in the MyTracks Android application from Figure 1.1. 65 2. It is clearly an undesirable nondeterministic behavior. An example involves the GameOver event in KLAX, which can be preempted by the GamePaused event, re- sulting in an additional life for the player. 3. It is potentially an undesirable nondeterministic behavior. An example is a Stoxx event that changes a threshold on a stock's price while another event requests a computation using the threshold and the stock's change history. While not all of the EAs reported by DEvA will have the same eect, they all have the potential to cause undesired behavior in a system and should be carefully examined by the engineers [30]. This dissertation's ranking approach considers event interactions and their eects at the level of components. Some anomalies can have eects at the lower levels that can also be harmful. As an example, consider part of the code from the ToDoWidget android application that is shown in Figure 4.4. There is an event anomaly in this gure between events onDraw and onMeasure over eld variable center radius. The read to center radius that occurs at line 6 is used as an argument for a call to method drawCircle. This method does not change any eld variable and also does not publish any event, but depending on the semantics of the system, the method's output, which can be aected by the value of center radius, can be considered harmful. For example, if center radius has a specic value relative to other parameters and it changes without changes to other parameters, the drawn circles may be considered esthetically inferior and this can be interpreted as undesirable (i.e., harmful) in the system. To investigate the harmfulness of anomalies such as the one shown in Figure 4.4, we need information about the semantics of the systems. This information is hard to extract from the code 66 1 public class ColorCircle extends View { 2 private float center_radius; 3 4 protected void onDraw(Canvas canvas) { 5 ... 6 canvas.drawCircle(0, 0, center_radius + mCenterPaint.getStrokeWidth(), mCenterPaint); 7 ... 8 } 9 protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) { 10 ... 11 this.center_radius = CENTER_RADIUS_SCALE * size/2; 12 ... 13 } 14 ... 15 } Figure 4.4: Excerpt from the ToDoWidget Android application and should be provided by the developers of the system. DEvA can easily be customized to incorporate such information if provided by developers. Comparison to Multi-Threaded Analysis In a multi-threaded environment, a data race occurs when two or more threads access the same memory location without proper synchronization. Because this is similar to the EA problem, I investigated whether standard data race detection techniques could be applied to nd EAs. To determine whether this is the case, I studied the literature on race detection and selectedChord [76] as a state-of-the-art static analysis technique to apply on the subject systems that I have. Chord was chosen because it is the only technique of its kind that has a reliable, actively maintained implementation. The purpose of this study was to establish the extent of overlap between the outputs produced by Chord and DEvA. The numbers of data races found by Chord are shown in Table 4.4. Since Chord starts its 67 Table 4.4: Results of applying Chord on the subject systems System Data Races Planner 14 KLAX 115 DRADEL 0 ERS 1418 Troops 78 Stoxx 31 JMSCHAT 0 analysis from a main method and Project.net has no such method,Chord was unable to analyze Project.net. Furthermore, I was unable to apply Chord on Android applications because Chord is not designed to consider the methods used in Android as event entry points. Despite this, the comparative analysis was revealing. I found that the result sets produced by Chord and DEvA do not overlap: none of the EAs reported by DEvA were among the data races reported by Chord, or vice-versa. DEvA was unable to detect any of the data races in the subject systems for the simple reason that it does not target traditional data races. Similarly, Chord does not consider the conditions that result in EAs. This suggests that DEvA and a static race detection technique such as Chord are complementary, and can be used eectively alongside each other. A deeper analysis sheds further light on why the two techniques yield such dierent results. EAs are caused by implicit concurrency. They are independent of the number of threads in a system and may occur even in single-threaded EBSs. On the other hand, data races only happen in the presence of at least two threads. EAs detected by DEvA are potentially more dangerous in that it is harder to track accesses to shared data when those accesses are impacted by a number of factors and actors in a distributed system: the local 68 state of the component processing an event, the network, event routers and dispatchers, and all the other distributed components in the EBS along with their respective states. Note that it is possible, in principle, to implement event-handlers in EBS such that each event is processed in a separate thread. This would render a technique such asChord more eective in detecting event anomalies. However, this is not a practical approach for constructing EBS since it would require a lot of resources for synchronizing and managing threads, without providing signicant benets. This is why existing EBS do not use such a technique. Comparison to Existing Android Analysis I also evaluated DEvA against CAFA [56], a recent dynamic analysis technique for de- tecting Use-After-Free (UF) EAs in Android applications. In order to compare DEvA withCAFA, I congured DEvA to report only UF anomalies among the EAs it detects. I then applied DEvA on the same versions of applications used inCAFA's evaluation [56]. Table 4.5 shows the relevant results. Note that the CAFA results did not include the two Android support libraries shown in Table 4.1, so for this reason, they are omitted from Table 4.5. Also in this table I have considered the results of DEvA after ranking. The CAFA All and DEvA All columns include all UF anomalies reported by the two techniques. As discussed by the authors of CAFA, certain UF anomalies will not actually result in null-pointer exceptions because of checks placed in the code [56]. Using the classication established for evaluating CAFA, the CAFA Harmful and DEvA Harmful columns in Table 4.5 show the respective numbers of discovered UF anomalies that result in actual runtime exceptions. 69 Table 4.5: Results of UF analysis by CAFA, and DEvA after Ranking CAFA DEvA j CAFA DEvA App Name All All j Harmful Harmful ToDoWidget 8 1 j 8 1 FBReader 3 2 j 1 2 MyTracks 1 5 j 1 2 ZXing 1 0 j 0 0 Firefox 4 0 j 0 0 ConnectBot 1 2 j 0 1 VLC 0 4 j 0 1 Browser 1 23 j 0 1 Camera 1 0 j 1 0 Music 4 17 j 2 9 For all but two of the applications, DEvA was able to identify more harmful UF anomalies thanCAFA. For two of the ten applications, Camera and ToDoWidget,CAFA performed better. In the case of Camera, CAFA reported a single harmful UF anomaly that DEvA could not detect. After inspecting the source code of this application, I were not able to locate CAFA's reported anomaly. It is possible that the anomaly occurred because of problems in the libraries that Camera uses, as CAFA did not analyze libraries separately from the application, while DEvA did not analyze these particular libraries. DEvA also had seven false negatives in the case of ToDoWidget. The reason was already discussed above: ToDoWidget relies on non-standard mechanisms to access component state. With regards to false positives, the performance of DEvA was also stronger. CAFA reported non-harmfulUF EAs in six out of ten Android applications. As discussed in [56], these are false positives: they are not actual EAs. As congured for this comparative evaluation, DEvA reported non-harmful UF anomalies in ve of the ten applications. However, these false positives were actually EAs (just not UF anomalies): while they 70 will not produce runtime null-pointer exceptions, they do result in nondeterministic ap- plication behavior and it is therefore still important for the developers to be aware of them. In summary, out of the combined 25 actual harmful UF anomalies that exist in the analyzed Android applications, CAFA was able to discover 13, while DEvA was able to discover 17. In addition to the comparison study described above, DEvA was able to uncover a number of EAs in the two Android libraries I analyzed (recall Table 4.2). Some of those were, in fact, harmful UF anomalies. For example, DEvA reported a harmful anomaly in the android.support.v4 library. This is a known bug [16], but after much investigation, DEvA is the rst to identify the bug's root cause. This bug happens in the android.support.v4.app.DialogFragment class due to the interaction between events ActivityCreated (processed by the method onActivityCreated) and DestroyView (pro- cessed by onDestroyView) over the eld variable mDialog. Because the order of the two events is nondeterministic, onActivityCreated is able to access mDialog after onDe- stroyView sets it to null. Overall, the results of this study were very positive. When DEvA congured to identify only UF anomalies, DEvA was able to uncover more such anomalies than a leading technique in eight of the ten applications. The number of false positives yielded by DEvA was lower and, while these were not necessarily UF anomalies, they were actual EAs. Finally, DEvA was able to nd the root cause of a previously unsolved bug in a widely used Android library. 71 Execution Time Given the objective of constructing an ecient technique, I designed DEvA to generate and use only the subset of system information that is required for event-anomaly anal- ysis. Thus, for example, DEvA generates PDGs for component sink methods only, and considers only those CETs that are consumed at a given sink (recall Section 3). Tables 4.2 and 4.3 show the execution Time required by DEvA to analyze each subject system with and without ranking respectively. These measurements include the time used by Soot to generate call graphs, CFGs, and PDGs; the Soot analysis averaged around 42s per system. The analysis by DEvA without ranking ranged between 24s and 122s, with an average of 55s. The ranking phase adds, on average, 10 seconds to the execution time. This execution time is reasonable, especially when we take into account that analyzing each system manually took over 15 hours. As a further comparison, CAFA's execution time for the ten Android applications shown in Table 4.5 varied between 30 minutes and 16 hours; DEvA's maximum execution time for the same applications was 93 seconds. 4.2 Evaluation of DSA I have evaluated DSA both empirically and analytically. The goal of the empirical eval- uation was to study the commonality of occurrence of SAs in real world systems. I also examined the execution time of DSA. Using probability and queuing theory concepts, I provided an analytical evaluation of DSA. This analytical evaluation provided useful guidelines for the situation in which removing SAs is more benecial. In the last part of 72 the evaluation of DSA, I used those analytical guidelines to provide more useful informa- tion about the SAs that exist in the subject systems. Subject Systems and Implementation Table 4.6 contains information about the 14 subject systems that I have used in the evaluation of DSA. All the subjects are implemented in Java but are from dierent ap- plication domains (App Type) and are dierent sizes (SLOC ). Also, the last 5 systems in Table 4.6 are selected randomly from public repositories such as GitHub. The goal of choosing random systems was to show that SAs can occur in all of the system, and my subject systems are not biased with regard to the occurrence of SAs. DSA is implemented in Java and Scala, and it uses the Soot [103] program analysis library to generate call graphs and control ow graphs. I ran DSA on a quad-core Intel i7 2.80GHz system with 8GBs of memory, running Windows 10 Professional. Table 4.6: Systems used in the evaluation of DSA App Name App Type SLOC Planner AI Planner [82] 6K KLAX Arcade Game [96] 5K DRADEL Software IDE [72] 11K ERS Crisis Response [68] 7K Troops Simulator [69] 9K Stoxx Stock Ticker [74] 6K JMSCHAT Chat System [10] 12K jms2009-PS JMS Benchmark [91,92] 18.6K Spark Chat Client [8] 85K JFreeChart Chart Library [9] 226K MarkShark Grading System [20] 28K JavaGame Game [19] 7K WebSocket Comunication protocol [22] 14K Chronicle In-Memory Database [17] 27K 73 Results of Applying DSA In this section, I discuss the results of applying DSA to the subject systems that are listed in Table 4.6. Tables 4.7 - 4.19 contain the results of applying DSA on the systems in Table 4.6. Each table is about one system. The Component column contains the component names, and the column Number of SAs indicates the number of identied SAs by DSA in each component. The last row of each table shows the total number of discovered SAs in that system. In Figure 1.2 in chapter 1, an example of a method with SAs was shown. In Fig- ures 4.5 - 4.7, additional examples of methods that contain SAs from three other subject systems are shown. Figure 4.5 shows part of the code for the Planner system, which provides an AI-based solution for smart routing of cargo. Figure 4.5 shows part of the system that is responsible for handling events that are related to building dierent user interfaces in the system. In this code, it can be seen that inside the method handle, there are ve other methods that are called at lines 5, 9, 13, 17, and 21. These methods are independent from each other and can be executed concurrently. In this case, DSA reports that method handle contains 5 SAs. Figure 4.6 shows a method, runScript, in the JMS2009-ps system. JMS2009-ps is a benchmark for publish/subscribe-based messaging systems. The runScript method that is shown in Figure 4.6 sets up and congures the environment for the execution of JMS2009-ps. There are three independent methods that are called inside runScript at lines 3, 5, and 7. Since these methods are not accessing the same memory locations, they can be executed concurrently, and DSA reports three SAs for runScript. Figure 4.7 shows part of the WebSocket system. This part is responsible 74 for the handshake process between clients and the server. There are three method calls inside method processHandshake at lines 12, 18, and 22 that are not accessing the same memory locations, and consequently DSA reports three SAs for the processHandshake method. 1 public class RouterArtist_C2_Component extends ComponentThread{ 2 public void handle(Request req_msg){ 3 if (req_msg.name().equals("AcceptTextFieldEvent")) { 4 ... 5 handleRequest_AcceptTextFieldEvent(parent_id, value, text_field); 6 } 7 else if (req_msg.name().equals("AcceptButtonEvent")){ 8 ... 9 handleRequest_AcceptButtonEvent(parent_id, button); 10 } 11 else if (req_msg.name().equals("AddTextField")){ 12 ... 13 handleRequest_AddTextField(width, height, foreground, label, background, parent_id, y, x); 14 } 15 else if (req_msg.name().equals("AddButton")){ 16 ... 17 handleRequest_AddButton(width, height, foreground, label, background, parent_id, y, x); 18 } 19 else if (req_msg.name().equals("CreateViewport")){ 20 ... 21 handleRequest_CreateViewport(width, height, foreground, title, background, id, y, x); 22 } 23 } 24 ... 25 } Figure 4.5: Excerpt from the Planner system In total, DSA was able to discover 556 SAs across all the systems. All except 10 of these 556 SAs actually existed in the subject systems, meaning that there were 10 false positives in the analyzed systems. These false positives occurred because the source code of some of the libraries that were used by dierent systems were not available to DSA, 75 1 public class jms2009PSControlDriver extends ControlDriver { 2 public int runScript() throws Exception { 3 configureSelf(); 4 ... 5 configureSatellites(); 6 ... 7 connectToAgents(); 8 9 } 10 ... 11 } Figure 4.6: Excerpt from the JMS2009-ps system 1 public class SSLSocketChannel2 implements ByteChannel, WrappedByteChannel { 2 private synchronized void processHandshake() throws IOException { 3 ... 4 if( !tasks.isEmpty() ) { 5 Iterator<Future<?>> it = tasks.iterator(); 6 while ( it.hasNext() ) { 7 Future<?> f = it.next(); 8 if( f.isDone() ) { 9 it.remove(); 10 } else { 11 if( isBlocking() ) 12 consumeFutureUninterruptible( f ); 13 } 14 } 15 } 16 if( sslEngine.getHandshakeStatus() == SSLEngineResult.HandshakeStatus.NEED_UNWRAP ) { 17 ... 18 unwrap(); 19 ... 20 } 21 ... 22 consumeDelegatedTasks(); 23 } 24 ... 25 } Figure 4.7: Excerpt from the WebSocket system 76 Components Number of SAs c2.PlannerSystem.Warehouse C2 Component 7 c2.PlannerSystem.RouterArtist C2 Component 5 c2.PlannerSystem.LayoutManager C2 Component 11 c2.PlannerSystem.Map C2 Component 1 c2.PlannerSystem.InPortArtist C2 Component 10 c2.PlannerSystem.WarehouseArtist C2 Component 7 c2.fw.Connector 10 c2.PlannerSystem.InPort C2 Component 6 c2.fw.graphics.GraphicsBinding 35 c2.PlannerSystem.CargoRouter C2 Component 8 c2.fw.ltering.EventVisualization 8 c2.PlannerSystem.Vehicle C2 Component 5 c2.PlannerSystem.Clock C2 Component 2 c2.PlannerSystem.VehicleArtist C2 Component 8 Total 123 Table 4.7: Summary of the application of DSA on the Planner system Components Number of SAs c2.apps.klax.comp.LetterTileArtist 2 c2.apps.klax.comp.StatusComponent 2 c2.apps.klax.comp.MatchingLogic 4 c2.apps.klax.comp.RelativePositionLogic 2 c2.apps.klax.comp.ChuteComponent 2 c2.fw.Connector 10 c2.fw.graphics.GraphicsBinding 35 c2.apps.klax.comp.StatusArtist 5 c2.fw.ltering.EventVisualization 8 c2.apps.klax.comp.PaletteComponent 2 c2.apps.klax.comp.WellComponent 6 c2.apps.klax.comp.TileArtist 2 c2.apps.klax.comp.TileRectArtist 2 c2.apps.klax.comp.PaletteArtist 4 c2.apps.klax.comp.WellArtist 4 c2.apps.klax.comp.ChuteArtist 2 Total 92 Table 4.8: Summary of the application of DSA on the KLAX system Components Number of SAs c2.apps.dradel.TypeCheckerComponent 6 c2.apps.dradel.TypeChecker 6 c2.apps.dradel.ComponentSpec 4 c2.fw.Connector 10 c2.fw.graphics.GraphicsBinding 35 c2.apps.dradel.TypeMismatchHandlerComponent 2 c2.fw.ltering.EventVisualization 8 c2.apps.dradel.ParserComponent 2 c2.apps.C4.comp.OnLineDB C2 Component 4 c2.apps.dradel.ArithmeticExpressionOperand 2 c2.apps.C4.comp.NOSS Mgr C2 Component 3 c2.apps.dradel.CodeGenComponent 39 c2.apps.dradel.ArchADTComponent 16 Total 137 Table 4.9: Summary of the application of DSA on the DRADEL system 77 Components Number of SAs Prism.core.EDFScheduler 2 tdspp.ResourceManager.ResourceManagerImp 3 Prism.extensions.component.deployment.Admin 9 tdspp.RenderingAgent.TopPanel 12 tdspp.hq.RenderingAgent.TopPanel 5 tdspp.StrategyAnalyzerAgent.StrategyAnalyzerAgentImp 1 Total 32 Table 4.10: Summary of the application of DSA on the ERS system Components Number of SAs C3.conn.distribution.RoutingConnector 2 C3.TroopsDemo.WarManager 1 C3.conn.scaolds.WindowedMessageMonitor 10 Total 13 Table 4.11: Summary of the application of DSA on the Troops system Components Number of SAs stoxx.DBPortfolioItem 6 stoxx.StockAbsLine 2 stoxx.DBRelLimit 4 stoxx.DBAbsLimit 5 stoxx.StockRelLineFactory 5 stoxx.PortfolioQuoteFactory 5 Events.SimpleRouting 5 stoxx.CurrentQuotes 2 stoxx.QuoteIntervalClient2 1 stoxx.events.DBEvents 2 stoxx.events.QuoteMinMax 4 stoxx.QuoteIntervalClient 1 stoxx.Portfolio 5 Total 47 Table 4.12: Summary of the application of DSA on the Stoxx system Components Number of SAs es.deusto.ingenieria.ssd.chat.jms.controller.Controller 6 Total 6 Table 4.13: Summary of the application of DSA on the JMSChat system Components Number of SAs org.spec.jms.framework.jms2009PSControlDriver 3 org.spec.jms.results.DetailReporter 8 org.spec.jms.preauditor.Auditor 2 org.spec.jms.setup.BenchmarkSetup 2 org.spec.jms.agents.SPECAgent 2 org.spec.jms.reporter.util.ChartGenerator 1 Total 18 Table 4.14: Summary of the application of DSA on the JMS2009-PS system 78 Components Number of SAs org.jfree.chart.editor.DefaultPolarPlotEditor 4 org.jfree.chart.plot.XYPlot 11 org.jfree.chart.plot.PiePlot 1 org.jfree.data.DefaultKeyedValues 4 org.jfree.chart.editor.DefaultAxisEditor 3 org.jfree.chart.editor.DefaultPlotEditor 6 org.jfree.data.xy.DefaultTableXYDataset 2 org.jfree.chart.renderer.category.BoxAndWhiskerRenderer 4 org.jfree.chart.plot.PolarPlot 4 org.jfree.data.time.DynamicTimeSeriesCollection 6 org.jfree.chart.renderer.WaferMapRenderer 2 org.jfree.chart.title.ImageTitle 2 org.jfree.chart.axis.AxisState 1 org.jfree.chart.editor.DefaultTitleEditor 3 org.jfree.chart.editor.DefaultValueAxisEditor 5 org.jfree.chart.ChartPanel 11 org.jfree.chart.plot.CategoryPlot 8 org.jfree.chart.StandardChartTheme 7 org.jfree.chart.renderer.xy.XYBoxAndWhiskerRenderer 4 Total 88 Table 4.15: Summary of the application of DSA on the JFreeChart system or systems didn't use standard ways to access memory locations. There could be false negatives in the output of DSA, however since there was no ground truth available for the systems, I was not able to report a number. To obtain the ground truth, one must read the whole source code of each system, since SAs can occur in any part of the system and are not limited to specic parts{ like, for example, event handlers. This will be a time-consuming job, since the size of the smallest system that I analyzed was about 5K in SLOC. It is a well-known fact that Soot library, which DSA is based on, can introduce false negatives to the analysis. One interesting observation in the results of the evaluation was that the ambiguous interfaces were mostly reported to contain SAs. This is because in an ambiguous interface, there exist dierent execution paths, and the decision concerning which one should be executed will be made based on the input of the interface. Whenever those execution paths are not accessing a shared memory location, they will be considered as SAs by 79 Components Number of SAs org.jivesoftware.sparkimpl.plugin.alerts.BroadcastPlugin 3 org.jivesoftware.spark.PluginManager 6 org.jivesoftware.spark.ui.conferences.GroupChatInvitationUI 2 org.jivesoftware.sparkimpl.plugin.privacy.list.PrivacyPresenceHandler 2 org.jivesoftware.sparkimpl.prole.AvatarPanel 2 org.jivesoftware.smackx.jingle.mediaimpl.JMFInit 3 org.jivesoftware.smackx.packet.VCard 4 org.jivesoftware.sparkimpl.preference.notications.NoticationPlugin 2 org.jivesoftware.spark.component.tabbedPane.SparkTabbedPane 2 org.jivesoftware.sparkimpl.plugin.phone.JMFInit 3 org.jivesoftware.sparkimpl.preference.notications.NoticationAlertUI 1 org.jivesoftware.spark.ui.ContactList 18 org.jivesoftware.smackx.bytestreams.socks5.Socks5BytestreamRequest 2 org.jivesoftware.MainWindow 2 org.jivesoftware.smackx.jingle.JingleSession 4 org.jivesoftware.smack.XMPPConnection 1 org.jivesoftware.spark.ui.conferences.ConferenceUtils 2 org.jivesoftware.spark.ui.conferences.GroupChatParticipantList 2 org.jivesoftware.sparkimpl.plugin.letransfer.transfer.ui.ReceiveFileTransfer 1 org.jivesoftware.sparkimpl.plugin.alerts.SparkToaster 1 org.jivesoftware.spark.ui.ContactGroup 7 com.jcraft.jzlib.ZOutputStream 4 org.jivesoftware.smackx.muc.MultiUserChat 11 org.jivesoftware.smackx.workgroup.user.Workgroup 4 org.jivesoftware.spark.component.browser.EmbeddedBrowserViewer 2 com.jcraft.jzlib.In ate 2 org.jivesoftware.spark.ui.rooms.ChatRoomImpl 4 org.jivesoftware.spark.Workspace 2 org.jivesoftware.smackx.debugger.EnhancedDebugger 1 org.jivesoftware.spark.ui.themes.ColorSettingManager 3 org.jivesoftware.smackx.jingle.media.MediaNegotiator 2 org.jivesoftware.sparkimpl.prole.VCardManager 2 org.jivesoftware.spark.ui.ChatArea 4 org.jivesoftware.Spark 2 org.jivesoftware.spark.ui.ChatContainer 10 org.jivesoftware.smack.util.collections.AbstractHashedMap 6 org.jivesoftware.smack.PacketReader 5 org.jivesoftware.smackx.jingle.ContentNegotiator 2 de.javawi.jstun.test.demo.ice.ICENegociator 2 org.jivesoftware.smackx.workgroup.agent.AgentSession 4 com.jcraft.jzlib.De ate 12 org.jivesoftware.smackx.jingle.mediaimpl.sshare.api.OctTreeQuantizer 2 Total 156 Table 4.16: Summary of the application of DSA on the Spark system Components Number of SAs com.redomar.game.net.GameClient 2 com.redomar.game.Game 2 com.redomar.game.net.GameServer 3 com.redomar.game.script.Printing 2 com.redomar.game.lib.Music 2 Total 11 Table 4.17: Summary of the application of DSA on the JavaGame system 80 Components Number of SAs org.java websocket.SSLSocketChannel2 6 org.java websocket.WebSocketImpl 1 Total 7 Table 4.18: Summary of the application of DSA on the WebSocket system Components Number of SAs com.higherfrequencytrading.chronicle.datamodel.ListWrapper 58 com.higherfrequencytrading.chronicle.impl.AppendParseTest 8 com.higherfrequencytrading.chronicle.impl.IndexedChronicleTest 14 com.higherfrequencytrading.chronicle.datamodel.MapWrapper 24 com.higherfrequencytrading.chronicle.examples.ExampleKeyedExcerptMain 3 com.higherfrequencytrading.chronicle.datamodel.SetWrapper 30 Total 137 Table 4.19: Summary of the application of DSA on the Chronicle system DSA. In the subject systems that were event-based (the rst nine systems in Table 4.6), these ambiguous interfaces were mostly event sinks. DSA reported tens of SAs for each of the systems. The SAs exist in the random chosen systems as well as the systems that were available to me from our previous studies. This means that SAs are a common problem. It can be argued that the overhead of using concurrent method calls may force the existence of SAs in the systems. This is a valid point, and therefore providing more information about the benets of removing SAs will be useful. In the analytical evaluation, I will provide more information about the situation in which having concurrency will be more benecial. Using that information, it will be possible to pinpoint the parts of the system in which it will be benecial to remove SAs. Execution Time As discussed in the Chapter 3 of this dissertation, to make DSA an ecient technique, I made use of summary-based techniques to store computations in order to avoid redundant 81 computations, and also I made use of on-demand generation of required information such as CFGs. Table 4.20 shows the execution Time required by DSA to analyze each subject system. These measurements include the time used by Soot to generate call graphs, CFGs; the Soot analysis averaged around 50s per system. The analysis by DSA ranged between 32s and 224s, with an average of 78s. The highest execution time was for Spark system and the lowest one was for Chronicle. Although Spark has a large size among the subject systems, the analysis time doesn't have a direct correlation with the size of the system. Systems that contain long methods more often also have longer analysis times. These methods have larger CFGs and usually call many other methods, which will cause a larger call graph as well. Having large CFGs and call graphs will increase the analysis time. It is considered as a code smell to have long methods [44]. In summary, the execution time of DSA is reasonable, since it was, on average, less than two minutes, and at its peak, it was less than four minutes. Analytical evaluation of DSA In this section, I will provide an analytical evaluation of DSA. The goal of this analytical evaluation is to provide more information for developers to help them better identify the situations in which removing a SA will be more benecial to the performance of the systems. To provide such information, I make use of queuing theory [49] and probability theory [84]. A queuing system deals with the situations in which there exist customers requiring a specic service. If they cannot be served immediately, they will wait in the system until they can be served, and after they receive their service, they leave the 82 Table 4.20: DSA's Analysis time for each system App Name Time(s) Planner 56 KLAX 53 DRADEL 69 ERS 58 Troops 49 Stoxx 51 JMSCHAT 57 jms2009-PS 119 Spark 224 JFreeChart 97 MarkShark 133 JavaGame 52 WebSocket 51 Chronicle 32 system [49]. The goal of queuing theory is to provide predictions about queuing systems in the presence of random customer arrivals. A call to a method either in response to an event or because of the implementation logic of the system can be considered as a customer arriving at the system. The server in this system is the CPU. If there is a single thread that will perform all the computations, our queue system has just one server. If there is more than one thread, then our system can be considered a queuing system with more than one server. If multiple methods calls are concurrent, then they are related to dierent customers, and they can be served using dierent servers. Depending on the probability distribution of the occurrence of method calls and the probability distribution of execution time of each method, we can have dierent queuing systems. In the simplest form of queuing systems, the input distribution is Poisson with variable, and the service time distribution is exponential with scale parameter . Such 83 a queuing system is called a M=M=k system, where k is the number of servers. If the distribution of incoming method calls and service time is unknown, we need to know the average and standard deviation of incoming method calls and service time to be able to use queuing theory concepts. Such a queuing system is called a G=G=k system, with k indicating the number of servers. Studying dierent properties of the system, such as waiting time, throughput, and queue length, is much harder in G=G=k systems. The main properties of the queuing systems that are important in the analytical evaluation of DSA are the length of the queue and the length of the system. The length of a queue can be mapped to the number of method calls in the system that are waiting to be handled, and the length of the system can be mapped to the number of method calls that are in the system, including those that are currently being served. These metrics are important because as they increase in their value, the waiting time in the system will also increase, and this will cause a reduction in the performance of the system. Consider that there exists a method m that calls methods m 1 ;:::;m n during its exe- cution. For eachi;j wherei6=j,m i andm j are independent of each other, meaning that they are not accessing the same memory locations. Now consider that the duration of execution ofm i is an exponential distribution with rate parameter i , or scale parameter i where i = 1 i , and it is also independent of duration of execution of other methods like m j where j6= i. Depending on the behavior of m, the distribution of its execution time can also be dierent. Here I will study two dierent cases. The rst case deals with the situation in which execution ofm requires a call to eachm i method. The second case considers the situation in which a call tom just requires a call to one of them i methods. 84 For a case in which a call to m will require a call to each method in a set m 1 ;:::;m j and a call to just one of the methods m j+1 ;:::;m n , which are the rest of the methods that are called inside the body of m, we can consider a queuing model that consists of two independent queues, since the methods m i are independent from each other. One queue will represent calls to m 1 ;:::;m j (rst case) and the other queue will represent a call to one of m j+1 ;:::;m n (second case). We can then study each of these two queues independently, as discussed in the following subsections. At the end, the length of the system's queue and the length of the system that consists of the two queues will be the maximum of those values among the two queuing models. In other words, every other case can be transformed to the combination of the two main cases. To combine, we just need to compute the maximum value among the two cases. For each of the two main cases, there can be several sub-cases that I will discuss in the following subsections. The Case in which m Calls Each m i This section will present the case where a call to method m will require the execution of eachm i method. In this scenario, the duration of execution ofm should be computed by considering the distribution of the sum of n independent exponential random variables with rate parameters 1 ;:::; n . This distribution has the following form [11]: f m 1 +m 2 ++mn (x) = n Y i=1 i ! n X j=1 e x j n Y k=1 k6=j ( k j ) ;x 0 (4.1) 85 In probability theory, the mean or expectation (E(x)) of a distribution f can be determined by computing R +1 1 xf (x) dx. Using this and equation 4.1, the mean for the execution time of m is: Mean(m) = n Y i=1 i ! n X j=1 1 2 j n Y k=1 k6=j ( k j ) (4.2) The variance of a distribution like f can be computed using E[x 2 ]E[x] 2 . TheE[x] for the execution time of m which is Mean(m) is shown in equation 4.2. To compute E[x 2 ] the following integral should be computed: R +1 1 x 2 f (x) dx. Using these formulas and equation 4.2, the variance for execution time of m will be: Var(m) = n Y i=1 i ! n X j=1 2 3 j n Y k=1 k6=j ( k j ) Mean(m) 2 (4.3) Considering the principles of queuing theory, our queuing system is a M=G=1 with a Poisson arrival with mean and a general service distribution like equation 4.1 with the mean of equation 4.2 and the variance of equation 4.3. As I said before, the arrival in this evaluation means a call to method m, and service means execution of m. The length of the queue in such a queuing system can be determined using the following formula [49]: L q = 2 + 2 2 2 (1) (4.4) Equation 4.4 is often referred to as the PollaczekKhintchine or (PK) formula. In 86 this formula is the arrival rate mean of the service time distribution . If we apply DSA to the above-mentioned system, since the m i are independent of each other, DSA will report them as synchronization anomalies. If we decide to execute m i concurrently, the distribution of the execution time of m will be a random variable like Max(m 1 ;m 2 ;:::;m n ). The reason for this is that when m i methods are executing concurrently, m just needs to wait for the method that has the maximum duration to nish. After that method is nished, all other methods also have nished their operations. The distribution for the maximum of n independent exponential random variables is the following equation: f Max(m 1 ;m 2 ::::;mn) (x) = n X i=1 i e i x n X i;j=1 i<j ( j + i )e ( j + i )x + + (1) n+1 n X i=1 i ! e ( P n i=1 i)x (4.5) The mean and variance for this distribution can be computed using the following formu- las: Mean Max(m 1 ;m 2 ::::;mn) = n X i=1 1 i n X i;j=1 i<j 1 ( j + i ) + + (1) n+1 1 P n i=1 i (4.6) 87 Var Max(m 1 ;m 2 ::::;mn) = n X i=1 2 2 i n X i;j=1 i<j 2 ( j + i ) 2 + + (1) n+1 2 ( P n i=1 i ) 2 Mean 2 Max(m 1 ;m 2 ::::;mn) (4.7) As you can see, the formulas for the mean and variance of the maximum of n inde- pendent exponential random variable are complex. Because of this, I will consider two special cases that make the computations easier. For the rst case, consider the simplest case, where we just have two random variables. This simple case means that methodm calls two methodsm 1 andm 2 , where each of them has a duration with an exponential distribution with parameters 1 and 2 respectively. If m 1 and m 2 are synchronized, then distribution of execution of m will be the sum of distributions of m 1 and m 2 . Using equation 4.2, the mean for the duration of m will be 1 1 + 1 2 . The variance of this distribution based on equation 4.3 will be 1 2 1 + 1 2 2 . Now if we decide to executem 1 andm 2 concurrently insidem because DSA recommends it, the distribution for the duration of m will be of the form of equation 4.5. In this case, using equation 4.6, the mean for the distribution will be 1 1 + 1 2 1 1 + 2 , and the variance using equation 4.7 will be 1 2 1 + 1 2 2 3 ( 1 + 2 ) 2 . In both situations the arrival rates are Poisson and the service times have a general form. As a result, both systems areM=G=1 systems, and the length of their queue can be computed using equation 4.4. By comparison, you can see that the length of the queue for the case that m 1 and m 2 are concurrent would be shorter. This means that in the concurrent case, there would always be fewer calls to 88 m waiting to be handled in the steady state, or in other words, the system is handling the requests faster and is able to provide more service. The second special case that I study is when the rate parameters or i s for all of the random variables are the same and is equal to . Considering this situation with the case thatm i are synchronized, the sum ofm i 's distributions, which is the distribution for m's execution time, will be anErlang distribution [84] with rate parameter and shape parametern. Mean for them's execution time will be n and the variance will be n 2 . The corresponding queuing model for this system will be a system that is called M=E [n] =1. E [n] means that the service time hasn steps and its distribution is an Erlang distribution. The queue length in this system can be computed using the following formula [49]: L q = n + 1 2n 2 1 (4.8) For the situation in which m calls m i concurrently, and all the m i have the same rate as , we can use equations 4.6 and 4.7 to compute the mean and variance for the service time distribution of the queuing system. This system can be considered a M=G=1 system, and its queue length can be computed using the PK formula, which is shown in equation 4.4. The length of the queue in the second special case for synchronized and concurrent m i are shown in gures 4.8 and 4.9 respectively. In each gure, the queue length (y axis) is depicted for the cases where the number of m i varies between 2 and 5. The x axis, which is the parameter, varies between 1.5 and 2.5. The arrival rate is also set to be 89 1.5 1.6 1.7 1.8 1.9 2 2.1 2.2 2.3 2.4 2.5 0 1 2 3 4 5 6 7 8 9 10 αParameter Queue Length n=2 n=3 n=4 n=5 Figure 4.8: Queue length for synchronized methods 1.4. As can be seen, the concurrent situation has a much shorter queue length compared to the synchronized situation. As the number of m i methods increases, the queue length will become closer to zero for both systems; however, the length will be still shorter for the concurrent scenario. In summary, based on the above analytical evaluation, in the cases that a call to method m requires a call to each of the m i methods, executing m i concurrently will always enhance the performance of the system. When the execution times of the methods are close to each other, this enhancement can be more signicant if we execute a smaller number ofm i methods concurrently. As the number of concurrent methods increases, the queue length will converge to a specic number [50]. This fact suggests that there is a threshold on the number of concurrent methods that can cause a gain in the performance 90 1.5 1.6 1.7 1.8 1.9 2 2.1 2.2 2.3 2.4 2.5 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 αParameter Queue Length n=2 n=3 n=4 n=5 Figure 4.9: Queue length for concurrent methods of the system. Increasing the number of the concurrent methods above than that threshold will not yield a signicant improvement in the performance of the system. In real-world software systems, where there are multiple threads and the number of threads in the system becomes larger than the number of cores that the processing unit provides, an overhead is introduced in the system because of context switches. A context switch occurs because the operating system will make use of the processing unit through a time-share policy. As the number of concurrent methods increases, the overhead of context switching will also grow. We can consider this overhead as an increase in the mean of service time for each method. If the overhead becomes signicant compared to the original service time of each method, the waiting time of the jobs in the queue will be increased even for small jobs, and as a result, the queue length of the system will grow. 91 In other words, increasing the number of concurrent methods until the overhead is not signicant is acceptable, but if the overhead increases such that even service times for original small jobs become very high, the queue length will increase. The Case in which m Calls Just One m i This section will present the scenario where a call to methodm will require the execution of just one of the m i methods. This scenario usually happens in ambiguous interfaces and also sink interfaces for event-based systems. In such systems, method m will decide which m i should be called based on the arguments that are sent to it. Using Markov models, we can show the transition between dierent states of such a system. Figure 4.10 shows the Markov model for the situation where there are two methods, since it is hard to show the general Markov model of such a system. In Figure 4.10, each state like (i;j) represents that there are i calls to m 1 and j calls to m 2 in the system. In this model, it is considered that calls to method m i arrive in a Poisson distribution with the rate i and the execution time of m i is Exponential with the scale parameter i . A Markov model will reach a steady sate when the rate of entering to a state is equal to the rate of exiting that state. This means that for a state (i;j), if the probability of being in this state is P i;j , then we will have 2 P i;j1 = 2 P i;j and 1 P i1;j = 1 P i;j . Using this relation and considering all possible values ofi andj, we arrive at the following equation: P i;j = ( 1 1 ) i ( 2 2 ) j P 0;0 = i 1 j 2 P 0;0 (4.9) 92 ... 0,0 1,0 2,0 λ 1 λ 1 3,0 4,0 5,0 λ 1 λ 1 λ 1 μ 1 μ 1 μ 1 μ 1 μ 1 ... 0,1 1,1 2,1 λ 1 λ 1 3,1 4,1 5,1 λ 1 λ 1 λ 1 μ 1 μ 1 μ 1 μ 1 μ 1 λ 2 λ 2 λ 2 λ 2 λ 2 λ 2 μ 2 μ 2 μ 2 μ 2 μ 2 μ 2 0,2 1,2 λ 1 μ 1 0,3 1,3 λ 1 μ 1 λ 2 λ 2 μ 2 μ 2 λ 2 μ 2 λ 2 μ 2 ... ... ... ... ... ... Figure 4.10: Markov model for the system with two dierent arrival rates and two dierent service times In this equation, i is equal to i = i . Since P P i;j = 1, using equation 4.9, P 0;0 will have the following form: P 0;0 = (1 1 )(1 2 ) (4.10) Also, we can compute the probability of havingn customers in the system,P n , which is equal to P i+j=n P i;j by using equation 4.9. In the case that 1 6= 2 , P n will be: P n = n+1 1 n+1 2 1 2 P 0;0 (4.11) For the case where 1 = 2 =, P n can be computed as: P n = (n + 1) n P 0;0 (4.12) 93 The average length of the system, L, is equal to the expectation value of P n , or in other words, the following formula holds: L =E[n] = X nP n (4.13) In the case where 1 6= 2 , by using equations 4.10, 4.11, and 4.13, equation 4.14 can be computed for average length of the system or L. L = 1 1 1 + 2 1 2 (4.14) In the case where 1 = 2 =,L can be computed as shown in equation 4.15 by using equations 4.10, 4.12, and 4.13. L = 2 (1) 3 (4.15) Also the probability of having some requests in the system, or 1P 0;0 , for the syn- chronized scenario, is: P (r6= 0) = 1 (1 1 ) (1 2 ) (4.16) If DSA analyzes this system, it will suggest to execute m 1 andm 2 concurrently, since they are not accessing any shared memory locations and their executions are completely independent from each other. In this situation, method m can handle a new call as soon 94 as either m 1 or m 2 nishes its job. This means that we have two servers that each have a service time with the distribution of Min(m 1 ;m 2 ). Since the execution times of m 1 and m 2 are exponential with scale parameters 1 and 2 , based on probability theory the execution time of their minimum is also exponential with parameter 1 + 2 . This means that our queuing system is a M=M=2 system. This queue has a Poisson input rate of 1 + 2 and two servers that each have an exponential execution time with scale parameter 1 + 2 . The number of requests in the system in a M=M=c system can be computed using the equation 4.17 [49]. In this equation r =c and = 1 + 2 1 + 2 . L =r + r c c! (1) 2 c1 X n=0 r n n! + r c c!(1) ! 1 (4.17) If we consider c = 2 in equation 4.17, the L value for a M=M=2 system would be: L = 1 1 1 1 + (4.18) The probability of having at least one request in the systems for the concurrent scenario can be computed using the following formula [49]: P (r6= 0) = 1 1 1 + (4.19) 95 Comparison of equations 4.15 and 4.18 reveals that the length of the queue in the concurrent situation would be smaller than the synchronized situation whenever 1 = 2 in the synchronized scenario. It should be mentioned that generally, if we consider 1 2 , we will have 1 2 and because of this 1 = 2 forces 1 = 2 =. 0.1 0.11 0.12 0.13 0.14 0.15 0.16 0.17 0.18 0.19 0.2 0 1 2 3 4 5 6 7 8 9 10 ρ Parameter Number of Requests in System Concurrent Synchronized Figure 4.11: Comparing number of requests in the system when the parameters have a large dierence For the cases where there is a large dierence between i , there will be a signicant dierence in the length of the system between the two scenarios. When the dierence between i is very large, that means one of the i values is very low and the other one is very high. In this situation, the of the concurrent scenario will be closer to the minimum i . Having a low for the concurrent system means that fewer requests will stay in the system. The main reason for this is that, based on the equation 4.16, in such a situation 96 the probability of having some requests in the system using the synchronized scenario will be closer to one. However, in the concurrent scenario, by considering equation 4.19 and the fact that will be closer to the minimum i , this probability is lower. This means that in the concurrent scenario the system is more often empty and ready to serve new requests. This situation is shown in in Figure 4.11. The x-axis in this gure shows the values for the concurrent scenario, and the y-axis shows the length of the system for each scenario. To compute the length of the system for the synchronized scenario in this gure, I chose the value of 1 to be close to and the value of 2 to be much larger than . As can be seen in this gure, the number of requests in the system is lower than one and close to zero for the concurrent scenario, but for the synchronized scenario this value is high. This result can help developers to derive more benets from a concurrent scenario. To do this, they must add concurrency in a method like m that has an ambiguous interface or is the sink of an event-based system, whenever the i of the methods that are called by m are signicantly dierent from each other. I will use this guideline in the next section to provide better insight into SAs for DSA users. Finding Most Important SAs As discussed earlier, the results of applying DSA on the subject systems used in evaluating the results of this dissertation showed that SAs are common across dierent systems. In the previous section, it was shown by using probability and queuing theory that a method m will benet more from the concurrent execution of the m i that it calls whenever the ratios, i , of the rate of calls to m i to the mean of their execution times' distributions 97 are signicantly dierent among m i . In this section, I will use this guideline to provide related information to DSA users. To do this, I consider a metric to rank methods according to the extent to which they are aected by SAs. This metric takes into account the length of the sequence of method calls in a SA. If the methods' execution times are independent, the variance of the execution time of the sequence of method calls will be the sum of variances of each method in the sequence. As a result, as the length of the sequence grows, so will the diversity in its execution time compared to shorter sequences. This will make concurrent execution of the longest sequences more benecial for the performance of the system, according to the analytical evaluation in the previous section. To investigate the application of the longest sequence metric, I computed the perfor- mance gain of the system after removing SAs that were reported by DSA for each method in nine of the subject systems. Subsequently, I compared this performance gain with the performance gain from removing SAs of methods with the longest sequence of method calls. The goal of these experiments was to check whether removing SAs that are marked by the longest sequence metric actually introduces more performance gain in comparison with the gain introduced by removing other SAs that are not marked by this metric. To compute the performance gains, I used VisualVM, a proling tool that is included in the distribution of JVM. To compute the performance gain, I compared the time to nish the execution of the main thread before and after removing the SAs. In cases where I observed a performance gain, I repeated measuring the time it took for the main thread to nish before and after removing each SA until my data passed a statistical signicance test. This test is necessary to ensure that the reported performance gain is not due to 98 random or accidental factors. To investigate the statistical signicance property, I use both the T-test (in case the time distributions is normal) and also the Mann-Whitney test (in case the time distribution has any other form), with probability parameter being equal to 0.05. On average for each method that contained a SA, I executed the corresponding system and gathered data 30 times before and after removing the SAs to pass the sta- tistical signicance test. The total number of executions among all the nine systems was 10,889. Figure 4.12 shows a simple way of removing a SA. To remove a SA, I made use of the Executor framework, Callable, and Future objects, which are all provided by Java. First, a thread pool should be built using the Executor framework (line 3 of Figure 4.12), and then for each method m that needs to be executed concurrently, an anonymous Callable object will be dened, and inside the call method of that object,m will be called (lines 6 - 10). After this, the callable object should be submitted to the thread pool, and the submission should be stored in a Future object (line 11). Using the Future object, we can check whether the concurrent execution of m is nished (line 12). The way that we used the Executor framework may dier depending on the SA, but the general structure re ects that shown in Figure 4.12. As can be seen, this suggests less than 10 lines of code will be added per SA. The results of the experiments are shown in tables 4.21 and 4.22. Table 4.21 reports the average, the standard deviation, and also the maximum gain in performance after removing SAs that were not marked by the longest sequence metric. The minimum gain in performance for each of the systems was zero. Table 4.22 reports the average, minimum, and also maximum gain in performance after removing SAs that were marked 99 1 class C extends Component { 2 ExecutorService executor = 3 Executors.newFixedThreadPool(PoolSize); 4 public void methodContainedSA(ActionEvent event) { 5 Callable<ReturnType> task= 6 new Callable<ReturnType>(){ 7 public ReturnType call(){ 8 return m(); 9 } 10 }; 11 Future<ReturnType> f=executor.submit(task); 12 if(f.isDone()) 13 .... 14 } 15 } Figure 4.12: An illustration of how to remove a SA by the longest sequence metric. The standard deviation was not reported, since for many of the systems, there was just one sequence with the longest length. As can be observed, for all but one of the systems, the average performance gain of removing SAs with longest sequence property is much higher compared to the average gain from removing other SAs. The only exception is Chronicle. In this system, the SA that was related to the longest sequence was a false positive because there exist accesses to memory locations that are non-standard. However, among ordinary SAs, the ones whose removal caused the largest gain in performance were ranked higher based on their length of sequence compared to other SAs. For JMSChat system, all the sequences have the same length, and that is why there is no dierence between tables 4.21 and 4.22 for this system. To perform further investigations, I compared the impact of the longest sequence metric with the impact of a second metric on performance. The second metric considers methods with loops as the main source of diversity in the execution time and marked 100 the sequences with loops as more diverse sequences. The intuition behind the second metric was that when there is a loop in a method, the execution time of the method will be dependent on the size of the loop, and if the size of the loop is not xed, it will cause a diverse execution time. The result of considering the second metric is shown in table 4.23. This table reports the average, minimum, and maximum gain in performance after removing SAs that were marked by the loop metric. Comparing tables 4.22 and 4.23 reveals that for all but three of the systems, the longest sequence metric introduces a greater gain in performance. For the rst two systems that are listed in these tables, both metrics mark the same SAs. For the DRADEL system, we get the highest gain from a SA that is marked by both the longest sequence and the loop metrics; however, there are two other SAs that were marked by the longest sequence metric that do not contain loops. In general, my empirical study shows that the longest sequence metric introduced a greater gain in performance compared to the loop metric. Using the longest sequence metric, there was, on average, a 10% performance gain across the nine subject systems. Table 4.21: The results of removing SAs without considering the longest sequence metric System Average percentage gain in performance Standard Deviation Maximum gain Planner 1.38 1.34 5 KLAX 1 1.44 4 DRADEL 1.21 2.16 10 JMSCHAT 2 1.73 3 jms2009-PS 1.25 2.05 5 JFreeChart 1.16 1.43 3 JavaGame 0.75 0.86 1.5 WebSocket 5 7 10 Chronicle 0.88 2.6 12 101 Table 4.22: The results of removing SAs by considering the longest sequence metric System Average percentage gain in performance Minimum gain Maximum gain Planner 10 10 10 KLAX 7 7 7 DRADEL 5 4 6 JMSCHAT 2 0 3 jms2009-PS 25 25 25 JFreeChart 7 6 8 JavaGame 10 10 10 WebSocket 15 15 15 Chronicle 0 0 0 Table 4.23: The results of removing SAs by considering the loop metric System Average percentage gain in performance Minimum gain Maximum gain Planner 10 10 10 KLAX 7 7 7 DRADEL 6 6 6 JMSCHAT 0 0 0 jms2009-PS 1 1 1 JFreeChart 0 0 0 JavaGame 0 0 0 WebSocket 0 0 0 Chronicle 0 0 0 4.3 Limitations and Threats to Validity I have identied three limitations that were specic to DEvA in my evaluation. The rst limitation is the largely manual construction of the ground truth, aided only by the code-search and visualization features of Eclipse. Human error in this process could aect the reported results. Our research group and I tried to mitigate this issue by carefully validating the ground truth through inspection, as re ected in the very long time required to construct it. This is further mitigated by the comparison to CAFA, which indicates that, with the exception of the single application that relies on a non-standard event- processing mechanism, DEvA did not miss any harmful UF EAs. The second limitation 102 is that, in analyzing EBSs with ambiguous interfaces, Eos occasionally yields a small number of false positives in reporting event types [47]. In turn, this may aect DEvA's results. Note that Eos's inaccuracy did not have any impact on the Android systems I analyzed since DEvA's Algorithm 1, rather than Eos, was used there. The third limitation is that DEvA assumes that callback methods are dened in the source code of an Android application to handle events that can occur nondeterministically. If this is not the case, DEvA will report false positives. Also, there were limitations that were specic to DSA. In the analytical evaluation of DSA, I assumed that calls to methods occur based on a Poisson distribution. This assumption may not always be applicable to real world systems, but it simplies my ana- lytical models. I also hold similar assumptions about the execution time of the methods. Despite these assumptions, my evaluations veried that removing SAs based on DSA suggestions would actually increase the performance of the system. Beyond the limitations that are specic to DEvA and DSA, there are at least two additional limitations that impact the accuracy of static analysis techniques in general: aliasing and detecting feasible paths. Aliasing can take the form of data aliasing, in the case of references to variables, or method aliasing, in the case of virtual invocation. Data aliasing can aect data- ow analysis, and virtual invocation can aect call graph generation by introducing false edges between methods. Inaccuracies in data ow anal- ysis and call graph generations can aect the generation of ICFGs and PDGs. As it is mentioned earlier, to generate ICFGs and PDGs, both of my proposed approaches re- lied on Soot, but Soot is occasionally unable to analyze certain systems. This happened 103 with two components of the KLAX system, four components of Spark, and three com- ponents of WebSocket for which Soot was unable to provide the CFG or PDG. In the subject systems that I used for DEvA evaluation, virtual invocation is used for creating and building events, but not for their consumption. This was the primary reason why aliasing inaccuracies did not cause DEvA to report false positives. For the ground truth I tried to detect infeasible paths by manual inspection of the subject systems' code. I did not nd any infeasible paths that would have resulted in falsely identied EAs by DEvA. Checking the feasibility of paths in the subject systems was time consuming, but doable, since the extent of branching after event consumption was low in these systems. As I mentioned earlier, DSA can suer from false negatives, but since preparing the ground truth was time-consuming and infeasible, this dissertation does not report a number that indicates the rate of false negatives in DSA. 104 Chapter 5 Related work My research is related to several approaches that have been proposed to aid the analysis of concurrent, distributed, and loosely-coupled systems. Here, I highlight the most closely related work. Section 5.1 discusses the related work to DEvA and Section 5.2 discusses the related work to DSA. 5.1 Related Work to DEvA Multi-Threaded Systems Event anomalies bear resemblance to a well-known problem in multi-threaded systems| data race. Data races occur when two or more execution threads access the same variable. If at least one of those threads writes to the variable, then dierent runtime scheduling of the threads may result in errors. Even though they are well known, data races are hard to nd and debug because of the nondeterministic nature of the systems that contain 105 them. Race detection techniques for multi-threaded systems deal with explicit concur- rency between threads. On the other hand, DEvA is dealing with implicit concurrency that happens as a result of nondeterminism in the order of event consumptions. There are a number of dynamic race detection techniques for multi-threaded systems [32,43,80]. Most of these techniques are based on checking the happens-before relationship. This relationship introduces a partial ordering among events in a system [59]. There are also dynamic race detection techniques that are based on checking shared memory references and verifying proper locking (lockset checking) on them [94]. Smaragdakis et al. [95] introduced a sound dynamic analysis technique based on a generalization of happens-before, called causally-precedes. Dynamic analysis techniques are dependent on the size of the execution trace and cannot be applied to non-executable programs, such as libraries [76]. Static analysis techniques are either ow-sensitive versions of lockset analysis [41], ow-insensitive [87], or path-sensitive model-based [88] techniques. Most static race de- tection techniques rely on computationally complex analyses such as may-alias. Because of this, they may suer from precision, soundness, or scalability problems. Chord [76] is a ow-insensitive analysis that makes use of conditional-must-not-alias analysis instead of the may-alias analysis to reason about shared memory locations. As described in Section 4.1, I compared DEvA with Chord in the evaluation. Web Systems and Applications There are techniques that try to solve similar problems to EAs in other domains. Zheng and Zhang [107] proposed an approach to extract race problems that may happen in 106 PHP-based web applications. Their approach is a static analysis that reports races that might happen as a result of atomicity violations in accessing external resources such as database tables or les. Their approach also takes explicit concurrency into account. Paleari et al. [83] proposed an analysis that also looks into atomicity violations oc- curring due to the inherent concurrency in web applications. Their approach focuses on interactions between an application and a DBMS. It logs SQL queries and looks for specic interleaving patterns in them. Since this is a dynamic analysis, it is unable to detect races unless they occur in an execution. Petrov et al. introduced WebRacer [85], a dynamic analysis technique for race detection in client-side web applications. WebRacer denes the happens-before rela- tionship to check for nondeterministic accesses to the same object. Raychev et al.'s EventRacer [90] improved WebRacer to decrease the number of reported harmless races. Event-Based Systems Panini [65] is a language that uses asynchronous typed events to make designing for mod- ularity and concurrency easier. Panini only considers the overlapping that may happen between event-handler methods. Each event-handler consumes a specic type of event, similarly to Android. A common practice in event-based systems is that a component has a single event handler, and inside that handler dierent events are processed using dispatching and implicit invocation. Neither EventRacer [90] nor Panini can support such systems. An added disadvantage is that Panini is not applicable to legacy systems written in a general-purpose language such as Java. 107 P [39] is a domain-specic language for event-driven asynchronous systems. A program written in P consists of a set of state machines that interact via events. There is no specic analysis built into P for data races or event anomalies. Instead, a developer has to design policies inside state machines to avoid such situations using P's concepts such as deferred and ignored events. Android Systems CAFA [56] is a dynamic analysis technique targeting Android applications. Android applications react to events that originate from sources such as the user or Android kernel. CAFA considers the causal order between events to check if a race can happen because of the resulting memory accesses. CAFA focuses on uncovering a specic class of problems, UF , in which a memory location is accessed by an event without proper reference checking after it is freed by another event. As discussed in Section 4.1, I compared DEvA toCAFA over the same set of applications. DroidRacer [67] is also a dynamic analysis technique for race detection in Android applications. This technique is based on dening happens-before relationships between Android events and provides a formal semantic model for Android concurrency by study- ing the Android framework. However, DroidRacer neither supports user-dened asyn- chronous callback events nor categorizes races based on their harmfulness. Based on the evaluation results, about 50% of callback-related races reported byDroidRacer were false positives. 108 GATOR [105] is a static analysis technique for Android applications that extracts user-dened GUI-based callback methods. This technique provides an ICFG of the ana- lyzed Android application that shows interactions between user-dened GUI-based call- backs, as well as Android's onCreate and onDestroy life-cycle callbacks. Unlike DEvA, GATOR does not consider other kinds of user-dened callbacks, such as asynchronous callbacks, nor does it take into account the remaining life-cycle callbacks dened by An- droid. Program Slicing Techniques Since I also consider the control dependency relation between event consumptions and eld accesses, one can argue that my work is related to program slicing techniques. Pro- gram slicing techniques try to extract program statements that would aect a variable's value at a specic location in the code or will be aected by the value of a variable at a specic point in code [26, 99, 104]. To compute slices, these techniques must consider control and data dependencies, and usually they make use of graphs such as program dependence graphs (PDG) or system dependence graphs (SDG) [55]. There are some ap- proaches to extract control and data dependencies for concurrent systems [48] and object oriented systems [60,64,100]. For most of the event-based systems and distributed systems, the scalability of a program slicing is the main concern, since a PDG or SDG could easily acquire a very high number of nodes and edges. There are some approaches that attempt to solve the scalability problem through optimization strategies [25]. An eective and scalable program slicing technique can help us to nd dependencies between event consumptions 109 and eld accesses even in the presence of large PDGs and SDGs. There is no slicing technique that has been designed for detecting event anomalies. As discussed in Chapter 2, I built the required PDGs and CFGs on demand to overcome the scalability problem. 5.2 Related Work to DSA Performance Bugs Performance bugs refer to the situation where program executes correctly, but with simple changes in the code the performance of the program will increase. Detecting performance bugs is a hard job, since the performance of the system depends on dierent factors, in- cluding input data and the environment, and cannot be determined automatically without human intervention. It is possible to classify into two main categories existing techniques to detect performance bugs. The rst category of techniques is dynamic analysis tech- niques and the second category is the static analysis techniques. In the dynamic analysis techniques category, Toddler [79] extracts patterns of re- dundant computations by instrumenting the code and monitoring accesses to memory locations. Toddler may introduce an overhead to the runtime of the system that it ana- lyzes. This overhead for the systems that are analyzed by Toddler can sometimes be very high{ up to 50% longer than their normal execution. Toola et al. in [38] proposed a dynamic analysis technique, MemoizeIt, that identies the methods that may benet from storing their computation. Their idea is to compare the input/output pairs for a method in an iterative fashion to check whether there is a repetition pattern for that method. As the number of iterations increases, the overhead 110 of using MemoizeIt will also increase. There are other dynamic analysis techniques that are based on analyzing execution traces [40,106]. The quality of these techniques is highly dependent on the quality of execution traces. To have high-quality execution traces some sort of human intervention is required. Nistor et al. in [78] performed an empirical study of performance bugs in open source software systems, and their study suggests that most of the performance bugs have been discovered through code reasoning and not through proling. This means that static analysis techniques are more likely to nd performance bugs in comparison to dynamic analysis techniques. The diculties in detecting performance bugs forced researchers to focus on specic types of performance bugs. In the static analysis category, Olivo et al. in [81] focused on the performance bugs that are caused by redundant traversals of a data structure and proposed a static analysis technique with the name CLARITY to automatically detect those bugs. A traversal, such as looping over a list of objects, is redundant if the traversed data structure is not changed since the last traverse, and the new traverse can be avoided by storing the result of previous traverses. CLARITY requires manual eort to determine if a method is traversing a data structure with a runtime complexity of in average (n), and this can limit its usability. Chen et al in [34] proposed a static analysis technique for detecting performance anti-patterns in systems that use Object-Relational Mapping or ORM. This means that this approach extracts patterns of interaction with the database that will harm the per- formance of the system. Some of the anti-patterns that are reported by this technique cannot easily be xed, since removing them may require widely used APIs to be changed. 111 None of the existing static analysis techniques for detecting performance bugs consider unnecessary synchronization as a cause of reduction in performance, but DSA does. Compiler Optimization There are several compiler optimization techniques that try to optimize the generated code by a compiler [23, 58, 66, 98, 102]. The goal of these techniques is to modify the generated code by a compiler without changing the semantic and functionality of the code. Usually the optimization involves a data ow and dependency analysis [23,58]. The most well-known optimization techniques deal with optimizing loops in a pro- gram. Loop-Invariant Code Motion technique [73], or LICM technique, optimizes a loop by detecting invariants inside the body of the loop. Moving out those invariants will not change the semantic of the loop and will avoid redundant computations. Strength reduc- tion technique [24] optimizes a loop by replacing a time-consuming statement inside the loop with a lighter one. A common example is to replace a multiplication with an equiva- lent summation. There are also some techniques to automatically parallelize a sequential code at the compile time through parallelizing loops [37, 51]. There are also speculative approaches that use predictions of thread overlapping or predictions of the value of hard- ware registers to speed up computations [33, 52, 89]. In general, full parallelization of a sequential code at the compiler level is a hard job, since it usually needs some sort of feedback from users to decide whether it is safe to parallelize a specic part of the code. I will discus techniques for application level parallelizing in the next subsection. 112 Automatic Parallelization Prabhu et al. in [86] proposed a technique to parallelize a sequential C# code through speculative analysis. In this technique, rst, data dependencies between dierent sequen- tial parts will be extracted. A sequential partp 2 is data dependent on another sequential part p 1 if p 2 will be waiting for p 1 to nish its job and return a value so that p 2 can use that value and start its execution. The idea is that by predicting that value beforehand, we can run the two sequential codes in parallel. Later on, after the execution of p 1 if it is discovered that the predicted value was wrong,p 2 will be rolled back to its starting point and will be executed again with the corrected value. Like DSA, this technique also needs to reason about accesses to memory locations across dierent methods; however, unlike DSA, this technique assumes that the formal parameters of a method do not alias. The authors rely on programmers' annotations concerning the location in the code where it is OK to perform speculative analysis. Roll-backing can cause a reduction in performance and developers should be careful about choosing the locations where speculations should be done. This will harm the usability of this approach. Tournavitis et al. in [101] proposed a hybrid dynamic/static analysis technique to parallelize sequential code. In this technique, data dependency relations will be extracted from the runtime of the system through proling. These data dependencies provide insight for a static analysis technique to better determine possible parallelizations for the loops that exist in the code. Then, by getting feedback from users about protable loop parallelizations and applying a machine learning approach, the parallelization will 113 be performed. Because this approach just focuses on parallelization of the loops and relies heavily on the user's feedback, this may limit the usability of this approach. DSA will also recommend parallelizations to the user but without requiring feedback from the user. This makes DSA more automatic than existing techniques. 114 Chapter 6 Conclusion Event-based interaction is an attractive paradigm because of its promise of highly exible systems in which components are decoupled and can \come and go" as needed. However, this paradigm could introduce complications into the system because of the nature of underlying event processing mechanisms. In an EB system, components make use of events for inter-component interactions, and they make use of method calls to modularize the intra-component interactions. In inter-component interactions, the order in which events are sent, received, and processed can be highly unpredictable. This is caused by implicit invocation and implicit concurrency, which, in turn, may result in event anomalies (EAs). Also, in intra-component interactions, it is possible that dierent sequences of method calls that were initiated from the same method introduce independent execution paths into the system, meaning that those sequences of method calls do not access a shared memory location. Having these execution paths synchronized with each other can harm the performance of the system, since they can be executed concurrently. In this situation we will have synchronization anomalies (SAs). 115 Several existing approaches have targeted dierent facets of EAs and SAs. Existing approaches that target EAs have notable shortcomings. A long-standing class of data-race detection techniques for concurrent systems is unable to deal with the implicit concurrency and implicit invocation present in EBSs. Approaches that target EBSs directly have tended (1) to rely on dynamic analysis with its inherent limitations, (2) to use new language constructs that cannot be applied to existing systems, or (3) to exploit domain- specic characteristics that lend themselves to simplifying assumptions holding only in relatively narrow settings. Existing approaches that target problems similar to SAs also have some shortcomings. They have tended (1) to rely on dynamic analysis and proling with their well-known limitations, (2) to heavily depend on human intervention, or (3) to make assumptions to simplify the problem. To address problems related to EAs, I have developed DEvA, a new static analysis technique that targets EBSs. DEvA receives as its input the source code of an EB system along with some information about the system's underlying event-handling mechanism, and it will report all the EAs that exist in that system. DEvA's approach is a four-phase approach. In the rst phase, extraction, the analysis identies all of the consumed event types (CETs) and elds accessed by each component of the system. In the second phase, causation, a path-based analysis will be performed using information from the extraction phase to determine if there is a connection between the CETs and accessed elds. In the third phase, joining, the analysis identies CETs that will lead to an access of the same eld; that is, a possible EA. The CETs and elds identied in the last phase are returned to the developer as potential EAs for more investigation. In the last phase, ranking, DEvA provides a ranking of EAs based on their potential harmfulness. This 116 ranking is based on EAs' potential harmfulness at the level of component interactions. This will reduce the amount of investigation needed to be conducted by developers. I have designed DEvA to be scalable by building necessary graphs when needed. Also my ranking is customizable in case the priorities of system changes. My empirical evaluation shows that DEvA is ecient and scalable while exhibiting high precision with respect to the manually identied EAs in 20 subject systems. I have demonstrated that DEvA can be used in tandem with existing analysis techniques such as Chord [76]. A comparison with CAFA [56], a recent technique that targets a specic class of EAs, UF , shows that, in most cases, DEvA is able to detect a greater number of such anomalies in a fraction of the time. DEvA was also able to nd previously unknown bugs in well-known Android libraries and applications. To address problems related to SAs, I have developed DSA, a new static analysis technique to detect SAs from the source code of a Java-based system. DSA's approach is also a three-phase approach. In the rst phase, extraction, all the locations in the code that access a memory location will be extracted. In addition to this, the sequences of method calls that start from each method of each component of the system will be identied in this phase. In the second phase, checking, each non-primitive argument of each method will be checked to see whether it is changed inside the method. In the third phase, categorization, the analysis will perform a categorization for each method of the independent sequences of method calls that start from that method. The independence of sequences of method calls will be determined based on the information that was obtained in the previous phases of the analysis. The output of this phase will be the list of 117 independent sequences of method calls that are synchronized, or in other words, the list of SAs. I have evaluated DSA both empirically and analytically. My empirical evaluation shows that SAs are a common problem and can occur in any randomly chosen system. Also, DSA is ecient and scalable while exhibiting high precision. My analytical eval- uation of DSA provides some guidelines for the situations where removing a SA can be more benecial. Those guidelines suggested that we can rank SAs based on the length of sequences of method calls. I performed several experiments over nine of the subject systems to verify the applicability of this metric and to check whether there could be other more benecial metrics with which to rank the SAs. The experiments showed that the longest sequence of method calls is the most benecial metric to rank SAs and can introduce an enhancement of, on average, 10% in the performance of the subject systems. In summary, this dissertation's contributions are: 1. Design and evaluation of DEvA|a static analysis technique that achieves high pre- cision and recall (i.e., a measure of completeness) when extracting event anomalies from a broad range of existing EBSs. 2. Design and evaluation of a ranking approach to help developers rank event anomalies based on their harmfulness. 3. Design and evaluation of DSA|a static analysis technique that can provide useful information to users concerning possible points of performance improvement in the code. 118 4. Creation of an analytical evaluation to determine situations where removing syn- chronization anomalies is most benecial. 6.1 Future Work There are a number of remaining research challenges that can be addressed in future work. I divide future work into two main categories: future work related to event anomalies and future work related to synchronization anomalies. With regards to event anomalies, there are three main directions that it would be benecial for future work to take: 1. Monitoring, storing, and enabling the replay of event sequences in order to reproduce anomalies will be useful in investigation of the eects of anomalies. In general, it would be useful to provide a suite of ready-made remedies to dierent types of event anomalies, as well as a set of automated wizards to guide developers in selecting and applying these remedies. 2. As discussed in Section 3.1, feedback from developers and designers of the systems is necessary to better determine the harmfulness of an anomaly. It is possible to enhance DEvA with a machine learning approach based on previous feedback from developers and designers. 3. As discussed in Section 4.3, DEvA assumes that callback methods are dened in the source code of an Android application to handle events that can occur nondeter- ministically. This assumption is not always right. Existing techniques for callback 119 detection also have other limitations as discussed in Section 5.1. Proposing a new callback detection that enhances the relevant algorithm used by DEvA will be useful. There are also three main directions that it would be benecial for future work on synchronization anomalies to take: 1. My evaluations have indicated that many of the methods that contain SAs are ambiguous interfaces. By implementing some small changes in DSA, we can use it for detecting ambiguous interfaces. This can help architectural smell and recovery research [62], [45]. 2. Proposing a self-adaptive framework based on the information that was provided in the analytical evaluation of DSA will be useful. In this framework, we can rst analyze a system using DSA to extract its SAs. Then we can instrument the source code of the system to implant gauges in the methods that contain SAs in order to measure the call rates for the sub-methods and also the execution time of those sub-methods. If those call rates and execution times follow the guidelines that were provided by DSA, the framework will execute the sub-methods concurrently. Using this framework, we can greatly improve the performance of the system. 3. Another approach can involve proposing a speculative framework to predict the rates of method calls and their execution time based on the input data and the environment of the system. Using these speculations, we can determine which SAs we should remove in order to improve the performance of the system the most. 120 References [1] ConnectBot, Version 1.7: http://code.google.com/p/connectbot/, 2011. [2] ZXing, Version 4.5.1: http://code.google.com/p/zxing/, 2011. [3] FBReader, Version 1.9.6.1: http://fbreader.org/fbreaderj, 2012. [4] MyTracks, Version 1.1.7: http://code.google.com/p/mytracks/, 2012. [5] ToDoWidget, Version 1.1.7: https://github.com/chrispbailey/todo-list-widget, 2012. [6] VLC, Version 0.2.0: http://www.videolan.org/vlc/, 2012. [7] Firefox, Version 25: http://www.mozilla.org/en-us/refox/fx/, 2013. [8] Ignite Realtime: Spark IM Client, 2013. [9] JFreeChart: http://www.jfree.org/jfreechart/, 2013. [10] JMSChat: https://github.com/julentv/chatjms.git, 2013. [11] Notes on the sum and maximum of independent exponentially distributed ran- dom variables with dierent scale parameters. http://arxiv.org/abs/1307.3945, 2013. [12] Project.net: https://www.project.net/, 2013. [13] Spring Framework: http://projects.spring.io/spring-framework/, 2013. [14] UF bug 1070795 in refox for android: https://bugzilla.mozilla.org/show bug.cgi?id=1070795, 2013. [15] UF bug 923407 in refox for android: https://bugzilla.mozilla.org/show bug.cgi?id=923407, 2013. [16] Discussion about the bug in android.support.v4. http://stackoverflow.com/questions/12265611/ dialogfragment-nullpointerexception-support-library, 2014. [17] Chronicle:https://github.com/peter-lawrey/java-chronicle.git, 2015. 121 [18] DEvA's Implementation. http://www-scf.usc.edu/ ~ gsafi/FSE2015/DEvA/, 2015. [19] JavaGame: https://github.com/redomar/javagame.git, 2015. [20] MarkShark: https://github.com/donnervoegel/java.git, 2015. [21] Results for DEvA's Evaluation. http://www-scf.usc.edu/ ~ gsafi/FSE2015/ AnalysisResults/, 2015. [22] WebSocket: https://github.com/tootallnate/java-websocket.git, 2015. [23] Alfred Aho et al. Compilers, Principles, Techniques, and Tools. Addison-Wesley, 1986. [24] David F Bacon, Susan L Graham, and Oliver J Sharp. Compiler transformations for high-performance computing. ACM Computing Surveys (CSUR), 26(4):345{ 420, 1994. [25] D. Binkley et al. Empirical study of optimization techniques for massive slicing. ACM Transactions on Programming Languages and Systems (TOPLAS), 2007. [26] D. Binkley and M. Harman. A Survey of Empirical Results on Program Slicing. Advances in Computers: Advances in Software Engineering, 2004. [27] F. Biscotti et al. Market Share: AIM and Portal Software, Worldwide, 2009. Gart- ner Market Research Report, April 2010. [28] F. Biscotti and A Raina. Market Share Analysis: Application Infrastructure and Middleware Software, Worldwide, 2011. Gartner Market Research Report, April 2012. [29] Bruno Blanchet. Escape analysis for object-oriented languages: Application to java. In Proceedings of the 14th ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications, OOPSLA '99, pages 20{34, New York, NY, USA, 1999. ACM. [30] Hans-J. Boehm. Nondeterminism is unavoidable, but data races are pure evil. In Proceedings of the 2012 ACM Workshop on Relaxing Synchronization for Multicore and Manycore Scalability, RACES '12, pages 9{14, New York, NY, USA, 2012. ACM. [31] J Brederekea and R Gotzheinb. Increasing the concurrency in estelle. 1993. [32] Yan Cai and W. K. Chan. Loft: Redundant synchronization event removal for data race detection. In Proceedings of the 2011 IEEE 22Nd International Symposium on Software Reliability Engineering, ISSRE '11, pages 160{169, Washington, DC, USA, 2011. IEEE Computer Society. 122 [33] Simone Campanoni, Glenn Holloway, Gu-Yeon Wei, and David Brooks. Helix- up: Relaxing program semantics to unleash parallelization. In Proceedings of the 13th Annual IEEE/ACM International Symposium on Code Generation and Opti- mization, CGO '15, pages 235{245, Washington, DC, USA, 2015. IEEE Computer Society. [34] Tse-Hsun Chen, Weiyi Shang, Zhen Ming Jiang, Ahmed E. Hassan, Mohamed Nasser, and Parminder Flora. Detecting performance anti-patterns for applications developed using object-relational mapping. In Proceedings of the 36th International Conference on Software Engineering, ICSE 2014, pages 1001{1012, New York, NY, USA, 2014. ACM. [35] JM Correira and F. Biscotti. Market Share: AIM and Portal Software, Worldwide, 2005. Gartner Market Research Report, 2006. [36] Gianpaolo Cugola et al. The JEDI Event-Based Infrastructure and Its Application to the Development of the OPSS WFMS. IEEE TSE, 2001. [37] Chirag Dave, Hansang Bae, Seung-Jai Min, Seyong Lee, Rudolf Eigenmann, and Samuel Midki. Cetus: A source-to-source compiler infrastructure for multicores. Computer, 42(12):36{42, 2009. [38] Luca Della Toola, Michael Pradel, and Thomas R. Gross. Performance problems you can x: A dynamic analysis of memoization opportunities. In Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2015, pages 607{622, New York, NY, USA, 2015. ACM. [39] Ankush Desai, Vivek Gupta, Ethan Jackson, Shaz Qadeer, Sriram Rajamani, and Damien Zuerey. P: Safe asynchronous event-driven programming. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI '13, pages 321{332, New York, NY, USA, 2013. ACM. [40] Amer Diwan, Matthias Hauswirth, Todd Mytkowicz, and Peter F. Sweeney. Trace- analyzer: A system for processing performance traces. Softw. Pract. Exper., 41(3):267{282, March 2011. [41] Dawson Engler and Ken Ashcraft. Racerx: eective, static detection of race con- ditions and deadlocks. In ACM SIGOPS Operating Systems Review, volume 37, pages 237{252. ACM, 2003. [42] P.T. Eugster et al. The Many Faces of Publish/Subscribe. ACM Computing Surveys (CSUR), 2003. [43] Cormac Flanagan and Stephen N. Freund. Fasttrack: Ecient and precise dynamic race detection. In Proceedings of the 2009 ACM SIGPLAN Conference on Program- ming Language Design and Implementation, PLDI '09, pages 121{133, New York, NY, USA, 2009. ACM. 123 [44] M. Fowler and K. Beck. Refactoring: Improving the Design of Existing Code. Object Technology Series. Addison-Wesley, 1999. [45] J. Garcia, I. Ivkovic, and N. Medvidovic. A comparative analysis of software ar- chitecture recovery techniques. In Automated Software Engineering (ASE), 2013 IEEE/ACM 28th International Conference on, pages 486{496, Nov 2013. [46] Joshua Garcia et al. Toward a Catalogue of Architectural Bad Smells. In Interna- tional Conference on Quality of Software Architectures, 2009. [47] Joshua Garcia, Daniel Popescu, Gholamreza Sa, William G. J. Halfond, and Ne- nad Medvidovic. Identifying message ow in distributed event-based systems. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2013, pages 367{377, New York, NY, USA, 2013. ACM. [48] D. Gihorn and C. Hammer. An evaluation of slicing algorithms for concurrent programs. In Proc. SCAM, 2007. [49] Donald Gross, John F. Shortle, James M. Thompson, and Carl M. Harris. Funda- mentals of Queueing Theory. Wiley-Interscience, New York, NY, USA, 3rd edition, 2004. [50] Varun Gupta, Mor Harchol-balter, Karl Sigman, and Ward Whitt. Analysis of join-theshortest-queue routing for web server farms. In In PERFORMANCE 2007. IFIP WG 7.3 International Symposium on Computer Modeling, Measurement and Evaluation, 2007. [51] Mary W. Hall, S.P. Amarasinghe, B.R. Murphy, Shih-Wei Liao, and M.S. Lam. Detecting coarse - grain parallelism using an interprocedural parallelizing compiler. In Supercomputing, 1995. Proceedings of the IEEE/ACM SC95 Conference, pages 49{49, 1995. [52] Liang Han, Wei Liu, and James M. Tuck. Speculative parallelization of partial reduction variables. In Proceedings of the 8th Annual IEEE/ACM International Symposium on Code Generation and Optimization, CGO '10, pages 141{150, New York, NY, USA, 2010. ACM. [53] Mary Jean Harrold, Gregg Rothermel, and Saurabh Sinha. Computation of inter- procedural control dependence. In Proceedings of the 1998 ACM SIGSOFT Inter- national Symposium on Software Testing and Analysis, ISSTA '98, pages 11{20, New York, NY, USA, 1998. ACM. [54] Manfred Hauswirth and Mehdi Jazayeri. A Component and Communication Model for Push Systems. In ESEC/FSE. Springer, 1999. [55] S. Horwitz et al. Interprocedural slicing using dependence graphs. In Proc. PLDI '88. ACM, 1988. 124 [56] Chun-Hung Hsiao, Jie Yu, Satish Narayanasamy, Ziyun Kong, Cristiano L. Pereira, Gilles A. Pokam, Peter M. Chen, and Jason Flinn. Race detection for event-driven mobile applications. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI '14, pages 326{336, New York, NY, USA, 2014. ACM. [57] Guoliang Jin, Linhai Song, Xiaoming Shi, Joel Scherpelz, and Shan Lu. Under- standing and detecting real-world performance bugs. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementa- tion, PLDI '12, pages 77{88, New York, NY, USA, 2012. ACM. [58] David J Kuck, Robert H Kuhn, David A Padua, Bruce Leasure, and Michael Wolfe. Dependence graphs and compiler optimizations. In Proceedings of the 8th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, pages 207{218. ACM, 1981. [59] Leslie Lamport. Time, clocks, and the ordering of events in a distributed system. Commun. ACM, 21(7):558{565, July 1978. [60] Loren Larsen and Mary Jean Harrold. Slicing object-oriented software. In Proc. 18th ICSE, pages 495{505, 1996. [61] Thomas D. LaToza and Brad A. Myers. Developers Ask Reachability Questions. In ICSE, 2010. [62] Duc Minh Le, Pooyan Behnamghader, Joshua Garcia, Daniel Link, Arman Shah- bazian, and Nenad Medvidovic. An empirical study of architectural change in open-source software systems. In Proceedings of the 12th Working Conference on Mining Software Repositories, MSR '15, pages 235{245, Piscataway, NJ, USA, 2015. IEEE Press. [63] Ond rej Lhot ak and Laurie Hendren. Scaling java points-to analysis using spark. In Proceedings of the 12th International Conference on Compiler Construction, CC'03, pages 153{169, Berlin, Heidelberg, 2003. Springer-Verlag. [64] D. Liang and MJ Harrold. Slicing objects using system dependence graphs. In Proc. ICSM, 1998. [65] Yuheng Long, Sean L. Mooney, Tyler Sondag, and Hridesh Rajan. Implicit invo- cation meets safe, implicit concurrency. In Proceedings of the Ninth International Conference on Generative Programming and Component Engineering, GPCE '10, pages 63{72, New York, NY, USA, 2010. ACM. [66] Fabio Luporini, Ana Lucia Varbanescu, Florian Rathgeber, Gheorghe-Teodor Bercea, J. Ramanujam, David A. Ham, and Paul H. J. Kelly. Cross-loop opti- mization of arithmetic intensity for nite element local assembly. ACM Trans. Archit. Code Optim., 11(4):57:1{57:25, January 2015. 125 [67] Pallavi Maiya, Aditya Kanade, and Rupak Majumdar. Race detection for android applications. In Proceedings of the 35th ACM SIGPLAN Conference on Program- ming Language Design and Implementation, PLDI '14, pages 316{325, New York, NY, USA, 2014. ACM. [68] S. Malek, N. Medvidovic, and M. Mikic-Rakic. An extensible framework for improv- ing a distributed software system's deployment architecture. Software Engineering, IEEE Transactions on, 38(1):73{100, Jan 2012. [69] Sam Malek, Marija Mikic-Rakic, and Nenad Medvidovic. A style-aware architec- tural middleware for resource-constrained, distributed systems. IEEE Trans. Softw. Eng., 31(3):256{272, March 2005. [70] Ravi Mangal, Xin Zhang, Aditya V. Nori, and Mayur Naik. A user-guided approach to program analysis. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2015, pages 462{473, New York, NY, USA, 2015. ACM. [71] N. Medvidovic et al. The Role of Middleware in Architecture-Based Software De- velopment. Int. J. of Softw. Eng. and Knowl. Eng., 2003. [72] Nenad Medvidovic, David S. Rosenblum, and Richard N. Taylor. A language and environment for architecture-based software development and evolution. In Pro- ceedings of the 21st International Conference on Software Engineering, ICSE '99, pages 44{53, New York, NY, USA, 1999. ACM. [73] Etienne Morel and Claude Renvoise. Global optimization by suppression of partial redundancies. Communications of the ACM, 22(2):96{103, 1979. [74] G. M uhl. Large-Scale Content-Based Publish/Subscribe Systems. PhD thesis, Darmstadt University of Technology, 2002. [75] G. M uhl et al. Distributed Event-Based Systems. Springer-Verlag New York, Inc., 2006. [76] Mayur Naik, Alex Aiken, and John Whaley. Eective static race detection for java. In Proceedings of the 2006 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI '06, pages 308{319, New York, NY, USA, 2006. ACM. [77] Satish Narayanasamy, Zhenghao Wang, Jordan Tigani, Andrew Edwards, and Brad Calder. Automatically classifying benign and harmful data races using replay anal- ysis. In Proceedings of the 2007 ACM SIGPLAN Conference on Programming Lan- guage Design and Implementation, PLDI '07, pages 22{31, New York, NY, USA, 2007. ACM. [78] Adrian Nistor, Tian Jiang, and Lin Tan. Discovering, reporting, and xing perfor- mance bugs. In Proceedings of the 10th Working Conference on Mining Software Repositories, MSR '13, pages 237{246, Piscataway, NJ, USA, 2013. IEEE Press. 126 [79] Adrian Nistor, Linhai Song, Darko Marinov, and Shan Lu. Toddler: Detecting performance problems via similar memory-access patterns. In Proceedings of the 2013 International Conference on Software Engineering, ICSE '13, pages 562{571, Piscataway, NJ, USA, 2013. IEEE Press. [80] Robert O'Callahan and Jong-Deok Choi. Hybrid dynamic data race detection. In Proceedings of the Ninth ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP '03, pages 167{178, New York, NY, USA, 2003. ACM. [81] Oswaldo Olivo, Isil Dillig, and Calvin Lin. Static detection of asymptotic perfor- mance bugs in collection traversals. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2015, pages 369{378, New York, NY, USA, 2015. ACM. [82] Peyman Oreizy, Nenad Medvidovic, and Richard N. Taylor. Architecture-based runtime software evolution. In Proceedings of the 20th International Conference on Software Engineering, ICSE '98, pages 177{186, Washington, DC, USA, 1998. IEEE Computer Society. [83] Roberto Paleari, Davide Marrone, Danilo Bruschi, and Mattia Monga. On race vulnerabilities in web applications. In Detection of Intrusions and Malware, and Vulnerability Assessment, pages 126{142. Springer, 2008. [84] Athanasios Papoulis. Probability, random variables, and stochastic processes. McGraw-Hill series in electrical engineering. McGraw-Hill, New York, 1991. [85] Boris Petrov, Martin Vechev, Manu Sridharan, and Julian Dolby. Race detection for web applications. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI '12, pages 251{262, New York, NY, USA, 2012. ACM. [86] Prakash Prabhu, Ganesan Ramalingam, and Kapil Vaswani. Safe programmable speculative parallelism. In Proceedings of the 31st ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI '10, pages 50{61, New York, NY, USA, 2010. ACM. [87] Polyvios Pratikakis, Jerey S. Foster, and Michael Hicks. Locksmith: Context- sensitive correlation analysis for race detection. In Proceedings of the 2006 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI '06, pages 320{331, New York, NY, USA, 2006. ACM. [88] Shaz Qadeer and Dinghao Wu. Kiss: keep it simple and sequential. In ACM SIGPLAN Notices, volume 39, pages 14{24. ACM, 2004. 127 [89] Arun Raman, Hanjun Kim, Thomas R. Mason, Thomas B. Jablin, and David I. August. Speculative parallelization using software multi-threaded transactions. In Proceedings of the Fifteenth Edition of ASPLOS on Architectural Support for Pro- gramming Languages and Operating Systems, ASPLOS XV, pages 65{76, New York, NY, USA, 2010. ACM. [90] Veselin Raychev, Martin Vechev, and Manu Sridharan. Eective race detection for event-driven programs. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applica- tions, OOPSLA '13, pages 151{166, New York, NY, USA, 2013. ACM. [91] Kai Sachs et al. Performance Evaluation of Message-Oriented Middleware Using the SPECjms2007 Benchmark. Performance Evaluation, 2009. [92] Kai Sachs et al. Benchmarking Publish/Subscribe-Based Messaging Systems. In Proc. BenchmarX, 2010. [93] Alexandru Salcianu and Martin Rinard. Pointer and escape analysis for multi- threaded programs. In Proceedings of the Eighth ACM SIGPLAN Symposium on Principles and Practices of Parallel Programming, PPoPP '01, pages 12{23, New York, NY, USA, 2001. ACM. [94] Stefan Savage, Michael Burrows, Greg Nelson, Patrick Sobalvarro, and Thomas Anderson. Eraser: A dynamic data race detector for multithreaded programs. ACM Trans. Comput. Syst., 15(4):391{411, November 1997. [95] Yannis Smaragdakis, Jacob Evans, Caitlin Sadowski, Jaeheon Yi, and Cormac Flanagan. Sound predictive race detection in polynomial time. In Proceedings of the 39th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL '12, pages 387{400, New York, NY, USA, 2012. ACM. [96] Richard N. Taylor, Nenad Medvidovic, Kenneth M. Anderson, E. James Whitehead, Jr., and Jason E. Robbins. A component- and message-based architectural style for gui software. In Proceedings of the 17th International Conference on Software Engineering, ICSE '95, pages 295{304, New York, NY, USA, 1995. ACM. [97] RN Taylor et al. A Component- and Message-Based Architectural Style for GUI Software. IEEE TSE, 1996. [98] Xiaonan Tian, Rengan Xu, Yonghong Yan, Sunita Chandrasekaran, Deepak Eachempati, and Barbara Chapman. Compiler transformation of nested loops for general purpose gpus. Concurrency and Computation: Practice and Experience, 2015. [99] F. Tip. A Survey of Program Slicing Techniques. Journal of Programming Lan- guages, 1995. [100] P. Tonella et al. Flow Insensitive C++ Pointers and Polymorphism Analysis and its application to slicing. In Proc. ICSE, pages 433{443, 1997. 128 [101] Georgios Tournavitis, Zheng Wang, Bj orn Franke, and Michael F.P. O'Boyle. To- wards a holistic approach to auto-parallelization: Integrating prole-driven paral- lelism detection and machine-learning based mapping. In Proceedings of the 30th ACM SIGPLAN Conference on Programming Language Design and Implementa- tion, PLDI '09, pages 177{187, New York, NY, USA, 2009. ACM. [102] Spyridon Triantafyllis, Manish Vachharajani, Neil Vachharajani, David August, et al. Compiler optimization-space exploration. In Code Generation and Optimiza- tion, 2003. CGO 2003. International Symposium on, pages 204{215. IEEE, 2003. [103] Raja Vall ee-Rai et al. Soot - a Java Bytecode Optimization Framework. In Con- ference of the Centre for Advanced Studies on Collaborative research, 1999. [104] B. Xu et al. A Brief Survey of Program Slicing. ACM SIGSOFT Software Engi- neering Notes, 2005. [105] Shengqian Yang, Dacong Yan, Haowei Wu, Yan Wang, and Atanas Rountev. Static control- ow analysis of user-driven callbacks in Android applications. In Interna- tional Conference on Software Engineering, 2015. [106] Xiao Yu, Shi Han, Dongmei Zhang, and Tao Xie. Comprehending performance from real-world execution traces: A device-driver case. In Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS '14, pages 193{206, New York, NY, USA, 2014. ACM. [107] Yunhui Zheng and Xiangyu Zhang. Static detection of resource contention prob- lems in server-side scripts. In Proceedings of the 34th International Conference on Software Engineering, ICSE '12, pages 584{594, Piscataway, NJ, USA, 2012. IEEE Press. 129

Abstract (if available)

Abstract The event-based paradigm allows developers to design and build systems that are highly flexible and can be easily adapted. There are two main complications that can occur in the systems that are based on this paradigm. The first complication concerns inter-component interactions. Events that are used by components for this purpose are sent, received, and processed nondeterministically, due to the systems' reliance on implicit invocation and implicit concurrency. This nondeterminism can lead to event anomalies, which occur when an event-based system receives multiple events that lead to the write of a shared field or memory location. Event anomalies can lead to unreliable, error-prone, and hard-to-debug behavior in an event-based system. The second complication concerns intra-component interactions that usually occur through method calls. Each sequence of method calls introduces an execution path to the system. It is possible that there exist multiple execution paths that are not accessing the same memory locations or sharing data but are unnecessarily synchronized with each other despite the fact that they can be executed concurrently. I call this situation synchronization anomalies. ❧ To detect event anomalies, this dissertation presents a new static analysis technique, DEvA, for automatically detecting event anomalies. DEvA has been evaluated on a set of open-source event-based systems against a state-of-the-art technique for detecting data races in multi-threaded systems and a recent technique for solving a similar problem with event processing in Android applications. DEvA exhibited high precision with respect to manually constructed ground truths and was able to locate event anomalies that had not been detected by the existing solutions. ❧ Also, this dissertation presents a new static analysis technique, DSA, for automatically detecting synchronization anomalies. I have evaluated DSA both empirically and analytically. My empirical evaluation shows that synchronization anomalies are a common problem and can occur in any randomly chosen system. Also, DSA is efficient and scalable while exhibiting high precisions after being applied to fourteen subject systems. The analytical evaluation of DSA provides guidelines for the situations where removing a synchronization anomaly can be more beneficial. By removing just one synchronization anomaly from nine of our subject systems based on the provided guidelines, on average there was an enhancement of 10% in the performance of those systems.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Reducing inter-component communication vulnerabilities in event-based systems

PDF

Automatic detection and optimization of energy optimizable UIs in Android applications using program analysis

PDF

Constraint-based program analysis for concurrent software

PDF

Detection, localization, and repair of internationalization presentation failures in web applications

PDF

Detecting SQL antipatterns in mobile applications

PDF

Proactive detection of higher-order software design conflicts

PDF

Deriving component‐level behavior models from scenario‐based requirements

PDF

Energy optimization of mobile applications

PDF

Automated repair of presentation failures in Web applications using search-based techniques

PDF

Prediction of energy consumption behavior in component-based distributed systems

PDF

Behavior-based approaches for detecting cheating in online games

PDF

Design-time software quality modeling and analysis of distributed software-intensive systems

PDF

Techniques for methodically exploring software development alternatives

PDF

Analysis of embedded software architecture with precedent dependent aperiodic tasks

PDF

A user-centric approach for improving a distributed software system's deployment architecture

PDF

Software connectors for highly distributed and voluminous data-intensive systems

PDF

Automated repair of layout accessibility issues in mobile applications

PDF

Event detection and recounting from large-scale consumer videos

PDF

Vision-based studies for structural health monitoring and condition assesment

PDF

Policy based data placement in distributed systems

Asset Metadata

Creator Safi, Gholamreza (author)

Core Title Detecting anomalies in event-based systems through static analysis

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Computer Science

Publication Date 09/22/2016

Defense Date 01/19/2016

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag distributed systems,event anomalies,event-based systems,OAI-PMH Harvest,performance bug detection,program analysis,queuing theory,race detection,synchronization anomalies

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Medvidovic, Nenad (committee chair), Gupta, Sandeep K. (committee member), Halfond, William G. J. (committee member)

Creator Email gsafi@usc.edu,rezasafi@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c40-305111

Unique identifier UC11280466

Identifier etd-SafiGholam-4794.pdf (filename),usctheses-c40-305111 (legacy record id)

Legacy Identifier etd-SafiGholam-4794.pdf

Dmrecord 305111

Document Type Dissertation

Format application/pdf (imt)

Rights Safi, Gholamreza

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA