University of Southern California Dissertations and Theses

QoS-based distributed design of streaming and data distribution systems

(USC Thesis Other)

QoS-based distributed design of streaming and data distribution systems

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content QOS-BASED DISTRIBUTED DESIGN OF STREAMING AND DATA DISTRIBUTION SYSTEMS by Lik Hang Chow A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (COMPUTER SCIENCE) May 2010 Copyright 2010 Lik Hang Chow Table of Contents List of Tables iv List of Figures v List of Algorithms viii Abstract ix Chapter 1: Introduction 1 1.1 Improving BitTorrent . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 BitTorrent Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 Multi-path Streaming Optimization . . . . . . . . . . . . . . . . . . . . 5 1.4 Encoded Multi-path Live Streaming . . . . . . . . . . . . . . . . . . . 8 Chapter 2: Related Work 11 2.1 BitTorrent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1.1 BitTorrent Modeling . . . . . . . . . . . . . . . . . . . . . . . 12 2.2 Multi-path Streaming . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.2.1 Multi-path Live Streaming . . . . . . . . . . . . . . . . . . . . 17 Chapter 3: Improving BitTorrent: a Simple Approach 19 3.1 BitTorrent Protocol Description . . . . . . . . . . . . . . . . . . . . . . 19 3.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.2.1 Motivating Example . . . . . . . . . . . . . . . . . . . . . . . 21 3.3 Proposed Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.4 Seeding Approach Evaluation . . . . . . . . . . . . . . . . . . . . . . . 25 3.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 Chapter 4: BitTorrent: an Extensible Heterogeneous Model and Applications 34 4.1 BitTorrent Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4.1.1 Seeds and Free-Riders . . . . . . . . . . . . . . . . . . . . . . 37 4.1.2 More Realistic Model . . . . . . . . . . . . . . . . . . . . . . . 39 4.2 Model Validation and Insight . . . . . . . . . . . . . . . . . . . . . . . 44 ii 4.2.1 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.2.2 Real-world Experiments . . . . . . . . . . . . . . . . . . . . . 53 4.3 Model Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Chapter 5: Multi-path Streaming Optimization 72 5.1 Load Distribution Problem . . . . . . . . . . . . . . . . . . . . . . . . 72 5.1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 5.1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 5.1.3 Optimization Objectives . . . . . . . . . . . . . . . . . . . . . 76 5.2 Performance Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 Chapter 6: Encoded Multi-path Streaming for Real-time Live Streaming Applications 92 6.1 Asymptotically Optimal Load Splitting . . . . . . . . . . . . . . . . . . 92 6.1.1 Asymptotically Optimal Validation . . . . . . . . . . . . . . . . 93 6.2 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 6.2.1 EMS Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.2.2 Online Load Splitting (OLS) . . . . . . . . . . . . . . . . . . . 97 6.2.3 FEC Redundancy Adaptation . . . . . . . . . . . . . . . . . . . 98 6.2.4 FEC Group Size Adaptation . . . . . . . . . . . . . . . . . . . 101 6.3 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . 101 6.3.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 6.3.2 Comparison with TCP-based DMP Streaming . . . . . . . . . . 104 6.3.3 Comparison with Model-based UDP Multi-path Streaming . . . 108 6.3.4 Effectiveness of OLS . . . . . . . . . . . . . . . . . . . . . . . 112 6.3.5 Effectiveness of FEC Redundancy Adaptation . . . . . . . . . . 115 6.3.6 Effectiveness of FEC Group Size Adaptation . . . . . . . . . . 116 Chapter 7: Conclusions 121 References 123 iii List of Tables 3.1 Simulation Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.2 Class Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 4.1 Notation Used in Our model . . . . . . . . . . . . . . . . . . . . . . . 35 4.2 Simulation Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.3 Class Description (Two Contributing Classes) . . . . . . . . . . . . . . 46 4.4 Class Description (with Free-Riders) . . . . . . . . . . . . . . . . . . . 49 4.5 Fraction of Free-Riders . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.6 Download Time (min) using the Optimal Threshold (CL) . . . . . . . . 69 5.1 Default Experimental Settings . . . . . . . . . . . . . . . . . . . . . . 80 5.2 FEC overhead needed to achieve information loss rate of≤ 0.1%; Path 1 loss rate = 5%. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 6.1 Optimal splitting (3 paths) . . . . . . . . . . . . . . . . . . . . . . . . 94 6.2 Default EMS Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 6.3 FEC settings used by EMS:(K,N) and redundancy . . . . . . . . . . . . 107 iv List of Figures 3.1 Effect of Seeding on CL and FR . . . . . . . . . . . . . . . . . . . . . 22 3.2 Example Chunk Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.3 Different Seeding Schemes . . . . . . . . . . . . . . . . . . . . . . . . 27 3.4 Large View Exploit: Download Rate . . . . . . . . . . . . . . . . . . . 29 4.1 Slow Node Imperfect Clustering . . . . . . . . . . . . . . . . . . . . . 41 4.2 Different Mix of Two CL Classes . . . . . . . . . . . . . . . . . . . . . 47 4.3 Imperfect Clustering Fraction . . . . . . . . . . . . . . . . . . . . . . . 48 4.4 Different Mix of Three Contributing Classes . . . . . . . . . . . . . . . 49 4.5 Different Seed Time (Three Classes) . . . . . . . . . . . . . . . . . . . 50 4.6 Different Fraction of Free-Riders . . . . . . . . . . . . . . . . . . . . . 51 4.7 Different Seed Time (with FR) . . . . . . . . . . . . . . . . . . . . . . 52 4.8 Different Peer Set Size . . . . . . . . . . . . . . . . . . . . . . . . . . 53 4.9 Different Mix of Two CL Classes (PlanetLab) . . . . . . . . . . . . . . 55 4.10 Imperfect Clustering Fraction (PlanetLab) . . . . . . . . . . . . . . . . 56 4.11 Different Seed Time (PlanetLab) . . . . . . . . . . . . . . . . . . . . . 57 4.12 Different Peer Set Size (PlanetLab) . . . . . . . . . . . . . . . . . . . . 58 4.13 Different Mix of Three CL Classes (PlanetLab) . . . . . . . . . . . . . 59 4.14 Different File Size (PlanetLab) . . . . . . . . . . . . . . . . . . . . . . 60 4.15 Fast Nodes Download Rate vs Download % (PlanetLab) . . . . . . . . . 61 4.16 Large View Exploit . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 v 4.17 Different Threshold (Free-Riders) . . . . . . . . . . . . . . . . . . . . 66 4.18 Different Thresholds (Contributing Leechers) . . . . . . . . . . . . . . 67 4.19 Optimal Threshold . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.20 Download Time using the Optimal Threshold . . . . . . . . . . . . . . 70 5.1 Optimal Load Distribution for Different Objective Functions (without FEC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 5.2 Optimal Load Distribution for Different Objective Functions Zoomed (without FEC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 5.3 Mean Loss Rate at Optimal . . . . . . . . . . . . . . . . . . . . . . . . 83 5.4 Mean Burst Length at Optimal . . . . . . . . . . . . . . . . . . . . . . 83 5.5 Mean Lag 1 Autocorrelation at Optimal . . . . . . . . . . . . . . . . . 84 5.6 Optimal Load for Different Objective Functions (with FEC) . . . . . . . 85 5.7 Information Loss Rate at Optimal . . . . . . . . . . . . . . . . . . . . . 86 5.8 Optimal Load for Different Objective Functions (with FEC, Path 1 loss rate = 10%) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 5.9 Information Loss Rate at Optimal (Path 1 loss rate = 10%) . . . . . . . 87 5.10 Sensitivity (Path 1 loss rate = 5%, Best Single Path) . . . . . . . . . . . 90 5.11 Sensitivity (Path 1 loss rate = 5%, Optimizing MLR*MBL) . . . . . . . 90 6.1 Optimal splitting (2 paths) . . . . . . . . . . . . . . . . . . . . . . . . 93 6.2 EMS overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.3 Information loss vs. FEC redundancy . . . . . . . . . . . . . . . . . . . 99 6.4 Network topology used in our simulations . . . . . . . . . . . . . . . . 102 6.5 DMP comparison: Information loss vs path loss (3 or 5 paths) . . . . . 104 6.6 DMP comparison: Information loss vs # paths (Path loss 0.06%) . . . . 105 6.7 DMP comparison: Information loss vs # paths (Path loss 2.24%) . . . . 106 6.8 DMP comparison: Average delay vs # paths (Path loss 2.24%) . . . . . 108 vi 6.9 Model-based scheme comparison: Information loss vs Path loss (10% FEC redundancy) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6.10 Model-based scheme comparison: Information loss vs Path loss (40% FEC redundancy) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 6.11 Model-based scheme comparison: Load splitting vs Path loss (10% or 40% FEC redundancy) . . . . . . . . . . . . . . . . . . . . . . . . . . 111 6.12 Model-based scheme comparison: Load splitting vs. Path loss (40% FEC redundancy) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 6.13 OLS: Load splitting under different schemes . . . . . . . . . . . . . . . 113 6.14 OLS: Information loss under different schemes . . . . . . . . . . . . . 114 6.15 FEC Redundancy Adaption: Amount of FEC Redundancy . . . . . . . 115 6.16 FEC Redundancy Adaption: Information Loss . . . . . . . . . . . . . . 116 6.17 FEC Redundancy Adaption: Average Delay . . . . . . . . . . . . . . . 117 6.18 FEC Redundancy Adaption: Background FTP Speed . . . . . . . . . . 118 6.19 FEC Group Size Adaption: Information Loss . . . . . . . . . . . . . . 118 6.20 FEC Group Size Adaption: Information loss . . . . . . . . . . . . . . . 119 6.21 FEC Group Size Adaption: Average delay . . . . . . . . . . . . . . . . 120 6.22 FEC Group Size Adaption: Joint FEC adaptation . . . . . . . . . . . . 120 vii List of Algorithms 1 Online Load Splitting (OLS) . . . . . . . . . . . . . . . . . . . . . . . 98 2 FEC Redundancy Adaptation . . . . . . . . . . . . . . . . . . . . . . . 100 viii Abstract Along with the bandwidth and popularity growth of the Internet, the scale of the con- tent delivered on the Internet also expanded. Large file downloading and online media streaming will soon, if not already, become the two most resource consuming applica- tions on the network. In the case of file downloading, the peer-to-peer (P2P) paradigm has been used extensively in recent research literature to improve the system’s scalability. One repre- sentative example is BitTorrent (BT), which utilizes the upload capacity of participating users with the help of its tit-for-tat mechanism. BT has been used widely in distributing movies, software, various Linux distributions, etc. It is reported that BT is responsible for 35% of all Internet traffic. In the first part of this thesis, we study problems related to the seeding behavior in the current BT protocol and propose enhancements that can (a) discourage free-riding behavior and at the same time (b) improve the performance of other leechers. In order to aid understanding of BT and possible protocol changes, we propose a simple yet accurate and extensible model of BT in the second part of this thesis. Our model includes (measured) characteristics of the protocol that have previously been left unmodeled. Our model can be used to explore BT parameter setting as well as help ix design improvements. We validate our model extensively through a rigorous simulation- based study and PlanetLab experiments and show that our model is quite accurate in predicting BT performance. In addition to file downloading, real-time online media streaming is receiving more and more attention as well. Video content sharing web-sites such as YouTube have gained significant popularity - it is reported that YouTube has 100 million downloads per day. TV broadcasting stations are also beginning to put their programs online for real-time viewing. Unlike file downloading, online media streaming has strict real-time requirements, which translates into more stringent constraints on network resources. Instead of requiring another infrastructure for delivering the streaming traffic, our work focuses on streaming over the current best-effort Internet, which allows systems to be deployed more easily. However, if we use a traditional streaming approach, through a single path, the current best-effort Internet often fails to provide a good quality of service for such applications. To remedy this, we study the use of multiple paths existing in the network to improve the quality of media streaming. In the third part of the thesis, we focus on the load optimization problem in the context of multi-path streaming. Followed by that, we focus on practical issues in designing a multi-path streaming system that is suitable for real-time live applications, e.g., video conferencing. We propose adaptive schemes with the aim of satisfying the typically stringent loss and delay constraints of such applications and, at the mean time, maintaining a low overhead. x Chapter 1 Introduction Along with the bandwidth and popularity growth of the Internet, the scale of the con- tent delivered on the Internet also expanded. Large file downloading and online media streaming will soon, if not already, become the two most resource consuming applica- tions on the network. In this thesis, we first look at a popular file downloading applica- tion, BitTorrent(BT). We study problems related to the seeding behavior in the current BT protocol and proposed our simple yet effective enhancements in Chapter 3. Chap- ter 4 of this thesis presents our simple yet accurate and extensible BT model which includes (measured) characteristics of the protocol that have previously been left unmod- eled. Next, we shift our focus to the second application, online media streaming. Instead of following the traditional streaming schemes which only utilize a single path to deliver the data, we exploit the idea of using multiple paths for data delivery. In Chapter 5, we focus on a load optimization problem in the context of multi-path streaming. Followed by that, in Chapter 6, we study practical issues in designing a multi-path streaming sys- tem that is suitable for real-time live streaming applications. In the reminder of this chapter, we give an overview of each of these works. 1.1 Improving BitTorrent In Chapter 3 of this thesis, we present our simple yet effective approach to improving the performance BitTorrent. One important thread in the design and evaluation of P2P systems has been the provision of incentives for nodes to contribute their resources. An 1 interesting aspect of BitTorrent (BT) [13], which makes it stand out in that respect, is the “forcing” of peers to share their resources while attempting to complete their downloads (through the TFT mechanism) – this is done “locally” and without a need for centralized or distributed mechanisms for implementing an incentive system. Although this works nicely, there are still opportunities for free-riders (i.e., peers who do not contribute resources) to download files (e.g., [30, 47, 64]). These opportuni- ties exist in two forms: (1) capacity provided by leechers (nodes who have not completed their download yet) through the optimistic unchoking mechanism (used by leechers to probe for new downloading opportunities), and (2) capacity provided by seeds (nodes that have completed their download but continue to contribute their uploading capacity to the system). A number of research studies (e.g., [15, 44, 56, 65, 63]), have focused on the fairness, robustness, and performance characteristics of BT, mainly resulting from the TFT mech- anism. However, very few studies have considered seeding behavior effects. Given typ- ical user behavior and the design of most BT clients, seeding is a common phenomenon seen in the BT ecosystem. Many users leave their BT clients unmonitored while per- forming a large download, and the clients seamlessly transition to being seeds once the download completes. Also, some sharing communities enforce a download/upload ratio to enable seeding and a better performing system. Real-world measurements (e.g., [37, 67, 18, 7]) suggest that, in most torrents, there exists a significant number of seeds, with a large variety of seeding capacity. Some of the torrents measured have more than twice the number of seeds than the number of leechers. We believe that sufficient evidence exists to indicate that the following two state- ments are true: (1) opportunities for free-riding hurt the system’s performance, and (2) a non-negligible number of nodes in BT stay around as seeds for some time after completing their downloads. That is, many users would only contribute resources if 2 their performance was quite poor. And, the capacity provided by the seeds can signifi- cantly contribute to providing reasonable performance for the free-riders. Therefore, we believe that the behavior of seeds is an important factor in facilitating free-riding. Moreover, a leecher’s progress through the file downloading process is not uniform. The progress is slower at the beginning, when having few chunks prevents a node from being selected for uploads through tit-for-tat. It can also be slower at the end, when it is more difficult to find peers with the few remaining pieces of needed data (although provisions exist to aid with that). In both cases, seeds can contribute significantly, as they are no longer playing the tit-for-tat game, and they have all the data pieces. Thus, intuitively, appropriate use of seeding capacity can lead to overall system performance improvements. Given this, our main idea in Chapter 3 is to explore alternative approaches for seeds to contribute their upload capacity to the system, with the goal of degrading free-rider performance while (if possible) improving other leechers’ performance. The contribu- tions of Chapter 3 are as follows: • We propose several simple approaches to modifying the seeds’ uploading algo- rithms, where the goal is to (a) discourage free-riding behavior and at the same time (b) improve (or at least not degrade) the performance of other leechers. The details of these schemes are given in Section 3.3. • We show that simple schemes can provide significantly poorer performance to free-riders while at the same time improving (or not hurting) the other leechers’ performance. We do this through a detailed simulation study in Section 3.4. The poor free-rider performance should serve as a deterrent, which will encourage free-riders to contribute resources. 3 1.2 BitTorrent Model In Chapter 4 of this thesis, we focus on a simple model of BT, as a tool for providing insight into its behavior and possible future improvements in its architecture. A number of works have focused on modeling BT or BT-like systems (see Sec- tion 2.1) - some model flash crowd behavior and some steady state behavior. In Chap- ter 4, we focus on steady state behavior. Within the class of efforts that model steady state behavior, our model is distinct as it accounts for: (1) seeds and (2) free-riders. Moreover, to improve the accuracy of our model, we account for: (a) imperfect cluster- ing behavior in regular (TFT-based) unchokes and (b) bias in optimistic unchokes. As discussed in the previous section, there are significant seeding resources observed in typical torrents, which have significant effect on BT performance, and there are sig- nificant opportunities for free-riders to hurt BT performance. We believe that it is impor- tant to include such behavior in an accurate BT model. Thus, in Chapter 4, we provide a simple and extensible model that can characterize the effects of seeding and free-riding in BT. We also note that our model is of a heterogeneous BT system (in terms of node bandwidths) and that the only other work that considers a heterogeneous system in the context of steady state behavior is [15], but without the distinctions stated above. We present our model in Section 4.1 and illustrate its accuracy in Section 4.2 (where we also compare our model to the only other available model of a heterogeneous BT system in steady state, given in [15]). Another aspect that we include in our model is that of imperfect clustering 1 . Previous approaches have assumed that operationally BT decomposes cleanly into clusters, where nodes “find” like nodes that are similar in bandwidth capacity and exchange chunks 1 Our notion of clustering is similar to that in [36]. 4 amongst each other. That however is not the case in practice, and we model that, with details given in Section 4.1. Specifically, the contributions of Chapter 4 are as follows: • We provide a simple and complete steady state model of a heterogeneous BT sys- tem. To the best of our knowledge, it is the first analytical performance model of a heterogeneous BT system that includes the behavior of seeds and free-riders (see Section 4.1). Our model predictions confirm that BT is indeed quite exploitable (see Section 4.2), which further confirm the motivation of our work in Chapter 3. • We extend it to include important BT characteristics, namely (i) imperfect clus- tering in regular (TFT-based) unchokes and (ii) bias in optimistic unchokes (see Section 4.1). • We validate our model using simulation results and PlanetLab experiments. We demonstrate the importance of including imperfect clustering and biased opti- mistic unchoking in having accurate prediction of nodes’ download time in BT (see Section 4.2). • Our model’s extensibility is demonstrated by explicitly modeling two variations of BT, (a) the well known large view exploit [46, 64], and (b) our proposed fix for the large view exploit in Chapter 3 (see Section 4.3). 1.3 Multi-path Streaming Optimization We look at another Internet application, namely that of continuous media (CM) stream- ing, in Chapter 5 of this thesis. Quality of service (QoS) in streaming of CM over the Internet is still poor and inconsistent. The degradation in quality of CM applications, involving delivery of video and audio, is partly due to variations in delays, bandwidth limitations, as well as losses experienced by packets sent through wide-area networks. 5 Although many such applications can continue to operate with some degree of missing data, non-recoverable information loss degrades these applications’ quality of service. Consequently, a number of application areas (e.g., those related to the entertainment industry) have backed away from streaming of their content over the Internet. Even if streaming service is provided, contents offered are often limited to low bitrate version (also low quality), for example as in YouTube [27] and Hulu [22]. Besides running cost consideration, inability to control the resulting visual and auditory quality of the streamed presentation is an important reason for such a trend. One approach to providing QoS for CM applications over the Internet is to use the IntServ model for signaling (e.g., RSVP) and resource reservation in all routers along the streaming path. However, this approach suffers from scalability and deployment problems. In contrast, in this work we focus on providing QoS guarantees in CM deliv- ery through the exploitation of multiple paths existing in the network between a sender (or a set of senders) and a receiver. The basic idea is that CM data destined for a partic- ular receiver is fragmented into packets and the different packets take alternate routes to the receiver. One advantage of this approach is that the complexity of QoS provision can be pushed to the network edge (an original design principle of the Internet) and hence improve the scalability and deployment characteristics while at the same time provide a certain level of QoS guarantees. There are many other advantages of multi-path (MP) streaming over the Internet, some of which have been explored in previous works, as described in Section 2.2. There are a number of approaches to accomplishing multi-path data delivery; the specific approach considered in this work is described in Section 5.1.1. We note that the multiple paths do not have to be completely disjoint — it is sufficient for them to have disjoint bottlenecks. Existence of multiple paths with disjoint bottlenecks can result in 6 many benefits[17] including (a) increased bandwidth, (b) improved loss characteristics, and (c) ability to better adjust to changes in network conditions. In this work, we focus on the loss characteristics only. We also note that in Chapter 5 we narrow the scope by considering (a) delivery of pre-stored CM (delivery of live content is studied in Chapter 6) and (b) application- level schemes which are deployable today over the current Internet (without changes to the network). We obtain the multiple paths to the same receiver by distributing servers across wide-area networks and streaming data from multiple senders simultaneously. Here we are concerned with streaming and network performance issues only, rather than, e.g., considering server-related problems such as data placement issues. To reap the benefits of multi-path streaming one must first consider and solve a number of research problems[17]. One such problem is the appropriate load distribu- tion among the multiple path. Optimal load distribution in the context of multi-path streaming is the focus of this chapter. We define this problem formally and in more detail in Section 5.1. The contributions of this work are as follows. We first focus on determining an appropriate optimization objective for computing an optimal load distribution, where the high level goal is to improve the perceptual quality of the resulting CM. We then conduct a performance study to understand the goodness of our optimization objective as well as gain insight into the resulting benefits of using multiple paths to deliver CM data. The contributions of this work as compared to the existing literature are discussed in Section 2.2. Lastly, we note that one should also consider the potential costs or detrimental effects of multi-path (MP) streaming. For instance, MP streaming might have an adverse effect on the resulting delay characteristics observed at the receiver. As a result, it might require a large amount of receiver buffer space. Evaluating such effects is the topic of 7 future work. Also, a simple implementation of MP streaming would require replicas of the CM data to be distributed to the servers (although, mirroring servers is a common practice). Advances in data placement approaches are needed to reduce such costs. In addition, there are overheads associated with sending data over multiple paths and then assembling it into a single stream at the receiver. Moreover, the overheads and com- plexity due to measurements needed to achieve better performance with MP streaming should also be considered. For instance, in our case, we need to detect shared points of congestion, e.g., using passive techniques[60]. Other approaches to MP streaming might require even more detailed information about the network which is likely to result in a need for more “intrusive” and complex measurements (refer to Section 2.2). Lastly, scalability of such measurement schemes is an issue as well. 1.4 Encoded Multi-path Live Streaming Chapter 6 of this thesis studies the problem of live streaming under the use of multi- path and FEC. Live streaming applications have become increasingly popular nowadays, driven by the widespread adoption of broadband networks. Such applications typically have stringent quality of service requirements, and the transport protocol must ensure the streaming content is delivered in time, with minimal losses. As discussed in the previous section, multi-path streaming is one promising approach that leverages the availability of multiple paths between end-hosts and exploits such path diversity to improve the streaming quality. Most of the multi-path streaming schemes also apply Forward Error Correction (FEC) encoding in the stream, so that the receiver can recover from packet losses without retransmission. With the joint benefits of multi-path and FEC, these schemes can successfully provide high-quality streaming of pre-stored or live media content. However, in order to handle bursty losses, most existing streaming systems 8 typically require a long startup delay, several seconds or even longer (e.g., [66, 21]), at the receiver. In Chapter 6, we focus on real-time live streaming applications that have stringent end-to-end delay requirements, say a few hundreds of milliseconds. Examples of such applications include video conferencing, remote surgery, tele-presence, and virtual real- ity. The interactive nature of these applications precludes the use of a large startup buffer, and the stream content must be delivered within a very short time window after it is generated. According to the ITU-T/G.114 recommendation [61], the end-to-end delay should be no more than 150 ms for highly interactive tasks. Moreover, these live streaming applications also require high-fidelity stream delivery for quality user expe- rience. For example, with MPEG-4 encoding, there will be noticeable video quality degradation when the loss rate is above 1%. We first revisit the central issue of how to split the stream across multiple paths so as to minimize the information loss. From our experimental studies in Chapter 6, we observed an asymptotically optimal load splitting in multi-path streaming with FEC. We observe that in a general setting of heterogeneous paths, it is asymptotically optimal to split the traffic such that the load on each path is inversely proportional to its loss rate. These surprisingly simple results can offer valuable insights for the protocol design. The asymptotic analysis and the proof of the asymptotically optimal can be found in [10]. Inspired by the asymptotically optimal results, we have designed a novel Encoded Multi-path Streaming (EMS) scheme for real-time live streaming. EMS addresses prac- tical issues, such as network dynamics and the coupling effects of multi-path and FEC, through a set of adaptation mechanisms. Specifically, EMS uses an adaptive search process to dynamically adjust the load splitting decision around the asymptotically opti- mal one. It also dynamically adjusts the FEC settings, including both, FEC group size and FEC redundancy level, based on the current network conditions and the application 9 requirements. For real-time streaming, these FEC settings present multiple tradeoffs between the error correction capability and the delay performance. Through simple adaptation mechanisms, EMS strives to find a good balance where it satisfies the appli- cation requirements with minimal overhead. We have evaluated the performance of EMS using extensive simulations, and com- pared it to two existing multi-path streaming schemes, namely the TCP-based DMP scheme [66] and another Gilbert Model-based UDP scheme [55]. Our results show that EMS consistently outperforms these existing schemes under different network settings, with orders of magnitude lower information loss rate. The results also confirm the effec- tiveness of the load splitting and FEC setting adaptation mechanisms in EMS. In summary, the main contributions of Chapter 6 are as follows: • We observed an asymptotic and closed-form solutions for optimal load splitting in multi-path streaming with FEC. • We design a novel real-time live streaming scheme with adaptive load splitting and FEC settings. • We perform extensive simulation experiments which provide a comparison of dif- ferent multi-path load splitting approaches and the impact of network dynamics. 10 Chapter 2 Related Work In this chapter, we give an overview of literature related to this thesis, first in the context of BitTorrent (BT) and then in the context of multi-path streaming. 2.1 BitTorrent BitTorrent (BT), originally described in [13], has been the topic of a large number of research efforts. Thus, here we give a brief overview of those works most related to ours. A number of works have examined the incentive mechanisms of BT. For instance, Jun et al. [30] argue, using game-theoretic analysis, that the incentives mechanisms in the original BT protocol are ineffective in rewarding and punishing downloaders properly. Slow uploaders are capable of obtaining high download rates. Liogkas et al. [45] con- sider various approaches to how selfish peers can select neighbors from which download (e.g., fast peers or seeds) such that the their download performance is improved. Piatek et al. [56] develop a strategic BT client, “BitTyrant”, which attempts to get the most reciprocation through better assignment and scheduling of upload capacity to peers. In fact, free-riding (without uploading any data) is possible in BT. For instance, Locher et al. [46] demonstrate the use of a free-riding client they developed, ”BitThief”, where the main idea is for free-riders to open more connections (using proposed “tricks”) so that they can increase the probability of being unchoked by seeds or (optimistically) by leechers. The idea is similar to the “Sybil attacks” in [14]. Locher et al. proposed a source coding based solution to the problem in [47]. The idea they employ is to use 11 random linear combinations of chunks in the system to increase diversity and promote tit-for-tat. We note our approach achieves the same by reassigning seed capacity and needs no coding/decoding of chunks. Sirivianos et al. study a similar large view exploit and present their results based on a simpler client (fewer modification from the mainstream client) in [64]. They also suggest approaches to mitigating the large view exploit, e.g., by limiting the view of free-riders through the use of more sophisticated trackers. All these works suggest that free-riding in the current BT systems is feasible and can be achieved fairly easily. To mitigate this problem, our work provides a simple approach which does not require the use of sophisticated trackers or advanced coding schemes. Moreover, we illustrated that these schemes can help guard against the large view exploit as well. 2.1.1 BitTorrent Modeling A number of works have focused on modeling BT. Yang et al. [67] consider a BT sys- tem in two phases - initial transient phase and steady state - and propose the use of a simple Markov model (to study steady state performance) which describes the system’s state using the number of leechers and seeds in the system. A number of works followed that effort, including [58] (looking at stability of BT), [18] (looking at BT’s lifetime), [65] (looking at a somewhat more detailed Markov model), and [39] (looking at includ- ing free-riders in the Markov model). A number of efforts have also modeled BT-like systems. For instance, Massoulie et al. [49] model coupon replication systems and con- sider the make-span of a batch of nodes. A nice model of upload capacity of BT-like systems (under the assumption of it being the only constraint) is given [52]; this work also focuses on optimal make-span. However, all these works model homogeneous BT systems, i.e., where all (contributing) nodes are of the same class. 12 Since real-world torrents are typically heterogeneous, in this work we focus on mod- eling heterogeneous BT systems. Other models of heterogeneous systems are given in [44] and [15]. Liao et al. [44] provide a detailed model of BT with 2 heterogeneous classes of nodes, but for a flash crowd scenario. They consider the nodes’ make-span, where all nodes join the system simultaneously; moreover, their model does not include seeds. Although they show their model to be accurate in predicting download times, it is not clear how difficult it would be to extend it to an arbitrary number of classes. In contrast, our work focuses on steady state behavior, where we use a simpler model and allow for node arrivals as well as seeding. We also allow an arbitrary number of node classes. Fan et al. [15] use a simple heterogeneous model for evaluating the tradeoff between performance and fairness and focus on illustrating that the number of opti- mistic unchokes and regular unchokes correspond to important tuning parameters, for trading off fairness and performance. We use a similar model but include seeding and free-riding behavior, which have a significant effect on BT’s performance. Also, our validation results (in Section 4.2) indicate that it is important to account for BT’s imper- fect clustering characteristics as well as a bias in the optimistic unchoking mechanism, both of which are also not considered in the model of [15]. A recent effort [40] also con- siders the process of clustering in BT and models it in a more detailed manner (using a Markovian model); a similar approach can be adopted in our modeling framework, for additional accuracy. However, in this work, we have focused on a simple and extensible model, which we have demonstrated to be accurate. We now briefly describe several other efforts which do not focus on analytical mod- els of BT; however, these works provide evidence and motivation for the modeling choices made in our work. Legout et.al [37] provide measurement studies of real BT tor- rents and suggest that the rarest first chunk strategy results in high diversity of chunks in the system - this provides evidence for use of simpler models, like ours, i.e., that it might 13 be reasonable not to model efficiency of chunk exchange between peers, e.g., as is done in [65, 58, 67]). In [36], Legout et al. suggest that nodes with the same upload capacity tend to cluster with nodes of the same class. However, their results indicate (based on flash crowd arrivals), that clustering is still far from perfect (even in their closed system). Existence of imperfect clustering (or as they call it, imperfect bandwidth matching) is also argued in [56]. Moreover, Levin et al. [38] look at BT from a game theoretic perspective - they observe that BT is actually not (strictly) doing TFT and propose an alternative reciprocation scheme to improve BT’s download speed. Thus, these works ([36], [56], [38]) support the need for explicit modeling of imperfect clustering, which is done in our model. Lastly, Bharambe et al. [6] study BT through simulations under flash crowds. We adapt their simulator for validation purposes (see Section 4.2). Their study indicates that BT can achieve high upload bandwidth utilization, which supports our model’s assumption of upload capacity being the bottleneck. 2.2 Multi-path Streaming Multi-path streaming has recently attracted much attention for improving the quality of streaming applications, and a broad overview of this research area can be found in [3]. We now give a brief survey of existing work on streaming; specifically, we mostly focus on those that consider loss characteristics of continuous media and can be deployed over best-effort networks (as these are considerations in our work in Chapter 5 and 6 as well). Other works[12, 59], consider different applications of multi-path transmission, such as simultaneous downloads and network-level issues. Earlier efforts on dealing with losses through the use of multiple independent paths include dispersity routing[50], where the focus was on reducing delay in file transfers. An important difference in our work is that we focus on streaming applications (where 14 the data transmission rate is determined by the application’s needs) rather than on deliv- ering the data to its destination as fast as possible. Hence, in our case the data is sent through the network at a specific rate, and that has an effect on loss characteristics, which we investigate here. Another set of works considers higher level mechanisms but requires assistance from the lower layers and/or assumes significant knowledge of network topology and/or link capacities and delays (on all links used for data delivery). Given such knowledge, algo- rithms are proposed for selecting paths which can avoid congested routes[28, 8, 32]. In contrast, our approach does not rely on specific knowledge of topologies, capacities, delays, etc., and only considers whether a set of paths do or do not share joint points of congestion, as can be detected at the end-hosts[60]. Recent literature on this topic also includes works on voice-over-IP type applica- tions. For instance some works[43, 42] propose a scheme for real-time audio transmis- sion using multiple independent paths between a single sender and a single receiver, where multiple description coding (MDC) is used in multi-path delivery and a FEC approach is used in single-path delivery. In the case of using multiple paths, the authors use two equal-sized descriptions of the audio, sending one on each path. In contrast, we believe that it is important to understand the effects of multi-path delivery on loss characteristics, even without the use of coding techniques (an advantage being that it can then be made to work well with many coding techniques or media codecs). We also focus on cases where traffic is not necessarily distributed equally on separate paths, i.e., on the load distribution problem. Another work [4] uses multiple description coding in addition to multiple paths, as opposed to using FEC, thereby tying the benefits of the technique to a particular coding scheme. In contrast, as already stated we believe that it is important to understand the 15 benefits of MP streaming independently of data recovery schemes used. The applica- bility of this study (at least to future streaming applications and network conditions) is also somewhat limited in that (a) it constraints the bursts on paths to no more than 3 and (b) it only considers cases where the loss rates across multiple paths are fixed at 5%. Similar approach is also taken in [16, 31]. To the best of our knowledge, Golubchik et al. [17] is the first to provide an analytic framework for illustrating the benefits of MP streaming as compared to SP streaming, using metrics which describe both mean loss rates as well as correlations between losses (as illustrated in our study as well, it is important to consider both). Nguyen et al. [53] also consider delivery of pre-recorded video from multiple senders distributed across the network. However, this work focuses on a TCP-friendly optimization algorithm for (a) rate distribution among the paths (i.e., how much data to send over each path) and (b) packet distribution among the paths (i.e., which packet should be sent over which path), with the objective of minimizing the loss rate at the receiver and the added delay due to the distribution of the media source. Nguyen et al. [55] also add FEC techniques (as compared to their earlier work[53]). Again, distribution algorithms are considered but with the objective of minimizing the probability of irrecoverable error. In contrast, due to the nature of the application, we believe that in investigating when multi-path streaming is beneficial, it is also important to consider loss characteristics even when the losses cannot be fully recovered because, e.g., video can be displayed under some losses. In a later work [54], the authors use relay nodes to provide multiple paths for delivery of streaming media (this is also suggested as a possibility by Golubchik et al. [17]). To ensure diverse paths, their system requires detailed information about the underlying network topology, e.g., router names and link properties. In contrast, we do not require such information. 16 2.2.1 Multi-path Live Streaming Our EMS approach proposed in Chapter6 targets live streaming from a single source, while most existing schemes (e.g., [17, 9, 55, 41]) assume that the stream is pre-stored at the sender. Secondly, EMS strives to provide real-time streaming with stringent delay requirements, whereas existing schemes have largely ignored the delay characteristics as the receiver can maintain a large playback buffer. Lastly, existing schemes all use static FEC settings. However, as shown in Section 6.2, the FEC settings play a key role in the end-to-end delay, thus EMS uses adaptive schemes to dynamically adjust the FEC redundancy and group size. While several recent works, such as DMP [66] and MPLOT [62], have proposed multi-path data delivery schemes between a single source and a receiver, they cannot be directly applied in real-time live streaming. As shown in Section 6.3.2 and [66], the startup delay with DMP is on the order of seconds, which is not suitable for real-time applications. MPLOT is a transport protocol which focuses on maximizing application throughput, yet it does not address tight delay constraints. While MPLOT can greatly aid applications with bulk data transfers, a live streaming source cannot effectively leverage such throughput benefits because its streaming rate is typically fixed or bounded, e.g., by the encoding schemes. Optimal load splitting in multi-path streaming with FEC has been studied by several efforts listed earlier in this section. However, most approaches involve significant com- binatorics computation and do not directly offer engineering insight. Their optimal solu- tions often require fairly accurate knowledge of the path loss model parameters, which would be difficult to obtain in practice. In contrast, we design a practical load splitting scheme in Chapter 6, which is based on our observation of asymptotically optimal load splitting in multi-path streaming with FEC and dynamically adaptive searching based 17 on the network conditions. As shown in Section 6.3.3, EMS can achieve near-optimal load splitting without any assumptions about the path loss model. Under single path streaming, adaptive FEC has been studied in [11] for V oIP appli- cations. These works are closely tied to the media compression scheme used and are based on the knowledge of path loss model. Similarly to Chapter 5, our EMS approach in Chapter 6 does not assume any specific type of streaming data or specific loss model. Also, the FEC adaptation schemes in EMS address the joint effects of FEC and multi- path. Kurant [34] proposes to utilize the delay differences between multiple paths by assigning more workload (or sending packets with larger spacing) on shorter delay paths. Similarly to other model-based approaches, [34] requires the knowledge of a path loss model, which is not required in EMS. Furthermore, EMS exploits the delay in multi-path streaming by adapting the FEC settings. Finally, many P2P streaming systems such as PPLive [25], PPStream [26], and Cool- Streaming [68] have been widely used on the Internet. Their goal is to broadcast video streams to a large number of users, with the users exchanging data among themselves to reduce the source workload. Measurement studies (e.g., [21, 48]) have shown that such systems incur large end-to-end latency, ranging from several seconds to a few minutes; thus they are not suitable for real-time live streaming. 18 Chapter 3 Improving BitTorrent: a Simple Approach BitTorrent (BT) [13] was proposed to achieve efficient file downloadings by utilizing uploading resources of the users. However, seeds staying in the system hurt the built-in tit-for-tat fairness mechanism and make free-riding possible. In Section 1.1, we already provided an overview of our work on this problem. A detailed discussion is presented in this chapter. We first provide a background of BT in Section 3.1. We then motivate our work by looking at the problem of how free-riders can take advantages from seeds and how preventing free-riders can improve the system performance in Section 3.2. In Section 3.3, we propose several simple approaches to modifying the seeds’ uploading algorithms, where the goal is to (a) discourage free-riding behavior and at the same time (b) improve (or at least not degrade) the performance of other leechers. The performance study of these approaches is given in Section 3.4, which is followed by a discussion of our approach in Section 3.5. 3.1 BitTorrent Protocol Description In BT, nodes join the system (after receiving “start-up” information from the tracker) and begin requesting chunks of data from their neighbors. Nodes that do not have a complete copy of the file are termed “leechers” and those that do are termed “seeds”. Nodes that do not contribute their upload capacities are termed “free-riders”. Each leecheri picks a 19 number (typically 5 or 6) of nodes to whose requests it will respond with an upload of an appropriate chunk, i.e., these nodes are “unchoked”. A subset of these nodes (typically 4) are picked based on the tit-for-tat mechanism, i.e., those neighbors that have provided the best service (in terms of download rate) to node i recently. And a subset (typically 1 or 2) is picked randomly, i.e., they are “optimistically unchoked” (to explore better neighbors). Seeds also pick a subset of neighbors (typically 5 or 6), and upload data to them. In past versions of BT, seeds chose neighbors that could download data from them the fastest. In a more recent protocol (as described in [37]) the seeding capacity is distributed more uniformly to the neighboring peers. These choices are re-evaluated periodically. 3.2 Motivation A number of research studies (e.g., [30, 56, 45, 46, 64]), have focused on the fairness, robustness, and performance characteristics of BT, mainly resulting from the tit-for-tat mechanism. However, very few studies have considered the effects of seeding behavior. Given typical user behavior and the design of most BT clients, seeding is a typical behavior in the BT system. Many users may leave their clients running after completing downloads, possibly due to users not monitoring their clients during the download. Also, some sharing communities enforce a download/upload ratio to enable seeding. Real- world measurements such as (e.g. [5, 7, 18, 29, 37, 57]) also suggest that there exist a significant number of seeds in most torrents; they suggest that there is a large variety of seeding capacity in real-world torrents – some of the torrents measured have twice as many seeds as they do leechers. Availability of seeding capacity can have a significant effect on BT, e.g., it can com- pensate for the asymmetric bandwidth scenarios in the Internet. At the same time, it can 20 degrade the fairness and incentive properties of the system, as free-riders can finish their downloads with reasonable performance by relying on the seeds. (Not only do they not contribute to the systems’ upload capacity, they also effectively reduce the performance gains that seeds provide.) Thus, intuitively, appropriate use of seeding capacity can have a significant effect on performance of both, contributing leechers as well as free-riders. Existing studies on seeding behavior (e.g., [6, 36]) mainly concentrate on the initial seeding behavior, such as reducing sending of duplicate data. In this chapter, we focus on precisely that. Specifically, we ask a simple question – how can the seeding capac- ity be better utilized to discourage free-riding behavior and at the same time improve performance of contributing leechers. We now give a motivating example to illustrate opportunities for “closing a loop-hole” in BT which might encourage free-riding behav- ior. 3.2.1 Motivating Example Consider the following example, where we have two classes of nodes: Contributing Leechers (CL) and Free-Riders (FR), defined in Table 3.2. Figure 3.1 depicts the down- load times of each class as a function of average seeding time with 20% of users being free-riders and with no free-riders. These results are obtained by simulation (refer to Section 3.4) with settings given in Table 3.1. The 20% free-riders case illustrates that the average download time of free-riders is more sensitive to the seeding time than that of the contributing leechers. For example, with no seeding time, the average down- load time is≈ 166 mins and≈ 863 mins for contributing leechers and free-riders, respectively. When the average seeding time is increased to60 mins, the average down- load time is reduced to≈ 107 mins (≈ 36% speed up) for contributing leechers and ≈ 214 mins (≈ 75% speed up) for free-riders. This suggests that alternative approaches to distributing seeds’ upload capacity might lead to significant effects on free-riders’ 21 0 100 200 300 400 500 600 700 800 900 0 20 40 60 80 100 120 140 Average Download Time (min) Average Seed Time (min) FR (20% Free-Riders) CL (20% Free-Riders) CL (No Free-Riders) Figure 3.1: Effect of Seeding on CL and FR performance while possibly improving (or at least not hurting) the performance of con- tributing leechers. Moreover, if we degrade free-riders’ performance sufficiently, many of them might choose to become contributing leechers, i.e., in order to obtain reasonable performance. This would result in an improved overall system performance – e.g., with an average seeding time of 60 mins (in our example), the average download time can decrease from≈ 107 mins to≈ 74 mins (≈ 31% speed up), if free-riders contribute resources similarly to other users. 3.3 Proposed Approach There can be many ways to change how the seeding capacity is utilized, instead of having to modify the BT communication protocol and requiring tracker support, e.g., in [39], we explore the following simple approach to modifying how seeds distribute their upload capacity to leechers. Currently, seeds unchoke leechers regardless of how far along they are in their down- loading process. Measurements and simulations indicate that the downloading rate of 22 0 0.2 0.4 0.6 0.8 1 0 10 20 30 40 50 60 70 80 90 100 Download Rate (# chunk/sec) Total Data Downloaded (%) CL (Original) CL (Sort) FR (Original) FR (Sort) Figure 3.2: Example Chunk Rate leechers is the slowest in two regimes: (i) right at the beginning when they don’t have enough chunks to participate effectively in tit-for-tat with other peers, and (ii) right at the end when finding peers with the missing few chunks becomes difficult. (Possible causes of this are explored in Section 3.5.) In both situations seeds can help since they have no requirement for tit-for-tat and they have all the chunks to supply the last few pieces. Our simple insight is to prioritize the use of seeding capacity to only certain portions of a file’s downloading process. For example, we can unchoke only those leechers who are at the beginning (i.e., have a few chunks) and those at the end (i.e., have most of the chunks) of their download process. (Below we describe several approaches based on this idea.) We can motivate this as follows. Consider a plot of the chunk rate (i.e., the rate at which a node acquires the next chunk) as a function of the number of chunks that node has – this is depicted in Figure 3.2 which is obtained through simulation (refer to Section 3.4). Here we plot the the chunk rate behavior of contributing leechers (Fast class in Table 3.2) and free-riders using the original BT seeding scheme as well as our modified 23 scheme in which we prioritize leechers that are towards the two ends of the process. using the same settings as in in motivating exampling in Section 3.2), Both, contributing leechers and free-riders experience an increase in seeding capacity at the end points (where we shifted this capacity) and a decrease in the middle portion. However, the free-riders have a far greater dependence on seeds than contributing leechers. Thus, the free-riders get hurt more by being “banished” from using the seeds. The contributing leechers on the other hand benefit by ramping up fast and being able to participate in tit- for-tat earlier than in the original BT scheme. In the example of Figure 3.2, the average download time of free-riders is increased from≈ 214 mins to≈ 389 mins (≈ 82% slow down), while the average download time of contributing leechers remain at≈ 107 mins. Armed with this intuition, we now give details of the specific schemes explored in the remainder of this chapter. These are just samples of possible approaches. Our goal in this work is to evaluate the potential of this general approach (refer to Section 3.4) rather than to exhaustively consider all possible similar schemes. We note first that our approach does not require identification of free-riders. And, it is a simple approach to implement and deploy in the current BT system. Sort-based (N): In sort-based schemes, a seed sorts the requesting neighbors based on the number of chunks each has. It then unchokesN (we use the BT default ofN = 6) of them based on the sorting order. Motivated by the need to help end points (as explained above), we choose the N that are furthest from the middle (i.e., having half of all the chunks). Threshold-based (K,N): Threshold-based schemes are similar, except that we choose to unchoke N requesting neighbors that only have K% of the total number of chunks. Specially, we unchoke nodes that have either [0.. K 2 ]% or [(100− K 2 )..100%] of the chunks. 24 Table 3.1: Simulation Settings Filesize 500 MB (2000 Chunks, 256 KB each) Average node inter-arrival ( 1 λ ) 1 min Peer Set Size 80 Number of Leecher Unchokes 4 Regular + 2 Optimistic Number of Seed Unchokes 6 A number of variations on this schemes are possible; for clarity of exposition we only focus on the above mentioned one. 3.4 Seeding Approach Evaluation We use the BT simulator provided by [6], with modifications described below. The simulator is event-based and simulates the chunk exchanging mechanism of the BT pro- tocol. We extended the simulator to support: (a) nodes staying around as seeds, with different seeding approaches as proposed in Section 3.3, (b) node arrivals, and (c) nodes acting as free-riders (i.e., nodes that do not unchoke and leave the system upon down- load completion). The seeding times and node inter-arrival times follow an exponential distribution. We also modified the original seeding scheme to be more uniform, in-line with the BT version 4.0.0. Moreover, we also fixed a bug in the original simulator that affected the selection of peers for unchoking - the original simulator incorrectly imple- mented that part of the BT protocol, which resulted in a higher probability of unchoking previously unchoked peers. Unless specified otherwise, the following results correspond to the simulation set- tings in Table 3.1. The system starts with 1 origin seed with a 1000 kbps upload capacity, that stays in the system for12 hours. Nodes arrive to the system from a Poisson process with a rateλ and are assigned to a particular class according to a given distribution. The classes differ 25 Table 3.2: Class Description Class Upload Capacity Class Upload Capacity Super-Fast 1000 kbps Slow 128 kbps Fast (CL) 512 kbps Free-Riders (FR) 0 kbps in their upload capacities, which are defined in Table 3.2. The download capacities are set to 5000 kbps for all the nodes. We consider the steady state behavior of the system. Each simulation run corresponds to 63 hours, where we only compute our results over the last 48 hours. (We check our results to make sure the system passes the ramp up stage during the first15 hours.) We experimented with a number of seeding approaches. The results presented below use the following ones. Original refers to the current BT uploading scheme. Ideal refers to the scheme where the seeds can identify which of their neighbors are con- tributing leechers and which are free-riders, and they only upload to contributing neigh- bors. We only implement this scheme in the simulator as a baseline for evaluating other approaches (i.e., this gives us an idea of a possible bound on the improvements we can offer). Sort refers to the sort-based scheme (see Section 3.3) where the6 neighbors fur- thest away (in either direction) from having half of the chunks are unchoked. T 10% (5% to each end) refers to the threshold-based scheme (see Section 3.3) where the 6 neighbors are chosen for unchoking, using the same algorithm as “original”, except that the set of neighbors that quality for an unchoking is restricted to those that have at most the5% of the chunks or at least95% of the chunks. T 20% (10% to each end) is similar to T 10% but with the threshold size being doubled. T 40% (20% to each end) is similar to T 10% but with the threshold size being quadrupled. We performed a comprehensive study, exploring a variety of parameters. Effect of seeding approaches: We consider the effects of different seeding approaches (described above) with the Fast, Slow and Free-Riders classes, where each class carries 26 0 500 1000 1500 2000 2500 Original Ideal Sort T 10% T 20% T 40% Average Download Time (min) Fast Slow Free-Riders Original Ideal Sort T 10% T 20% T 40% Seed Time: 60 mins Seed Time: 120 mins Figure 3.3: Different Seeding Schemes 40%,40%, and20% of the arrival, respectively. Figure 3.3 depicts the download times of different node classes, with60 mins or120 mins average seeding times, under different seeding schemes. From these results, we observe the following: • Under “mild” seeding capacities, e.g., 60 mins average seed time, sort-based scheme already does a good job of slowing down free-riders while having a relatively smaller effect on contributing leechers. For instance, free-riders have an average download time of≈ 737 mins under the original scheme and it is increased to≈ 1109 mins (≈ 50% slow down) by using the Sort scheme. Under the same settings, the download time for the slow class is increased a little bit from≈ 420 mins under the original scheme to≈ 438 mins (≈ 4% slow down) in the Sort scheme. The download time for the fast class is improved from≈ 151 mins to≈ 144 mins (≈ 5% speed up) by using the Sort scheme. 27 • When there are more seeding capacities our schemes have a stronger effect. For instance, when the average seeding time is 120 mins, compared to the original scheme, the free-riders experience≈ 115% slow down under the Sort scheme. • Threshold-based approaches can slow down the free-riders more than sort-based schemes when a small threshold is used. However, they also have more of a detrimental effect on the contributing leechers – this becomes more critical as the seeding capacity increases since the threshold ends up wasting the seeds capacity. For instance, when the average seed time is 60 mins, the average download time for the free-riders have a≈ 123% slow down by using the T 10% scheme. Under the same settings, the download time for the slow class and the fast class are slowed down by≈ 31% and≈ 5%, respectively. When the seeding capacities is larger, a larger threshold is also required to reduce the detrimental effect on the contributing leechers. How to determine the optimal threshold is explored in Chapter 4. • The Ideal scheme has the strongest effect, however, it is not as practical as other schemes as it requires the identification of free-riders. Under the Ideal scheme, the free-riders experienced a stronger slow down when there are more seeding capacities, it is because the contributing leechers finish their downloading faster (by receiving more helps from seeds). The shorter staying time of the contributing leechers reduce the the download of free-riders through optimistic unchokes. Increasing peer set size (large view exploit): Interesting studies of free-riders’ behav- ior, in the context of the “large view exploit”, are given in [46, 64]. The basic idea is for free-riders to increase their peer set size, to increase the probability of being optimisti- cally unchoked by a leecher or picked by a seed’s unchoking mechanism. 28 100 200 300 400 500 600 700 800 900 80 100 120 140 160 180 200 220 240 Average Download Rate (kbps) Free-Riders Peet Set Size CL (Original) CL (Sort) CL (T 40%) FR (Original) FR (Sort) FR (T 40%) Figure 3.4: Large View Exploit: Download Rate We experiment with adding the large view exploit to our simulations in order to illustrate its effect on our schemes. For ease of illustration, we present our results with 80% of the arrival coming from the Contributing Leecher (CL) class and the remaining 20% from the Free-Riders (FR) class. The average seeding time is60 mins. We increase the average arrival rate to 2 per minute, thus sufficiently increasing the average number of nodes in the system for the large view exploit to work. Contributing leechers and seeds strive for a peer set of size 80 in all cases, and we vary the peer set size of free- riders. Figure 3.4 depicts the download rates of contributing leechers and free-riders, as a function of the free-riders “desired” peer set size, and illustrates that: • The large view exploit does indeed have a significant effect on the original scheme – free-riders can increase their download rate linearly as a function of increasing peer set size. Note that free-riders can even achieve faster downloads than con- tributing leechers when the peer set size is high enough (in this case200 or240). 29 • Our schemes can still degrade the performance of free-riders and essentially prevent free-riders from increasing their download rates through the large view exploit. We reiterate, the exploit is prevented by our schemes in a completely scalable way since no explicit identification or tracking of free-riders is required. 3.5 Discussion The focus of this work is on better utilization of seeding capacity. The significance of our approache’s effects depends on the amount of seeding capacity available. Specifically, with a “medium” amount of seeding capacity (relative to the download capacity), our schemes result in a significant difference, i.e., they degrade the performance of free- riders significantly while either improving or having little effect on the performance of contributing leechers. When the system’s seeding capacity is small relative to download capacity, our schemes do not have a significant effect, which makes sense, as seeding behavior is all that is being modified here. In this case, the performance is mainly affected by the utilization of leecher capacity. At the other extreme, when the system’s seeding capacity is very high relative to download capacity (either due to long seeding times or to having a large fraction of seeds with high upload capacities), our schemes do not have a significant effect. This also makes sense, as in that case resources are plentiful. A formal analysis of our seeding approaches using our proposed BT model can be found in Chapter 4. Additionally, our scheme can easily work with other schemes that have been pro- posed to handle free-riders (e.g., [47, 64]). As these schemes do not focus on approaches to distributing seeding capacity, our schemes can serve as a good complement. Overall Performance Improvement Our simulations indicate (refer to Figure 3.3) that re-directing seeding capacity to end 30 points improves the performance of contributing nodes, especially the slower ones. This confirms our intuition that appropriate prioritization of seeding capacity is useful. We speculate that in a real system our approaches will result in an even greater performance improvement. Consider Figure 3.2, where nodes using the original protocol have a slower downloading rate at the end points of the downloading process. In a real system, the slow start of newly joined nodes is even worse because it takes awhile to gather the peer list and successfully connect to enough peers, which leads to wasted upload resources of new nodes. Using our schemes the seeds can “kick start” a new node faster (as it rises to the top of their list for unchoking) so that it can start contributing its upload capacity quickly. Since simulation abstractions hide many of the delays, the performance gains of our technique are not visible to the fullest extent. Possible Consequence Even if our schemes do not improve the performance of the contributing leechers much in some scenarios, they can slow down free-riders significantly in many cases. Conse- quently, in order to have reasonable performance, free-riders may choose to contribute their resources, thereby increasing the system capacity and improving overall system performance (refer to Section 3.2). Malicious Behavior One way to circumvent our scheme by free riders is for them to lie about the chunks they have so that they would fall in the “right” region from a seeds’ perspective. Using current BT clients can act as free-riders easily by limiting their upload rate – most available BT clients allow this. On the other hand, cheating in our scheme requires source code modification, which is harder. Also, modifying source code can result in exploits in the original BT protocol as well, e.g., as in [45, 56]. Of course, contributing leechers can cheat in a similar manner. 31 Specifically, one malicious exploit would be for a free-rider to lie about the chunks it has by using a modified client. We believe that it would be tough to get a huge performance gain that way. That is, to prevent a node from always claiming to be within the “right” region, a seed can keep a counter of how many chunks it uploaded to each peer such that they are eventually forced to exit the right region. In this simple scheme, the amount of data a liar can download from a particular seed is no more than the size of the right region. Of course, a node can lie differently to each seed in order to collect more chunks. However, this requires more sophisticated client modifications. Moreover, a seed can estimate the seeding capacity of a peer by looking at its own peer distribution. A seed could take the view that (roughly and conservatively) it needs to upload at most own upload capacity total seed peers capacity of the “right region” to a peer; this can be used as a reference for the limit on the amount of data it uploads to a peer. Then, ideally, a liar (on aver- age) would download the amount of data corresponding to the “right region” from all its neighboring seeds, which is essentially equivalent to the non-lying case. Another way to prevent such attach is to employ the “Allow Fast” mechanism in the BT Fast Extension [20]. The underlying idea of the “Allow Fast” extension is similar to the Threshold-based seeding approach we discussed above. Instead of having the seeds to upload only to peers within certain downloading progress, the “Allow Fast” extension allows a node to download a small set of chunks “for free” from the seeds. The set of chunks are determined based on some pseudo-random function using the IP address of the node and all seeds agree on the same set of chunks for a particular node. Our seeding approach evaluation in Section 3.4 and our optimal seeding threshold study in Chapter 4 serve as a compensation for choosing a right size of “free-chunks set” in the “Allow Fast” extension. 32 We could also consider a simple system that globally keeps track of the total number of chunks uploaded to a peer. Light weight reporting and checking mechanisms can be used, as only loosely synchronized information is enough to serve our purpose. Lastly, a free-rider can lie by assuming multiple identities in the system, which is known as Sybil attack [14]. In a BT system, a node’s IP address can be used as its identity. Thus, a node requires access to multiple IP addresses in order to achieve this exploit. This is fairly difficult to achieve for common users. Practical Implementation Our proposed approach can be easily adopted in the current BT system. No protocol changes (in terms of messages) are required as all the information is already available in the current BT protocol, and it also allows for incremental deployment. 33 Chapter 4 BitTorrent: an Extensible Heterogeneous Model and Applications Peer-to-peer (P2P) systems in general, and BitTorrent (BT) specifically, have been of significant interest to researchers and Internet users alike. A detail description of the BT protocol can be found in Section 3.1. Existing models of BT abstract away certain characteristics of the protocol that are important, which we address in this chapter. We present a simple yet accurate and easily extensible model of BT in Section 4.1. The model’s accuracy is validated through a rigorous simulation-based study and PlanetLab experiments in Section 4.2. In Section 4.3, we study the extensibility of our model by incorporating recently proposed approaches to protocol changes in BT. Particularly, we use our model to study the seeding approaches we proposed in Chapter 3. 4.1 BitTorrent Model We use a simple rate balance model, similar to [15], but we account for a number of BT characteristics (which are not included in [15]). Table 4.1 gives a summary of notation used in our model. Let h be the number of node classes, where each class is defined by its upload and download capacities, U i and D i , respectively. New nodes arrive to the system at an average rate of λ, and there is a probability of p i that a newly arrived 34 Table 4.1: Notation Used in Our model h Number of node classes m File size s Peer set size λ Average new nodes arrival rate p i Probability a newly arrived node belongs to classi u i Average upload rate of a classi node d i Average download rate of a classi node N i l Average number of classi leechers in the system N i s Average number of classi seeds in the system T i l Average download time of a classi node T i s Average seeding time of a classi node x r Number of regular (T.F.T.) unchokes a node performs simultaneously x o Number of optimistic unchokes a node performs simultaneously x Number of unchokes a node performs simultaneously,x = x r +x o q i,j Average fraction of regular unchokes from classi that will go to classj o i,j Average fraction of optimistic unchokes from classi that will go to classj t reg Regular unchoking re-evaluation interval t opt Optimistic unchoking re-evaluation interval t win History window size for re-evaluation of both unchoking mechanisms f opt Average fraction of regular unchoking capacity a slow node spends on a fast node due to a fast node’s optimistic unchoke g i,j Average number of optimistic unchokes that a classi node receives from classj nodes node belongs to class i - thus, class i nodes arrive at a rate of λ i = p i λ. A class i node downloads at the rate of d i chunks per time unit (d i ≤ D i ), and it uploads at the rate ofu i chunks per time unit. Since (a) Internet upload and download capacities are often asymmetric and (b) some users tend to limit their upload capacities in BT, we assume that the upload links are the bottlenecks, i.e.,u i = U i . This is a fairly typical assumption in the literature, e.g., it is also made in [15]. In steady state, there are N i l class i leechers in the system and N i s class i seeds, whereN i = N i l +N i s represents the total number of classi nodes. LetT i l be the average amount of time a class i leecher takes to download a file. Let m be the number of data 35 chunks in the file; then, d i = m T i l . Let T i s be the average amount of time a class i node stays in the system after becoming a seed. Thus, T i = T i l +T i s is the average amount of time a classi node stays in the system. Then, we can state that at any time, the total download rate of all leechers is equal to the total upload rate from all leechers and all seeds, i.e.: h X i=1 u i N i = h X i=1 d i N i l . Next, we build on this fairly coarse model while making some standard assumptions, including that the system is in steady state and that it has a sufficiently large number of nodes where each peer has a sufficiently large number of neighbors 1 . Since in BT a node receives its downloads from other leechers (via regular and optimistic unchokes) and from seeds, what is needed is a model of how much download capacity a node will receive due to each. That is what we proceed to derive next. We do this first by assuming that there is (a) perfect clustering among nodes of the same class and (b) that the optimistic unchokes are unbiased. (This also corresponds to the assumptions made in [15].) We then show how to relax these assumptions in order to obtain a more accurate model. Regular Unchokes: We first assume perfect clustering, i.e., the exchange of chunks due to TFT only occurs between nodes of the same class. Let x be the number of simultaneous unchokes performed by a node, where x r of these correspond to regular unchokes (due to TFT) and the remaining x o correspond to optimistic unchokes. Since (given perfect clustering) the upload capacity due to regular unchokes of a classi node 1 We also do not model the initial slow startup of newly arrived nodes, and assume that leechers do not abort in the middle of a download. 36 is distributed to nodes of class i only, we describe the download rate of a class i node due to regular unchokes (from leechers of the same class),d i reg , as: d i reg = N i l u ixr x N i l = u i x r x . Optimistic Unchokes: Here, we assume optimistic unchokes are unbiased, i.e., that they are uniformly distributed among all leechers in the system. Let N l be the total number of leechers in the system, i.e.,N l = P h j=1 N j l ; we can then describe the down- load rate that a classi node receives due to optimistic unchokes (from all other leechers), d i opt , as: d i opt = P h j=1 N j l u jxo x N l . 4.1.1 Seeds and Free-Riders Seed Unchokes: We assume seeds upload uniformly to all leechers (approximating BT version4.0.0) and describe the download rate of a classi node from all seeds,d i seed , as: d i seed = P h j=1 N j s u j N l . Given above, we can state the following for the class i node’s download rate, d i = d i reg +d i opt +d i seed : d i = u i x r x + P h j=1 N j l u jxo x N l + P h j=1 N j s u j N l . (4.1) 37 Next, we apply Little’s Result to obtain: N i = λ i T i ,N i l = λ i T i l , N i s = N i −N i l = λ i (T i −T i l ) = λ i T i s . Since,T i l = m d i , we can plug this, along withλ i = p i λ, into Equation (4.1), to obtain: d i = u i x r x + P h j=1 p j λ m d j u jxo x P h j=1 p j λ m d j + P h j=1 p j λT j s u j P h j=1 p j λ m d j = u i x r x + P h j=1 p j u j xo d j x P h j=1 p j d j + P h j=1 p j T j s u j P h j=1 p j m d j . (4.2) Thus, we have a set ofh equations withh unknowns,d 1 ,d 2 ,...,d h , which can be solved to obtain the average download rate of each class. Note that the above model degenerates to the one presented in [15], if we ignore the seeds. However, as noted in Section 1.2 and Chapter 3, seeds have a significant effect on BT systems and thus need to be modeled. In addition, free-riders have a significant effect as well; thus, we address inclusion of free-riders in the model next. Free-Riders: A simple approach to including free-riders in this model is to view them as another user class (or multiple classes, if they have different download capacities), all with upload capacity of 0. Then, the download rate of a free-riding class is described as that of a contributing leecher class, but with the first term (corresponding to TFT) in Equation (4.2) dropped; e.g., if we have one free-riding class, say classh, then d h = P h j=1 p j u j xo d j x P h j=1 p j d j + P h j=1 p j T j s u j P h j=1 p j m d j , 38 would be its download rate, where classes 1 through h−1 correspond to contributing leechers - their download rate equations remain as in Equation (4.2), withu h = 0. This, of course, can be done for multiple free-riding classes. We believe that the basic model presented above is quite adaptable; thus future archi- tectural and protocol changes can be incorporated and studied through it (refer to Sec- tion 4.3). However, we first refine our model to make it more accurate. 4.1.2 More Realistic Model The above model assumes that regular unchokes are perfectly clustered and that opti- mistic unchokes are unbiased (i.e., as in [15]). However, evidence, based on measure- ments of real BT systems in [36] and our simulations and PlanetLab experiments in Section 4.2, clearly indicates that significant imperfect clustering (in regular unchokes) and biased optimistic unchokes exist. Thus, here we remove these assumptions to more accurately account for how the real BT protocol works. Specifically, we first modify the model in a more abstract manner, and then further develop it based on potential causes of imperfect clustering and biased optimistic unchokes. Firstly, we take into account the fact that a fraction of regular unchokes will go to nodes in other classes. Specifically, we defineq i,j to be the fraction of regular unchokes from class i that will go to class j, with P h j=1 q i,j = 1. Given q i,j , the download rate that a classi node receives from regular unchokes from all nodes,d i ′ reg , can be described as: d i ′ reg = P h j=1 q j,i N j l u jxr x N i l . Secondly, we consider the fact that optimistic unchokes of a node are not distributed evenly to all leechers. Specifically, we define o i,j to be the fraction of optimistic 39 unchokes from class i that will go to class j, where P h j=1 o i,j = 1. Given o i,j , the download rate a classi node receives from optimistic unchokes from all nodes,d i ′ opt , can be described as: d i ′ opt = P h j=1 o j,i N j l u jxo x N i l . Given these adjustments to regular and optimistic unchokes, we can express our new model for the download rate of a classi node,d i ′ = d i ′ reg +d i ′ opt +d i seed , as: d i ′ = P h j=1 q j,i N j l u jxr x N i l + P h j=1 o j,i N j l u jxo x N i l + P h j=1 N j s u j N l = d i ′ p i h X j=1 (q j,i x r +o j,i x o )p j u j d j ′ x + P h j=1 p j T j s u j P h j=1 p j m d j ′ , (4.3) where the main challenge is in determiningq i,j ando i,j . To do this, we examine possible causes for imperfect clustering and biased optimistic unchokes. Imperfect Clustering: Although there are a number of potential causes, we conjecture that the main cause of imperfect clustering is that optimistic unchokes from faster classes of nodes destroy clustering of slower classes of nodes. Specifically, when a fast node optimistically unchokes a slow node, the slow node reciprocates with a regular unchoke (as it is getting a high download rate from that fast node). For ease of exposition, we first focus on a two class (slow and fast) scenario. We assume that the fast nodes are perfectly clustered (q fast,fast = 1 and q fast,slow = 0). We then derive the effects of imperfect clustering on slow nodes by considering the timing of regular and optimistic unchokes between slow and fast nodes 2 . Specifically, 2 This is similar to the derivation in [44], which was done in the context of a flash crowd, rather than a steady state, model. 40 Fast Node Unchokes Slow Nodes Slow Node Unchokes Fast Node Case (i) Case (ii) T T+t reg t opt -t reg +t win t opt time t opt t opt +t win T T+t reg time Figure 4.1: Slow Node Imperfect Clustering the reciprocation of slow nodes with regular unchokes to optimistic unchokes of fast nodes is temporary. As a fast node realizes that it unchoked a slow node (due to a relatively slow download rate from that node), it discards the slow node. Since the TFT mechanism re-evaluates choices of best download rates using a sliding window, the effect of the fast node’s optimistic unchoke eventually wears off, at which point the slow node stops the regular unchoke of the fast node. The extreme cases of how this can happen are illustrated in Figure 4.1 3 . Here we define t opt as the optimistic unchoking re-evaluation interval, t reg as the regular unchoking re-evaluation interval, and t win as the history window size for re-evaluation of both unchoking mechanisms. We now examine the two cases in Figure 4.1. If the optimistic unchoke arrives right after the regular unchoke re-evaluation period of a slow node, say T (as depicted by the shaded box in Case (i)), then the slow node reciprocates with a regular unchoke to the fast node, starting in the next re-evaluation interval,T+t reg , for a time interval oft opt −t reg +t win , as depicted by the clear box in the figure. At the other extreme, if the unchoke arrives right before the next re-evaluation interval, i.e., Case (ii), the slow node reciprocates with a regular unchoke to the fast node, for a time interval of t opt + t win . Here, we assume that the unchokes arrive uniformly during the regular unchoking re-evaluation interval. As the slow node does x r regular unchokes at the same time, on average, the 3 A similar method can be used to model the effect of fast nodes doing regular unchokes to slow nodes. However, since measurement and simulation evidence indicates that this is a less frequent case, we omit it here as it would further complicate the model. 41 fraction of the regular unchoking capacity of a slow node that is spent on a fast node due to a fast node’s optimistic unchoke,f opt , can be described as: f opt = 1 2 (t opt −t reg +t win )+(t opt +t win ) x r t opt . We can then approximateq slow,fast , as q slow,fast = min(g opt f opt ,1), (4.4) where g opt is the average number of optimistic unchokes that a slow node is receiving from fast nodes 4 . Note that, q slow,slow = 1− q slow,fast . We can approximate g opt by looking at the average number of fast and slow peers of a node. We defines as the peer set size of a node (i.e., the number of peers to which that node connects and exchanges data with). Let N be the total number of nodes in the system, i.e., N = P h j=1 N j , We can approximateg opt by: g opt = s N fast l N x o (o fast,slow ) 1 s N slow l N = N fast l N slow l x o (o fast,slow ), whereo fast,slow is the fraction of optimistic unchokes from the fast class that goes to the slow class (as discussed in more detail below). Here,s N fast l N ands N slow l N are the average number of fast peers and slow peers of a node, respectively. 4 In cases where g opt f opt > 1, all the regular unchoking capacity of a slow node is spent on g opt fast nodes. Although effectively this means that the fraction of regular unchoking capacity spent on each fast node would be less than f opt , this will not affect our results as in our model we are only interested in q slow,fast . 42 We now show how to extend this to more than two classes. Without loss of general- ity, we assume that the class indices are in descending order of their uploading capaci- ties. As before, we assume that faster nodes do not experience imperfect clustering with slower nodes, i.e., q i,j = 0,∀j > i. Thus, there are no regular unchokes from a faster node to a slower node. For the regular unchokes of faster nodes by slower nodes, we can use the following approximation: q i,j = min min(g i,j f opt ,1),1− j−1 X k=1 q i,k , (4.5) where q i,i = 1− P i−1 j=1 q i,j , q i,1 = min(g i,1 f opt ,1), and g i,j is the average number of optimistic unchokes that a class i node is receiving from class j nodes, which can be described as: g i,j = s N j l N x o o j,i 1 s N i l N = N j l N i l x o o j,i . The idea is that when we consider the regular unchoking capacity due to imperfect clustering of class i, we assume that as much as possible of it first goes to the fastest class in the system, i.e., min(g i,1 f opt ,1). Then, as much as possible of the remainder goes to the second fastest class, and so on. Biased Optimistic Unchoking: Although usually modeled as being uniform among all leechers (as in [15]), optimistic unchokes are in fact biased in the real world 5 , with the 5 We also observe this in our experiments. 43 reason being that optimistic unchokes are only performed on peers that are not (cur- rently) unchoked through regular unchokes. Thus, given q i,j , we can approximate o i,j as: o i,j = s N j l N −x r q i,j P h k=1 (s N k l N −x r q i,k ) . (4.6) Here, s N j l N is the average number of class j peers and x r q i,j is the average number of regular unchokes that a class i node performs towards class j peers. The difference between these two represents, for a class i node, the average number of class j peers that are not unchoked using regular unchokes, i.e., potential class j peers that can be optimistically unchoked. We assume that s N j l N is larger than x r q i,j , for all i,j – this is reasonable since s (the peer set size) is typically much larger than x r (the number of regular unchokes a node performs simultaneously). Note that, the more perfect is the clustering (due to regular unchokes), the more biased are the optimistic unchokes. Also, the bias in optimistic unchokes favors nodes in other classes, which further destroys the clustering in regular unchokes of slower nodes. However, the biasing effect becomes less significant when the peer set size, s, grows. The model can now be solved numerically, e.g., using fixed point iteration. 4.2 Model Validation and Insight In this section, we validate the model proposed in Section 4.1 using simulations and PlanetLab experiments. We also illustrate how our model can be used to obtain insight into the BT system. We use both, simulations and PlanetLab experiments - simulations 44 Table 4.2: Simulation Settings Filesize (m) 500 MB (2000 Chunks, 256 KB each) Average node inter-arrival ( 1 λ ) 1 min Peer Set Size (s) 80 Number of Leecher Unchokes 4 Regular (x t ) + 2 Optimistic (x o ) Number of Seed Unchokes 6 Unchoke Re-evaluation Interval Regular (t reg ): 5 sec; Optimistic (t opt ): 30 sec Re-evaluation History (t win ) 20 sec enable larger scale studies while PlanetLab experiments enable more realistic experi- ments, as they include protocol and network properties that are abstracted by the simu- lator. 4.2.1 Simulations We use the BT simulator provided by [6] (also used by other researchers in the commu- nity), with modifications described below. The simulator is event-based and simulates the chunk exchange mechanism of the BT protocol. We extended the simulator to support: (a) nodes staying around as seeds, (b) node arrivals, and (c) nodes acting as free-riders (i.e., nodes that do not unchoke and leave the system upon download completion). The seeding times and node inter-arrival times follow an exponential distribution. We also modified the original seeding scheme to be more uniform, in-line with the BT version 4.0.0. Moreover, we also fixed a bug in the original simulator that affected the selection of peers for unchoking - the original simulator incorrectly implemented that part of the BT protocol, which resulted in a higher probability of unchoking previously unchoked peers. Unless specified otherwise, the following results correspond to the simulation set- tings in Table 4.2. The system starts with1 origin seed with a1000 kbps upload capacity, that stays in the system for12 hours. Nodes arrive to the system from a Poisson process 45 Table 4.3: Class Description (Two Contributing Classes) Class Download Capacity Upload Capacity Fast 5000kbps 512kbps Slow 5000kbps 128kbps with a rate λ and are assigned to a particular class according to a given distribution. The classes differ in their upload and download capacities. We consider the steady state behavior of the system. Each simulation run corresponds to 63 hours, where we only compute our results over the last 48 hours. (We check our results to make sure the system passes the ramp up stage during the first15 hours.) The model-based results are obtained numerically, using a fixed point iteration method. These solutions converge quickly, even for multiple class cases. We note that a single simulation run (i.e., one point in a figure) takes more than 10 hours on a reasonable Core 2 Duo machine while our model can compute the entire figure in less than 1 sec. Thus, our model provides a much faster way (than simulation or real experiments) of exploring system parameters and design choices. In what follows, “Model-PC” refers to the basic model, i.e., described by Equa- tion 4.2, and “Model-IC” refers to our final model with imperfect clustering and biased optimistic unchoking enhancements, i.e., described by Equation 4.3. Experiment 1: Two Leecher Classes: We consider a system with two classes and bandwidth settings given in Table 4.3 and no seeding time. Figure 4.2 depicts the result- ing download times as a function of the percentage of node arrivals corresponding to the fast class, where we observe the following. Model-IC is much more accurate in predicting the download time of the slow class. For example, when80% of the arriving nodes are from the fast class, Model-PC’s prediction for the slow class download time differs from simulation by≈ 22% while Model-IC’s prediction differs by less than1%. The error in Model-PC is mainly due to neglecting imperfect clustering. This can be 46 100 150 200 250 300 350 400 450 500 550 20 30 40 50 60 70 80 90 Average Download Time (min) Fast Node Arrival % Slow (Sim) Slow (Model-IC) Slow (Model-PC) Fast (Sim) Fast (Model-IC) Fast (Model-PC) Figure 4.2: Different Mix of Two CL Classes observed from there being a larger error with a higher percentage of fast nodes and from Figure 4.3 which depicts the fraction of the regular unchoking capacity that is not dis- tributed to nodes of the same class, obtained from the simulation and from Model-IC. As can be seen, slow nodes have a higher fraction of imperfect clustering when there is a higher percentage of fast nodes among the arrivals. Model-IC can predict the trend in the imperfect clustering of slow nodes; how- ever, it under-estimates the number, which can be explained as follows. While in the model we assumed that fast nodes have perfect clustering, simulation results indicate that some degree of imperfect clustering exists among fast nodes as well. Thus, the under-estimation of imperfect clustering of slow nodes is due to the imperfect clustering of fast nodes - the regular unchokes from fast nodes to slow nodes further degrade clus- tering of slow nodes, similarly to the degradation due to fast nodes’ optimistic unchokes 47 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 20 30 40 50 60 70 80 90 Imperfect Clustering Fraction Fast Node Arrival % Slow (Sim) Slow (Model-IC) Fast (Sim) Figure 4.3: Imperfect Clustering Fraction of slow nodes. To correct for this phenomenon, we can use a similar derivation to the one given in Section 4.1 6 . Experiment 2: Three Leecher Classes: Next, we look at a system with three classes, where we add a “Super-Fast” class, with upload (download) capacity of 1000kbps (5000kbps), to the classes given in Table 4.3 (again, with no seeding time at first). Fig- ure 4.4 depicts the average download times of the three classes as a function of the per- centage of super-fast node arrivals (with the remainder of arrivals split evenly between the fast and slow classes). Observe that Model-IC predicts the download times quite accurately, while Model-PC results in a larger error - e.g., when 70% of the arrivals are from the super-fast class, Model-PC has an error of≈ 20% and≈ 16% for the slow and the fast class, respectively, while the corresponding errors in Model-IC are only≈ 2% and≈ 5%. This is mainly due to neglecting of imperfect clustering in Model-PC. 6 Although the degree of imperfect clustering is under-estimated, the download time prediction is accu- rate, e.g., because the loss of regular unchoking capacity within the slow class is compensated by the fast nodes’ regular unchoking capacity given to the slow class. 48 50 100 150 200 250 300 350 400 450 500 10 20 30 40 50 60 70 80 90 Average Download Time (min) Fast Node Arrival % Slow (Sim) Slow (Model-IC) Slow (Model-PC) Fast (Sim) Fast (Model-IC) Fast (Model-PC) Super-Fast (Sim) Super-Fast (Model-IC) Super-Fast (Model-PC) Figure 4.4: Different Mix of Three Contributing Classes Table 4.4: Class Description (with Free-Riders) Class Fraction Download Capacity Upload Capacity Contributing 80% 5000kbps 512kbps Free-Riders 20% 5000kbps 0 To further illustrate the difference between Model-IC and Model-PC, in Figure 4.5 we depict the average download times of the three classes as a function of the average seeding time, with70%, 15%, and15% of the arrivals corresponding to super-fast, fast, and slow classes, respectively. From the results, we observe that Model-IC can predict the download times of all classes quite well. Again, we note that in this figure, Model- PC would degenerate to the model in [15], at0 seeding time, but since the model in [15] does not include seeding behavior, its prediction would not change as the seeding time is increased. Experiment 3: Free-Riders: We now focus on free-riders. For clarity of presentation, we look at the case with one contributing leecher class (CL) and one free-riders class 49 50 100 150 200 250 300 0 5 10 15 20 25 30 Average Download Time (min) Average Seed Time (min) Slow (Sim) Slow (Model-IC) Slow (Model-PC) Fast (Sim) Fast (Model-IC) Fast (Model-PC) Super-Fast (Sim) Super-Fast (Model-IC) Super-Fast (Model-PC) Figure 4.5: Different Seed Time (Three Classes) (FR), with bandwidth settings given in Table 4.4. Figure 4.6 depicts the average down- load times of the contributing leechers and free-riders as a function of the fraction of percentage of free-riders in the arriving nodes, where contributing nodes have an aver- age seeding time of 120 min. We observe the following. Both models can predict the download times of contributing leechers and free-riders quite accurately. The existence of free-riders slows down contributing leechers. Thus, it is important to include free- riders in the model. Moreover, improvements in contributing leechers’ download times can be achieved by discouraging free-riders, e.g., if we can reduce the percentage of free-riders from20% to5%, we can speed up the downloads of contributing leechers by more than50%. Thus, it is worth considering schemes which discourage free-riding, as discussed in Chapter 3. Figure 4.7 depicts the average download times of contributing leechers and free- riders as a function of the contributing nodes’ average seeding time, with bandwidth settings given in Table 4.4. Observe that both models can predict download times quite well when the seeding time is relatively high. Since the seed unchokes are modeled 50 0 100 200 300 400 500 600 0 10 20 30 40 50 Average Download Time (min) Free-Riders Arrival % FR (Sim) FR (Model-IC) FR (Model-PC) CL (Sim) CL (Model-IC) CL (Model-PC) Figure 4.6: Different Fraction of Free-Riders fairly accurately in both models and since the effect of seeding capacities dominates as that capacity grows, Model-PC exhibits similar accuracy to Model-IC at higher seeding capacity. Model-IC is more accurate when the seeding time is shorter, e.g., with no seeding time, Model-PC’s prediction of free-riders’ download time differs from that of simulation by≈ 16%, whereas that of Model-IC’s only differs by≈ 2%. As there is only one contributing class, the regular unchokes are close to perfect (as also observed in the simulations); thus, the improvements in Model-IC are due to us modeling the bias in optimistic unchokes. Moreover, the free-riders’ download time is quite sensitive to the seeding time, e.g., the download times of free-riders is reduced by≈ 54% when the average seeding time of contributing nodes increases from 0 to 30 min, while the download time of contributing nodes is only reduced by≈ 18%. This suggests the need for re-considering appropriate use of seeding capacity, e.g., as discussed in Chapter 3 and modeled in Section 4.3. In Figures 4.6 and 4.7, the model in [15] would give the same results as Model-PC, at the points where the fraction of free-riders and the seeding time are both0. However, 51 0 100 200 300 400 500 600 700 800 900 1000 0 20 40 60 80 100 120 140 Average Download Time (min) Average Seed Time (min) FR (Sim) FR (Model-IC) FR (Model-PC) CL (Sim) CL (Model-IC) CL (Model-PC) Figure 4.7: Different Seed Time (with FR) it would not be difficult to include free riders in the model in [15], similarly to the approach we have taken in Section 4.1. Experiment 4: Peer Set Size: Figure 4.8 depicts the download times of free-riders and contributing leechers as a function of a node’s peer set size 7 . We consider the case with no seeding and bandwidth settings given in Table 4.4. Observe that the peer set size has a significant effect on free-riders’ download time (especially when it is small), while having little or no effect on contributing leechers. The free-riders download faster when the peer set size is smaller, e.g., the free-riders’ download time is reduced by≈ 40% when the peer set size is reduced from80 to20. This occurs because under smaller peer set sizes, biased optimistic unchoking favors free-riders more. Model-IC captures this behavior well through biased optimistic unchoking. Model-PC does not consider peer set sizes, and thus in Figure 4.8 it results in an upper bound for Model-IC as the peer set size approaches infinity. 7 Here we vary the peer set size of all nodes while only the free-riders’ peer set size is varied in the large view exploit experiment. 52 100 200 300 400 500 600 700 800 900 1000 1100 10 20 30 40 50 60 70 80 90 100 110 120 Average Download Time (min) Peer Set Size FR (Sim) FR (Model-IC) FR (Model-PC) CL (Sim) CL (Model-IC) CL (Model-PC) Figure 4.8: Different Peer Set Size 4.2.2 Real-world Experiments In this subsection, we validate the accuracy of our model through real-world experi- ments using PlanetLab. Based on our observations from these experiments, we suggest extensions to our model that would further improve its accuracy. The BT client we use in our PlanetLab experiments is BitTornado (version 0.3.18), which is an open-source implementation of the standard BT protocol. The seed unchokes have been changed to be more uniform, in-line with BT version 4.0.0 (as in our simulations). Unless specified otherwise, the settings in the PlanetLab experiments remain the same as in our simulations, i.e., as given in Table 4.2. A controller server is used to schedule the node arrivals to the system from a Poisson process with a rate λ. Each node is assigned to a particular class according to a given distribution. The classes differ in their upload capacities (by setting the upload rate limit in the BT client), which are given in Table 4.3. We do not limit the downloading rates of the BT client in our PlanetLab experiments. 53 The system starts with 1 origin seed with a 2048 kbps upload rate, that stays in the system for7 hours. Each experiment runs until1400 nodes finish downloading the target file. After a node leaves the system (either upon download completion or after staying as a seed, depending on the experiment settings), it may rejoin the system several times (as a “new” node) since only a limited number of nodes is under our control in the PlanetLab testbed. That is, the scale of our experiments is limited by the PlanetLab usage policies, i.e., our experiments cannot exceed (i) PlanetLab’s daily quota for network traffic as well as (ii) the number of nodes under our control (i.e.,≈ 360). The results given below are computed after the system has been running for 20 hours, as our focus is on steady state behavior. In what follows, “Model-PC” refers to the basic model, i.e., described by Equa- tion 4.2, and “Model-IC” refers to our final model with imperfect clustering and biased optimistic unchoking enhancements, i.e., described by Equation 4.3. Although most PlanetLab nodes have average uploading capacities that are larger than the desired upload rates given in Table 4.3, the achieved upload rates in the BT client are normally lower than the desired upload rates. The measured average upload- ing rates are approximately 90% of the desired values (i.e., upload rate limits set in the BT clients). This is due to the TCP congestion control mechanism in the BT data con- nections and the occasional heavy cross-traffic in the network. We use the measured average uploading rate as input parameters in Model-IC and Model-PC for an accurate comparison of model and PlanetLab results. Similar observations are made in the peer set sizes of nodes. The measured average peer set size of nodes are approximately80% of the desired values. This is due to the peer list updating frequency limit in the tracker. Again, we use the measured average peer set sizes as input parameters in Model-IC. Experiment 5: Two Leecher Classes (PlanetLab): We consider a system with two classes and bandwidth settings given in Table 4.3 and no seeding time. Figure 4.9 depicts 54 0 100 200 300 400 500 600 20 30 40 50 60 70 80 Average Download Time (min) Fast Node Arrival % Slow (PlanetLab) Slow (Model-IC) Slow (Model-PC) Fast (PlanetLab) Fast (Model-IC) Fast (Model-PC) Figure 4.9: Different Mix of Two CL Classes (PlanetLab) the resulting download times as a function of the percentage of node arrivals correspond- ing to the fast class, where we observe the following. Similarly to the simulation-based Experiment 1 in Section 4.2.1, we observe that Model-IC is much more accurate in pre- dicting the download time of the slow class in our real-world experiment. For example, when 60% of the arriving nodes are from the fast class, Model-PC’s prediction for the slow class download time differs from PlanetLab experiment by≈ 20% while Model- IC’s prediction differs by less than 1%. Again, the error in Model-PC is mainly due to neglecting imperfect clustering. This can be observed from there being a larger error with a higher percentage of fast nodes and from Figure 4.10 which depicts the frac- tion of the regular unchoking capacity that is not distributed to nodes of the same class, obtained from the experiment and from Model-IC. As can be seen, slow nodes have a higher fraction of imperfect clustering when there is a higher percentage of fast nodes among the arrivals. Model-IC can predict the trend in the imperfect clustering of slow nodes, however, it under-estimates the number, as discussed in Section 4.2.1. 55 0 0.1 0.2 0.3 0.4 0.5 0.6 20 30 40 50 60 70 80 Imperfect Clustering Fraction Fast Node Arrival % Slow (PlanetLab) Slow (Model-IC) Fast (PlanetLab) Figure 4.10: Imperfect Clustering Fraction (PlanetLab) Experiment 6: Average Seed Time (PlanetLab): Figure 4.11 depicts the average download times of fast and slow nodes as a function of the average seeding time. We use the bandwidth settings given in Table 4.3 with 40% of node arrivals from the slow class. Again, we observe that Model-IC can predict the download times for both classes quite well. We also note that Model-PC would degenerate to the model in [15], at 0 seeding time in Figure 4.11. However, since the model in [15] does not include seeding behavior, its prediction would not change as the seeding time is increased and hence its accuracy would suffer greatly. We observe that there is a larger error when the average seeding time is long. For example, when the average seeding time is 120 min, Model-PC and Model-IC have an error of≈ 19% and≈ 27% for the slow class, respectively. This is due to the shorter average download time (when the average seeding time is longer) and the relatively greater effect of the download rate ramp up period when a node just joins the system. A more detailed study of this effect is presented in Experiment 9 below. 56 0 100 200 300 400 500 600 0 20 40 60 80 100 120 Average Download Time (min) Average Seed Time (min) Slow (PlanetLab) Slow (Model-IC) Slow (Model-PC) Fast (PlanetLab) Fast (Model-IC) Fast (Model-PC) Figure 4.11: Different Seed Time (PlanetLab) Experiment 7: Peer Set Size (PlanetLab): Figure 4.12 depicts the download times of free-riders and contributing leechers as a function of a node’s peer set size 8 . We consider the case with 60 min average seeding time and bandwidth settings given in Table 4.4. Similarly to Experiment 4 in Section 4.2.1, we observe that the peer set size has a significant effect on free-riders’ download time (especially when the peer set size is small), while having little or no effect on contributing leechers. The free-riders download faster when the peer set size is smaller, e.g., the free-riders’ download time is reduced by≈ 15% when the peer set size is reduced from80 to20. This occurs because under smaller peer set sizes, biased optimistic unchoking favors free-riders more. Again, Model-IC captures this behavior well through biased optimistic unchoking. The effect of peer set size is less significant here than in Experiment 4 in Sec- tion 4.2.1 because we introduced seeding time in the PlanetLab experiment (to prevent nodes from downloading too slowly and thus exhausting all available nodes). Com- pared to other PlanetLab experiments above, we also observe a larger error in download 8 Here again we vary the peer set size of all nodes while only the free-riders’ peer set size is varied in the large view exploit experiment. 57 100 150 200 250 300 350 20 30 40 50 60 70 80 Average Download Time (min) Peer Set Size FR (PlanetLab) FR (Model-IC) FR (Model-PC) CL (PlanetLab) CL (Model-IC) CL (Model-PC) Figure 4.12: Different Peer Set Size (PlanetLab) time prediction. This is due to the biased peer distribution, which particularly affects the seeds. As shown in Table 4.5, on average, the seeds have more free-riders as their neigh- bors than the system wide free-riders distribution. This is the case not only for seeds, but for “older” nodes (i.e., nodes that have been in the system longer) in general. This biased peer distribution is due to the peer list update artifact of the BT client. Specif- ically, among a node’s neighbors, those that contribute upload faster, thus also leaving the system earlier. This effectively decreases a node’s the peer set size (which we also measured) and affects the peer distribution. A node’s peer set is only replenished when (1) that node is contacted by new nodes or (2) that node’s peer set size falls below a certain threshold. All this contributes to a further bias in optimistic unchokes of a seed, i.e., in a real system seeds have a higher probability of unchoking a free-rider than in our model. This effect is more pronounced when the downloading times of nodes from dif- ferent classes have a larger disparity. In order to adjust for this behavior, we can model the peer set classes distribution as a function of the average download time of nodes in different classes. 58 Table 4.5: Fraction of Free-Riders Peer Set Size 20 40 60 80 System Wide 27% 31% 31% 32% Neighbors of Seeds 34% 37% 38% 38% 50 100 150 200 250 300 350 400 450 500 550 20 30 40 50 60 70 80 Average Download Time (min) Fast Node Arrival % Slow (PL) Slow (Model-IC) Slow (Model-PC) Fast (PL) Fast (Model-IC) Fast (Model-PC) Super-Fast (PL) Super-Fast (Model-IC) Super-Fast (Model-PC) Figure 4.13: Different Mix of Three CL Classes (PlanetLab) Experiment 8: Three Leecher Classes (PlanetLab): Next, we look at a system with three classes, where we add a “Super-Fast” class, with upload capacity of 1000kbps to the classes given in Table 4.3 with no seeding time. Figure 4.13 depicts the average download times of the three classes as a function of the percentage of super-fast node arrivals (with the remainder of arrivals split evenly between the fast and slow classes). We observe that, in general, Model-IC predicts the download times more accurately than the Model-PC. This is mainly due to neglecting of imperfect clustering in Model-PC. Compare to the two classes experiments in Experiment 1, we observe a larger error. It is because many of the Super-Fast nodes can not sustain a upload rate of 1000kbps. The measured average upload rate is only≈ 70% of the desired value. The higher 59 50 100 150 200 250 300 350 400 450 500 550 200 250 300 350 400 450 500 Average Download Time (min) Fast Node Arrival % Slow (PlanetLab) Slow (Model-IC) Slow (Model-PC) Fast (PlanetLab) Fast (Model-IC) Fast (Model-PC) Figure 4.14: Different File Size (PlanetLab) variance in the upload rate of Super-Fast nodes together with the smaller difference in the Fast and the Super-Fast nodes upload rate caused the larger errors. Experiment 9: Different File Sizes (PlanetLab): Here, we study the effect of different file sizes using the two-class settings given in Table 4.3. Figure 4.14 depicts the average download times of the two classes when the file sizes are 200 MB, 300 MB, 400 MB, and 500 MB. We observe that Model-IC is more accurate than Model-PC in predicting the download times of nodes. However, the errors are larger when the file sizes are smaller. This is due to the shorter average download time (when the file size is small) and the relatively greater effect of the download rate ramp up period, which occurs when a node just joins the system. Figure 4.15 shows the download rate of the fast nodes along with the percentage of the file downloaded, for each file size. We observe that the time for the download rate to reach a “steady rate” becomes a larger part of the total download time when the file size is smaller. This type of ramp up behavior is not considered in our model. In order to account for this phenomenon, we can model the download rate as a time dependent function. A simple approach would be to divide the download process 60 0 0.05 0.1 0.15 0.2 0.25 0 20 40 60 80 100 Download Rate (# chunk/sec) Download Progress (%) 200MB 300MB 400MB 500MB Figure 4.15: Fast Nodes Download Rate vs Download % (PlanetLab) into the initial part (where the rate increases with time) and the steady part (where the rate remains the same). Summary: The results in this section demonstrated that our model is accurate, using both, simulations and PlanetLab experiments. Hence, we believe that our model can be used to study the BT protocol as well as protocol variations, as shown in the next section. Specifically, we observed the followings in our experiments: • Imperfect clustering in regular (TFT-based) unchokes exists in simulations and real world experiments. A significant amount of regular unchokes’ capacity of the slow nodes is spent on faster nodes. Our proposed model captures the essence of imperfect clustering and provides a more accurate download time prediction as compared to a model that assumes perfect regular unchokes clustering, e.g., as in [15]. • Optimistic unchokes are biased in BT (i.e., favoring nodes that receive fewer regu- lar unchokes), particularly when the peer set size is small. Our model accounts for 61 this behavior and provides a more accurate download time prediction as compared to a model that assumes uniform optimistic unchokes, e.g., as in [15]. • The existence of seeds and free-riders results in significant effects on the system’s performance. Thus, it is important to include seeds’ and free-riders’ behavior in a heterogeneous BT model, which is a unique characteristic of our model as well. We also note that PlanetLab experiments indicate that our model has a larger error under certain conditions, e.g., small file sizes or surplus upload capacities, which indi- cates that there is room for model improvement. 4.3 Model Applications In this section we demonstrate the extensibility of our model by explicitly modeling a well known exploit of BT and our proposed fix in Chapter 3. We also discuss how to adapt our model for other BT variations. These protocol changes are used as an example; and other protocol variations can be easily incorporated. Modeling an Exploit: We first show how to incorporate the so-called large view exploit [46, 64] into our model. The basic idea behind the large view exploit is for free-riders to increase their peer set size in order to increase the probability of being optimistically unchoked by a leecher or picked by a seed’s unchoking mechanism. Theoretically, a free-rider can increase the download rate linearly with the increase in the peer set size. We can adapt both models to include large view exploit behavior of free-riders; For clarity of presentation, we illustrate the adaptation of Model-PC here (adaptation of Model-IC can be done in a similar manner). We do this by adjusting Equation (4.1) as follows. Again, let class h be the free-riding class with download rate d h . And, let N LV l be the number of leechers in steady state under the large view exploit scheme. Specifically, we setN LV l = P h−1 j=1 N j l +αN h l , whereα is a function of the free-riders’ 62 200 300 400 500 600 700 800 900 1000 80 100 120 140 160 180 200 220 240 Average Download Time (min) Free-Riders Peer Set Size FR (Sim) FR (Model-IC) FR (Model-PC) CL (Sim) CL (Model-IC) CL (Model-PC) Figure 4.16: Large View Exploit desired peer set size. In the remainder of this chapter, we set α to be the ratio of free- riders’ desired peer set size to the peer set size of contributing leechers. Then, for contributing leecher classes,1≤ i≤ h−1, we have: d i = u i x r x + P h j=1 N j l u jxo x N LV l + P h j=1 N j s u j N LV l , while d h = α P h j=1 N j l u jxo x N LV l +α P h j=1 N j s u j N LV l . We validate the resulting model in Figure 4.16 where we depict the average down- load rate of contributing leechers and free-riders, as a function of increasing free-riders’ peer set size (due to the large view exploit). We consider the bandwidth settings in Table 4.4, with contributing nodes having an average seeding time of60 min, and where contributing leechers and seeds strive for a peer set of size 80 in all cases. We increase the average arrival rate to 2/min, thus sufficiently increasing the average number of nodes in the system for the large view exploit to work. Observe that free-riders can 63 improve their performance (essentially linearly) through a larger peer set size, i.e., the large view exploit works. When the free-riders’ peer set size is sufficiently large (e.g., larger than≈ 170 in this case), they can do better than contributing leechers. Overall, our model makes accurate predictions in the case of large view exploits 9 . Modeling an Exploit Fix: Real-world measurements, such as [37, 67, 18, 7], suggest that there exists a significant number of seeds in most torrents. And, as shown in Chap- ter 3 and Section 4.2: (a) performance of free-riders is quite sensitive to seeding capacity, and (b) encouraging free-riders to contribute their capacity can improve overall system performance. Thus, it is important to consider alternative approaches to distributing seeding capacity. An approach which mitigates the large view exploit through alterna- tive schemes for distribution of seeding capacity is proposed in Chapter 3. We, again, illustrate extensibility and flexibility of our model by incorporating these schemes. We first give a brief review of the schemes proposed in Chapter 3. These schemes are motivated by observing that the progress through the file downloading process is not uniform. For example, the download is slower at the beginning, when having few chunks prevents a node from being selected for uploads through tit-for-tat. The approach then is to prioritize the use of seeding capacity to those portions of the download process where it is most needed. The goal there is to (a) help contributing leechers while (b) hurting free-riders as they depend heavily on seeding capacity (as shown in Section 4.2). The specifics of the schemes in Chapter 3 are as follows. Sort-based (N): where a seed sorts its requesting neighbors based on the number of chunks they have and then unchokes the N which are furthest from the middle (based on sorting order). (Below we use the BT default ofN = 6). 9 We note that Model-IC shows a larger error than Model-PC in Figure 4.16. This is due to the achieved peer set size of the contributing leechers in simulations being actually larger than the target value, which is an artifact in the simulator when we increase the peer set size of the free-riders. This decreases the ratio of peer set size between the free-riders and contributing leechers which results in slower downloading of free-riders than that is predicted in the model. 64 Threshold-based (K,N): Threshold-based schemes are similar, except that theN cho- sen are from those which have a certain percentage of the total number of chunks, e.g., in the experiments below we unchoke those nodes which have either [0.. K∗100 2 ]% or [(100− K∗100 2 )..100]% of the chunks. Threshold Optimization: One parameter which can have a significant effect on perfor- mance is the threshold K. Intuitively, the smaller the value of K, the more aggressive is the degradation of free-riders’ performance, but the greater is the danger of degrading contributing leechers’ performance as well. To model the effect of K, we conceptually divide the file with m chunks into two parts, ofm a andm b chunks each, wherem a = mK andm b = m(1−K). The download of the part of sizem a is assisted by seeds while the part of sizem b is downloaded without assistance from seeding capacity. We define d i a as the download rate corresponding to the sub-file of size m a and d i b as the download rate for the sub-file of size m b . We also define the corresponding download times asT i la andT i lb , whereT i la = ma d i a andT i lb = m b d i b . The total download time,T i l , for the entire file is thenT i la +T i lb and the average download rate for a classi node is m T i l . We then have: d i a = P h j=1 q j,i N j l u jxr x N i l + P h j=1 o j,i N j l u jxo x N i l + P h j=1 N j s u j N la andd i b = P h j=1 q j,i N j l u jxr x N i l + P h j=1 o j,i N j l u jxo x N i l , whereN la = P h i=1 N i la andN i la = λ i T i la , i.e., hered i a benefits from the seeding capacity and that seeding capacity is shared by all leechers within the threshold (N la ) while d i b only benefits from capacity due to regular and optimistic unchokes. 65 200 300 400 500 600 700 800 900 1000 10 20 30 40 50 60 70 80 90 FR Average Download Time (min) Threshold (%) Sim Model-IC Model-PC Figure 4.17: Different Threshold (Free-Riders) Figures 4.17 and 4.18 depict the download times of free-riders and contributing leechers, respectively, as a function ofK, with the bandwidth settings given in Table 4.4 and the average seed time of 60 min. Here, we observe that our models are reasonably accurate in predicting the download time, with Model-IC again being more accurate, particularly in predicting the download time of free-riders. A number of metrics can be used to determine an optimal value ofK, e.g., one may want to slow down free-riders as much as possible. However, doing so may slow down the contributing leechers quite a bit, while at the same time wasting seeding capacity. Another possibility would be to optimize some notion of fairness, e.g., using fairness metrics in [15]. Another optimization objective might be to slow down free-riders as much as possible while not hurting the contributing leechers. We focus on refining this objective below. When we examine Figures 4.17 and 4.18, we observe that there exists a knee in the threshold curves, such that when the threshold is larger than this value, reducing the threshold has no effect on contributing leechers while free-riders continue to be slowed 66 100 110 120 130 140 150 160 10 20 30 40 50 60 70 80 90 CL Average Download Time (min) Threshold (%) Sim Model-IC Model-PC Figure 4.18: Different Thresholds (Contributing Leechers) down to some degree. Setting the threshold to this knee would be one approach to slow- ing down the free-riders as much as possible while not hurting contributing leechers. The above mentioned knee can be approximated as the point where K is small enough for the seeding capacity (together with the capacity due to unchokings from leechers) to just “saturate” the downloading capacity of the m a chunks. Reducing the threshold to a smaller value would not result in additional improvements inT i la (as well asT i l ). To simplify our discussion, we assume (i) the download rate is dominated by the seed capacities when downloading them a chunks and (ii) nodes do not have download capacity limits. Discussion of how to remove these assumptions is presented later in this section. Under the above assumptions, downloading using an optimal threshold will be similar to not having seeding capacity and having a file of sizem b . Thus, we observe that with further decrease in K (i.e., increase in m b ), the download time increases linearly. We can determine the threshold value at this knee,K ′ , from our model as follows. 67 0 10 20 30 40 50 60 70 80 90 0 20 40 60 80 100 120 Optimal Threshold (%) Average Seed Time (min) 10% FR (Model-IC) 20% FR (Model-IC) 30% FR (Model-IC) 40% FR (Model-IC) Figure 4.19: Optimal Threshold Since we assume that the downloading time of the m a chunks is dominated by the seeding capacity, we have: d i a ≈ P h j=1 N j s u j N la = P h j=1 N j s u j P h j=1 N j la = P h j=1 N j s u j P h j=1 λ j T j la = P h j=1 N j s u j P h j=1 λ j ma d j a = P h j=1 N j s u j P h j=1 λ j mK ′ d j a . Also, since we assume that there is no download capacity limit, we have T i la ≈ 0,∀i. We can expressK ′ as: K ′ ≈ min P h j=1 N j s u j P h j=1 λ j m ,1 = min P h j=1 p j λT j s u j P h j=1 p j λm ,1 = min P h j=1 p j T j s u j m ,1 . Figure 4.19 depicts the optimal threshold found using this method, on a system with bandwidth settings given in Table 4.4. We illustrate cases with 10%,20%,30%, and 68 Table 4.6: Download Time (min) using the Optimal Threshold (CL) Average Seed Time (min) 20 40 60 80 100 120 Optimal (Model-IC) 147 127 107 87 67 47 Optimal (Sim) 151 133 115 96 78 59 40% of free-riding nodes in the arrivals. We observe that the optimal threshold (based on our model) increases linearly with the seeding time and smaller free-rider populations (higher contributing leecher populations) require a larger threshold. This occurs because larger thresholds are required with larger seeding capacities (i.e., longer seeding times or more contributing nodes) so that the seeding capacity is not wasted - such waste results in degraded contributing nodes’ performance. We now focus on the case of having20% of the arrivals being free-riders. Figure 4.20 and Table 4.6 give the download times of free-riders and contributing nodes, respectively - these are computed using our model as well as simulation (where we use the optimal threshold computed from the model in the simulation settings). Addition simulation results for the original BT protocol and the sort-based scheme (described above) are also included for comparison 10 . In the simulation results, our optimal threshold setting turns out to be a bit too aggressive, as it slows down the contributing nodes as well. However, this slow down of the contributing nodes is much less sensitive than that of the free-riders when aggressive thresholds are used. Moreover, in Figures 4.17 and 4.18, the optimal threshold value there would be larger than the one obtained from the model. Thus, we relax our threshold, i.e., we set the threshold toK ′ +θ (instead ofK ′ ), whereθ is the relaxation factor. In Figure 4.20 we also show the simulation results obtained using this relaxed value with θ = 0.15. 10 We do not include these for contributing leechers as those results are nearly identical to the Model-IC results. 69 0 100 200 300 400 500 600 700 800 20 40 60 80 100 120 FR Average Download Time (min) Average Seed Time (min) Optimal (Model-IC) Optimal (Sim) Original (Sim) Sort (Sim) Relaxed Optimal (Sim) Figure 4.20: Download Time using the Optimal Threshold Interestingly, the simulation download time using the relaxed value is very close to the download time of the sort-based approach. It can be explained by the fact that the underlying idea of the sort approach is quite similar. That is, this approach attempts to slow down free-riders as much as possible while not wasting seeding capacity - i.e., sorting is similar to finding the smallest value of the threshold which can still utilize the upload capacity of the seeds. We note that the insights match the observations made from simulation experiments given in Chapter 3, whereas the computation time reduces to seconds from hours. Thus, our model provides a fast, clean and flexible approach to explore the design space of BT variations. In the above threshold optimization, we obtained an optimal threshold value by ignoring (i) the download capacity contributed by the contributing leechers and (ii) the download capacity limit of nodes, which resulted in an over-aggressive threshold. If we remove these assumptions, the download capacity will be “saturated” earlier, i.e., a larger (“more relaxed”) threshold is needed in order not to waste the seeding capacity. Above, we utilized a simple offset to adjust an optimal threshold. One can compute an 70 optimal threshold more accurately as follows. First, for each classi, we assume that it is the first class to hit its download capacity limitD i . Under such an assumption, we find the corresponding optimal threshold by letting d i a = D i . This is done by solving the h download rate equations, which contain an optimal thresholdK ′ i and the download rate of the other classes as unknowns. Finally, we pick the maximum threshold among all K ′ i from each solved case. Other Seeding Approaches: The “Allowed Fast” mechanism in the BT Fast Extension [20] is similar to the Threshold-based seeding approach described above. Specifically, the “Allow Fast” extension allows a node to download a small set of chunks from the seeds. The set of chunks are determined based on a pseudo-random function using the IP address of the node where all seeds agree on the same set of chunks for a particular node. As in the Threshold-based approach, we can use the above threshold optimization framework to compute an optimal size of the allowed fast set. Another seeding approach aimed at prohibiting free-riding is proposed in [39]. The authors propose a scheme in which a seed allocates its uploading bandwidth according to the uploading rate of the leecher peers. Such information is reported by the neighbors of those leecher peers. We can model this seeding scheme using our framework by approximating the downloading rate obtained by the seeds to be proportionally fair [33] to the uploading rate of peers. Other Model Applications: Our model can easily be used to answer other “what if” type questions, e.g., how the number of regular vs. optimistic unchokes affects perfor- mance (as studied in [15]), as well as effects due to different bandwidth settings, seeding times, re-evaluation intervals, history window sizes, and so on (as these are parameters in our model). We can also model upload enforcement schemes, e.g., such as the share ratio enforcement schemes found in some private trackers, by modifying a node’s total staying time accordingly. 71 Chapter 5 Multi-path Streaming Optimization Quality of service (QoS) in streaming of continuous media (CM) over the Internet is still poor and inconsistent. The degradation in quality of CM applications, involving delivery of video and audio, is partly due to variations in delays, bandwidth limitations, as well as losses experienced by packets sent through wide-area networks. In this chapter, we focus on providing QoS guarantees in CM delivery through the exploitation of multiple paths existing in the network between a sender (or a set of senders) and a receiver. The basic idea is that CM data destined for a particular receiver is fragmented into packets and the different packets take alternate routes to the receiver. Optimal load distribution in the context of multi-path streaming is the focus of this work. In Section 1.3, we provided an overview of our work on this problem. The details are given in this Chapter. A background of this work is provided in Section 5.1.1. We define the load distribution optimization problem formally and in more detail in Section 5.1.3. A detailed performance study is presented in Section 5.2. 5.1 Load Distribution Problem In this section we formulate the optimal load distribution problem and analyze a set of possible corresponding optimization objectives. 72 5.1.1 Background Consider streaming of pre-stored CM data from N senders, where sender i sends frac- tionα (i) of the data expected by the receiver. The streaming traffic is split among these N senders according to a traffic splitting vector α = [α (1) ,α (2) ,...,α (i) ,...,α (N) ], whereα (i) ≥ 0 and P N i α (i) = 1. Assume the original packet sending rate, when using a single path is λ (in units of packets/sec). In our multi-path streaming scheme, sender k streams data at the rate ofλ (k) = α (k) λ, andδ (k) = 1/λ (k) represents the time interval between consecutive packets from senderk. We assume that the N senders are chosen such that there are N paths, with dis- joint bottlenecks, between the senders and the client. Moreover, we follow previous work[11, 17] and use a stationary continuous time Gilbert model (GM) to characterize the potential correlations between consecutive packet losses on each network path. That is, each path is characterized by its own GM. Since it is reasonable to characterize a path using its bottleneck link [11], what we need to be able to do is determine whether one path shares a point of congestion with another path and then choose paths for MP streaming which do not share bottlenecks. This can be done using previously proposed approaches[19, 60]. Since the paths are disjoint, the GMs are assumed to be indepen- dent. For a stationary continuous time Gilbert model, the packet loss process along path k is described by a two state continuous time Markov chain{X (k) (t)} whereX (k) (t)∈ {0,1}. If a packet is transmitted at time t when the state of path k is X (k) (t) = 0, then the transmitted packet is received correctly by the receiver. On the other hand, the 73 transmitted packet is considered lost ifX (k) (t) = 1. The infinitesimal generator for the Gilbert model characterizing pathk is: Q (k) =   −μ (k) 0 μ (k) 0 μ (k) 1 −μ (k) 1   whereμ (k) 0 is the transition rate from state0 to state1 andμ (k) 1 is the transition rate from state 1 to state 0 corresponding to path k. The stationary distribution of this Gilbert model isπ (k) = [π (k) 0 ,π (k) 1 ] whereπ (k) 0 = μ (k) 1 μ (k) 0 +μ (k) 1 andπ (k) 1 = μ (k) 0 μ (k) 0 +μ (k) 1 . Letp (k) i,j (τ) be the probability that pathk is in statej at timet+τ, given that it was in state i at time t, i.e., p (k) i,j (τ) = P(X (k) (t + τ) = j|X (k) (t) = i). From previous work[51], we have that p (k) i,j (τ) =                    μ (k) 1 μ (k) 0 +μ (k) 1 1−e −[μ (k) 0 +μ (k) 1 ]τ i = 1,j = 0, μ (k) 0 μ (k) 0 +μ (k) 1 1−e −[μ (k) 0 +μ (k) 1 ]τ i = 0,j = 1, μ (k) 0 +μ (k) 1 e −(μ (k) 0 +μ (k) 1 )τ μ (k) 0 +μ (k) 1 i = 1,j = 1, μ (k) 1 +μ (k) 1 e −(μ (k) 0 +μ (k) 1 )τ μ (k) 0 +μ (k) 1 i = 0,j = 0 for allτ > 0. Golubchik et al. [17] study and compare the performance of single path (SP) stream- ing vs. multi-path (MP) streaming using the Gilbert model and the following metrics: • Loss rate: P N is the fraction of lost packets as seen by the receiver when one uses N≥ 1 paths for streaming. • Lag-1 autocorrelation: the lag-1 autocorrelation function, R[X(t)X(t+δ)], measures the degree of dependency of consecutive packet losses as seen by the receiver, where X(t) is a random variable indicating whether the packet sent at time t is lost or received properly (depending on the state of the Gilbert model) 74 andδ = 1/λ. For instance, a high positive value ofR[X(t)X(t+δ)] implies that a lost packet is very likely to be followed by another lost packet. If the statistics of the consecutive packet losses are not correlated, thenR[X(t)X(t+δ)] = 0. • Conditional burst length of lost packets: this is the probability mass function of consecutively lost packets, conditioned on there being a burst of packet loss, as seen by the receiver. Intuitively, long bursts will significantly affect the viewing quality of the multimedia object. Moreover, they will reduce the effectiveness of an error correction scheme, if some form of an erasure code is deployed. The potential benefits of using MP streaming are analyzed using the above given GM and the above given metrics in Golubchik et al. [17] An experiment motivating the importance of considering loss correlations is given in the technical report [1]. This experiment illustrates how correlated bursty losses affect video quality. 5.1.2 Motivation Golubchik et al. [17] study MP streaming benefits using a round-robin approach to split- ting the CM traffic among the multiple paths. In contrast, here we focus on the problem of optimally assigning CM traffic to the multiple paths, according to the path charac- teristics, such that a certain specified performance metric is optimized. That is, given the Gilbert models for the N streaming paths, we compute the optimal traffic splitting vector,α ∗ according to an optimization objective, where P N i α (i) = 1. We refer to this as the load distribution problem, and we study both appropriate optimization objectives as well as the resulting streaming performance, all with a high level goal of improving the perceptual quality of the streamed media. We note that it is not trivial to pick an appropriate optimization objective. If we simply use one of the metrics mentioned above, undesirable effects might arise. For 75 example, optimizing the lag-1 autocorrelation may result in higher loss rates observed at the receiver, when the paths are not homogeneous. Determining and gaining insight into suitable optimization objectives is therefore an important part of this work. In order to obtain a basic understanding of the problem, we first study the resulting performance in an analytical setting, without the use of erasure codes. Subsequently, we present simulation results with erasure codes as well. 5.1.3 Optimization Objectives In order to attain the desirable performance properties of multi-path streaming, appro- priate load distribution, α, needs to be chosen, as the load distribution will affect the quality of the streamed media. In this work, we will approximate media quality using information loss rate (as defined in Section 5.2). Since most streams are expected to be transmitted with some form of an erasure code 1 , the information loss rate will be a function of the erasure code used as well. However, the analysis below first focuses on a setting without an erasure code, in order to gain insight into the problem independently of a particular error correcting scheme. Of course, choosing the optimal α is a factor of the optimization objective. The resulting quality of the stream, however, is a factor of how well that optimization objec- tive approximates media quality under packet losses. Since Golubchik et al. [17] pro- pose to describe stream quality using the three metrics stated above, we begin with those as our optimization objectives. We evaluate the considered optimization objec- tives under the above described Gilbert model, using two paths. 1 Here we do not consider retransmissions, partly because there is usually little opportunity to retrans- mit data in such real-time applications, and hence some amount of lossiness may have to be tolerated. In a complete system a hybrid approach (with retransmissions and erasure codes) might be useful. How- ever, this work is focused on exploring loss correlations which have a more significant effect on error correcting schemes; we believe that adding retransmission to our evaluation process would obscure the characteristics being illustrated here. 76 Mean Loss Rate Since our approximation of quality here is the mean information loss rate, it is nat- ural to start our search for an appropriate optimization objective with the Mean Loss Rate (MLR). The mean loss rate, ( ¯ X ), observed at the receiver when streaming over multiple paths is given by ¯ X = Pr[loss] = α (1) π (1) 1 +α (2) π (2) 2 . That is, optimizing on MLR is equivalent to streaming over the best (in terms of loss rate) single path (in what follows we use the terms MLR-based streaming and SP streaming interchangeably). And, as illustrated in Section 5.2, this does not always result in the best streaming qual- ity; specifically, this does not allow us to take advantage of the reduced correlations in packet losses achieved when using multiple paths[17]. Reducing such correlations (i.e., packet loss burst lengths) is important when erasure codes are applied — intuitively, the more losses are “spread out”, the better the performance of an erasure code. Conditional Mean Burst Length Motivated by the need to consider burst lengths, we examine the conditional Mean Burst Length (MBL) next (conditioned on there being a loss), as an optimization objective. The conditional mean burst length, ( ¯ B), is derived as follows: ¯ B = ¯ X Pr[burst] =              [α (1) π (1) 1 +α (2) π (2) 1 ] [α (2) −α (1) ]π (2) 0 p (2) 0,1 (δ (2) )+α (1) π (1) 0 π (2) 1 +α (1) π (2) 0 π (1) 1 α (1) < α (2) [π (1) 1 +π (2) 1 ] π (1) 0 π (2) 1 +π (2) 0 π (1) 1 α (1) = α (2) = 1 2 [α (1) π (1) 1 +α (2) π (2) 1 ] [α (1) −α (2) ]π (1) 0 p (1) 0,1 (δ (1) )+α (2) π (1) 0 π (2) 1 +α (2) π (2) 0 π (1) 1 α (1) > α (2) In our experiments, optimizing on MBL resulted in a round-robin approach. And, as illustrated in Section 5.2, this does not always result in the best streaming quality, e.g., this can occur when one of the paths has a high (relative) loss rate. In such cases, the losses due to “equal use” of highly heterogeneous paths become the dominant factor, i.e., 77 the reduced correlations achieved by optimizing on MBL are not sufficient to overcome the corresponding losses. Hence, there is a fundamental tradeoff between mean losses and the corresponding loss correlations — this tradeoff is explored further below. Lag-1 AutoCorrelation Still motivated by the need to reduce loss correlations in order to improve the ability to correct losses through erasure codes, we consider the lag-1 autocorrelation (L1AC) as an optimization objective, which is derived as follows: R[X t X t+δ ] = 1− Pr[burst] ¯ X(1− ¯ X) Pr[burst] =          [α (2) −α (1) ]π (2) 0 p (2) 0,1 (δ (2) )+α (1) π (1) 0 π (2) 1 +α (1) π (2) 0 π (1) 1 α (1) < α (2) 1 2 π (1) 0 π (2) 1 + 1 2 π (2) 0 π (1) 1 α (1) = α (2) = 1 2 [α (1) −α (2) ]π (1) 0 p (1) 0,1 (δ (1) )+α (2) π (1) 0 π (2) 1 +α (2) π (2) 0 π (1) 1 α (1) > α (2) under the assumption of a specific class of sending patterns, i.e., packets are distributed among paths such that no two consecutive packets are sent on the less loaded path. (In our experiments, such a sending pattern gave lower correlations than other sending patterns.) As illustrated in Section 5.2, optimizing on L1AC 2 expresses the need to consider correlations in packet losses better than optimizing on MBL. However, it suffers from the same shortcomings pointed out in the context of MBL above, when paths are highly 2 More specifically, we optimize on the square of lag-1 autocorrelation, so as to deal with positive values. 78 non-homogeneous. Another problem with L1AC is that it can be a fairly sensitive met- ric, and hence may not lead to robust load distribution approaches. Mean Loss Rate× Mean Burst Length Since there is a fundamental tradeoff between the frequency of losses and the corre- sponding loss correlations, as described above, we naturally consider an optimization objective which encompasses both metrics. There are a number of ways to include both in a single metric. Here we consider MLR× MBL as our optimization objective, derived as follows: ¯ X· ¯ B = ¯ X· ¯ X Pr[burst] =              [α (1) π (1) 1 +α (2) π (2) 1 ] 2 [α (2) −α (1) ]π (2) 0 p (2) 0,1 (δ (2) )+α (1) π (1) 0 π (2) 1 +α (1) π (2) 0 π (1) 1 α (1) < α (2) 1 2 [π (1) 1 +π (2) 1 ] 2 π (1) 0 π (2) 1 +π (2) 0 π (1) 1 α (1) = α (2) = 1 2 [α (1) π (1) 1 +α (2) π (2) 1 ] 2 [α (1) −α (2) ]π (1) 0 p (1) 0,1 (δ (1) )+α (2) π (1) 0 π (2) 1 +α (2) π (2) 0 π (1) 1 α (1) > α (2) We evaluate the goodness of the above described optimization objectives and the result- ing performance of multi-path streaming next. 5.2 Performance Study In this section, we present a performance study of SP and MP streaming under our load distribution optimization techniques. Results without the use of FEC are discussed first. Unless otherwise stated, the experiments below perform MP streaming on two paths with SP streaming using the “best path”, i.e., the one with the lower loss rate (we will use the terms SP and Best Single Path streaming interchangeably). Given a desired path mean loss rate, we achieve it by fixing μ 1 (i) = 70 and varying μ 0 (i) in the Gilbert 79 Table 5.1: Default Experimental Settings Path 1 Loss Rate 5% Path 2 Loss Rate Varies FEC k (except Experiment 1 & 2) 32 FEC n (except Experiment 1 & 2) 38 0 0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Loss Rate on Path 1 (π 1 (1) ) = 0.05 Loss Rate on Path 2 (π 1 (2) ) Optimal Load on Path 1 (α (1) ) best path (SP) optimizing L1AC 2 optimizing MBL optimizing MLR × MBL Figure 5.1: Optimal Load Distribution for Different Objective Functions (without FEC) model accordingly. The loss rate for Path 1 is set to 5% by default. Table 5.1 gives the default settings for the experiments presented below. Results in Experiments 1 and 2 are computed using the analytical results of Section 5.1. Experiment 1: Effect on Optimal Load Distribution: Here we study the optimal traffic distribution,α ∗ , resulting from using the objective functions proposed in Section 5.1. Figure 5.2 shows the optimal load assignment on Path 1, α ∗ 1 , as a function of loss rate on Path 2. Each curve corresponds to a different objective function. Figure 5.2) shows a smaller segment of the results (where Path 2 loss rate is≤ 20%) for clarity of presentation. From these results, we make the following observations: 80 0 0.05 0.1 0.15 0.2 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Loss Rate on Path 1 (π 1 (1) ) = 0.05 Loss Rate on Path 2 (π 1 (2) ) Optimal Load on Path 1 (α (1) ) best path (SP) optimizing L1AC 2 optimizing MBL optimizing MLR × MBL Figure 5.2: Optimal Load Distribution for Different Objective Functions Zoomed (with- out FEC) • The objective functions considered in Section 5.1 can produce significantly dif- ferentα ∗ s. • Optimizing by minimizing mean burst length results in a distribution where each path carries 50% of the load no matter how different in terms of loss rates Path 1 and Path 2 are, which can be implemented using a simple Round-Robin approach. (This turns out to be the case in all our experiments.) • When optimizing MLR*MBL,α ∗ remains constant when paths are close to being homogeneous in terms of loss rates. This can be explained as follows. When the paths are nearly homogeneous, the MLR factor has a smaller effect and hence the optimization is dominated by the MBL factor. This results in a flat region which behaves similarly to optimizing on MBL alone. This result supports the intuition 81 that when paths are nearly homogeneous, a simple Round-Robin approach should work well. • Optimizing the Lag 1 autocorrelation results in α ∗ less responsive to changes in loss rates, as compared to optimizing on MLR*MBL. This relative insensitivity can lead to high loss rates, which is illustrated in the next experiment. Lastly, we can also observe that the more interesting results occur when Path 2 loss rate is in a reasonable range, e.g,≤ 20% in the above experiment. Note that the loss rate on Path 1 is relatively low as well. Since the lower loss rates are reasonable to consider, given the current Internet, For clarity of presentation, in what follows we do not include the full range of Path 2 loss rates. Experiment 2: Effect on Performance Metrics: Using the previous experiment’s setup, we now investigate effects of the objective functions on several performance met- rics of interest. Figures 5.3, 5.4, and 5.5 show the resulting mean loss rate, mean burst length, and mean Lag 1 autocorrelation, respectively. From our results, we make the following observations: • Optimizing on MLR*MBL may result in higher loss rates as compared to the Best Single Path. The maximum observed difference (around 3%) occurred when Path 2 loss rate was 10%. However, using the Best Single Path gives significantly poorer performance in other metrics such as the mean burst length. We believe this will result in poorer perceptual quality; we elaborate on this below. • For most metrics, performance obtained from optimizing on MLR*MBL stays between the results obtained from SP streaming and results obtained by optimiz- ing on Lag 1 autocorrelation. While SP results in the lowest loss rate and opti- mizing on Lag 1 autocorrelation results in shorter mean burst lengths, optimizing 82 0 0.05 0.1 0.15 0.2 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 Loss Rate on Path 1 (π 1 (1) ) = 0.05 Loss Rate on Path 2 (π 1 (2) ) Mean Loss Rate at Optimal α (1) best path (SP) optimizing L1AC 2 optimizing MBL optimizing MLR × MBL Figure 5.3: Mean Loss Rate at Optimal 0 0.05 0.1 0.15 0.2 1 1.5 2 2.5 Loss Rate on Path 1 (π 1 (1) ) = 0.05 Loss Rate on Path 2 (π 1 (2) ) Mean Burst Length at Optimal α (1) best path (SP) optimizing L1AC 2 optimizing MBL optimizing MLR × MBL Figure 5.4: Mean Burst Length at Optimal 83 0 0.05 0.1 0.15 0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 Loss Rate on Path 1 (π 1 (1) ) = 0.05 Loss Rate on Path 2 (π 1 (2) ) Lag−1 Autocorrelation at Optimal α (1) best path (SP) optimizing L1AC 2 optimizing MBL optimizing MLR × MBL Figure 5.5: Mean Lag 1 Autocorrelation at Optimal MLR*MBL results in a reasonable tradeoff between these two metrics, both of which have an effect on the perceptual quality. In the following experiments, we study the effects of using FEC on SP and MP streaming. Unless otherwise stated, we set FEC parameters to k = 32, n = 38, i.e., every 32 media packets are grouped together to generate 6 redundant packets. Thus, the FEC overhead is≈ 20%. CSIM [24] simulations, where losses are generated using an appropriate Gilbert model, are carried out to study the performance of SP and MP streaming with different optimization objective functions. Each simulation emulates the streaming of a 2 hour media file which requires a sending rate of 120 pkts/sec. The resulting data sending rate is increased to account for the addition of redundant informa- tion, i.e., based on the percentage of FEC overhead. The optimal load distribution,α ∗ , is obtained analytically using this adjusted sending rate. FEC is performed on the received packet sequence and the mean information loss rate (MILR), which accounts for loss 84 0 0.05 0.1 0.15 0.2 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Loss Rate on Path 1 (π 1 (1) ) = 0.05 Loss Rate on Path 2 (π 1 (2) ) Optimal Load on Path 1 (α (1) ) best path (SP) optimizing MILR optimizing L1AC 2 optimizing MBL optimizing MLR × MBL Figure 5.6: Optimal Load for Different Objective Functions (with FEC) of media data (i.e., not including loss of redundant packets) after FEC processing, is measured. Experiment 3: Optimal Load Distribution and Information Loss with FEC: To compare the effects of different objective functions on α ∗ when FEC is used, we com- pute an optimal traffic load distribution using MILR as our optimization objective by a brute force search on simulation results. Figure 5.6 shows an optimal α ∗ 1 as a function of loss rate on Path 2. Figure 5.7 shows the corresponding MILR. The same experiment is repeated with Path 1 loss rate set to 10%; Figures 5.8 and 5.9 show the corresponding α ∗ 1 and MILR, respectively. Each experiment is repeated 10 times with different random seeds, and the results are reported with 95%± 10% confidence intervals. From these results, we make the following observations: 85 0 0.05 0.1 0.15 0.2 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Loss Rate on Path 1 (π 1 (1) ) = 0.05 Loss Rate on Path 2 (π 1 (2) ) Information Loss Rate at Optimal α (1) (%) best path (SP) optimizing MILR optimizing L1AC 2 optimizing MBL optimizing MLR × MBL Figure 5.7: Information Loss Rate at Optimal 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Loss Rate on Path 1 (π 1 (1) ) = 0.10 Loss Rate on Path 2 (π 1 (2) ) Optimal Load on Path 1 (α (1) ) best path (SP) optimizing MILR optimizing L1AC 2 optimizing MBL optimizing MLR × MBL Figure 5.8: Optimal Load for Different Objective Functions (with FEC, Path 1 loss rate = 10%) 86 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0 2 4 6 8 10 12 14 16 18 20 Loss Rate on Path 1 (π 1 (1) ) = 0.10 Loss Rate on Path 2 (π 1 (2) ) Information Loss Rate at Optimal α (1) (%) best path (SP) optimizing MILR optimizing L1AC 2 optimizing MBL optimizing MLR × MBL Figure 5.9: Information Loss Rate at Optimal (Path 1 loss rate = 10%) • When Path 1 loss rate is relatively low, i.e., 5%, optimal load distribution on information loss increases relatively smoothly as a function of loss rate on Path 2; it eventually flattens out as it reaches SP setting. • When Path 1 loss rate is low, optimizing on MLR*MBL results in a load distribu- tion closer to the optimal based on MILR. • When Path 1 loss rate is higher, i.e., 10%, the MILR-based optimal is closer to the load distribution given by SP. • If a reasonable amount of FEC redundancy is used, optimizing on MLR*MBL gives good performance, i.e., lower information loss rate, and an MLR*MBL sys- tem operates closer to the MILR-based optimum. If the path loss rate is higher and the amount of FEC redundancy is insufficient, using SP may result in better performance. 87 Experiment 4: Overhead Requirements for Fixed Information Loss Rate: Here we compare the effect of using FEC on SP (case i) and MP streaming, where we use a sim- ple Round-Robin approach (case ii) and the MBL*MLR-based optimization approach (case iii). Path 1 has a fixed loss rate of 5%. Three scenarios where Path 2 has0.5%,5%, and10% loss rates are studied. For SP streaming, we choose the better path, i.e., with a lower loss rate. The objective of this experiment is to find the minimum FEC overhead (n−k)/k required to achieve a required quality of service, which for this experiment we define as a mean information loss rate of≤ 0.1%. (We have tried other values, and the results are qualitatively similar.) For each scenario, different FEC group sizes are tested by varying k. Table 5.2 gives the mean FEC overhead requirements, averaged over 10 experiments; the results are reported with95%±11% confidence intervals. From these results, we make the following observations: • Performance of MLR*MBL is similar to that of RR when the difference in path loss rates of the two paths is not large. This is due to the fact that MLR*MBL results in even traffic splitting in these cases, which is also shown in Figure 5.6. • SP, RR, and MLR*MBL have different “best” performance ranges. For example, when path 2 loss rate is 0.5%, SP performs best (requires the lowest FEC over- head) when the FEC group size is larger (k≥ 32). For the same path loss rate, MLR*MBL performs better than RR when we use k≥ 16 for FEC. When path 2 loss rate is 5% or 10%, MLR*MBL (or RR) perform better than SP streaming unless the FEC group size is quite large. Experiment 5: Sensitivity: Here we study the sensitivity of different optimization objective functions to inaccurate measurements (which are likely to occur in an Internet- type environment), in order to understand the robustness characteristics of the proposed technique. Since our optimization approach is a function of the paths loss rates, we 88 Table 5.2: FEC overhead needed to achieve information loss rate of≤ 0.1%; Path 1 loss rate = 5%. Path 2 Loss k=8 k=16 k=32 k=64 k=128 0.5% (i) 98.75% 40.63% 18.44% 9.22% 5.00% 0.5% (ii) 77.50% 41.88% 21.88% 13.75% 9.06% 0.5% (iii) 86.25% 38.75% 18.75% 9.38% 5.16% 5% (i) ≥ 1000% 147.50% 58.75% 31.88% 19.92% 5% (ii) 97.50% 55.63% 31.25% 20.31% 14.14% 5% (iii) 97.50% 55.63% 31.25% 20.31% 14.14% 10% (i) ≥ 1000% 147.50% 58.75% 31.88% 19.92% 10% (ii) 128.75% 68.75% 40.63% 26.72% 19.53% 10% (iii) 128.75% 68.75% 40.63% 26.72% 19.53% consider cases where the loss rate information is inaccurate. And, specifically, we focus on a comparison between SP streaming and the MLR*MBL-based approach, as above experiments indicate that MLR*MBL is the better of the considered objective functions. We present results of two cases (1) where Path 2 loss rate measurements are 5% greater than the real path loss rates and (2) where Path 2 loss rate measurements are 5% smaller than the real path loss rates. Figures 5.10 and 5.11 depict the resulting MILR using SP streaming and MLR*MBL-based approach, respectively. In each figure we report MILR curves corresponding to the accurate measurements as well as to the two cases of inaccurate measurements. The results are reported with95%±6% confidence intervals. From these results, we make the following observations: • When the loss rate is over-estimated, SP streaming chooses the “wrong” path when the real Path 2 loss rate is relatively low; in this case it is less robust than the MLR*MBL-based approach. • When the loss rate is under-estimated, both SP streaming and the MLR*MBL- based approach suffer from inaccurate measurements, although the measurement inaccuracies have a somewhat smaller effect on the MLR*MBL-based approach. 89 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Loss Rate on Path 1 (π 1 (1) ) = 0.05 Loss Rate on Path 2 (π 1 (2) ) Information Loss Rate at Computed Optimal α (1) (%) Correct measurement (SP) Measured 5% more (SP) Measured 5% less (SP) Figure 5.10: Sensitivity (Path 1 loss rate = 5%, Best Single Path) 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0 0.5 1 1.5 2 2.5 3 3.5 Loss Rate on Path 1 (π 1 (1) ) = 0.05 Loss Rate on Path 2 (π 1 (2) ) Information Loss Rate at Computed Optimal α (1) (%) Correct measurement (MLR × MBL) Measured 5% more (MLR × MBL) Measured 5% less (MLR × MBL) Figure 5.11: Sensitivity (Path 1 loss rate = 5%, Optimizing MLR*MBL) 90 We have also performed experiments where the Path 2 loss rates are inaccurate by±10% and±20%; those results are qualitatively similar. Summary: Overall, we observe from the above results that the MLR*MBL-based approach, which attempts a compromise between mean loss rate and loss correlations (which effect the utility of erasure codes), performs well with respect to information loss rate when FEC is used. However, we also observed that the MILR-based optimum load distribution, obtained through brute force searching, shows a smoother transition in load distribution between two paths as compared to the MLR*MBL-based approach. Hence, we believe our approach can be improved in the future. Another observation is that SP streaming performance can be improved with larger FEC group sizes, i.e., values of k, which is not surprising. That is, instead of achieving spatial diversity from mul- tiple paths, one can attempt temporal diversity with the downside of additional delays. However, such additional delays may not be suitable for some classes of applications which require tighter real-time constraints. A comparison between the delay character- istics of MP streaming and SP streaming with larger FEC group sizes as well as with packet interleaving techniques (which could also achieve temporal diversity) is a topic of future research. In the above experiments, MP streaming is performed with two paths only. These experiments illustrate a noticeable enhancement over SP streaming. 91 Chapter 6 Encoded Multi-path Streaming for Real-time Live Streaming Applications In this chapter, we focus on real-time live streaming applications with stringent end- to-end latency requirements, say several hundreds of milliseconds. To satisfy these requirements, we take a joint multi-path and FEC approach that intelligently splits the FEC-encoded stream among multiple available paths. We first present our observation on asymptotically optimal load splitting under multi-path streaming with FEC in Section 6.1. Based on the insights obtained from the asymptotically optimal splitting, we design and implement a novel Encoded Multi- path Streaming (EMS) scheme for real-time live streaming. Section 6.2 describes the design of our EMS scheme. EMS strives to continuously satisfy an application’s QoS requirements by dynamically adjusting the load splitting decisions and the FEC settings. Our simulation results in Section 6.3 show that EMS can not only outperform existing multi-path streaming schemes, but it can also adapt to the dynamic loss and delay char- acteristics of the network with minimal overhead. 6.1 Asymptotically Optimal Load Splitting In this section, we present our observation on an asymptotically optimal load splitting for the multi-path streaming problem using FEC. In [17, 55, 41] and Chapter 5, the 92 0 1 2 3 4 5 6 7 8 0 20 40 60 80 100 Info Loss (%) Load on Path 1 (%) 0% 10% 20% 30% 40% 100% 80% 68% 58% 58% 61% Brute-force Optimal Asymptotic Optimal Figure 6.1: Optimal splitting (2 paths) focus was on exact distribution analysis, which involves heavy combinatorics computa- tion. Here we take another approach and advocate an asymptotic approximation-based method which we can provide useful engineering insights. Specifically, we observe the following under multi-path streaming using FEC: As FEC redundancy grows, an optimal load on a path becomes inversely proportional to the path loss rate. This observation together with detailed asymptotic analysis can be found in [10]. 6.1.1 Asymptotically Optimal Validation We validate the above observation using extensive simulations, in which we use brute- force search to find an optimal splitting under different scenarios. For clarity of pre- sentation, we present the results of a 2-path case in Figure 6.1. We defer the detailed simulation settings to Section 6.3. Here, it suffices to note that the loss rates on Path 1 93 Table 6.1: Optimal splitting (3 paths) Path 1 Path 2 Path 3 Path Loss rate 3.69% 6.89% 9.97% Asymptotically Optimal load 52.4% 28.1% 19.4% Brute-forece Optimal load 51% 24% 25% and Path 2 are 5.29% and 8.26%, respectively. In this case, we can calculate an asymp- totically optimal splitting as<x 1 =61%,x 2 =39%>. In these simulations, we fix the FEC group size at 20 and gradually increase the FEC redundancy from 0% to 60%. For each redundancy value, we vary the load on Path 1 (x 1 ) from 0 to 100%, with an increment of 3%. The load on Path 2 is always x 2 = 1−x 1 . The results of such brute-force search are shown in Figure 6.1, which plots how the information loss rate (data loss rate after FEC recovery) changes as we vary x 1 . We can see that as the FEC redundancy grows, an optimal splitting vector is indeed approaching the asymptotically optimal one. We have also validated the asymptotically optimal splitting using 3-path cases, and a subset of the results is summarized in Table 6.1 (the FEC group size and redundancy are 20 and 40%, respectively). In all cases, we have observed that the asymptotically optimal load splitting matches the simulation results very well. 6.2 Design In this section, we present the design of our Encoded Multi-path Streaming (EMS) pro- tocol for live streaming applications. EMS is inspired by the asymptotically optimal load splitting in Section 6.1 and takes an adaptation oriented approach in addressing practi- cal issues such as network dynamics and FEC settings. As a result, EMS can exploit the benefits of both, multi-path delivery and FEC to ensure high-quality streaming with minimal overhead. 94 EMS Sender Live Real-time Source Sender Buffer EMS Receiver Receiver Buffer FEC Encoder Monitoring Agent Packet Scheduler FEC Decoder Client Application Load Split Vector FEC Parameters Load Split EMS Adaptation Paths Quality Decode Stat Info Loss Rate Packet Receiver Multiple Paths FEC Group Size FEC Redundancy Figure 6.2: EMS overview 6.2.1 EMS Overview EMS assumes that the sender and the receiver can establish multiple paths between them through external mechanisms. In practice, this can be done by using multiple network interfaces or overlay networks (e.g., [2, 35]), where the traffic is relayed at multiple overlay nodes. However, the establishment and optimization of these overlay paths are out of the scope of EMS. Figure 6.2 shows an overview of our proposed EMS protocol. An EMS sender takes a live stream, generated in real time, and encodes it using a FEC encoder. The encoded stream is split among multiple available paths and arrives at the receiver with different delay and loss characteristics. The EMS receiver buffers all the received packets and, if it has received enough packets within any FEC group, uses a FEC decoder to recover any lost packets in this group. Any packet that has missed its deadline, even if it is successfully recovered by FEC, will be discarded. The above seemingly simple EMS operation poses several research challenges in terms of its effectiveness and efficiency. Specifically, we need to address the following design issues: • How to split load among multiple paths under practical and dynamic net- work settings? While the asymptotically optimal load splitting in Section 6.1 95 and the analytical results in [10] offer invaluable insights, they are obtained based on the assumptions of stationary loss processes and a large FEC group size. In practice, however, the path quality is highly dynamic as the background traffic changes, and the FEC group size is typically small. Thus it remains a design chal- lenge for EMS to operate consistently with optimal or near-optimal load splitting vectors. • How to properly set the FEC parameters such as redundancy and group size? Many existing FEC-based streaming studies use static, empirical settings. How- ever, when coupled with the use of multi-path, these FEC parameters present mul- tiple dimensions of tradeoff between delay, loss and overhead, thus playing a key role in the overall performance of EMS. For example, we have seen in Section 6.1 that the FEC redundancy has direct impact on the optimal load splitting. We will elaborate on these design tradeoffs in Sections 6.2.3 and 6.2.4. To address these two issues, we take an adaptation oriented approach as follows. As shown in Figure 6.2, an EMS receiver continuously monitors the quality of each path in terms of loss and delay, as well as the aggregate information loss rate after FEC recovery. These monitoring results are periodically fed into the EMS Decision Engine, which adapts the load splitting vector as well as the FEC parameters. Subsequently, the sender receives these decisions via feedback from the receiver, and then updates its local settings for the FEC encoder and packet scheduler accordingly. Note that EMS places most of the intelligence at the receiver, because it can readily observe the path quality without any extra signaling mechanism. In the rest of this section, we describe how EMS adapts the load splitting, FEC redundancy, and FEC group size. 96 6.2.2 Online Load Splitting (OLS) The Online Load Splitting (OLS) scheme in EMS is inspired by the asymptotically optimal load splitting in Section 6.1 and the analytical results in [10] yet addresses the practical issues of finite FEC group size and network dynamics through an adaptive search process. In order to bootstrap the system, OLS initially splits the load evenly over all available paths, so that the receiver can measure the loss rate along each path. After collecting T s seconds of such measurements, the receiver computes the asymptotically optimal splitting presented in Section 6.1 and passes the resulting splitting vector to the sender, which then changes its packet scheduler accordingly. Note that the asymptotically optimal solution is only an approximation of the true optimal, which itself is unknown and dynamically changing. Therefore, after applying the asymptotically optimal solution, OLS starts to search for the true optimal solution and adapt to network dynamics in an iterative manner. As illustrated in Algorithm 1 below, OLS sorts the paths in the increasing order of loss rate. It operates in either of two possible states: PickPath and AdaptLoad. In the PickPath state, OLS chooses the first path in the list, i.e., the one with the lowest loss rate among the candidates, and switches to the AdaptLoad state. In the AdaptLoad state, OLS increases the load on the chosen path by Δ L , a pre-defined increment term (we set Δ L as 3% of the stream rate in our implementation). OLS also decreases Δ L amount of traffic from the other paths collectively, with the load decrement on each path proportional to its loss rate. After applying this load adjustment, OLS measures the information loss rate within a window of T s seconds and compares it to the previously recorded one. If the loss rate decreases, OLS stays in the AdaptLoad state and repeats the load adjustment. Otherwise, it removes the previously chosen path from the list, reverts to the previous load splitting, and switches to the PickPath state to choose another path. Finally, when all paths have been tried, OLS sorts the paths again and restarts the process. 97 Algorithm 1 Online Load Splitting (OLS) 1: Compute the asymptotic optimal solution and split the load accordingly 2: Sort the paths in the increasing order of loss rate 3: repeat 4: Pick the first path in the list 5: repeat 6: Increase the load on the chosen path byΔ L 7: Decrease the load on each of the remaining paths by a fraction of δ, propor- tional to their respective loss rates 8: until measured information loss rate increases 9: Remove the chosen path from the list 10: Revert to the previous load splitting 11: until the path list is empty 12: goto Step 2 Note that the load splitting vector only decides the amount of traffic that each path should carry. The sender can enforce it through different packet scheduling algorithms, resulting in different characteristics of the split traffic. The packet scheduler in EMS (Figure 6.2) seeks to minimize the burstiness of traffic as follows. With a given load splitting vector, it keeps track of the offered load and expected load on each path, both in terms of the number of packets. Whenever a new packet arrives, it is scheduled on the path which has the largest gap between the offered and expected loads. This way, EMS can enforce the splitting vector yet ensure the packets sent on each path are spaced out as evenly as possible. 6.2.3 FEC Redundancy Adaptation There is an inherent tradeoff between the FEC overhead and its error correction power. With more redundant packets, the receiver can recover from more severe losses, at the cost of more traffic offered on the network. One may view this tradeoff as one between a user and the network (or other users in the network). That is, if a user increases the FEC redundancy for its own traffic, it can always enjoy the enhanced error correction 98 0 1 2 3 4 5 6 0 10 20 30 40 50 60 70 Info Loss (%) FEC Redundancy (%) with Delay Constraint without Delay Constraint Figure 6.3: Information loss vs. FEC redundancy capability, while the network or other competing users may see unpleasant side-effects due to the increased load. However, our simulation results show that this is not the case, once we take into account the latency requirement, and it is to a user’s own disadvantage to inject excessive FEC redundancy. Figure 6.3 depicts how the information loss rate changes as the FEC redundancy increases. In these simulations (detailed settings are given in Section 6.3), there are two paths between the sender and the receiver, with 5.4% losses on each path. The load is evenly split between the two paths as suggested in the analytical results in [10]. As we can see from Figure 6.3, with the absence of delay constraint, larger FEC redundancy can always improve the information loss rate despite the diminishing returns. However, with a 150 ms delay constraint, there is a sudden increase in the information loss rate as the FEC redundancy goes beyond 50%. This is because excessive redundant pack- ets cause queue buildup at bottleneck links and longer queuing delays for all packets traversing those links. Because a packet arriving after its deadline is discarded, such delay-induced losses become apparent as the end-to-end latency approaches the bound, 99 and the receiver cannot use FEC to recover these delayed packets either. Similar phe- nomenon has also been reported in the literature using prototype experiments [9]. To balance between the loss recovery capability and the latency, we have designed a FEC redundancy adaptation scheme, illustrated in Algorithm 2. Our basic idea is to use “just enough” FEC redundancy to meet the application’s loss requirements. Sup- pose L req is the maximum loss rate that the application can tolerate. An EMS receiver keeps track of the percentage of packets that arrived after their deadlines. If such delay- induced loss exceeds αL req , where α is a pre-defined threshold, EMS decreases the FEC redundancy. Otherwise, the receiver compares the current information loss rateL c with L req . If L c is low (i.e., smaller than βL req ), EMS also decreases the FEC redun- dancy. However, ifL c is relatively high (i.e., larger thanγL req ), EMS will increase the FEC redundancy. In our implementation, we set α = 0.5, β = 0.5, γ = 0.75 and the redundancy incrementΔ R = 5%. Algorithm 2 FEC Redundancy Adaptation 1: backoff = (delay-induced losses> αL req )? true:false 2: if (backoffk information loss rate< βL req ) then 3: Decrease FEC redundancy byΔ R 4: else if (information loss rate> γL req ) then 5: Increase FEC redundancy byΔ R 6: end if 7: goto Step 1 It should be noted that EMS adapts the FEC redundancy in a continuous manner, based on both, loss and delay measurements. As such, it can react quickly to network dynamics (e.g., a sudden increase of background traffic) and minimize the FEC overhead subject to the application’s loss requirements. 100 6.2.4 FEC Group Size Adaptation The FEC group size also affects the tradeoff between loss recoverability and delay in the presence of bursty losses. With a larger FEC group, the impact of bursty losses becomes less significant, thus the receiver has a better chance of recovering the lost packets. How- ever, a larger FEC group also results in longer delays for the recovered packets, because recovery only starts after the receiver has receivedK (data or redundant) packets. In the extreme cases, a packet may already have passed its deadline when it is recovered, thus making it effectively useless. To address this issue, EMS seeks to use the maximal FEC group size that can still ensure recovering the lost packets before their deadlines. Specifically, for every recov- ered packet, the receiver records its slack time, defined as the difference between its recovery timestamp and its deadline. Note that a packet that has missed its deadline has a negative slack time. The receiver keeps track of the minimum slack time, denoted by T s , within a sliding window. At the end, it adapts the FEC group sizeK as follows: K← K +⌊T s /λ⌋−b (6.1) whereλ is the stream source rate in units of packets per second, andb is a constant that prevents the increase in group size from using up the slack time. In essence, we estimate the number of packets that the source can generate during the minimum slack time, and then adjust the FEC group size accordingly. Note that in the case of a negative T s , the FEC group size will be effectively decreased. 6.3 Performance Evaluation In this section, we evaluate the performance of EMS and compare it to two existing multi-path streaming schemes, one based on TCP and the other on UDP. Our results 101 Sender Receiver bottleneck FTPs UDP on/off bottleneck FTPs UDP on/off Figure 6.4: Network topology used in our simulations show that EMS consistently outperforms these existing schemes under different set- tings, and EMS can adapt to network dynamics and provide high-quality streaming with minimal overhead. 6.3.1 Methodology We evaluate different multi-path streaming schemes using ns-2 simulations [23]. Fig- ure 6.4 shows the network topology in use, which consists of multiple virtual paths between a sender and a receiver. In practice, these paths can be established through multi-homing or network overlays. We evaluate the impact of different network charac- teristics by varying the propagation delay and background traffic on these paths. Specif- ically, each path consists of one bottleneck link, which has 10 ms propagation delay, 50 Mbps capacity, and a router queue length of 50 packets. The total propagation delay along a path is set to 20, 40, or 100 ms, which represents the typical delay found on Internet paths within the same coast, across the coast, and across the continent, respec- tively. To simulate the bursty loss behavior, each bottleneck link is shared with background traffic that is composed of 10 FTP sessions and one UDP session consisting of On/Off 102 Table 6.2: Default EMS Settings Parameter Value Loss Requirement,L req 1% Delay Constraint 150ms OLS Step Size,Δ L 3% OLS Adapt Window 60s FEC Adapt Window 30s FEC Redundancy Adapt Step Size,Δ R 5% FEC Redundancy Adapt Threshold,α andβ 0.5 FEC Redundancy Adapt Threshold,γ 0.75 FEC Redundancy Initial Value 10% FEC Group Size Initial Value 10 FEC Group Size Adapt Threshold,b 3 traffic. The On time and Off time of the UDP traffic each follows an exponential dis- tribution with a given mean value. We vary the two mean values (On and Off times) to introduce different loss characteristics, but fix the sum of them at 0.1 seconds. When the UDP traffic is active, it sends 1440 Byte UDP packets at 50 Mbps. Except for the results in Section 6.3.3 (to be explained later), we do not directly set the loss rate for a path. Instead, we control it by varying the UDP traffic and report the measured loss rates. The streaming sender generates CBR traffic at 1900 Kbps, which is a typical for the 720p HD video streaming rate (e.g., in the iTunes Store). The packet size is 500 Byte. The end-to-end delay requirement is set as 150 ms, based on the ITU-T/G.114 recom- mendation for highly interactive tasks [61]. Only those packets that arrive before the deadline are considered to be useful. To ensure good video quality, the loss require- ment is set to be below 1%, which is the typical loss tolerance level for MPEG-4 video. Each simulation run starts with a 30-second warm-up phase (for the background traffic to ramp up), followed by a 3-hour streaming phase. Table 6.2 summarizes the default parameter settings in the EMS implementation. 103 0 5 10 15 20 25 30 35 40 45 0.5 1 1.5 2 Info Loss (%) Path Loss Rate (%) DMP (3 Paths) DMP (5 Paths) EMS (3 Paths) EMS (5 Paths) EMS (3, 5 Paths) Figure 6.5: DMP comparison: Information loss vs path loss (3 or 5 paths) The primary metric used to compare different schemes is the information loss rate. In order to account for the application delay requirements, we now redefine the infor- mation loss rate as the percentage of streaming packets that fail to arrive before their deadlines. It includes: (a) lost packets, (b) late-arrival packets, as well as (c) packets that are recovered after their corresponding deadlines. 6.3.2 Comparison with TCP-based DMP Streaming We first compare EMS with the TCP-based DMP streaming scheme [66]. DMP utilizes multiple paths by maintaining a TCP connection on each path. The sender puts data packets in a single sender queue. At any time, only one TCP connection can gain the access to the sender queue. The winning TCP connection will keep sending data until the connection is blocked. Another available TCP connection will then gain access to the sender queue and continue sending data. Figure 6.5 plots the information loss rates with EMS and DMP, as the path loss increases from 0.06% to 2.24%. The path propagation delays are fixed as 40 ms. To 104 0.01 0.1 1 10 100 1 2 3 4 5 Info Loss (%) Number of Paths DMP (20 ms) DMP (40 ms) DMP (100 ms) EMS (20 ms) EMS (40 ms) EMS (100 ms) EMS (20, 40, 100 ms) Figure 6.6: DMP comparison: Information loss vs # paths (Path loss 0.06%) yield different path loss rates, the on-time of background UDP traffic varies between 0.005 and 0.03 s. From Figure 6.5 we can make two observations as follows. Firstly, EMS can meet the demanding requirements of high-quality live streaming (delay < 150 ms, loss< 1%) in all scenarios. In contrast, DMP meets the requirements only when the paths are almost loss-free. Secondly, DMP can effectively exploit the existence of more paths to improve the streaming quality, but it is very sensitive to the underlying network losses. On the other hand, EMS can consistently maintain high rate of in-time delivery even though the paths become more lossy, thanks to the use of online load splitting and adaptive FEC mechanisms. Next we evaluate the impact of the number of available paths and the delay along each path. For illustration purposes, we focus on the two extreme cases, one with very light path losses (0.06% path loss rate) and the other with heavy path losses (2.24% path loss rate). Figure 6.6 shows the information loss rate when the network is under-utilized and the path loss rate is as low as 0.06%. In such cases, EMS can satisfy the loss requirement 105 0.1 1 10 100 1 2 3 4 5 Info Loss (%) Number of Paths DMP (20 ms) DMP (40 ms) DMP (100 ms) EMS (20 ms) EMS (40 ms) EMS (100 ms) Figure 6.7: DMP comparison: Information loss vs # paths (Path loss 2.24%) with any number of paths, but DMP can only achieve satisfactory performance when 3 or more paths are being used. As expected, as more paths are available, the information loss rate with DMP decreases, because it is less likely that all paths are congested at the same time. However, when there are only one or two paths, DMP is sensitive to the path delays, because the TCP throughput drops as the path delay increases. When the paths become more congested and lossy, as shown in Figure 6.7, EMS can still satisfy the loss requirements in all scenarios. In contrast, even with 5 paths, DMP has an information loss rate as high as 10%, which is not acceptable for multimedia streaming. The reason this happens is that the TCP congestion window is throttled by heavy losses, and many packets miss their deadlines as they are queued either at the sender, waiting for an available TCP connection, or at the receiver, waiting for an earlier packet to be retransmitted. While EMS uses FEC to proactively protect its traffic from network losses, one important question is how much overhead it incurs. To address this concern, Table 6.3 lists the FEC settings that EMS uses under different settings. We can see that the FEC 106 Table 6.3: FEC settings used by EMS:(K,N) and redundancy Number of Paths 1 2 3 4 5 20ms (53,63) (54,62) (53,58) (53,58) (53,58) 18.9% 14.8% 8.6% 8.6% 8.6% 40ms (44,52) (44,50) (43,47) (43,47) (43,47) 18.2% 13.6% 9.3% 9.3% 9.3% 100ms (25,33) (25,30) (25,28) (25,28) (25,28) 32.0% 20.0% 12.0% 12.0% 12.0% overhead in EMS is moderate, ranging from 8.6% to 32%. More importantly, EMS can effectively exploit the path diversity and reduce the FEC redundancy as more paths are available. For example, with 3 paths, the FEC overhead is at most 12%. EMS also adapts the FEC group size to the path delay, which further reduces the FEC overhead when the paths have short delays. A more detailed study of such FEC adaptation mechanisms will be presented later in this section. Finally, we compare the average packet delay with EMS and DMP under different settings, which is plotted in Figure 6.8. Here, the path loss rate is again 2.24%. Not surprisingly, the average packet delay with DMP is much higher than with EMS. Such a large delay is mostly caused by the TCP reliable and in-order delivery mechanisms. While DMP can utilize multiple paths to significantly reduce the packet delay, it still cannot satisfy applications that demand very short end-to-end delay, say a few hundred milliseconds. In summary, the above results indicate that EMS can provide live streaming with stringent delay and loss requirements, while DMP works well only in restricted settings where multiple nearly loss-free paths are available. However, we would like to stress that DMP was designed for video-on-demand type of streaming applications, which can tolerate several seconds of playback delay [66]. It is indeed well suited for such appli- cations, as it effectively exploits the path diversity to improve the streaming quality, and 107 10 100 1000 1 2 3 4 5 Avg Delay (ms) Number of Paths DMP (20 ms) DMP (40 ms) DMP (100 ms) EMS (20 ms) EMS (40 ms) EMS (100 ms) Figure 6.8: DMP comparison: Average delay vs # paths (Path loss 2.24%) the use of TCP makes DMP friendly to other competing TCP traffic. Nevertheless, one should take caution in applying such TCP-based schemes in live streaming applications, such as video conferencing, in which both ends need to be tightly synchronized. 6.3.3 Comparison with Model-based UDP Multi-path Streaming In this subsection, we compare EMS with an existing UDP-based multi-path streaming scheme similar to [55], which seeks to optimally allocate load over multiple available paths using a model-driven approach. It assumes the packet loss process on each path follows the Gilbert Model, and the streaming application can accurately estimate the Gilbert Model parameters. With such parameters in place, it finds an optimal load allo- cation vector numerically. In what follows, we refer to this scheme as the model-based scheme. In contrast, our EMS scheme does not assume or make use of any specific packet loss processes; instead, it takes an adaptive approach to dynamically adjusting the load on each path. 108 0 0.2 0.4 0.6 0.8 1 1.2 1 2 3 4 5 6 7 8 9 10 Info Loss (%) Path 2 Loss Rate (%) EMS Model (perfect) Model (+20ms) Model (-20ms) Figure 6.9: Model-based scheme comparison: Information loss vs Path loss (10% FEC redundancy) The model-based scheme suffers from practical limitations, e.g., the loss processes may not follow the Gilbert Model, and it is hard to accurately estimate the model param- eters in real time. Nevertheless, its theoretical optimality provides an important com- parison base to gauge how well EMS performs. For this purpose, we carry out this comparison using simulation settings that are ideal for the model-based scheme. Specif- ically, we remove all background traffic and emulate packet losses using the Gilbert Model. We consider both the “perfect” case, where the model parameters are precisely known, and the “imperfect” case, where the model parameters are estimated with slight errors. For simplicity, we only consider two paths in this set of experiments. In the Gilbert Model, the sum of average good time and average bad time is fixed at 0.1 sec. For Path 1, the model parameters are fixed to achieve a resulting path loss rate of 2.5%. We vary the model parameter for Path 2 such that the resulting path loss rate varies from 0.5% to 10%. Because the model-based scheme also uses FEC parameters in the computation, we disabled the FEC adaptation mechanism in EMS and experimented 109 0 0.002 0.004 0.006 0.008 0.01 0.012 0.014 1 2 3 4 5 6 7 8 9 10 Info Loss (%) Path 2 Loss Rate (%) EMS Model (perfect) Model (+20ms) Model (-20ms) Figure 6.10: Model-based scheme comparison: Information loss vs Path loss (40% FEC redundancy) with static FEC settings (40-packets group, either 10% or 40% redundancy). The path propagation delays are fixed at 40 ms. Figure 6.9 plots the information loss rate of EMS, perfect model, and imperfect models, with 10% FEC redundancy. Figure 6.10 plots the information loss rate of EMS, perfect model, and imperfect models, with 40% FEC redundancy. With imperfect mod- els, there is a±20 ms error in the estimated Gilbert Model parameter (average bad time) for Path 1. We can make the following two important observations from these two figures. Firstly, the information loss rates yielded by EMS and the perfect model are very close in all cases. In fact, when the FEC redundancy is set at 10%, the performances of EMS and the perfect model are almost identical. This shows that EMS can make near-optimal load splitting decisions, even though it does not utilize any knowledge of the loss processes. Secondly, the model-based scheme is sensitive to the accuracy of the model param- eters. With only a 20ms error in the estimation, the information loss rate may increase 110 10 20 30 40 50 60 70 80 90 100 1 2 3 4 5 6 7 8 9 10 Load on Path 1(%) Path 2 Loss Rate (%) EMS (10%) Model (10%) EMS (40%) Model (40%) Figure 6.11: Model-based scheme comparison: Load splitting vs Path loss (10% or 40% FEC redundancy) from 0.6% to 1%. Note that in our simulations, the loss processes of all paths indeed fol- low exactly the Gilbert Model and the parameters are assumed to be known. In practice, the application has to frequently send probe packets along each path and fit the probing results with the Gilbert Model. This process is inevitably error-prone and introduces traffic overhead, which casts doubts on the performance of such model-based schemes in practical scenarios. To validate these results, we also plot the load splitting vectors decided by EMS and the model-based scheme in Figure 6.11 and Figure 6.12. We can see from Fig- ure 6.11 that our EMS scheme indeed converges to the optimal load splitting solution, as computed by the model-based scheme with accurate parameters. When the loss rate of Path 2 increases, both schemes assign more traffic to Path 1. Furthermore, as the FEC redundancy increases from 10% to 40%, the load split becomes relatively more even. These observations are consistent with our observations in Section 6.1. On the other hand, Figure 6.12 shows that the inaccuracy in the model parameters can lead to 111 0 20 40 60 80 100 1 2 3 4 5 6 7 8 9 10 Load on Path 1(%) Path 2 Loss Rate (%) EMS Model (perfect) Model (+20ms) Model (-20ms) Figure 6.12: Model-based scheme comparison: Load splitting vs. Path loss (40% FEC redundancy) distorted load splitting decisions, which accounts for the performance degradation as we discussed earlier. In summary, EMS can achieve near-optimal traffic allocation among available paths, yet avoid the practical hurdle of accurately modeling the packet loss processes. We also note that the existing UDP-based schemes all use static FEC settings, while EMS can dynamically adjust the FEC parameters based on application requirements and network conditions. 6.3.4 Effectiveness of OLS Now we evaluate how well the Online Load Splitting (OLS) scheme in EMS works. Again, we consider two paths, where Path 1 has a loss rate of 2.9% (with background UDP on-time set at 40 ms) and Path 2 has a loss rate ranging from 0.1% to 5.2% (with background UDP on-time ranging from 5 ms to 80 ms). To stay focused, we disabled the FEC adaption in EMS and used a static FEC setting with 40-packet groups and 10% redundancy. The delays on both paths are 40 ms. We compare OLS to two alternative 112 0 20 40 60 80 100 1 2 3 4 5 Load on Path 1 (%) Path 2 Loss Rate (%) OLS Even Split Single Best Path Figure 6.13: OLS: Load splitting under different schemes schemes, namely Even-Split and Single-Best-Path. Even-Split assigns equal amount of traffic on each path; Single-Best-Path detects the path with the lowest loss rate and assigns all traffic on that single path. Figure 6.13 plots the amount of load on Path 1 with different schemes, and shows that OLS splits the load according to the loss characteristics of different paths. Figure 6.14 shows the corresponding information loss rates. The results show that OLS consistently gives the lowest information loss rate. It is better than Even-Split, especially when paths are more heterogeneous, and it also performs better than Single-Best-Path, especially when the paths have similar path loss rates. In summary, OLS can adjust the load splitting based on the path loss characteristics, thus achieving low information loss rate under all settings. 113 0 0.5 1 1.5 2 1 2 3 4 5 Info Loss (%) Path 2 Loss Rate (%) OLS Even Split Single Best Path Figure 6.14: OLS: Information loss under different schemes 114 0 5 10 15 20 25 1 2 3 4 5 6 7 8 FEC Redundancy (%) Loss Requirement (%) Auto Adapt Fixed 10% Fixed 20% Figure 6.15: FEC Redundancy Adaption: Amount of FEC Redundancy 6.3.5 Effectiveness of FEC Redundancy Adaptation Next we evaluate the performance of FEC redundancy adaptation in EMS, using two homogeneous paths with a path loss rate of 5.1%. We disabled OLS and FEC group size adaption in this experiment, and used Even-Split and 40-packet FEC groups. Figure 6.15 plots the amount of FEC redundancy decided by our adaptation scheme under different loss requirements. Figures 6.16, 6.17, and 6.18 show the corresponding information loss rate, average packet delay, and average throughput of the background FTP sessions, respectively. We can see from Figure 6.15 that EMS can adjust the amount of FEC redundancy according to the loss requirements, by using higher (or lower) redundancy for more stringent (or more relaxed) loss requirements. Figure 6.16 shows that, with EMS adap- tation, the information loss rate is always kept below the requirement. If we use a fixed FEC redundancy amount, e.g., 10% or 20%, it can either result in violation of the loss requirement or incur unnecessary traffic overhead. For example, if we use 10% FEC redundancy, it cannot satisfy any information loss requirement below 2.9%. If we use 115 0 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 Info Loss (%) Loss Requirement (%) Auto Adapt Fixed 10% Fixed 20% Loss Req Figure 6.16: FEC Redundancy Adaption: Information Loss 20% FEC redundancy, it can satisfy the loss requirement ranging from 0.5 to 8%. How- ever, it results in more overhead than the EMS adaptation scheme, while both can satisfy the information loss requirement. From Figure 6.17, we can see the delay reduced along with more relaxed loss requirements. This is due to i) shorter queuing delay in the network as less traffic is injected, and ii) less FEC recovered packets (which normally has long recovery delay). From Figure 6.18, we can see that the background FTP traffic has a faster transmission speed with a more relaxed loss requirement because less FEC redundant packets are sent on the network. In summary, by adaptively changing the amount of FEC redundancy, EMS can sat- isfy the loss requirement with minimum overhead. 6.3.6 Effectiveness of FEC Group Size Adaptation We also study the effectiveness of FEC group size adaptation in EMS using two homoge- neous paths (loss rate fixed as 2.9%). We disabled OLS and FEC redundancy adaptation 116 46 47 48 49 50 51 52 1 2 3 4 5 6 7 8 Avg Delay (ms) Loss Requirement (%) Auto Adapt Fixed 10% Fixed 20% Figure 6.17: FEC Redundancy Adaption: Average Delay in this experiment, and used Even-Split and 10% FEC redundancy. Figure 6.19 shows the FEC group size (K) decided by the EMS adaptation scheme along with different application delay constraints. Figures 6.20 and 6.21 show the corresponding informa- tion loss rate and average packet delay, respectively. As shown in Figure 6.19, EMS can adjust the FEC group size according to the delay constraint. A larger group size is used for longer delay constraints, which results in lower information loss rate. We notice from Figure 6.20 that, compared to a fixed FEC group size, our adaptation scheme can result in the lowest information loss rate for all different delay constraints. Generally, the larger the FEC group is, the more tolerant it is to the high path loss rate. However, the FEC recovery time also becomes longer, result- ing in larger delays. Thus, applications with stringent delay constraint will experience high information loss rates. Figure 6.21 shows that the packet delay increases along with the delay constraint for our adaptation scheme. It is because a larger FEC group size is used in our adaptation scheme when the delay constraint is more relaxed. 117 250 252 254 256 258 260 262 264 266 1 2 3 4 5 6 7 8 BG FTP Speed (kbps) Loss Requirement (%) Auto Adapt Fixed 10% Fixed 20% Figure 6.18: FEC Redundancy Adaption: Background FTP Speed 20 30 40 50 60 70 80 90 100 110 120 120 140 160 180 200 220 240 260 280 300 FEC K Delay Requirement (ms) Auto Adapt Fixed K=40 Fixed K=100 Figure 6.19: FEC Group Size Adaption: Information Loss Finally, we turned on the FEC redundancy adaption and repeated the above exper- iments. The joint effects of both adaptation mechanisms are illustrated in Figure 6.22, which plots the FEC redundancy level chosen by EMS as well as the transmission rate of the background FTP traffic. Interestingly, when the delay requirement is relaxed, EMS also reduces the FEC redundancy. This is because in such cases, EMS will choose 118 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 120 140 160 180 200 220 240 260 280 300 Info Loss (%) Delay Requirement (ms) Auto Adapt Fixed K=40 Fixed K=100 Figure 6.20: FEC Group Size Adaption: Information loss a larger FEC group size (as explained in Figure 6.19), which provides better recovery strength. As such, EMS can ensure the same loss requirement with a lower FEC redun- dancy level, which in turn reduces the network overhead and leads to throughput gains in the competing background FTP traffic. In summary, the FEC group size adaptation in EMS can efficiently exploit the delay/loss tradeoff and ensure both, in-time FEC recovery and maximum recovery strength 119 44.5 45 45.5 46 46.5 47 47.5 48 48.5 49 120 140 160 180 200 220 240 260 280 300 Avg Delay (ms) Delay Requirement (ms) Auto Adapt Fixed K=40 Fixed K=100 Figure 6.21: FEC Group Size Adaption: Average delay 6 8 10 12 14 16 18 20 22 24 26 140 180 220 260 300 640 645 650 655 660 FEC Redundancy (%) BG FTP Speed (kbps) Delay Requirement (ms) FEC Redundancy BG FTP Speed Figure 6.22: FEC Group Size Adaption: Joint FEC adaptation 120 Chapter 7 Conclusions We now give a summary of this thesis. We first considered an important problem of how to appropriately distribute seeding capacity in BitTorrent in Chapter 3 of this thesis. Bit- Torrent was widely used nowadays to achieve efficient file downloadings by utilizing uploading resources of the users. However, seeds staying in the system hurt the built-in tit-for-tat fairness mechanism and make free-riding possible. Our study indicates that, under a number of scenarios, our proposed seeding schemes are able to: (1) significantly degrade free-riders’ performance while (2) improving (or not affecting) the performance of contributing leechers. Thus, we believe that this is a promising direction for improv- ing the overall performance of BitTorrent while discouraging free-riding behavior. In order to aid understanding of BitTorrent and possible protocol changes, we pro- posed a simple yet accurate and extensible model for BitTorrent in Chapter 4 of this thesis. Our model includes (measured) characteristics of the protocol that have pre- viously been left unmodeled, and we demonstrated the importance of including these characteristics both via our model as well as simulations and PlanetLab experiments. Our validation study indicates that our model is quite accurate in predicting BitTorrent performance. Our modeling approach can aid in understanding of BitTorrent, which can in turn lead to improvements in parameter setting as well as protocol design. Further- more, our model can easily be extended to study protocol changes, e.g., our proposed seeding approach in Chapter 3 in this thesis. Thus, our model can be used by the com- munity to gain insights on the working of BitTorrent and help design improvements. 121 In addition to large file downloading, we also studied another rapidly growing Inter- net application, online media streaming. Specifically, we studied the use of multiple paths existing in the network to improve the quality of media streaming. In Chapter 5 of this thesis, we first considered the load distribution problem in the context of multi-path streaming. We focused on appropriate optimization objectives for splitting the contin- uous media traffic among multiple paths, based on path characteristics, where the high level goal was to improve the quality of the received media. We proposed several opti- mization objectives and evaluated their characteristics through analysis and simulations. Our results indicate that approaches which minimize the mean packet loss rate may increase the loss correlations, and vice versa. These results also indicated that both, fre- quency of losses and their correlations are important to the resulting quality of streamed media, when erasure codes are used. Within the context of online streaming, live streaming applications have become increasingly popular nowadays, driven by the widespread adoption of broadband net- works. Such applications typically have stringent quality of service requirements, and the transport protocol must ensure the streaming content is delivered in time, with min- imal losses. In Chapter 6 of this thesis, we studied how to adapt multi-path streaming for the emerging real-time live streaming applications with tight delay and loss require- ments. We designed a novel multi-path live streaming protocol, EMS, with built-in load splitting and FEC adaptation mechanisms. The effectiveness of EMS is confirmed by simulation results under various settings. 122 References [1] Bassem Abdouni, William W.C. Cheng, Alix L.H. Chow, Leana Golubchik, and John C.S. Lui. Multi-path streaming: Optimization and evaluation. Technical Report IMSC-04-001, IMSC, University of Southern California, 2004. [2] David G. Andersen, Hari Balakrishnan, M. Frans Kaashoek, and Robert Morris. Resilient overlay networks. In Proceedings of ACM SOSP, Banff, Canada, Oct 2001. [3] John Apostolopoulos and Mitchell Trott. Path diversity for enhanced media streaming. IEEE Communications Magazine, 42:80–87, Aug 2004. [4] John Apostolopoulos, Tina Wong, Susie Wee, and Dan Tan. On multiple descrip- tion streaming with content delivery networks. In Proceedings of IEEE INFO- COM, New York, NY , Jun 2002. [5] Anthony Bellissimo, Brian N. Levine, and Prashant Shenoy. Exploring the use of BitTorrent as the basis for a large trace repository. Technical Report 04-41, Dept. of Computer Science, University of Massachusetts, 2004. [6] Ashwin R. Bharambe, Cormac Herley, and Venkata N. Padmanabhan. Analyz- ing and improving BitTorrent performance. In Proceedings of IEEE INFOCOM, Barcelona, Catalunya, Spain, Apr 2006. [7] Justin Bieber, Michael Kenney, Nick Torre, and Landon P. Cox. An empirical study of seeders in BitTorrent. Technical Report CS-2006-08, Duke University, 2006. [8] Jiancong Chen and S.-H. Gary Chan. Multipath routing for video unicast over bandwidth-limited networks. In Proceedings of IEEE Globecom, San Antonio, TX, Nov 2001. [9] Alix L.H. Chow, Leana Golubchik, John C.S. Lui, and Adam W.-J. Lee. Multi- path streaming: Optimization of load distribution. Performance Evaluation, 62(1- 4):417–438, Oct 2005. 123 [10] Alix L.H. Chow, Hao Yang, Cathy H. Xia, Minkyong Kim, Zhen Liu, and Hui Lei. EMS: Encoded multipath streaming for real-time live streaming applications. In Proceedings of IEEE ICNP, Princeton, New Jersey, Oct 2009. [11] Jean chrysostome Bolot, Sacha Fosse-parisis, and Don Towsley. Adaptive FEC- based error control for internet telephony. In Proceedings of IEEE INFOCOM, New York, NY , Mar 1999. [12] Israel Cidon, Raphael Rom, and Yuval Shavitt. Analysis of multi-path routing. IEEE/ACM Transactions on Networking, 7(6):885–896, 1999. [13] Bram Cohen. Incentives build robustness in BitTorrent. In Proceedings of P2PECON, Berkeley, CA, Jun 2003. [14] John R. Douceur. The sybil attack. In Proceedings of IPTPS, Cambridge, MA, Mar 2002. [15] Bin Fan, Dah-Ming Chiu, and John C.S. Lui. The delicate tradeoffs in BitTorrent- like file sharing protocol design. In Proceedings of IEEE ICNP, Santa Barbara, CA, Nov 2006. [16] Venkat Raju Gandikota, Bheemajun Reddy Tamma, and C. Siva Ram Murthy. Adaptive FEC-based packet loss resilience scheme for supporting voice commu- nication over ad hoc wireless networks. IEEE Transactions on Mobile Computing, 7(10):1184–1199, Oct 2008. [17] Leana Golubchik, John C.S. Lui, Tak Fu Tung, Alix L.H. Chow, Adam W.-J. Lee, Giuliana Franceschinis, and Cosimo Anglano. Multi-path continuous media streaming: What are the benefits? Performance Evaluation Journal, 49(1-4):429– 449, Sep 2002. [18] Lei Guo, Songqing Chen, Zhen Xiao, Enhua Tan, Xiaoning Ding, and Xiaodong Zhang. Measurements, analysis and modeling of BitTorrent-like systems. In Pro- ceedings of ACM SIGCOMM IMC, Berkeley, CA, Oct 2005. [19] Khaled Harfoush, Azer Bestravos, and John Byers. Robust identification of shared losses using end-to-end unicast probes. In Proceedings of IEEE ICNP, Osaka, Japan, Nov 2000. [20] David Harrison and Bram Cohen. BitTorrent Fast Extension. http://bittorrent.org/beps/bep 0006.html. [21] Xiaojun Hei, Chao Liang, Jian Liang, Yong Liu, and Keith W. Ross. A measure- ment study of a large-scale P2P IPTV system. IEEE Transactions on Multimedia, 9(8):1672–1687, Dec 2007. 124 [22] http://www.hulu.com/. Hulu. [23] http://www.isi.edu/nsnam/ns/. The Network Simulator - ns-2. [24] http://www.mesquite.com/. CSIM. [25] http://www.pplive.com/. PPLive. [26] http://www.ppstream.com/. PPStream. [27] http://www.youtube.com/. YouTube. [28] Hao hua Chu and Klara Nahrstedt. Dynamic multi-path communication for video traffic. In Proceedings of IEEE HICSS, Maui, HW, Jan 1997. [29] Mikel Izal, Guillaume Urvoy-Keller, Ernst Biersack, Pascal Felber, Anwar Al Hamra, and Luis Garces-Erice. Dissecting BitTorrent: Five months in a torrent’s lifetime. In Proceedings of PAM, Antibes Juan-les-Pins, France, Apr 2004. [30] Seung Jun and Mustaque Ahamad. Incentives in BitTorrent induce free riding. In Proceedings of P2PECON, Philadelphia, PA, Aug 2005. [31] Dan Jurca and Pascal Frossard. Video packet selection and scheduling for multi- path streaming. IEEE Transactions on Multimedia, 9(3):629–641, Apr 2007. [32] Hema Tahilramani Kaur, Shivkumar Kalyanaraman, Andreas Weiss, Shifalika Kanwar, and Ayesha Ghandi. Bananas: An evolutonary framework for explicit and multipath routing in the internet. ACM SIGCOMM Computer Communication Review, 33(4):277288, Oct 2003. [33] Frank Kelly. Mathematical modelling of the internet. Mathematics Unlimited - 2001 and Beyond, pages 685–702, 2001. [34] Maciej Kurant. Exploiting the path propagation time differences in multipath trans- mission with FEC. In Proceedings of IEEE INFOCOM, Rio de Janeiro, Brazil, Apr 2009. [35] Sung-Ju Lee, Sujata Banerjee, Puneet Sharma, Praveen Yalagandula, and Sujoy Basu. Bandwidth-aware routing in overlay networks. In Proceedings of IEEE INFOCOM, Phoenix, AZ, Apr 2008. [36] Arnaud Legout, Nikitas Liogkas, Eddie Kohler, and Lixia Zhang. Clustering and sharing incentives in BitTorrent systems. In Proceedings of ACM SIGMETRICS, San Diego, CA, Jun 2007. 125 [37] Arnaud Legout, Guillaume Urvoy-Keller, and Pietro Michiardi. Rarest first and choke algorithms are enough. In Proceedings of ACM SIGCOMM IMC, Rio de Janeiro, Brazil, Oct 2006. [38] Dave Levin, Katrina LaCurts, Neil Spring, and Bobby Bhattacharjee. BitTorrent is an auction: Analyzing and improving BitTorrents incentives. In Proceedings of ACM SIGCOMM, Seattle, WA, Aug 2008. [39] Minglu Li, Jiadi Yu, and Jie Wu. Free-riding on BitTorrent-like peer-to-peer file sharing systems: Modeling analysis and improvement. IEEE Transactions on Par- allel and Distributed Systems, 19(7):954–966, Jul 2008. [40] Qiu Hui Li and John C.S. Lui. On modeling clustering indexes of BT-like systems. In Proceedings of IEEE ICC, Dresden, Germany, Jun 2009. [41] Yi Li, Yin Zhang, LiLi Qiu, and Simon Lam. SmartTunnel: Achieving reliability in the internet. In Proceedings of IEEE INFOCOM, Anchorage, AK, May 2007. [42] Yi J. Liang, Eckehard G, Steinbach, and Bernd Girod. Multi-stream voice over IP using packet path diversity. In Proceedings of IEEE MMSP, Cannes, France, Oct 2001. [43] Yi J. Liang, Eckehard G, Steinbach, and Bernd Girod. Real-time voice communi- cation over the internet using packet path diversity. In Proceedings of ACM MM, Ottawa, Canada, Sep/Oct 2001. [44] Wei-Cherng Liao, Fragkiskos Papadopoulos, and Konstantinos Psounis. Perfor- mance analysis of BitTorrent-like systems with heterogeneous users. Performance Evaluation Journal, 64(9-12):876–891, Oct 2007. [45] Nikitas Liogkas, Robert Nelson, Eddie Kohler, and Lixia Zhang. Exploiting Bit- Torrent for fun (but not profit). In Proceedings of IPTPS, Santa Barbara, CA, Feb 2006. [46] Thomas Locher, Patrick Moor, Stefan Schmid, and Roger Wattenhofer. Free riding in BitTorrent is cheap. In Proceedings of ACM SIGCOMM HotNets, Irvine, CA, Nov 2006. [47] Thomas Locher, Stefan Schmid, and Roger Wattenhofer. Rescuing tit-for-tat with source coding. In Proceedings of IEEE P2P, Galway, Ireland, Sep 2007. [48] Gustavo Marfia, Alex Sentivelli, Saurabh Tewari, Mario Gerla, and Leonard Klein- rock. Will IPTV ride the peer-to-peer stream? IEEE Communications Magazine, 45(6):86–92, Jun 2007. 126 [49] Laurent Massoulie and Milan V ojnovie. Coupon replication systems. IEEE/ACM Transactions on Networking, 16(3):603–616, 2008. [50] Nicholas F. Maxemchuk. Dispersity routing. In IEEE ICC, San Francisco, CA, Jun 1975. [51] Philip McCord Morse. Queues, Inventories, and Maintenance. John Wiley, 1958. [52] Jochen Mundinger, Richard Weber, and Gideon Weiss. Optimal scheduling of peer-to-peer file dissemination. Journal of Scheduling, 11(2):105–120, Apr 2008. [53] Thinh Nguyen and Avideh Zakhor. Distributed video streaming over internet. In Proceedings of SPIE MMCN, San Jose, CA, Jan 2002. [54] Thinh Nguyen and Avideh Zakhor. Path diversity with forward error correction (PDF) system for delay sensitive applications over the internet. In Proceedings of IEEE INFOCOM, San Francisco, CA, Mar 2003. [55] Thinh Nguyen and Avideh Zakhor. Multiple sender distributed video streaming. IEEE Transactions on Multimedia, 6(2):315–326, Apr 2004. [56] Michael Piatek, Tomas Isdal, Thomas Anderson, Arvind Krishnamurthy, and Arun Venkataramani. Do incentives build robustness in BitTorrent? In Proceedings of USENIX NSDI, Cambridge, MA, Apr 2007. [57] Johan A. Pouwelse, Pawel Garbacki, Dick H.J. Epema, and Henk J. Sips. The BitTorrent P2P file-sharing system: Measurements and analysis. In Proceedings of IPTPS, Ithaca, NY , Feb 2005. [58] Dongyu Qiu and R. Srikant. Modeling and performance analysis of BitTorrent- like peer-to-peer networks. In Proceedings of ACM SIGCOMM, Portland, OR, Aug 2004. [59] Pablo Rodriguez, Andreas Kirpal, and Ernst Biersack. Parallel-access for mirror sites in the internet. In Proceedings of IEEE INFOCOM, Tel Aviv, Israel, Mar 2000. [60] Dan Rubenstein, Jim Kurose, and Don Towsley. Detecting shared congestion of flows via end-to-end measurement. In Proceedings of ACM SIGMETRICS, Santa Clara, California, Jun 2002. [61] Telecommunication Standardization Sector. ITU-T recommendation G.114. Tech- nical report, International Telecommunication Union, 1993. [62] Vicky Sharma, Shivkumar Kalyanaraman, Koushik Kar, K.K. Ramakrishnan, and Vijay Subramanian. MPLOT: A transport protocol exploiting multipath diversity using erasure codes. In Proceedings of IEEE INFOCOM, Phoenix, AZ, Apr 2008. 127 [63] Kyuyong Shin, Douglas S. Reeves, and Injong Rhee. Treat-before-trick: Free- riding prevention for BitTorrent-like peer-to-peer networks. In Proceedings of IEEE IPDPS, Rome, Italy, May 2009. [64] Michael Sirivianos, Jong Han Park, Rex Chen, and Xiaowei Yang. Free-riding in BitTorrent networks with the large view exploit. In Proceedings of IPTPS, Bellevue, WA, Feb 2007. [65] Ye Tian, Di Wu, and Kam Wing Ng. Modeling, analysis and improvement for BitTorrent-like file sharing networks. In Proceedings of IEEE INFOCOM, Barcelona, Catalunya, Spain, Apr 2006. [66] Bing Wang, Wei Wei, Zheng Guo, and Don Towsley. Multipath live streaming via TCP: Scheme, performance and benefits. In Proceedings of ACM CoNEXT, New York, NY , Oct 2007. [67] Xiangying Yang and Gustavo de Veciana. Service capacity in peer-to-peer net- works. In Proceedings of IEEE INFOCOM, Hong Kong, Mar 2004. [68] Xinyan Zhang, Jiangchuan Liu, Bo Li, and Tak-Shing Peter Yum. Coolstream- ing/DONet: A data-driven overlay network for efficient live media streaming. In Proceedings of IEEE INFOCOM, Miami, FL, Mar 2005. 128

Abstract (if available)

Abstract Along with the bandwidth and popularity growth of the Internet, the scale of the content delivered on the Internet also expanded. Large file downloading and online media streaming will soon, if not already, become the two most resource consuming applications on the network.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

QoS based resource management for Internet applications

PDF

QoS-aware algorithm design for distributed systems

PDF

Adaptive resource management in distributed systems

PDF

Supporting multimedia streaming among mobile ad-hoc peers with link availability prediction

PDF

Satisfying QoS requirements through user-system interaction analysis

PDF

Distributed resource management for QoS-aware service provision

PDF

Design-time software quality modeling and analysis of distributed software-intensive systems

PDF

Measuring the impact of CDN design decisions

PDF

Scalable peer-to-peer streaming for interactive applications

PDF

High-performance distributed computing techniques for wireless IoT and connected vehicle systems

PDF

Scaling recommendation models with data-aware architectures and hardware efficient implementations

PDF

Modeling and optimization of energy-efficient and delay-constrained video sharing servers

PDF

Coded computing: Mitigating fundamental bottlenecks in large-scale data analytics

PDF

Improving user experience on today’s internet via innovation in internet routing

PDF

Enabling efficient service enumeration through smart selection of measurements

PDF

Resource scheduling in geo-distributed computing

PDF

Energy efficient design and provisioning of hardware resources in modern computing systems

PDF

Detecting and mitigating root causes for slow Web transfers

PDF

Efficient processing of streaming data in multi-user and multi-abstraction workflows

PDF

Optimal designs for high throughput stream processing using universal RAM-based permutation network

Asset Metadata

Creator Chow, Lik Hang (author)

Core Title QoS-based distributed design of streaming and data distribution systems

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Computer Science

Publication Date 02/05/2010

Defense Date 11/12/2009

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag BitTorrent,content distribution,Modeling,multi-path,OAI-PMH Harvest,peer-to-peer,streaming

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Golubchik, Leana (committee chair), Govindan, Ramesh (committee member), Kuo, C.-C. Jay (committee member), Misra, Vishal (committee member), Teng, Shang-Hua (committee member)

Creator Email alix@alixchow.com,lhchow@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-m2838

Unique identifier UC1184025

Identifier etd-Chow-3409 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-301441 (legacy record id),usctheses-m2838 (legacy record id)

Legacy Identifier etd-Chow-3409.pdf

Dmrecord 301441

Document Type Dissertation

Rights Chow, Lik Hang

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Repository Name Libraries, University of Southern California

Repository Location Los Angeles, California

Repository Email cisadmin@lib.usc.edu