University of Southern California Dissertations and Theses

Congestion control in multi-hop wireless networks

(USC Thesis Other)

Congestion control in multi-hop wireless networks

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content CONGESTION CONTROL IN MULTI-HOP WIRELESS NETWORKS by Sumit Rangwala A Dissertation Presented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (COMPUTER SCIENCE) December 2009 Copyright 2009 Sumit Rangwala TableofContents ListOfTables iv ListOfFigures v Abstract viii Chapter1: Introduction 1 Chapter2: BackgroundandRelatedWork 6 2.1 Internet Congestion Control . . . . . . . . . . . . . . . . . . . . 6 2.2 Congestion Control in Wireless Networks . . . . . . . . . . . . . 7 2.2.1 Congestion Control in Single Hop Wireless Networks . . 7 2.2.2 Congestion Control in Multi-hop Wireless Networks: Dis- tributed Schemes . . . . . . . . . . . . . . . . . . . . . . 8 2.2.2.1 AIMD Based Congestion Control . . . . . . . . 8 2.2.2.2 Congestion Control with Explicit Rate Feedback 11 2.2.3 Congestion Control in Multi-hop Wireless Networks: Cen- tralized Schemes . . . . . . . . . . . . . . . . . . . . . . 12 2.2.4 Congestion Control in Wireless Sensor Networks . . . . . 13 2.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Chapter3: CongestionControlinWirelessSensorNetworks 17 3.1 Motivation and Definitions . . . . . . . . . . . . . . . . . . . . . 21 3.1.1 Fair and Efficient Rate Allocation . . . . . . . . . . . . . 23 3.2 IFRC Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.2.1 IFRC Mechanisms . . . . . . . . . . . . . . . . . . . . . 27 3.2.1.1 Measuring Congestion Levels . . . . . . . . . . 27 3.2.1.2 Congestion Sharing . . . . . . . . . . . . . . . 29 3.2.1.3 Rate Adaptation . . . . . . . . . . . . . . . . . 31 3.2.1.4 Base Station Behavior . . . . . . . . . . . . . . 33 3.2.2 Extensions to IFRC . . . . . . . . . . . . . . . . . . . . . 34 3.2.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.3 Parameter Selection in IFRC . . . . . . . . . . . . . . . . . . . . 38 3.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 ii 3.4.1 Implementation and Methodology . . . . . . . . . . . . . 47 3.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 Chapter4: CongestionControlinMulti-hopWirelessNetworks 65 4.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 4.1.1 WCP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 4.1.2 WCPCap . . . . . . . . . . . . . . . . . . . . . . . . . . 79 4.2 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . 86 4.2.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . 86 4.2.2 Performance of WCP and WCPCap . . . . . . . . . . . . 91 4.2.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 97 4.2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . 114 4.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 Chapter5: ConclusionandFutureDirections 121 5.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 5.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 5.3 Open Problems and Future Directions . . . . . . . . . . . . . . . 124 5.4 Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 Bibliography 127 iii ListOfTables 3.1 IFRC parameters used in the experiments . . . . . . . . . . . . . 49 3.2 Validating the analysis of Section 3.3 . . . . . . . . . . . . . . . . 63 4.1 Parameters used in simulations . . . . . . . . . . . . . . . . . . . 88 4.2 Stack with auto-rate adaptation . . . . . . . . . . . . . . . . . . . 111 iv ListOfFigures 3.1 An example routing sub-tree. Dashed lines represent a neighbor that is neither a parent nor a child of a node. . . . . . . . . . . . . 22 3.2 Scenario with multiple base stations (nodes 0 and 10) . . . . . . . 35 3.3 AIMD behavior ofr i (t) . . . . . . . . . . . . . . . . . . . . . . . 40 3.4 Testbed connectivity graph . . . . . . . . . . . . . . . . . . . . . 48 3.5 Routing tree and link qualities used in the experiments . . . . . . 51 3.6 Per flow goodput in the 40-node experiment . . . . . . . . . . . . 52 3.7 Packet reception in the 40-node experiment . . . . . . . . . . . . 53 3.8 Rate adaptation in the 40-node experiment . . . . . . . . . . . . . 54 3.9 Instantaneous queue size in 40-node experiment . . . . . . . . . . 55 3.10 Optimality test for the 40-node experiment . . . . . . . . . . . . . 56 3.11 Node addition: Rate adaptation . . . . . . . . . . . . . . . . . . . 58 3.12 Node deletion: Rate adaptation . . . . . . . . . . . . . . . . . . . 59 3.13 Per flow goodput with weighted fairness . . . . . . . . . . . . . . 60 3.14 Per flow goodput with only a subset of senders . . . . . . . . . . . 61 3.15 Per flow goodput with multiple sinks . . . . . . . . . . . . . . . . 62 3.16 Per flow goodput with no link-layer retransmissions . . . . . . . . 63 v 4.1 Stack topology . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 4.2 The achievable rate region for Stack topology . . . . . . . . . . . 67 4.3 Congestion neighborhood . . . . . . . . . . . . . . . . . . . . . . 71 4.4 Pseudo-code for rate controller at linki→ j . . . . . . . . . . . . 84 4.5 Diamond topology . . . . . . . . . . . . . . . . . . . . . . . . . 89 4.6 Half-Diamond topology . . . . . . . . . . . . . . . . . . . . . . . 90 4.7 Chain-Cross topology . . . . . . . . . . . . . . . . . . . . . . . . 90 4.8 WCP without congestion and RTT sharing, Stack . . . . . . . . . 92 4.9 WCP and WCPCap, Stack . . . . . . . . . . . . . . . . . . . . . 93 4.10 WCP and WCPCap, Diamond . . . . . . . . . . . . . . . . . . . 94 4.11 WCP and WCPCap, Half-Diamond . . . . . . . . . . . . . . . . 95 4.12 WCP and WCPCap, Chain-Cross . . . . . . . . . . . . . . . . . . 96 4.13 Evolution of flow rates with WCP in Chain-Cross . . . . . . . . . 97 4.14 WCP and WCPCap over lossy links, Stack . . . . . . . . . . . . . 98 4.15 WCP without goodput correction, Stack . . . . . . . . . . . . . . 99 4.16 WCP with in-network rate adaptation, Chain-Cross . . . . . . . . 100 4.17 WCP with in-network rate adaptation, Stack . . . . . . . . . . . . 100 4.18 WCP with in-network rate adaptation, Diamond . . . . . . . . . . 101 4.19 WCP with in-network rate adaptation, Half-Diamond . . . . . . . 101 4.20 Delay as a function ofU for f 4→6 , Stack . . . . . . . . . . . . . . 102 4.21 WCP and WCPCap with no RTS/CTS, Stack . . . . . . . . . . . 103 4.22 WCP and WCPCap with no RTS/CTS, Half-Diamond . . . . . . . 104 vi 4.23 WCP with delayed flow arrival . . . . . . . . . . . . . . . . . . . 105 4.24 WCPCap with delayed flow arrival . . . . . . . . . . . . . . . . . 106 4.25 WCP with larger interference range, Chain-Cross . . . . . . . . . 107 4.26 WCP with larger interference range, Stack . . . . . . . . . . . . . 108 4.27 WCP with larger interference range, Diamond . . . . . . . . . . . 108 4.28 WCP with larger interference range, Half-Diamond . . . . . . . . 109 4.29 Stack topology with different link rates . . . . . . . . . . . . . . . 110 4.30 WCP over links with different link rates, Stack . . . . . . . . . . 111 4.31 WCP with no link layer retransmissions, Stack . . . . . . . . . . . 112 4.32 WCP with no link layer retransmissions, Diamond . . . . . . . . . 113 4.33 WCP with no link layer retransmissions, Half-Diamond . . . . . . 113 4.34 WCP with no link layer retransmissions, Chain-Cross . . . . . . . 114 4.35 Average end-to-end delay with WCP and WCPCap . . . . . . . . 115 4.36 Results from Stack experimental topology . . . . . . . . . . . . . 117 4.37 Arbitrary experimental topology of 14 nodes . . . . . . . . . . . . 118 4.38 Results from arbitrary topology of 14 nodes . . . . . . . . . . . . 119 4.39 Arbitrary experimental topology of 13 nodes . . . . . . . . . . . . 119 4.40 Results from arbitrary topology of 13 nodes . . . . . . . . . . . . 120 vii Abstract In this dissertation, we propose that it is necessary to consider wireless neighbor- hood as a single indivisible entity for congestion control in multi-hop wireless networks. We precisely define this neighborhood for a class of contention-based wireless networks and validate it with three protocols. We design IFRC, an open- loop rate-based fully distributed congestion control protocol for multi-hop wire- less sensor networks with tree-like traffic pattern. We design WCP, a closed-loop AIMD-based fully distributed end-to-end congestion control protocol for multi-hop wireless networks with arbitrary traffic pattern. In addition, we design WCPCap, an explicit-rate feedback congestion control protocol for multi-hop wireless net- work. While they differ in their local mechanisms to achieve congestion control all these protocols recognize neighborhood, as per our definition, as a single entity and achieve near optimal performance in both simulations and real deployments. viii Chapter1 Introduction The Internet, with its phenomenal growth of three-decades since its inception, is in- creasingly becoming the primary means of communication. However, in the recent decade the linear growth of Internet usage is dwarfed by the exponential growth of wireless networks with one-third of the current users accessing the Internet via a wireless connection [71]. With this growth wireless networks are increasingly be- coming part of the edge networks while pushing wired networks to the Internet’s core. Currently, majority of these networks of last hop wireless networks. However, with the advent of distributed wireless sensing, community networks, and city-wide video surveillance, wireless networks are increasing becoming multi-hop. Congestion control protocols are an integral part of current communication net- works. They seek to maximize utilization and achieve a desired allocation of net- work resources. TCP, the de-facto congestion control protocol of the Internet, is known to perform well in wired network but exhibits poor performance in wireless 1 networks. Early research in congestion control for wireless networks proposed so- lutions to address this performance degradation [7, 5]. Though effective in a last hop wireless network, these solutions are inadequate for congestion control in a multi-hop wireless network [74]. In this dissertation, we take a clean-slate approach to the design of congestion control protocol for a static, solely wireless multi-hop network. We show that due to the shared medium of communication in wireless networks congestion control for such network should consider a wireless neighborhood as a single entity. With the design of three protocols we validate this principle which we believe is integral to design of congestion control scheme for wireless networks. Congestion control is a resource management technique that tries to achieve the goal of efficiency, in the form of high link utilization and small queues, and re- source allocation in accordance to a desired allocation policy [34]. Achieving this goal requires knowledge of network resources and flows’ demands. However, in a communication network the source of this information is not a single entity. The end-host are cognizant of the flows’ demands while the routers have the knowledge of the network resources and their utilization. Congestion control in communication networks, thus, strive to achieve the goals of efficiency and resource allocation with minimum communication between these entities to attain low control overhead. In today’s networks congestion control is accomplished through a cooperative coordi- nation between the end hosts and the routers. 2 Central to the distributed design of congestion control is the identification of the local entity (e.g., link, node, etc.) on which a router enforces local policies to achieve the global objectives of congestion control. For e.g. in a wired network this local entity is a wired link. TCP along with various active queue management schemes locally enforce policies on a single link to globally achieve a desired band- width allocation. Other protocols [35, 18] while differing in their local policies still use the same local entity, a link. However, this notion of link as a local entity does not extend to wireless net- works due to the intrinsic characteristic of a shared medium in wireless communi- cations. In wireless networks, nodes share the wireless channel with their neighbors (and additionally with other nodes). Therefore, capacity 1 of a wireless link 2 de- pends on the capacity of the other links that interfere with the given link. Thus, TCP, using link as a local entity, is known to perform poorly for wireless networks [74] In this dissertation, we identify and show wireless neighborhood as the correct entity for local policy enforcement. We precisely define this neighborhood arguing that it should be defined with respect to a link rather than for a node. We demon- strate the effectiveness of this abstraction with the design of the following three congestion control protocols that we describe in subsequent chapters. IFRC: In Chapter 3 we describe the design of Interference-Aware Fair Rate Con- trol (IFRC), a congestion control protocol for wireless sensor networks where data 1 i.e., the number of packets that can be transmitted over single hop between a sender and a receiver 2 In this dissertation, we refer to a one-hop sender receiver pair as a wireless link 3 traffic is primarily from various nodes in the network to a single base station. IFRC addresses congestion as a neighborhood phenomenon, exploits many-to-one traffic pattern of wireless sensor networks to design a low-overhead congestion sharing mechanism, and converges to a fair and efficient rate using an AIMD control law. IFRC achieves a fair and efficient rate allocation that is within 20-40% of the opti- mal fair rate allocation on some network topologies. WCP: In Chapter 4 we present the design of Wireless Control Protocol (WCP) an AIMD-based rate-control protocol for multi-hop wireless networks. Unlike IFRC, WCP is designed for networks with arbitrary traffic pattern. During congestion, WCP signals all flows in a neighborhood of congestion and set the control inter- val (and thus these flows’ rate clocking) to the maximum RTT of any flow in the neighborhood. Using analysis, simulations, and real deployments, we find that our designs yield rates that are both fair and efficient, and for all the topologies that we study. WCP achieves this level of performance while being extremely easy to implement. WCPCap: In Chapter 4 we also present the design of Wireless Control Protocol with Capacity estimation (WCPCap), a distributed rate controller that estimates the available capacity within each neighborhood, and divides this capacity to contend- ing flows. WCPCap achieves within 15% of the max-min rates for our topologies, while still being distributed and amenable to real implementation. 4 Congestion control in wireless networks is a hard problem. In this disserta- tion, we make a small but significant step toward understanding it with the design of IFRC, WCP, and WCPCap. We do not claim to have addressed all issues of congestion control in multi-hop wireless networks. We demonstrate, however, that considering wireless congestion collectively over a neighborhood of a link is es- sential to any future design of wireless congestion control. 5 Chapter2 BackgroundandRelatedWork This chapter starts with a brief overview of Internet congestion control, followed by a discussion of previous works in wireless congestion control. We describe how WCP and WCPCap, our congestion control schemes for multi-hop wireless net- works, differs from these efforts. We conclude with a description of research in congestion control for wireless sensor networks in contrast to IFRC, our conges- tion control scheme for networks with a specific (many-to-one) traffic pattern. 2.1 InternetCongestionControl Congestion control problem in communication networks, belonging to general class of resource allocation, seek to attain efficiency and a desired resource allocation of the shared communication resources [34]. Distributed congestion control in today’s networks is realized with a coordination of the end hosts and the routers. These pro- tocols differ in amount of communication between these entities and the task they perform in resource allocation. 6 One end of the spectrum exemplified by TCP, the de facto Internet congestion control protocol, consist of end hosts that probe for network congestion and deter- mine their sending rate accordingly. The router, in contrast, provides a binary feed- back, primarily via a packet drop, of network congestion in a manner that leads the end hosts to converge to a rate leading to a particular distribution of resources [36]. Our design of WCP in similar in spirit to such protocols. The other end of the spectrum consisting of protocols, like XCP and RCP, in- volves routers that assess congestion in the network and calculate the rate as per the required allocation policy. The end hosts, thus, receive explicit feedback from the routers that determines their sending rate. Our design of WCPCap follows this approach. 2.2 CongestionControlinWirelessNetworks Extensive research has been done to understand the shortcoming and to improve the performance of TCP in wireless networks [63, 21, 12, 26, 76, 37, 43, 74]. We briefly discuss broad classes of research pertinent to our work while referring interested reader to [44] for a more comprehensive survey of congestion control in wireless networks. 7 2.2.1 CongestionControlinSingleHopWirelessNetworks Early work on improving TCP performance in wireless networks focused on distin- guishing between packet loss due to wireless corruption from loss due to conges- tion, in the context of last-hop wireless [7, 4] or wireless wide-area networks [60]. Examples include I-TCP [5] that splits a TCP connection into two separate connec- tions one between the source and the access point (base station) and one between the access point and the destination, distinct acknowledgments [11] to separate losses on wired and wireless links, the Snoop protocol [7] that caches packets and ACKs at the base station to recover from link error losses to improve TCP performance. In contrast, we address congestion control for multi-hope wireless networks. 2.2.2 CongestionControlinMulti-hopWirelessNetworks:Distributed Schemes 2.2.2.1 AIMDBasedCongestionControl More recent work, however, has addressed congestion control for mobile ad-hoc wireless networks. One class of work, exemplified by TCP-F [12], TCP-ELFN [26], TCP-BuS [37], ATCP [43], and EPLN/BEAD [76], concentrates on improving TCP’s throughput by freezing TCP’s congestion control algorithm during link- failure induced losses, especially when route changes occur. Individual pieces of work differ in the manner in which these losses are identified and notified to the 8 sender and in their details of freezing TCP. For example, TCP-ELFN [26] explic- itly notifies the TCP sender of routing failure causing the sender to enter a standby mode. The sender re-enters the normal TCP mode on route restoration, identified using periodic probe messages. Unlike WCP, these proposals do not explicitly rec- ognize and account for congestion within a neighborhood. As a result, they would exhibit the same shortcomings of TCP as discussed in Chapter 1. Another class of work related to WCP includes schemes like COPAS [15], LRED [21], and ATP [63], which discuss TCP performance issues even in ad- hoc networks with no link-failure induced losses. COPAS [15] proposes a route selection scheme that attempts to find disjoint paths for different flows by assign- ing weights to links proportional to the average number of backoffs on the link. LRED [21] uses a exponential weighted moving average of the number of retrans- missions at the MAC layer as a measure of congestion while marking packets in a manner similar to RED [20]. ATP [63], like WCP, is a rate-based congestion con- trol scheme that involves explicit rate feedback to the sources from the network. In ATP, a flow receives the maximum of the weighted average of the sum of the queueing and transmission delay at any node traversed by the flow. ATP uses the inverse of this delay as the sending rate of a sender. Even though these schemes do not recognize the need of congestion detection and signaling over a neighborhood, their congestion metric implicitly takes some degree of neighborhood congestion into account. However, congestion in wireless networks exhibits strong location dependency [74]i.e., different nodes in a congested neighborhoodlocally perceive 9 different degrees of congestion. In the above schemes, flows traversing different nodes in a single congested neighborhood would receive varying levels of conges- tion notification. In contrast, WCP explicitly shares congestion within a neighbor- hood, ensuring that each flow in a single congested neighborhood gets its fair share of the bottleneck bandwidth. Two other pieces of work, however, have recognized the importance of explic- itly detecting and signaling congestion over a neighborhood. NeighborhoodRandomEarlyDetection(NRED) NRED [74] identifies a sub- set of flows which share channel capacity with flows passing through a congested node. But, it identifies only a subset of contending flows: it misses flows that tra- verse two hop neighbors of a node without traversing its one hop neighbors . More- over, the mechanism to regulate the traffic rates on these flows is quite a bit more complex than ours (it involves estimating a neighborhood queue size, and using RED [20]-style marking on packets in this queue). Finally, unlike WCP, NRED requires RTS/CTS, is intimately tied to a particular queue management technique (RED), might require special hardware for channel monitoring, and has not been tested in a real implementation. Explicit Wireless Congestion Control Protocol (EWCCP) EWCCP [65] cor- rectly identifies the set of flows which share channel capacity with flows passing through a congested node. EWCCP is designed to be proportionally-fair, and its design as well as its proof of correctness assumes that the achievable rate region 10 of 802.11 is convex. However, we show in Chapter 3 that it is not necessarily true. Moreover, EWCCP [65] has also not been tested in a real implementation. Recently, DiffQ [70], a practical realization of the optimal theoretical solution of differential backlog based backpressure [66] has been proposed. It proposed pri- oritizing node (MAC) transmissions in a neighborhood based on the difference in the queue size of the per-destination queue between a node and the next hop. In addition to the huge overhead of per destination queues, DiffQ, unlike WCP, re- quires modification to the underlying MAC layer. Horizon [55] is another scheme based on differential backlog based backpressure that does not require any changes to TCP or 802.11 protocol. However, unlike WCP, Horizon addresses challenges in load balancing in multi-hop networks withmulti-path routing. 2.2.2.2 CongestionControlwithExplicitRateFeedback An alternative to AIMD based schemes are schemes in which intermediate routers send explicit and precise feedback to the sources. XCP [35] and RCP [18] are exam- ples of such schemes for wired networks. Such schemes cannot be directly extended to multi-hop wireless networks, since the available capacity at a wireless link de- pends on the link rates at the neighboring edges, and ignoring this dependence will overestimate the available capacity and lead to performance degradation [50] and eventually to instability. Variants of XCP for wireless multi-hop networks, like WXCP [62] and XCP-b [1], use heuristics based on measuring indirect quantities 11 like queue sizes and the number of link layer retransmissions, to reduce the over- estimation in the available capacity. If, instead, one can directly estimate the exact capacity of a link as a function of the link rates at the neighboring edges, then an accurate XCP-like scheme can be implemented for wireless multi-hop networks. In 802.11-scheduled multi-hop networks, the complex interference among nodes makes it very hard to estimate the capacity of a link. Results have been known either for multi-hop networks that use perfect MAC schedulers [30, 40], or for single-hop 802.11-scheduled networks under saturation traffic conditions [10, 59]. We have recently developed an analytical methodology which characterizes the achievable rate region of 802.11-scheduled multi-hop networks [33, 32]. Our second scheme, WCPCap, uses this prior work of ours to find the supportable per-flow rate in a neighborhood. Further, it uses a novel, decentralized mechanism that relies on mes- sage exchanges within local neighborhoods only, to calculate the end-to-end flow rates. 2.2.3 CongestionControlinMulti-hopWirelessNetworks:Centralized Schemes Related to WCPCap is an interesting line of work that has explored theoretical methods for jointly optimizing scheduling and rate assignment in wireless net- works [19, 42, 61, 49]. Unlike this body of work, we restrict the scheduler to be 802.11. Moreover, we restrict our explorations to plausibly implementable rate- control mechanisms, whereas this line of research yields schemes that require a 12 centralized implementation to optimize both scheduling and rate assignment. While optimized rate assignment (congestion control) can be done in a distributed fashion by using back-pressure techniques [67], it still requires every node in the network to maintain separate queues for each possible network destination. More recent practi- cal studies of the problem have not been able to relax [2] this requirement. Recently, Li et al. [41] have explored theoretical methods to set up centralized optimization problems for 802.11-scheduled multi-hop networks to find rate allocations achiev- ing a given objective, like max-min fairness. However, they do not discuss how to achieve these allocations through distributed rate control schemes. 2.2.4 CongestionControlinWirelessSensorNetworks However, our problem setting is quite unlike any considered in these areas; to our knowledge, the problem of achieving a fair and efficient rate when many wireless nodes send data, possibly over multiple hops, to a base station has not been studied in either the Internet or ad-hoc networks literature before. Prior work in the sensor networks literature has looked at two qualitatively dif- ferent problems:congestioncontrol, andcongestionmitigation. Broadly speaking, Congestion control involves addressing the objectives of efficiency and bandwidth allocation policy [34]. The former ensuring that the network resource are utilized to its full extend while the latter attaining a desired distribution of these resources. Congestionmitigation, in contrast, entails only the objective of efficiency. 13 Fusion [27] is a congestion mitigation scheme that uses queue lengths to mea- sure levels of congestion, as we do, but also uses a different set of mechanisms than the congestion sharing mechanism we have considered; it applies hop-by-hop backpressure using a token-based regulation mechanism, and a prioritized medium access layer that allows congestion at local nodes to drain quickly. With this com- bination, Fusion achieves higher goodput and fairness at high offered loads than in the absence of these mechanisms. CODA [69] is another congestion mitigation strategy that uses slightly different mechanisms than Fusion. It senses both channel and buffer occupancies for measuring congestion levels; Fusion uses only buffer occupancy because, to a first order, a CSMA MAC layer with link-layer retrans- missions will cause congestion to be reflected in backed up buffers. Beyond that, CODA considers two strategies: open-loop back pressure for transient congestion, and an end-to-end acknowledgment based approach for persistent congestion. Un- like Fusion, CODA does not explicitly focus on per-source fairness. Because they are focused on congestion mitigation, neither Fusion nor CODA implements an AIMD control law that IFRC uses to converge to a fair and efficient rate. Also, instead of their explicit hop-by-hop back-pressure, in IFRC, nodes exchange con- gestion indicators, and distributedly converge to a fair and efficient rate. IFRC is closer in spirit to prior work on sensor network congestion control. In early work, Woo et al. [72] examine an AIMD rate adjustment strategy in which the additive increase is proportional to the number of descendants of a node, and multiplicative decrease is performed whenever a node detects (by promiscuously 14 listening) that its parent is unable to forward its traffic. Thus, a parent indicates con- gestion by dropping some traffic from its child, forcing the latter into multiplicative decrease. IFRC is different from this work in that it detects incipient congestion and applies aggressive rate cutting to avoid dropping packets. It also employs a more sophisticated congestion sharing strategy beyond just a signal from a parent to its children. ESRT [58] allocates transmission rates to sensors such that an application- defined number of sensor readings are received at a base station, while ensuring that the network is uncongested (and therefore efficient). Unlike IFRC, ESRT’s rate allocation is centrally computed at the base station: periodically, the base sta- tion counts the number of received sensor readings and re-tasks the sensors by broadcasting a new transmission rate. ESRT uses a sophisticated control law based on empirically derived regions of operation, and does not attempt to find a rate allocation that is maximally efficient. More recently, Ee and Bajcsy [14] have designed a congestion control scheme in which each node estimates the channel capacity by measuring the time to trans- mit a packet, and then divides up the available capacity among the nodes in its sub- tree. Each node enforces overall fairness by using a non-work-conserving weighted- fair queue for each of its children. Besides mechanistic differences (IFRC does not measure channel capacity by measuring the time to transmit a packet because that measurement can be skewed by local packet processing overhead), IFRC is funda- mentally different from this scheme since it achieves both fairness and efficiency. 15 By contrast, this scheme’s non-work-conserving scheduler does not promote effi- ciency; nodes whose flows do not pass through the congested regions cannot fully utilize the available bandwidth. 2.3 Summary Congestion control in multi-hop wireless networks is still an open research prob- lem. Though previous work have addressed various issues important to conges- tion control none with the exception of NRED [74] and EWCCP [65] explicitly addresses congestion neighborhood, an aspect we believe is integral to conges- tion control in wireless networks. Motivated by this observation we first introduce the design of IFRC, the first neighborhood-centric congestion control protocol for wireless sensor network characterized by its many-to-one traffic pattern. We, then, describe WCP and WCPCap, our congestion control protocol for multi-hop wire- less network with arbitrary traffic pattern. 16 Chapter3 CongestionControlinWirelessSensorNetworks Congestion control has been an active area of networking research for several decades now. Relatively less attention has been paid to congestion control in the emerging area of wireless sensor networks. In this chapter, we examine the fol- lowing simple but important problem: In a sensor network of N sensor nodes each transmitting data, possibly over multiple hops, to a base station, what distributed mechanisms must the sensor network implement in order to dynamically allocate fair andefficient transmission rates to each node? One motivation for this problem comes from the application of wireless sensor networks for structural health monitoring [39, 52, 46]. In this application, a set of sensors is deployed on a large civil structure (such as a building or a bridge). Each sensor continuously measures structural vibrations at several hundred samples a second, a rate that is comparable to the nominal data rate of today’s low-power sensor radios. When a significant response is detected, as when the structure is vi- brated using a mechanical shaker, every sensor transmits a time series of recorded 17 samples to a base station. Civil engineers find an application such as this to be ex- tremely useful for conducting short to medium term experiments on test structures; they analyze the structural response to derive models of the structure, or assess the level of damage in the structure. In the absence of rate control, this sensor network could suffer congestion collapse. A more general motivation comes from an evolved understanding of the struc- ture and application of sensor networks. Early sensor network designs assumed a flat network of sensors supporting low-rate periodic sensing. More recently, tiered sensor networks [64] have been proposed for use in high data-rate applications (e.g., acoustic [25, 8], imaging [56]). In tiered networks, the lower-tier consists of tiny wireless sensors that transmit data to the closest upper-tier node (usually an embedded 32-bit system with an 802.1x radio) [24]. In such networks, when an event is sensed, a relatively large number of nodes might wish to transmit signifi- cant volumes of data (either raw samples, or processed information) along one or more trees towards base stations. Rate control will prevent congestion collapse in these situations. In this chapter, we examine the design of a distributed and adaptive mechanism for fair and efficient rate control in wireless sensor networks. In general, the design of such a mechanism is complicated by multiple nodes concurrently accessing a time varying, shared wireless channel. As such, the channel arbitration used by the MAC layer and the quality of paths determined by the routing protocol can impact the quality of any solution to rate control. For simplicity, we build our work upon 18 the sensor network de facto standard MAC layer (CSMA) and routing layer (link- quality based path selection). We defer to future work the examination of an optimal “cross-layer” design—one which jointly designs the MAC layer, the routing layer, and a rate allocation scheme. Within our chosen framework, one important challenge is to design a mecha- nism by which each node can locally detect all flows that can contend for channel capacity, fairly adapt its own rate such that the capacity is not exceeded, and signal all relevant flows to do so as well. In general, such interference-aware rate allo- cation in wireless networks is difficult for arbitrary communication patterns [31]. As we show in this chapter, we can leverage the tree-based traffic pattern prevalent in wireless sensor networks to obtain a distributed, adaptive, and fair rate control mechanism. Specifically, we make the following contributions: Design: Our interference-aware fair rate control (IFRC) technique (Section 3.2) is a collection of inter-related mechanisms for distributed and adaptive fair and ef- ficient rate allocation for tree-based communication in wireless sensor networks. IFRC monitors average queue lengths to detect incipient congestion, and uses an AIMD control law to ensure convergence to fairness. Our main contribution is the design of a low-overhead, yet surprisingly efficient,congestionsharing mechanism. Congestion sharing itself is not a new idea; it has been used to achieve MAC-layer fairness [9], and to help TCP connections exchange network state [6]. However, designing such a mechanism in the context of sensor networks is not a trivial ex- tension of prior efforts: to achieve fairness, all flows (even those originating at a 19 distant node) that pass through a congested node need to be throttled, and rapidly signaling all relevant nodes is a non-trivial task. What is more, any flow that affects the rate at which the congested node sends data to the base station may need to be throttled, but such flows may not traverse the congested node at all. Precisely identifying the set of these flows is an important contribution of this work. Analysis: We analyze the steady state behavior of IFRC to rigorously derive IFRC parameters for stability and fast convergence. We validate our analysis with exper- iments which demonstrate that parameter choices within the bounds predicted by our analysis results in a stable system, while parameter choices outside this bound result in instability. Implementationandexperimentation: We have implemented IFRC in TinyOS (the widely-used sensor node OS), and demonstrate, using extensive experiments on a 40-node testbed (Section 3.4), that IFRC achieves fair rate control. IFRC’s perfor- mance is within 20–40% of the maximum sustainable fair rate using a fixed tree and a CSMA MAC layer. IFRC can be trivially extended to more general versions of our problem: where each sensor node sends to the nearest of many base stations, where only a subset of the nodes transmit data, or where an application might re- quire a weighted fair rate allocation. Finally, IFRC’s rate adaptation is highly ef- fective: we noticed no packet drops due to queue overflows even with relatively small buffer sizes inany of our experiments. 20 3.1 MotivationandDefinitions Consider a network of N sensor nodes, with each node uniquely identified by an integer in the range 1. . .N. In the simplest version of our problem, each node has infinite data to send to a single base station. This data can traverse multiple hops before reaching the base station. Thus, a sensor node originates traffic (is asource), and may forward traffic sent by other nodes. The traffic originated by source i and sent to the base station is the i-th flow, denoted by f i . We seek to adaptively assign a fair and efficient rate r i to f i (or equivalently, to node i). Specifically, r i is the rate at which node i sources flow f i , and does not include the rate at which node i forwards traffic. In Section 3.2.2, we explore other problem formulations: supporting multiple base stations; allowing only a subset of the nodes to send; and, assigning different priorities to flows. We assume that the sensor nodes run a contention-based medium access layer. The default MAC layer in TinyOS (the widely-used sensor node operating system) uses carrier-sense for collision avoidance, and our implementation is built upon this MAC. Our design can be extended to other contention-based MACs, such as those that use RTS/CTS [75]. We have not considered whether IFRC extends easily to token-based and TDMA MACs. We also assume that the sensor nodes run a routing protocol [73] that builds a tree to the base station. The solid lines in Figure 3.1 depict such a tree. The correctness of IFRC is not sensitive to the particular choice of routing protocol, but its performance is. In general, IFRC will result in higher overall throughput on 21 routing protocols that use link-quality metrics to establish the tree than on those that do not. In what follows, we assume that a sink tree has been constructed by the routing protocol, and, for ease of exposition, that the sink tree is stable. IFRC is designed to adapt to changes in the underlying routing tree. 10 11 13 14 16 15 17 19 18 12 21 20 Neighbor Child/Parent Figure 3.1: An example routing sub-tree. Dashed lines represent a neighbor that is neither a parent nor a child of a node. Finally, we assume that the MAC layer provides link-layer retransmissions. Our current IFRC design performs well in a regime where the wireless loss rates on the tree links are such that link-layer retransmissions recover from most packet losses. Outside this regime, IFRC needs an end-to-end feedback mechanism to determine the effective goodput that each node receives. We explore the impact of wireless loss rates on IFRC’s performance in Sections 3.2.3 and 3.4.2. 22 3.1.1 FairandEfficientRateAllocation What do we mean by a fair and efficient rate? Intuitively, we would like each flow to receive a fair share of the total channel capacity. However, in non-uniform de- ployments, spatial reuse effectively defines several wireless contention domains, in each of which the fair share may be different depending upon the number of flows traversing the domain. IFRC’s goal is to assign to each flowatleast the most congested fair share rate. In addition, it allows flows passing through less restric- tive contention domains to send at higher rates in order to achieve overall network efficiency. It is possible to make this intuition more precise using a constructive definition, described below. Central to this constructive definition, and a major contribution of this work, is the precise identification of the set of nodes whose flows share the channel capacity at a given node. We call all such nodes potential interferers of the given node. Before defining potential interferers, we introduce the following definition: Interferinglinks: A linkl 1 interferes with a linkl 2 if a transmission along l 1 pre- vents the initiation or the successful reception of a transmission alongl 2 . Then, we define a potential interferer as follows: Potentialinterferer: A node n 1 is a potential interferer of node n 2 if a flow orig- inating from node n 1 uses a link that interferes with the link between n 2 and its parent. 23 In tree-based wireless communication, the potential interferers are not merely the set of neighbors of a node, as we now show. Figure 3.1 shows a network of wireless nodes; solid arrows describe the routing tree, and dashed lines indicate neighboring nodes (those that can overhear each others’ transmissions). Consider node 16 in Figure 3.1, which transmits data to its parent 14. Clearly, links 20→ 16, 21→ 16 and 14→ 12 interfere with the link 16→ 14. Moreover, links 13→ 11, 17→ 14 and 12→ 10 interfere with 16→ 14 as well: when a packet is being transmitted on any of these links, 14cannotsafelylistentotransmissionsfrom 16. Thus, any flow traversing these links shares the channel capacity at 16. For Figure 3.1 this includes flows originating from 16, 20, 21, 14, 13, 17, 12 as well as flows originating from 15, 18 and 19 as these flows traverse 12→ 10. These nodes form the set of potential interferers of node 16. Interestingly, node 11 is not a potential interferer of node 16, since the flow originating at node 11 does not traverse a link that interferes with transmissions along 16→ 14 and therefore does not share the channel capacity at node 16. This example illustrates an important point: in tree-based communication, the potential interferers of a node include nodes not just in the node’s subtree or its neighbor’s (parent included) subtrees, but also includes nodes in its parent’s neighbor’ssubtrees. We now constructively define the fair and efficient rate allocation that IFRC strives to achieve. (In effect, IFRC achieves a max-min fair rate allocation, but we resort to this constructive definition since it cleanly motivates IFRC’s novel congestion-sharing mechanism, which we describe in Section 3.2.1.2.) To do this, 24 consider the sink tree built by the routing protocol (as in Figure 3.1), and define F i to be the set of flows routed through node i. This set includes f i (the flow from source i), and all the flows originating at the descendants in the subtree rooted at i. Also define a neighbor of node i to be any node which is within a single hop of i; this includes i’s parent, its children, and other tree nodes that can hear i’s transmissions (represented by dashed lines in Figure 3.1). Finally, for the purpose of this definition, assume that radios have a well-defined nominal data rateB. Then, IFRC attempts to achieve the following rate allocation: 1. At each node i, defineF i to be the union of F i and all sets F j , where j is either a neighbor of i, or a neighbor of i’s parent. Following our discussion above, F i includes flows from all of i’s potential interferers. Allocate to each flow inF i a fair and efficient share of the nominal bandwidth at i, B. (We do not define how to compute this share, since we are merely interested in defining the network-wide fairness and efficiency. Note that more efficient allocations may be possible than those that merely evenly divide B across all the flows by, for example, carefully scheduling non-interfering flows.) Denote by f l,i the rate allocated at nodei to flow l. Repeat this calculation for each node. 2. Assign to f l the minimum of f l,i over all nodesi. We now apply this definition to node 16. Clearly, f 16 belongs toF 16 . f 20 , f 21 , f 14 , being flows from neighbors of node 16, belong toF 16 . f 13 , f 17 , f 12 , being flows 25 from neighbors of 16’s parent, also belong to this set. Finally f 15 , f 18 , f 19 also be- long toF 16 as they are routed through a neighbor of 16’s parent 14. If, in this network, 16 is the most congested node, then all nodes with flows inF 16 should be constrained to send at a rate no greater than that assigned to f 16 . f 11 can send at a higher rate since it is not inF 16 . Intuitively, if node 16 is congested, the network should perform two actions: (i) reduce the arrival rate to this node, that is, reduce the rate of flows from nodes 16, 20 and 21, and (ii) increase the service rate of this node, that is, reduce the contention for the wireless channel between node 16 and 14. For the latter to happen, the capacity of 16 to transmit should increase (its neighbors 20, 21 and 14, and their descendants 17 should reduce the rate of their flows) and the capacity at 14 to receive packets should increase (its neighbors 17, 13 and 12 and their descendants 15, 18 and 19 should decrease the rate of their flows). Before describing IFRC’s rate control mechanisms, we consider two questions. Is fairness the appropriate design goal for sensor network data transport? Unless we get more deployment experience, this question is difficult to answer. Fairness is a reasonable initial design goal, and design insights from IFRC will be useful in designing other rate adaptation mechanisms for sensor networks. Furthermore, simple extensions to fairness, such as weighted fairness, can be easily realized with IFRC (Section 3.2.2). Secondly, should sensor network congestion control be rate- based or window-based? The former seems more natural for sensor networks, given 26 sensors often generate periodic traffic. That said, many of IFRC’s mechanisms can be adapted to window-based congestion control. 3.2 IFRCDesign In this section, we discuss the design of IFRC, including the detailed node-level algorithms for congestion detection, signaling, and rate adaptation in IFRC. Exten- sions to, and limitations of, IFRC appear at the end of the section. 3.2.1 IFRCMechanisms In IFRC, each node i adaptively converges to a fair and efficient rate r i for flow f i using a distributed rate adaptation technique. It achieves this by accurately shar- ing congestion information with potential interferers. IFRC consists of three inter- related components: one that measures the level of congestion at a node, another that shares congestion information with potential interferers, and a third that adapts the rate using an AIMD control law. We describe these mechanisms in this subsection. 3.2.1.1 MeasuringCongestionLevels Various techniques have been proposed in the wireless and sensor network litera- ture to measure the level of congestion experienced by a node. Broadly speaking, these techniques either directly measure the channel utilization around a node [69], 27 or measure the queue occupancy at the node [27], or a combination thereof. How- ever, recent work [27] reports that, for a traffic pattern in which multiple sources send to a single sink, queue occupancy is a sufficiently accurate indicator of con- gestion. Intuitively, with a MAC layer that uses carrier-sense, MAC backoffs and retransmissions effectively cause queues to build up at a node, so queue lengths are a reasonable indicator of congestion. At each node, IFRC maintains a single queue of packets from all flows passing through the node. This includes flows from all descendants of the node in the rout- ing tree and that sourced by the node itself. Following much of the prior work on congestion control, IFRC uses an exponentially weighted moving average of the instantaneous queue length as a measure of congestion: avg q =(1−w q )×avg q + w q ×inst q . The average queue length is updated whenever a packet is inserted into the queue. Conceptually, IFRC detects incipient congestion by using a simple threshold- ing technique with hysteresis. Thus, if avg q exceeds a certain upper threshold U, the node is said to be congested, and the node halves its current r i (the precise rate adaptation behavior is described in detail in Section 3.2.1.3), and then starts addi- tively increasing its r i . The node, however, remains in a congested state until avg q falls below a lower thresholdL. In practice, a single upper threshold is too coarse-grained to effectively react to congestion. When avg q crosses U, halving r i and signaling others to accordingly adjust their rates may still leave node i in a congested state (for example, when 28 the network size doubles instantaneously). Thus, IFRC employs multiple thresh- oldsU(k), defined byU(k) = U(k− 1)+I/2 k−1 , where k is a small integer and I is a constant increment of the queue length. When avg q is increasing the node halves its r i whenever avg q crosses U(k) for any k. Since the difference between U(k) andU(k− 1) decreases geometrically with increasing k, the rate halving be- comes more frequent as the queue size increases. In this manner, a node continues to aggressively cut its r i until its queue starts to drain. (In TCP, the same effect is achieved when the sender treats each dropped packet as a congestion signal, and halves its window in response). Section 3.4 shows how this scheme effectively sig- nals congestion while eliminating any packet drop due to queue overflow in our experiments. 3.2.1.2 CongestionSharing In Section 3.1, we introduced a constructive definition for IFRC’s rate adaptation goal. To achieve that goal, it is necessary to share i’s congestion state (its current average queue length) with other potential interferers. Each node can do this by explicitly transmitting its queue length to its potential interferers. Since some of its potential interferers can be many hops away from node i, the design of such a mechanism is a little tricky. IFRC uses a simpler congestion sharing mechanism, described below, that achieves the same goal. In IFRC, node i includes the following information in the header of each out- going transmission packet: its current r i ; its current average queue length, using 29 which other nodes can infer i’s congestion state; a bit indicating whether any child of i is congested; the smallest rate r l among all its congested children (the reason for this information will become clear in a moment); andl’s average queue length. Including this information in every packet enables robust signaling in the face of packet loss. Using this information, all recipients of each packet (we assume that nodes snoop transmissions, a common assumption in many wireless protocols) can de- termine whether i (or any of its children) is congested or not. All neighbors of i (its children, its parent, and other nodes that can heari’s transmissions) receive this packet. However, some potential interferers of i (the neighbors of i’s parent) may not receive this packet. How, then, does IFRC converge to the fair rate? IFRC introduces two simple constraints on the value ofr i at nodei: Rule1: r i can never exceedr j , the rate ofi’s parent j. Rule2: Whenever a congested neighbor j ofi crosses a congestion thresholdU(k) (for any k), i sets its rate to the lower of r i and r j . The same rule is applied for the most congested child l of any neighbor of i, i.e., i sets its rate to the lower ofr i andr l wherel is the most congested child ofi’s neighbor. Why do these two constraints achieve the goal described in Section 3.1? Con- sider a congested node i. By rule 1 above, all descendants of i will eventually be notified of i’s congestion and reduce their rates to that of r i . By rule 2, any other neighbor of i, including its parent, will set its own rate to r i . By the same rule, 30 all neighbors of i’s parent will set their rates to r i since when i’s parent sends any transmission it indicates that one of its children (namely i) is congested. Finally, recursively, descendants of a non-parent neighbor, and descendants of the parent’s neighbors will also reduce their rate tor i , again by rule 1. More specifically, let us assume node 16 in Figure 3.1 is congested. We require all nodes with flows inF 16 to have the same rate asr 16 . By rule 1, 16’s children 20 and 21 will set their rates to r 16 . By rule 2, 16’s neighbor 14 will set its rate equal to r 16 . 14 will include information about a congested child in its outgoing packets. Based on this information and using rule 2, nodes 13, 17 and 12 will set their rate to equal tor 16 . All the remaining nodes with flows inF 16 , namely node 15, 18 and 19 will eventually have their rates set to r 16 by rule 1 since one of their ancestors has set its rate tor 16 . Finally, IFRC’s state scales well, since, with this congestion sharing mecha- nism, a node need only maintain state proportional to the number of neighbors. 3.2.1.3 RateAdaptation Finally, in this section, we describe some of the details of IFRC’s rate adaptation mechanism. Before we do this, it is worth emphasizing that the averaged value of r i (which oscillates within a narrow range during steady state operation) represents the long-term fair and efficient rate at which node i can originate traffic. r i itself is not the maximum instantaneous rate at which a node can originate traffic; the 31 instantaneous rate may be affected by MAC-layer transmission scheduling mecha- nisms. In IFRC, nodes increase their r i ’s additively. Specifically, every 1/r i seconds, a node i increases its rate by δ/r i . We discuss the choice of the parameter δ and the choice of the 1/r i control interval in Section 3.3. If i is congested, then when thresholdU(k) is crossed, the node halves its currentr i . It does this at most once for eachU(k) during one congestion epoch; an epoch begins when the node transitions to a congested state, and ends when it transitions to a non-congested state. The latter happens when, in a congested state, the average queue length goes below the lower thresholdL. When i discovers its r i to be higher than that of its parent j, it sets its rate to r j . When i overhears a neighbor l’s transmission that indicates that l, or one of l’s children p has crossed a threshold U(k), its sets r i to either r l or r p as necessary. Both these sets of rules follow from the discussions in the previous two sections. In either case,i doesnot change its own congestion state. All nodes start from a fixed initial rate r init . For faster convergence, IFRC im- plements a multiplicative rate increase initially. This technique is similar to TCP’s slow start. While nodei is in slow start, it addsφ to its rate every 1/r i seconds, thus multiplicatively increasing its r i . It exits this mode when one of three conditions is true. If, in slow start, i itself becomes congested, it halves r i . If its r i equals or exceeds that of its parent, it setsr i to its parent’s rate and transitions to additive in- crease. Finally, if its rate is constrained by congestion sharing,i sets itsr i to that of 32 the constraining node and transitions to additive increase. These three conditions are consistent with the congestion sharing rate adaptation mechanisms discussed above. Slow start behavior is also invoked, for rapid recovery, when r i equals or goes below r init . This can happen in one of two ways. If i’s average queue length in- creases continuously, it will repeatedly halve its rate (Section 3.2.1.1). Alternately, i’s rate can be constrained by a neighbor tor init . 3.2.1.4 BaseStationBehavior The base station is a distinguished node in IFRC, since it never sources any traffic, and therefore does not need a separate allocated rate. It does, however, need to facilitate congestion sharing between its children. For this, it maintains a “rate” r b and enforces congestion sharing by adaptingr b to congestion indicators from its children. Its children are constrained by rule 1 in Section 3.2.1.2 to a rate no greater thanr b . The base station implements a slightly different algorithm to adapt its rate r b . It decreases its rate only when any of its children j crossesU(k) for any k. Unlike other nodes, it doesnot decrease its rate when any of its non-child neighbors or any child of a neighbor is congested. Since the base station does not source or forward any traffic, the rate of its children (determined byr b ) does not affect the congestion status of the congested neighbor (or the congested child of a neighbor) of the base station. 33 Initially, IFRC sets r b equal to the nominal data rate of the wireless channel. IFRC is not sensitive to the choice of value for this rate, since each node inde- pendently measures congestion by monitoring their own queue lengths. The initial value of r b merely needs to be high enough that the base station’s children can de- termine their fair rates without being constrained by the configured value. The base stationalways additively increasesr b : every 1/r b seconds, it incrementsr b byδ/r b . Since the base station does not source any traffic, it broadcasts a control message containing this rate as a way of sharing congestion information. To limit control message overhead, the base station broadcasts this message after at least m pack- ets have been received from the child with the highest rate. Intuitively, this lets the fastest child perform m additive increases before hearing from the base station. In our implementation, m= 5. However, when r b is decreased in response to conges- tion at a child, the base station immediately broadcasts the control message. This message is broadcast twice, to increase the likelihood that the children receive the “bad news” quickly (in the event that a child doesn’t receive both these transmis- sions, it will eventually get one of the periodically transmitted control messages). 3.2.2 ExtensionstoIFRC So far, we have assumed that every node in the network has data to send to a single base-station. Our IFRC design is flexible enough to accommodate multiple base stations, weighted fairness, or only a subset of nodes transmitting data. 34 MultipleBaseStations: Consider a network with multiple base stations. Each sensor node may be able to send to more than one base station, but picks the one with the best path and the system converges to one routing tree for each base station. In this scenario, a potential interferer for node i may be on a different routing tree than the one i is on. IFRC must account for this in adapting to a fair and efficient rate. To accommodate multiple base stations, IFRC requires a simple modification to the base station behavior (Section 3.2.1.4). Figure 3.2 motivates this modifica- tion and shows two routing trees rooted at node 0 and node 10. When node 11 is congested, node 10, following the base station behavior described above, sets r 10 to r 11 . r 1 should also to be set to r 10 as node 1 is a potential interferer of node 11. But since node 10 is not congested (it is the base station), Rule 2 in Section 3.2.1.2 is never triggered at node 1. 10 12 Neighbor Child/Parent 0 1 2 11 14 13 Figure 3.2: Scenario with multiple base stations (nodes 0 and 10) 35 Our fix to this modifies the control packet transmitted by the base station to include, in place of the current rate of the most congested child, the base station’s own rate. With this change, when node 11 is congested, r 10 resets its rate to r 11 and sends a control packet indicating that r 11 is congested. This triggers Rule 2 at node 1 if r 1 is greater than r 11 . This modification is backward compatible to the single base station case. All our experiments have incorporated this modification. In discussing other extensions to IFRC below, we revert to a single base-station scenario, and believe that these other extensions will extend without modification to multiple base stations. WeightedFairness: Consider a network where node i is assigned a weight w i . We say a rate allocation scheme is fair if the normalized rate for each flow, f i w i , is fair according to our definition in Section 3.1. We have implemented weighted fairness as follows, without needing to change any aspect of IFRC’s rate adapta- tion, congestion sharing, or queue management behavior: each node sends traffic at a long-term rate w i ×r i , instead of just r i . This is equivalent to adding (w i − 1) leaf nodes to the parent of i each sending at rate r i ; this intuition explains why no modifications are needed to IFRC. We experimentally demonstrate this strategy in Section 3.4.2. Whenonlyasubsetofnodestransmit: Finally, IFRC works (without modi- fication) when only a subset of nodes are sending data in the network. Each node i maintains r i exactly as it would if it had data to send. Intuitively, a subset of nodes sending data can be considered as a special case of weighted fairness where nodes 36 that have data to send have weight 1, and nodes that don’t have weight 0. We ex- perimentally demonstrate this extension in Section 3.4.2. 3.2.3 Discussion As we discussed briefly in Section 3.1, IFRC currently works well in regimes where hop-by-hop loss recovery (using a limited number of retransmissions) suf- fices to recover from most end-to-end losses. Hop-by-hop loss recovery is used in many real-world scenarios today to improve delivery reliability [52, 64]. Fur- thermore, many, but not all, of our experiments in real radio environments fall in this regime. (Our experiments were conducted in relatively harsh wireless envi- ronments with loss rates in the 15-40% range, with paths of more than five hops, and where five retransmissions or less were sufficient to recover from losses in, relatively speaking, most—92% or more—cases.) In this regime, the average r i is effectively the goodput that the node receives. Outside this regime, however, since IFRC is essentially an open-loop system and does not get feedback about the ac- tual goodput each node receives, there is no way to ensure globally fair allocation of goodput. For this purpose, as well as for reliability reasons, IFRC needs to be complemented with an end-to-end reliability mechanism. Designing a scalable end-to-end reliability mechanism is a challenge, and we have left that to future work. When integrated with such a mechanism, we ex- pect the rate adaptation parameters and some of the details of those algorithms to change, but the congestion sharing algorithms to stay the same. 37 IFRC cannot detect the reduction of channel capacity caused by interfering transmissions from non-neighboring nodes. This kind of interference is harder to detect at the higher layers of software, and manifests itself as packet loss. The interference rejection capabilities of spread spectrum radios mitigates this problem somewhat, and hop-by-hop recovery deals with its effects. Node battery conservation is an important issue in the design of sensor network systems. Many existing sensor network systems duty-cycle network nodes [64] to conserve energy. As long as the underlying routing protocol maintains or re- constructs routing paths when nodes resume from sleep, IFRC will work unmod- ified. Although duty-cycling achieves significant energy conservation, other re- search has proposed finer-grain energy conservation methods [75], for example, by turning off radios to avoid overhearing transmissions. IFRC assumes promiscuous listening for congestion sharing, and so will not work with such methods. IFRC will work without modification when intermediate nodes perform in- network aggregation [29]. In general, aggregation reduces the volume of data trans- mitted by nodes; in IFRC, nodes will adaptively detect the availability of, and use, the available capacity. 3.3 ParameterSelectioninIFRC IFRC converges to a fair rate using AIMD, but its long-term stability, efficiency, and convergence properties crucially depend upon the choice of its parameters. The 38 choice of many IFRC parameters is influenced by IFRC’s open-loop design. In re- liable transport protocols like TCP, acknowledgments used for error recovery are received after one round trip time. This RTT is a natural timescale at which rate can be adapted. Since IFRC is not coupled with an end-to-end feedback mecha- nism (Section 3.1), it departs from traditional congestion control protocols in its parameter choices. Intensity of AIMD: In IFRC, node i increases its rate r i every 1/r i seconds byδ/r i . Other additive functions are possible. We choose this because, intuitively, 1/r i is the inter-packet transmission time, and thus it provides a natural timescale for making a control decision. Hence, we have: r i t+ 1 r i (t) =r i (t)+ δ r i (t) . It is easy to see that the function r i (t) is a linear function with slope δ, that is, r i (t)=δt. The choice ofδ dictates the intensity of the additive increase, and is thus crucial for both stability and speed of convergence. We now present a simple analysis of the steady state behavior of the system, which we use to deduce the appropriate value ofδ. Recall thatr i (t) is the rate at which nodei generates traffic at timet. Letr st,i be the maximum sustainable rate of node i, r min,i be the minimum rate of node i, and r max,i be its maximum rate. Note that all these rates do not include the rate by which 39 node i forwards traffic generated by other nodes. Figure 3.3 shows r i (t) during the additive increase phase. The rate increases from r min,i , crosses the maximum sustainable rate r st,i , and, once a congestion signal is received, it decreases from r max,i tor min,i by half. Time (sec) r max Excess Load Rate (pkts/sec) r st slope = δ r min / δ r min = r max /2 Multiplicative Decrease Additive Increase Underutilized Capacity Figure 3.3: AIMD behavior ofr i (t) For stability, we require that the amount of data transmitted when r i is above r st (the “excess load” in Figure 3.3) be no more than the unexploited transmission opportunity when r i is below r st (the “underutilized capacity” in Figure 3.3), i.e., r st,i > r min,i +r max,i 2 . Also, since, our multiplicative decrease factor equals 1/2, r max = 2×r min and we have: r st,i > 3r max,i 4 . (3.1) To avoid r i jumping from r min,i to r max,i in a single step, we requireδ/r min,i ≪ r min,i , or, δ =εr 2 min,i , 40 where 0<ε < 1 is a small positive number whose value we derive below. When the average rate of nodes in the network is below their sustainable rate, r st,i , the load on the network is less than the capacity and thus the network is able to support the load without any queue build-up at the nodes. When the rate of nodes increases beyond their sustainable rate, the excess load will cause queues to build up at the congested nodes. The excess number of packets that a node sends during this time is equal to the area of the shaded region in Figure 3.3, which equals (r max,i −r st,i ) 2 /(2δ) for each nodei. Let’s focus on one congested node, call it node j. LetI ij be an indicator function that equals one if the packets from nodei traverse node j, and equals zero otherwise. Then, the total number of excess packets that are accumulated at node j equals ∑ i (r max,i −r st,i ) 2 I ij 2δ . This assumes that the r i values increase and decrease in synchrony at all network nodes. This is an idealized assumption that would only be true if feedback were instantaneous. As Figure 3.8 shows, this assumption holds approximately in our experiments, because per-hop latencies are small relative to the timescale of addi- tive increase. Intuitively, the formula above measures the excess work on node j’s queue due to the increase in the packet arrival rate at this queue. Notice that, in the above calculation, there is an underlying assumption that the service rate of this queue 41 remains the same independently of the network congestion. But this assumption does not hold for wireless networks because of contention. In particular, contention reduces the service rate of the queue and creates further queue build up. To see this consider a flow from a potential interferer of j. This flow may use various links that interfere with the link between j and its parent. A transmission of an excess packet of this flow along one of these interfering links will prevent j from transmitting a packet it would otherwise transmit, which isasif one more packet is added at node j’s queue. Precisely analyzing such contention is a hard problem in general. Here, we model its effect by replacing the indicator functionI ij with a function f ij whose value reflects not only the arrival of an excess packet, but also the number of time slots that node j cannot transmit due to the transmission of an excess packet by one of j’s potential interferers. The congested node will signal congestion only when its average queue in- creases beyond the thresholdU(0). LetU 0 denote the instantaneous queue length required for the EWMA-averaged queue length to exceedU(0). Then, for conges- tion signaling, we require ∑ i (r max,i −r st,i ) 2 f ij 2δ >U 0 . (3.2) Further, IFRC decreases its rate every time a threshold is crossed. Therefore, to avoid raising multiple congestion signals, that is, to avoid multiple multiplicative 42 decreases, we also require that the excess load is less than the second threshold. Thus, ∑ i (r max,i −r st,i ) 2 f ij 2δ <U 1 , (3.3) whereU 1 is the instantaneous queue length required for the EWMA-averaged queue length to exceedU(1), starting from an empty queue. In the analysis so far we have ignored the latency associated with the time that it takes for the congestion signal to travel from node j to all its potential interferes. Leti be a potential interfere of j ands i be the number of rate updates performed at node i before it receives the congestion information from node j. Assume that the rate at nodei when congestion occurred at node j was at leastr st,i and approximate the increase inr i due to thes i updates bys i δ/r i . Then, we have r max,i >r st,i + s i δ r st,i . (3.4) Equations (3.1), (3.2), and (3.3) guarantee system stability, and, whenever there is congestion, that one and only one signal is going to be propagated to all relevant nodes. Together with Equation (3.4) that takes the propagation delay of the con- gestion signals into account, they dictate some operational constraints that must be satisfied. Without loss of generality, assume all node weights are equal, that is, in steady state r st,i , r max,i , and r min,i are the same for all i. By substituting r st in Equation (3.2) using its lower bound from Equation (3.1), and letting F j = ∑ i f ij we get 43 r 2 st F j 18δ >U 0 . Similarly, by substituting r max in Equation (3.3) using its lower bound from Equation (3.4) and lettings be the average value ofs i we get s 2 δF j 2r 2 st <U 1 . Now, recall thatδ =εr 2 min to get ε < F j (r st /r min ) 2 18U 0 , andε < 2U 1 (r st /r min ) 2 s j 2 F j . The value of r st /r min ranges between 1.5 and 2. (The lower bound comes from Equation (3.1) and the fact that r max = 2×r min , and the upper bound from the fact that r max = 2×r min >r st .) Any value in this range can be legitimately substituted into the above inequalities. However, a particular choice of value determines the ef- ficiency of the system. When the value is 1.5, the average rate assigned to each node will be r st , but when the value is 2, the system achieves only 75% efficiency. We choose the former, sacrificing convergence time for efficiency (notice that choosing 1.5 yields a lower value ofε, resulting in longer convergence times) which yields: ε < F j 8U 0 , (3.5) and ε < 9U 1 2s 2 F j . (3.6) Intuitively, when F j is small, the first inequality (Equation (3.5)) determinesε. This happens in small networks or in sparse networks with low contention. In larger networks, the second inequality (Equation (3.6)) determinesε. 44 In Section 3.4, we conduct experiments that validate our analysis. In particular, we show that choosing ε using the above inequalities results in stable operation, while choosingε that minimally violates them results in instability. In general, however, choosing an ε for an arbitrary network is tricky, since F j is a function of the topology and the tree. In our experiments, we have been using a rule-of-thumb value ofnlogn forF j , wheren is the number of nodes in the network. nlogn is a good approximation of F j for a reasonably balanced tree in a network where every node can hear every other node, and where the congested node is close to the root. In the worst case—in a chain topology in which all links interfere with each other—F j can ben 2 . ChoiceofOtherParameters: We now turn to a brief discussion of how to set other parameters. Our choices for these parameters are based on intuition, rather than rigorous analysis, but which we have validated through experimentation. We have left the analysis to future work. Clearly,r init should have a value that is smaller than the steady state rate achiev- able for a given network. We conservatively set r init to be one order of magnitude smaller than the steady state rate. Now, in a wireless network of n nodes, depend- ing upon the network topology and the routing tree, the maximum achievable rate, r st , can vary greatly. In a sparse network with a balanced tree, nodes can hear only their parent and children. In this case r st is O(1/n). When each node can hear all other nodes, and the tree is balanced,r st is of the order ofO(1/(nlogn)). If instead, the tree degenerates to a chain topology, r st is O(1/n 2 ). This is the worst case. We 45 expect most networks of interest to be closer to the second category, and thus con- servatively set r init to equal B/(10nlogn), where B is the nominal data rate of the radio. The parameter φ determines the rate of multiplicative increase during slow- start. We conservatively set it to r init /8 to ensure a small slow-start overshoot over the sustainable rate, without compromising the speed at which slow-start hunts for that rate. Finally, for a given choice ofU(0), the choice ofw q dictates the burst lengthsU 0 andU 1 that each node is able to accommodate. The relationship between these two parameters and the burst length has been explored in [20]. In our case, a reasonable rule of thumb forU 0 isN/2, andU 1 isN; intuitively, if the congested node is close to the root of the tree, we would like to be able to accommodate one additive increase step (which, in IFRC, is one packet) at either that node or its child (when a node is congested, queues start to build up at its child too), from each node in the network. Our parameter choices in Section 3.4 have been derived based on the discus- sions in this section. 3.4 Evaluation In this section, we present the main results from an extensive performance evalua- tion of our implementation of IFRC on a 40-node wireless sensor testbed. 46 3.4.1 ImplementationandMethodology We have implemented, in TinyOS 1.1, all the features of IFRC described in Sec- tion 3.2. Our implementation is about 1200 lines of code, and consists of two mod- ules, QueueManagement and RateControl. The former module maintains the packet queue, and implements packet forwarding functionality. It also measures the node’s congestion level as described in Section 3.2.1.1, and signals the RateControl mod- ule when the node’s congestion state changes. The RateControl module implements congestion sharing (Section 3.2.1.2) and rate adaptation (Section 3.2.1.3). In addi- tion, the module maintains a neighbor table to track neighbor congestion states and rates. The RateControl module also promiscuously snoops network packets for con- gestion sharing. In our hardware platform, putting the radio in promiscuous mode disables the chip-level acknowledgments usually provided by the radio hardware. Since hop-by-hop recovery is essential for IFRC, we added MAC-layer acknowl- edgments, but expect IFRC to work well with chip-level acknowledgments as well. We evaluated our IFRC implementation on a 40-node indoor wireless sen- sor network testbed. Each sensor node in our testbed is a Moteiv Tmote with an 8MHz Texas Instruments MSP430 microcontroller, 10KB RAM and a 2.4GHz IEEE 802.15.4 Chipcon Wireless Transceiver with a nominal bit rate of 250Kbps. These motes are deployed over 1125 square meters of a large office floor They have a USB back-channel that we use for logging experimental data. The motes collec- tively form a connected topology (Figure 3.4) with a diameter of eight hops. We only depict links that have a loss rate of 40% or lower. 47 1 2 3 4 5 6 7 8 9 10 11 12 13 14 17 16 15 19 20 18 29 30 21 24 32 23 22 26 25 27 31 33 35 28 34 36 37 38 39 41 40 Figure 3.4: Testbed connectivity graph We have tested IFRC on several routing trees constructed on various topologies with up to 40 nodes, and with different base station locations. Here, we present a subset of these results. Table 3.1 shows the parameters used in our experiments (unless otherwise specified); these parameter choices were driven by the discussion in Section 3.3. A routing protocol which computes routes using link reliability estimators can incur route changes under heavy traffic. During preliminary experiments, we have found IFRC to work well in dynamic topologies. However, in order to study the steady-state behavior of IFRC, we modified the existing TinyOS routing protocol (MultiHopLQI) to terminate the route computation after an initial, reasonable tree 48 Parameter Value Lower Threshold (L) 4 pkts Upper Threshold (U(0)) 8 pkts Queue Increment (I) 8 pkts Additive Increase Parameter (ε) 0.025 per pkt Slow-start initial rate (r init ) 0.02 pkt/sec Slow-start mult. incr. (φ) 0.0025 pkt/sec MAX RETRANS 5 Queue Size 64 pkts EWMA Weight (w q ) 0.02 Packet Size 32 bytes Table 3.1: IFRC parameters used in the experiments is found. For each experiment, then, we ran this modified routing protocol to fix an initial tree, after which we ran IFRC on it. This procedure results in the same rout- ing tree with substantially similar link qualities across different IFRC experiments. We ran each experiment for at least an hour. For each experiment, we logged every packet transmission and reception, and every change in rate, at each net- work node. In addition, we logged every packet received at the base station. This fairly detailed logging helps us visualize IFRC behavior in several ways. However, during and across runs, the quality of wireless links can and does change. This is beyond our control, of course, but we compensate by running long experiments at consistent times of day (usually late at night or in the early morning hours). We plot the per-flow goodput as the total number of packets received from a node at the base station divided by the duration of the experiment. We plot therate adaptation at each node, by plottingr i as a function of time. For some experiments, we visualize instantaneous queue size at nodes as a function of time. For some 49 experiments, we also present a per-flow packet reception plot, which shows the number of packets received at the sink as a function of time. 3.4.2 Results In this section, we present experimental results that validate our IFRC design, demonstrate that IFRC can be extended in the ways we have discussed in Sec- tion 3.2.2, and analyze the consequences of poor parameter choices for IFRC. IFRC on a 40-node network: We now discuss the performance of IFRC on a 40-node network. All 40 nodes are transmitting data, and there is one base station. All flows are treated equally. Figure 3.5 shows the routing tree generated during one experimental run. Packet reception ratios are depicted on each link and vary between 66% and 96%. This tree is 9 hops deep, which is slightly greater than the diameter of the underlying topology. This surprisingly large depth is because of the way the routing protocol can prefer shorter links with higher qualities over poorer but longer links. The depth of the tree, the variable fanout at various nodes, the high variability in packet loss rates, and the unbalanced layout make this a good routing topology to study IFRC performance. Figure 3.6 shows the per-flow goodput received at the sink. Each bar represents the average rate of transmitted packets from the corresponding node. The lighter section of each bar represents the average rate at which packets from that node were received at the base station. Packet losses account for the rest (the remaining, darkly 50 2 1 0.87 3 0.88 4 0.85 5 0.89 6 0.8 7 0.92 8 0.94 9 13 0.79 0.88 10 0.87 11 0.87 12 0.9 14 0.87 15 0.77 16 0.81 17 0.84 18 0.72 19 0.74 20 0.87 21 0.85 22 0.8 23 0.66 24 0.96 25 0.95 26 0.79 27 0.93 28 0.96 29 0.84 30 0.87 31 32 0.93 0.82 33 0.87 34 0.94 35 0.93 36 0.96 37 0.96 38 0.95 39 0.93 40 0.91 41 0.89 Figure 3.5: Routing tree and link qualities used in the experiments 51 shaded, section of the bar). We make several observations from this graph. First, it validates the basic design of IFRC, and shows that nodes receive approximately fair rates and fair goodput. In this topology, it turns out that nodes 13 and 8 are congested, and every other node is a potential interferer of one of these nodes. For this topology, hop-by-hop recovery resulted in fewer than 8% of the packets being lost end-to-end. Second, node 1 is the base station, and the bar corresponding to node 1 indicates the base station’s signaling control traffic rate. This represents pure overhead, and is less than 1 packet in 10 seconds. Figure 3.7 shows the packet 0 0.05 0.1 0.15 0.2 0.25 0 5 10 15 20 25 30 35 40 45 Throughput (pkts/sec) Node Id Figure 3.6: Per flow goodput in the 40-node experiment reception plot for all the nodes in the network. The slope of this curve around a particular point gives us an estimate of the instantaneous goodput received by the node at that point. Notice that the reception plot is relatively smooth, with minor variations attributable to AIMD. 52 0 100 200 300 400 500 600 700 800 12:30 12:40 12:50 13:00 13:10 13:20 13:30 13:40 Packet Count Time (Hour:Min) Figure 3.7: Packet reception in the 40-node experiment Figure 3.8 shows the rate adaptation at each node, including the base station. All nodes adapt their rates in near-synchrony, as the rate plots for various nodes overlap with each other (cf. the discussion in Section 3.3). The slow start phase and the classic AIMD saw-tooth behavior is clearly visible in the graph. A closer look (not shown) reveals that the rate adaptation at different nodes is slightly de- synchronized by network latencies, but this effect does not show up at the time scale of the graph. Finally, there are several small measurement artifacts in the graph, caused by loss of experimental data. For example, the horizontal line starting at about 12:45 shows that the rate plot for one of the nodes is slightly distorted. This results from an infrequent data logging queue overflow problem at the USB ports on one of our motes, due to software driver issues beyond our control. 53 0 500 1000 1500 2000 2500 3000 3500 4000 12:30 12:40 12:50 13:00 13:10 13:20 13:30 13:40 Rate X 10000 (pkts/sec) Time (Hour:Min) Figure 3.8: Rate adaptation in the 40-node experiment In subsequent experiments, unless otherwise specified, we use the same routing tree (Figure 3.5) as the one used for Figure 3.8. Figure 3.9 shows the instantaneous size of the queue at every node. While it is difficult to decipher the detailed behavior of each flow from the graph, we clearly see that the instantaneous size varies a fair bit, but never grows beyond 20. Each of our nodes has a buffer size of 64, and, so, in this experiment, no packets are lost due to queue overflow. In fact, in all the experiments that we have conducted, we have never seen a single instance of buffer overflow. This results from IFRC’s aggressive rate cutting at eachU(k) (Section 3.2.1.1). Optimalityof IFRC: How close is IFRC’s rate allocation to the optimal? There are several ways to answer that question, but the one we have chosen is to ask: What is the maximum fair rate sustainable on the routing tree of Figure 3.5? This can 54 0 5 10 15 20 25 12:30 12:40 12:50 13:00 13:10 13:20 13:30 13:40 Inst. Queue Length Time (Hour:Min) Figure 3.9: Instantaneous queue size in 40-node experiment help us understand how much inefficiency IFRC introduces. To understand this, we programmed each node to send at a fixed rateR, without IFRC, but kept buffer sizes the same as in our above experiment, and used link-layer retransmissions to recover from losses. Starting with a value of R corresponding to the fair rate in Figure 3.6, we then increased R and measured the goodput received by each node. Our conjecture was that beyond some rate R ′ , the network would result in unfair goodput, andR ′ would be the benchmark we could use to calibrate IFRC. Figure 3.10 shows two distinct plots in the same graph. The x-axis depicts the successive rates used in the experiment. The solid line is the ratio of the maximum goodput received by any node to the minimum goodput. The dotted line measures the largest queue seen at a node during an experiment. 55 0.6 0.8 1 1.2 1.4 1.6 2000 2500 3000 3500 4000 0 10 20 30 40 50 60 Max/Min Goodput Ratio Max Queue Length Per node configured rate X 10000 (pkts/sec) Max/Min Goodput Ratio Max Queue Length Figure 3.10: Optimality test for the 40-node experiment Consider the goodput ratio curve. It remains flat initially, and then starts in- creasing. If we define the point at which it starts increasing to be R ′ (about 0.27 packets/sec), then IFRC achieves about 80% of the maximum sustainable fair rate. Roughly speaking, that value ofR ′ also represents the knee of the maximum queue size curve. However, real-world experiments are often messy; in our graph, there are two higher rates that appear to be anomalously non-monotonic, but still unfair. If we conservatively assume thatR ′ is closer to 0.36 packets/sec (beyond which the largest queue always overflows, as can be seen from the second, queue size plot), then IFRC still achieves about 60% of the maximum sustainable fair rate. Either way, this is extremely encouraging, and validates the basic motivation and premise of our design. 56 As an aside, the nominal bandwidth of the Tmote radio is about 80-90 pack- ets/sec. However, IFRC’s fair rate for our topology amounts to a total goodput of about 8.8 packets/sec. Why this apparent discrepancy? First, our goodput num- bers do not account for loss in channel capacity due to lost transmissions and re- transmissions. Second, our link-layer acknowledgments reduce some of the net- work capacity. Finally, the TinyOS MAC layer implements a random, and not ex- ponential backoff. The default random backoff parameter in TinyOS is not suffi- cient for the densities and traffic levels we use, so we increased it by a factor of three. These last two changes effectively reduce the nominal bandwidth of the ra- dio to about 50 packets/sec. Now, in our experiments, we have measured the total traffic at each node (defined to be all packets received, overheard, and transmitted by that node, at the transport layer). We find that the busiest node has a total traffic of close to 32 packet/sec, close to the nominal bandwidth. Collision-related losses handled purely by the radio at the MAC layer partially contributed to the remaining difference. DynamicsandStartupBehavior: IFRC dynamically adapts to node addition and removal. We would have liked to validate this behavior by turning nodes on and off. However, as we have discussed above, IFRC can interact with link quality based route selection, and turning notes on and off would trigger changes to the routing tree. So, we conducted two simple experiments to demonstrate how IFRC adapts to the addition or removal of nodes. 57 In the first experiment, all the nodes were turned on at the beginning of the experiment, and the routing tree was fixed, but only a fourth of the nodes started sending data initially. The others started sending data about 20 minutes into the experiment. Figure 3.11 shows the rate adaptation of the nodes in this experiment. The reduced overall fair share rate when nodes are added to the network is clearly visible in this plot. The horizontal line in the left half of the picture represents the r i ’s of inactive nodes. The graph also shows another interesting feature: two succes- sive multiplicative decreases occur at around 9:09. Such a behavior is sometimes caused by a parent and a child reaching a congested state within a short time of each other. When the parent gets congested, the child reduces its rate. Again, when the child’s own congestion threshold is exceeded, the child halves its rate again. 0 5000 10000 15000 20000 25000 30000 09:00 09:10 09:20 09:30 09:40 09:50 Rate X 10000 (pkts/sec) Time (Hour:Min) Figure 3.11: Node addition: Rate adaptation 58 The second experiment was similar, but all nodes started sending data at the beginning, and three-fourths of the nodes stopped about 20 minutes into the exper- iment. Nodes that continue transmitting data obtain higher fair share rates, as shown in Figure 3.12. This figure also has an interesting feature: a slight de-synchronization at 10:30 of the additive increase rates of some nodes. This is caused by packet loss; if a node loses a couple of transmissions from its parent, its rate can then lag behind that of its parent for some time. 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000 10:00 10:10 10:20 10:30 10:40 10:50 11:00 Rate X 10000 (pkts/sec) Time (Hour:Min) Figure 3.12: Node deletion: Rate adaptation IFRC Extensions: In this section, we validate the performance of the IFRC ex- tensions discussed in Section 3.2.2. Figure 3.13 plots the per-flow goodput for an experiment in which some nodes were assigned different weights than others. Each node whose id is divisible by 4 was assigned a weight of 2 and all the other nodes were assigned a weight 1. The 59 figure clearly demonstrates that IFRC is effective in allocating rates conforming to the configured weights. All other nodes whose identifiers are divisible by 4 get twice the fair share rate dictated by the most congested node. (As with other good- put graphs, the lighter section of the bar measures goodput received by each node, while the bar for node 1 denotes the base station’s control traffic overhead.) 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0 5 10 15 20 25 30 35 40 45 Throughput (pkts/sec) Node Id Figure 3.13: Per flow goodput with weighted fairness Figure 3.14 shows the per-flow goodput for an experiment in which only nodes whose identifiers were divisible by 3 were allowed to transmit data. As expected, all transmitting nodes receive at least the fair rate. Furthermore, the fair share per-flow goodput is approximately a factor of four higher in this case than when all nodes transmit; IFRC adapts to the increased overall available capacity and allocates it fairly to the transmitting nodes. 60 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0 5 10 15 20 25 30 35 40 Throughput (pkts/sec) Node Id Figure 3.14: Per flow goodput with only a subset of senders Finally, we ran an experiment withtwo base stations (Figure 3.15). Nodes num- bered higher than 33 have node 41 as the sink, and the rest send to another sink (node 1). As Figure 3.15 shows, the two sets of nodes get different fair share rates. The most congested node in the larger cluster is 17; the smaller cluster is not con- strained by this node. This experiment further illustrates IFRC’s efficiency. Nodes 4, 5 and 6 receive a little less thantwice the fair share rate because, in this topology, these nodes are not the potential interferers of node 17. ValidatingDesignChoicesandParameterSettings: In Section 3.1, we discussed why link-layer retransmissions are essential for IFRC. In all of our experiments, we have found that the five retransmission limit we use is sufficient to ensure fairness. (We emphasize that our topology encounters fairly high loss rates, with some links experiencing upwards of 30% loss rates.) Figure 3.16 shows the effect of turning off 61 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 0 5 10 15 20 25 30 35 40 45 Throughput (pkts/sec) Node Id Figure 3.15: Per flow goodput with multiple sinks link-layer retransmissions. In this figure, notice that the rate allocation is fair, but the goodput that each node gets is not (as shown by the lighter section of each bar). This, as we have discussed in Section 3.2.3, is because nodes receive no feedback about the goodput they receive. Finally, we conducted two experiments to validate the analysis of Section 3.3. Table 3.2 summarizes our findings. Our first experiment involved running IFRC on two nodes. For this experiment, we chose a nominal value ofU 0 = 10 andU 1 = 20, which gives a value of w q < 0.32. We conservatively set w q to be 0.3. With these values, the first inequality determines ε, giving an ε < 0.0125. We then increased ε slowly, until the system became unstable—the queue didn’t drain completely after one multiplicative decrease and the subsequent additive increase, triggering another multiplicative decrease. This happened atε = 0.0167. For this experiment, 62 0 0.5 1 1.5 2 2.5 3 0 5 10 15 20 25 30 35 40 45 Throughput (pkts/sec) Node Id Figure 3.16: Per flow goodput with no link-layer retransmissions the ε value predicted by our analysis is conservative (in that the predicted value results in stability), but tight. Experiment Predictedε Smallest unstableε 2 nodes 0.0125 0.0167 30 nodes 0.147 0.2 Table 3.2: Validating the analysis of Section 3.3 For a larger, 30-node network, we conducted a similar experiment, but with a small difference. In a large network, as we have discussed before, it is difficult to estimate F j or s j , so a reasonable rule of thumb for the former is nlogn and for the latter is a small number (we chose 3, which is the about the average network radius). Following Section 3.3, we setU 0 = 15 andU 1 = 30, giving anε < 0.147. For this large network, we ran IFRC usingε values of 0.05, 0.1, and 0.2. Instability 63 occurred at 0.2. Thus, for this larger network, choosingε using our analysis and our rules of thumb resulted in a conservative, but stable choice. While these experiments are not conclusive, they are highly encouraging in that they both validate the analysis, and our rules of thumb for picking IFRC parameters for a given network. 3.5 Summary In this chapter, we have described the design and implementation of IFRC, the first practical interference-aware rate control mechanism for wireless sensor networks. IFRC incorporates a novel low-overhead congestion sharing mechanism that lever- ages the prevalent tree-based communication pattern in wireless sensor networks. IFRC is fair and efficient in realistic wireless environments with packet loss rate over 30%, and its queue management strategy completely prevents packet drops due to queue overflow (at least in the experiments we have conducted). Finally, IFRC can be extended easily to support situations where only a subset of the nodes transmit, where the network has multiple base stations, or where nodes are assigned different transmission weights. 64 Chapter4 CongestionControlinMulti-hopWirelessNetworks Static multi-hop wireless mesh networks, constructed using off-the-shelf omnidi- rectional 802.11 radios, promise flexible edge connectivity to the Internet, enabling low-cost community networking in densely populated urban settings [47]. They can also be rapidly deployed to provide a communications backbone where none exists, such as in a disaster recovery scenario. However, their widespread adoption has been limited by significant technical challenges. Finding high-quality routing paths was an early challenge addressed by the research community [16]. However, that alone is not sufficient to ensure good performance in mesh networks, where transport protocols like TCP can perform poorly because of complex interference among neighboring nodes. In particular, TCP does not explicitly account for the fact that congestion in a mesh network is a neighborhood phenomenon. Consider the topology of Figure 4.1, in which links connect nodes which can exchange packets with each other, perhaps with asym- metric reception rates. In this topology, it is easy to show in simulation and actual 65 experiments that the TCP connection in the middle is almost completely starved (gets extremely low throughput), since it reacts more aggressively to congestion than the two outer flows. As an aside, we note that research on TCP for last-hop wireless networks [7, 4] does not address this problem. 1 2 3 4 5 6 8 7 8 9 Figure 4.1: Stack topology To understand the properties of a desirable solution to this problem, consider Figure 4.2. The y-axis plots the rate achieved by the middle flow, and the x-axis for the outer two flows (by symmetry, these flows will achieve approximately the same rate for any scheme) of Figure 4.1. Now, with a perfect MAC scheduler that has the same overhead as 802.11, it is intuitively clear that the rates achievable lie on or below the straight line shown in the figure (since an optimal scheduler would either schedule the two outer flows simultaneously or the flow in the middle). With 802.11, there is some loss of throughput due to contention, and the corresponding achievable-rateregion bounds the rates achievable by the flows on this topology (in Section 4.2.1, we describe a methodology to compute the achievable-rate region). 66 0.4 0.5 0.6 0.7 0.8 0.9 1 Rate of Middle Flow (Mbps) Capacity Region with Optimal Scheduling Achievable Rate Region with 802.11 0 0.1 0.2 0.3 0.4 0 0.2 0.4 0.6 0.8 1 Rate of Middle Flow (Mbps) Rate of Outer Flows (Mbps) WCPCap Optimal WCP TCP Figure 4.2: The achievable rate region for Stack topology TCP achieves rates that lie at one corner of this plot. We contend that, for this topology, a desirable solution is one that gets us close to the max-min fair rate al- location point, which corresponds to the intersection of the 45 ◦ line and the 802.11 achievable-rate curve. In this chapter, we explore mechanisms for achieving such a solution in wireless mesh networks. Three considerations inform our choice of mechanisms. First, we do not make any changes to the widely-used 802.11 MAC. It may well be that such changes can improve the performance of our mechanisms, but we have deliber- ately limited the scope of our work to enable a clearer understanding of congestion control. Second, our approach is clean-slate. We conduct our explorations in the context of arate-based protocol that incorporates some of TCP’s essential features 67 (such as ECN, and SACK), yet allows us to explore more natural implementa- tions of the mechanisms for improving fairness and efficiency that we study in this chapter. However, our work makes no value judgement on whether a clean-slate transport protocol is necessary for mesh networks; it may be possible to retrofit our mechanisms into TCP. Finally, we restrict our explorations to plausibly im- plementable mechanisms, in contrast to other work that has explored theoretical methods for optimizing (separately or jointly) scheduling and rate assignment in wireless networks [19, 42, 61, 49]. Contributions. We make two contributions in this chapter. First, we design an AIMD-based rate-control protocol called WCP which explicitly reacts to conges- tion within a wireless neighborhood (Section 4.1.1). Specifically, we correctly iden- tify the precise set of nodes within the vicinity of a congested node that needs to reduce its rates. Signaling these nodes can be implemented using a lightweight congestion sharing mechanism. More interestingly, we find that congestion shar- ing alone is not enough, and that, to achieve fairness, sources also need to clock their rate adaptations at the time-scale of the highest RTTs of flows going through the congested region. This can be implemented using a local mechanism for RTT sharing. Figure 4.2 shows that, for the topology of Figure 4.1, WCP avoids starving the middle flow (we discuss methodology and more detailed experimental results in Sections 4.2 and 4.3). Our second contribution is the design of a distributed rate controller that esti- mates the available capacity within each neighborhood, and apportions this capacity 68 to contending flows. This scheme, which we call WCPCap (Section 4.1.2), has the property that it uses local information and can plausibly be implemented in a dis- tributed fashion. Techniques that perform congestion control by estimating capacity in wired networks have been proposed before, e.g., [35], but wireless capacity es- timation is significantly harder. WCPCap is the first attempt in that direction that does not rely on heuristics, but instead uses a precise analytical methodology to accurately estimate the available capacity. Using analysis, simulations, and real deployments, we find that our designs yield rates that are both fair and efficient. WCP is fairer than TCP, while an ide- alized version of WCPCap is max-min fair, and a practical implementation of the scheme allocates to each flow a rate within 15% of the rate allocated to it by the max-min fair rate allocation. WCP achieves consistently good performance in the topologies we study while being extremely easy to implement. In fact, our exper- iments using five flows in a 14-node testbed show that, while TCP starves one or two of these flows in each run, WCP assigns fair rates to all the flows. Finally, in addition to good throughput performance, WCPCap exhibits low end-to-end delay and fast convergence. 4.1 Design In this section, we first discuss the design and implementation of WCP, an AIMD- based rate-control protocol that incorporates many of the features of TCP, but dif- fers significantly in its congestion control algorithms. We then describe WCPCap 69 which incorporates wireless capacity estimation in order to assign fair and efficient rates to flows. 4.1.1 WCP WCP is a rate-based congestion control protocol for static multi-hop wireless mesh networks which use the 802.11 MAC. In WCP, for every flow, the source main- tains a rate r which represents the long term sending rate for the flow. WCP is AIMD-based, so that the source additively increasesr on every ACK reception and multiplicatively decreases r upon receiving a congestion notification from routers (intermediate forwarding nodes). Routers signal congestion by setting a conges- tion bit in the packet header of ongoing packets. Unlike existing congestion control techniques, WCP has novel algorithms for detecting and signaling congestion at the intermediate routers, as well as for adapting rates at sources in response to con- gestion signals. 6 7 9 8 5 2 4 1 3 10 Figure 4.3: Congestion neighborhood 70 Congestion in Multi-hop Wireless Networks. The central observation underly- ing the design of WCP is that the nature of congestion in a wireless network is qualitatively different from that in a wired network. In a wireless network, since neighboring nodes share the wireless channel, the available transmission capacity at a node can depend on traffic between its neighbors. More precisely, congestion in wireless networks is defined not with respect to a node, but with respect to transmissions from a node to its neighbor. In what follows, we use the term link to denote a one-hop sender-receiver pair. (We use the terms sender and receiver to denote one-hop transmissions, and source and destination to denote the endpoints of a flow). Thus, in Figure 4.3, we say that a transmission from 5 to 6 is along the link 5→ 6. Consider the following example. When 5 is transmitting to node 6 it shares the wireless channel with any transmission from node 7, say a transmission from node 7 to node 9, as that transmission can collide with a transmission from node 5 to node 6. However, when node 5 is transmitting to node 2 it does not share capacity with, for example, a transmission from node 7 to node 9. Thus, congestion in wireless networks is defined not with respect to a nodei, but with respect to a linki→ j. What, then, are the set of links (L i→j ) that share capacity with a given link (i→ j)? Consider link 5→ 6 in Figure 4.3. Clearly, all outgoing links from node 5 and node 6 share capacity with link 5→ 6. Moreover, every outgoing link from a one- hop neighbor of node 5 shares capacity with link 5→ 6 because any transmission from a neighbor of 5, say node 2, can be sensed by node 5 and would prevent 71 node 5 from capturing the channel while node 2 is transmitting. Additionally, any incoming link to any neighbor of node 5, say 1→ 2, also shares capacity with link 5→ 6 as the link-layer acknowledgement from node 2 to node 1 would also prevent node 5 from capturing the channel for transmission. Similarly, any outgoing link from a neighbor of node 6 shares capacity with link 5→ 6 as any transmission along the outgoing link of neighbor of 6 can collide with transmission along 5→ 6. Finally, any incoming link into a neighbor of node 6, say 8→ 7, also shares capacity with 5→ 6 as the link-layer acknowledgement from node 7 to node 8 can collide with transmissions along 5→ 6. Thus,L i→j , for a mesh network using an 802.11 MAC is defined as: the set of all incoming and outgoing links ofi, j, all neighbors ofi, and all neighbors of j. Note that L i→j includes i→ j. Moreover, this relationship is symmetric. If a link i→ j belongs to L k→l , k→l also belongs to L i→j . Furthermore, this definition is valid even when RTS-CTS is used. However, there is an important limitation in our definition. If a node is outside another node’s transmission range, but within its interference range, WCP cannot account for the reduction in channel capacity as a result of the latter’s transmissions. CongestionDetectionandSharing. In WCP, one key idea is congestion sharing: if link i→ j is congested, it shares this information with all links in L i→j . Pack- ets traversing those links (as well as link i→ j itself) are marked with an explicit 72 congestion notification, so that sources can appropriately adapt the rates of the cor- responding flows. We now describe how routers detect congestion, and how they share their congestion state. Congestion detection in WCP is deliberately simple. A router detects conges- tion on its outgoing link using a simple thresholding scheme. It maintains an ex- ponentially weighted moving average (EWMA) of the queue size, q avg i→j , for every outgoinglink i→ j as q avg i→j =(1−w q )∗q avg i→j +w q ∗q inst i→j where q inst i→j is the instantaneous queue size for link i→ j and w q is the EWMA weight. A link is congested when its average queue size is greater than a con- gestion threshold K. Various other congestion detection techniques have been ex- plored in wireless networks: channel utilization [74], average number of retrans- missions [22], mean time to recover loss [53], among others. We choose queue size as a measure of congestion for two reasons: it has been shown to work sufficiently well in wireless networks [57, 28]; and is a more natural choice for detecting con- gestion per link compared to, say, channel utilization which measures the level of traffic around a node. Also, more sophisticated queue management schemes are possible (e.g., RED or A VQ), but they are beyond the scope of this chapter. When a router i detects that i→ j is congested, it needs to share this infor- mation with nodes at the transmitting ends of links in L i→j (henceforth referred to 73 as nodes in L i→j ). This is achieved by piggybacking congestion information on each outgoing packets which neighbors snoop. Specifically, each node maintains the congestion state of all its outgoing and incoming links. Nodes can locally de- termine (from queue size) the congestion state of their outgoing links. For nodes to obtain congestion state on their incoming links, every node during an outgoing packet transmission along a link includes congestion state ofthat link in the outgo- ing packet. In addition, every node also includes the following information in each outgoing packet: a bit indicating if any outgoing or incoming link from the node is congested and the identity of the link (sender and receiver of the link); and a bit indicating if any outgoing or incoming link from any of the node’s neighbors is congested and the identity of the link. This latter bit is calculated from information obtained by snooping the former bit from neighbors and requires no additional in- formation exchange. In the event of more than one link being congested at a node, it is sufficient for the node to select (and inform its neighbors) of only one of these links. Information shared in this manner is sufficient for any node in L i→j to deter- mine if linki→ j is congested. Finally, when a node in L i→j detects that link i→ j is congested, it marks all outgoing packets on that link with an explicit congestion indicator (a single bit). Rate Adaptation. In WCP sources perform rate adaptation. While the principles behind our AIMD rate adaptation algorithms are relatively standard, our contri- bution is to correctly determine the timescales at which these operations are per- formed. The novel aspect of our contribution is that these timescales are determined 74 by the RTTs of flows traversing a congested neighborhood; without our innovations (described below), flows do not get a fair share of the channel, and sometimes react too aggressively to congestion. A source S in WCP maintains a rate r f for every flow f originating at S. It linearly increases the rate r f every t ai seconds, where t ai is the control interval for additive increase: r f =r f +α whereα is a constant. The choice oft ai is an important design parameter in WCP. In the above equation the rate of change of r f , dr f /dt is α/t ai . Intuitively, for stable operation, dr f /dt should be dependent on feedback delay of the network. Using the weighted average round-trip time of flow f , rtt avg f , of the flow seems an obvious choice fort ai as it satisfies the above requirement. But consider three flows 1→ 3, 4→ 6, and 7→ 9 in Figure 4.1. Packets of flow 1→ 3 share the wireless channel with nodes 1 through 6 while packets from flow 4→ 6 share wireless channel with all the nodes in the figure. As the rate of all the flows in the network increases, flow 4→ 6 experiences more contention as compared to flow 1→ 3 and the average RTT of flow 4→ 6 increases much faster than the average RTT of flow 1→ 3. Thus, even if these flows were to begin with the same rate, their rates would diverge with the choice of t ai = rtt avg f . For fairness, all flows sharing a congested neighborhood should have similardr f /dt. 75 To enable fairness, WCP introduces the notion of a shared RTT. Denote by rtt avg i→j the average RTT of all the flows traversing the link i→ j (the average RTT of each flow is computed by the source, and included in the packet header)i.e., rtt avg i→j = ∑ ∀f∈F i→j rtt avg f |F i→j | where F i→j is the set of flows traversing link i→ j. For each link i→ j, node i computes the shared RTT, rtt Smax avg i→j , as the maximum RTT among all links in L i→j i.e., rtt Smax avg i→j = max ∀k→l∈L i→j (rtt avg k→l ) In words, this quantity measures the largest average RTT across the set of links that i→ j shares the channel with. Why this particular choice of timescale? Previous work has shown that the average RTT of flows is a reasonable control interval for making congestion control decisions [35]. Our definition conservatively chooses the largest control interval in the neighborhood. For all flows traversing linki→ j, the router includesrtt Smax avg i→j in every packet header only if it exceeds the current value of that field in the header. The source uses this value fort ai : thus,t ai is the largest shared RTT across all the links that the flow traverses. This value of control intervalt ai ensures that all flows going through the same congested region increase their rates at the same timescale. If a flow traverses multiple congested regions, its rate increase is clocked by the neighborhood with the largest shared RTT. 76 Upon receiving a packet with a congestion notification bit set, a source reduces the rater f asr f =r f /2 and waits for the control interval for multiplicative decrease t md before reacting again to any congestion notification from the routers. t md must be long enough so that the source has had time to observe the effect of its rate re- duction. Moreover, for fairness, flows that traverse the congested region must all react at roughly the same timescale. To ensure this, WCP also computes a quantity for each link that we term the shared instantaneous RTT, denoted by rtt Smax inst i→j . This is computed in exactly the same way as the shared RTT, described above, ex- cept that the instantaneous RTT is used, rather than the average RTT. The former is a more accurate indicator of the current level of congestion in the network and is a more conservative choice of the timescale required to observe the effect of a rate change. As before, routers insert this shared instantaneous RTT into the packet header only if it exceeds the current value. Sources set t md to be the value of this field in the packet header that triggered the multiplicative decrease. If a flow tra- verses multiple congested regions, its multiplicative decreases are clocked by the neighborhood with the largest shared instantaneous RTT. Similar to congestion sharing, computation of rtt Smax avg i→j (rtt Smax inst i→j ) requires sharing “link RTTs”, rtt avg i→j (rtt inst i→j ), among all nodes in L i→j requiring significant overhead. However, its definition permits a natural optimization. Similar to con- gestion sharing discussed above, every node includes in each outgoing packet only the maximum of rtt avg i→j (rtt inst i→j ) over all incoming and outgoing links of the node and the maximum of rtt avg i→j (rtt inst i→j ) over all incoming and outgoing links of all the 77 neighbors of the node. This allows for a low-overhead distributed calculation of shared RTT over all the nodes inL i→j . Finally, we describe how the source uses the valuer f . WCP aims to assign fair goodputs. Naively sending packets at the rater f assigns fair throughputs, but packet losses due to channel error or interference can result in unequal goodputs. Instead, WCP sends packets at a rate r f /p, when p is the empirically observed packet loss rate over the connection. Intuitively, this goodput correction heuristic sends more packets for flows traversing lossy paths, equalizing flow goodputs. Other rate-based protocols [51] use more sophisticated loss rate computation techniques to perform similar goodput correction. As we show in Section 4.2, our approach works ex- tremely well for WCP. In that section, we also quantify the impact of turning off this “correction”. Implementation. We could have retrofitted congestion and RTT sharing in TCP. But, the complexity of current TCP implementations, and the fact that TCP per- forms error recovery, congestion control and flow control using a single window- based mechanism, made this retrofit conceptually more complex. Given that our goal was to understand the issues underlying congestion in mesh networks, incre- mental deployability was not paramount. So, at the cost of some additional packet header overhead, we decided to explore a clean-slate approach. Our implementa- tion uses a rate-based protocol for congestion control (as described above), but uses an implementation of TCP SACK for error recovery, a window-based flow control mechanism exactly like TCP, and the same RTO estimation as in TCP. In a later 78 section, we show that our implementation does not bias the results in any way: if we remove our sharing innovations from WCP, its performance is comparable to TCP. 4.1.2 WCPCap An alternative to WCP is a protocol in which the network sends explicit and precise feedback to the sources. In order to do this, it is important to be able to estimate the available capacity within a congested region, a non-trivial task. In this section, we describe WCPCap, a protocol that provides explicit feedback (in much the same way that XCP [35] and RCP [18] do for wired networks). An important goal in designing WCPCap is to explore the feasibility of capacity estimation using only local information, thereby making it amenable to distributed implementation. Determining the Achievable Link-Rate Region. At the core of WCPCap is a technique to determine whether a given set of rates is achievable in an 802.11 network; using this technique, WCPCap estimates the available capacity and dis- tributes this fairly among relevant flows. This technique is presented in detail in a prior work [32]. For completeness, we describe the main idea of the analytical methodology here, assuming IEEE 802.11 scheduling with RTS/CTS in the net- work. The precise goal of the technique is as follows. Given a link i→ j, and a set of candidate aggregate rates r l→m over links l→m belonging to L i→j (link i→ j belongs to this set), we seek a decision procedure that will enable us to determine if 79 these rates are achievable. The decision process assumes that the channel loss rates (losses not due to collisions) of links in L i→j , and the interference graph between links in L i→j are known. Channel losses are assumed to be independent Bernoulli random variables. The interference model neglects some physical layer phenomena like the capture effect [13] (where a receiver can correctly decode data despite of interference), situations where transmitters send data despite of interference and carrier sensing, and situations where remote links in isolation do not interfere with the link under study, but their aggregate effect may cause loses on the link [17]. The interested reader is referred to [32] for a detailed description of all the assumptions, an extensive evaluation of their effect on the accuracy of the model, and a discussion on how to remove them. The decision process first determines, for each link, the expected service time in terms of (a) the collision probability at the receiver and (b) the idle time perceived by the transmitter of that link. Given these service times, the given set of link-rates is achievable only if ∑ e∈O v λ e E[S e ]≤U,∀v∈V whereV is the set of all nodes, O v is the set of outgoing links from a node v∈V , λ e is the packet arrival rate at link e, E[S e ] is the expected service time of a packet at link e, and the utilization factorU is a fraction between 0 and 1 and reflects the desired utilization of the channel. In practice,U is usually set to less than 1 to keep the system stable. Otherwise, small non-idealities can drive the network beyond the capacity region. 80 The key challenge then, is to determine the collision and the idle time prob- abilities, made difficult because these values for a link depend on the rates at its neighboring links, which, in turn, depend on the rates at their neighboring links and so on. We use the following procedure: the sub-graph formed by the set of links in L i→j is decomposed into a number of two-link topologies and the collision and idle probabilities for each two-link topology is derived. The net probability is found by appropriately combining the individual probabilities from each two-link topol- ogy. Combining these probabilities is quite complicated due to the interdependence among links. For brevity, we will omit the analytical formulas and their complete derivations. The interested reader is referred to [32] for details. Estimating Available Bandwidth. WCPCap uses the achievable rate computa- tion technique to estimate achievable bandwidth and give precise rate feedback to sources. Conceptually, each router maintains, for each outgoing linki→ j, a rate R i→j which denotes the maximum rate allowable for a flow passing through the link. However, a flow traversing i→ j is actually only allowed to transmit at the minimum (denoted R min i→j ) of all rates R k→l such that k→ l belongs to L i→j (intu- itively, at the most constraining rate over all links that share channel capacity with i→ j). The rate feedback is carried in the packet header. When a packet traverses i→ j, the router sets the feedback field to R min i→j if R min i→j is lower than the current value of the field. This feedback rate is eventually delivered to the source in an end- to-end acknowledgement packet, and the source uses this value to set its rate. Thus 81 the source sets its rate to the smallest allowable rate in the wireless neighborhoods that it traverses. R i→j for each link is updated everyk·rtt Smax avg i→j , wherertt Smax avg i→j is the shared RTT defined in Section 4.1.1 and k is a parameter which trades-off the response time to dynamics for lower overhead. The duration between two successive updates ofR i→j is referred to as an epoch. During each epoch, transmitteri measuresx i→j , the actual data rate over link i→ j and n i→j , the number of flows traversing link i→ j. Using x k→l and n k→l for all k→ l in L i→j transmitter i computes the new value of R i→j (denoted by R new i→j ) to be used in the next time epoch, and broadcasts x i→j , n i→j , and R new i→j to all nodes in L i→j . (If the measured x i→j in the previous epoch equals zero at a link i→ j, it does not broadcast its current value of R new i→j . Thus links which become inactive due to network dynamics will not contribute in determining the maximum allowable rate in a neighborhood.) We now describe howR new i→j is determined (Figure 4.4). Note that the transmitter i has x k→l and n k→l for all links k→ l in L i→j . It uses this information, and the methodology described above, to determine the maximum value ofδ such that the rate vector~ x shown in Figure 4.4 is achievable. (δ can have a negative value if the current rates in the neighborhood are not achievable.) Then, node i sets R new i→j to R i→j +ρδ−βq inst i→j /rtt Smax avg i→j if δ is positive, else R new i→j is set to R i→j +δ− βq inst i→j /rtt Smax avg i→j . We use a scaling factorρ while increasing the rate to avoid big jumps, analogous to similar scaling factors in XCP and RCP. On the other hand, we remain conservative while decreasing the rate. q inst i→j denotes the instantaneous 82 queue at link i→ j, rtt Smax avg i→j is the shared RTT defined in Section 4.1.1, and β is a scaling parameter. To ensure that the rate goes down when the queue builds up, we subtract a fraction of the bandwidth required to drain the queue within one shared RTT (q inst i→j /rtt Smax avg i→j ). Each node independently computes R k→l for its links. These computations do not need to be synchronized, and nodes use the most recent information from their neighbors for the computation. The computational overhead of the WCPCap algorithm is very low. To deter- mineR new i→j , we perform a binary search to find the maximum value ofδ such that the rate vector~ x is achievable. Each iteration decomposes L i→j into two-link topolo- gies, computes collision and idle probabilities for each two-link topology, and com- bines the results. Overall, the algorithm requires a logarithmic number of iterations whose complexity is polynomial in|L i→j |. In practical topologies the cardinality of L i→j is small. For example, in our experiments (run on 3.06GHz Linux boxes) determining R new i→j takes as much time as it takes to send a data packet. Since each epoch consists of about 30 data packet transmissions and a single R new i→j computa- tion, the computational overhead per epoch is very low. Finally, we note that, if naively designed, WCPCap can impose significant com- munication overhead. For each link i→ j, the following information needs to be transmitted to all nodes in L i→j once every epoch: the maximum RTT across the flows passing through the link, the actual data rate at the link, the number of flows passing through the link and R i→j . There are ways to optimize this, by quantiz- ing the information or reducing the frequency of updates, but we have left these to 83 Every k·rtt Smax avg i→j sec Find max δ such that ~ x← x k→l +n k→l δ for k→l∈L i→j is achievable R new i→j ← ( R i→j +ρδ−βq inst i→j /rtt Smax avg i→j δ > 0 R i→j +δ−βq inst i→j /rtt Smax avg i→j δ≤ 0 Broadcast R new i→j , x i→j and n i→j to all links in L i→j Figure 4.4: Pseudo-code for rate controller at linki→ j future work. Instead, in our simulations, we assume that all the relevant informa- tion is available at each node without cost, since our goal has been to understand whether available bandwidth estimation using only local information is plausibly implementable in wireless networks. Properties. To understand the design rationale of the WCPCap algorithm, we char- acterize the fairness properties of an idealized WCPCap algorithm. The idealized WCPCap algorithm assumes that all control message broadcasts are exchanged in- stantaneously and without loss, and each node has complete information about the entire network instead of just its neighborhood. Specifically, each node is aware of the data rate at each link in the network and the global network topology. The last assumption is needed because residual capacity at a link depends on the global topology and not merely the local neighborhood topology [32]. Hence WCPCap obtains an approximate value of the residual capacity while idealized WCPCap will obtain an exact value of the residual capacity. We prove fairness properties for ide- alized WCPCap here, and evaluate how the non-idealities impact the performance of WCPCap through simulations in Section 4.2. 84 Theorem4.1.1. IdealizedWCPCapallocatesmax-minfairratestotheflows. Proof. At the max-min allocation, let the links whose queues are nearly fully uti- lized be referred to as congested links, and neighborhood of congested links be referred to as congested regions. Each flow may pass through several congested re- gions, however, the most congested region a flow passes through (the region which gets congested at the lowest throughput) dictates the throughput of that flow. The next lemma states a property of the max-min rate allocation relating the throughputs of flows which share the most congested region they traverse. Lemma4.1.2. If two pass flows pass through the same most congested region, the max-minrateallocationwillallocateequalratestothesetwoflows. Proof. Let there ben flows passing through the congested regionC R . Additionally, letk of thesen flows haveC R as the most congested region they pass through. Con- sider the following rate allocation. Fix the rate of the other n−k flows as dictated by the most congested region they pass through, and then assign the maximum pos- sible equal rate to the k flows. Label the equal rate assigned to these k flows as r eq . Then, by definition of the most congested region a flow passes through, the other n−k flows have a rate smaller thanr eq . Let i→ j denote the congested link in the congested regionC R . (That is, link i→ j operates at full utilization.) Increasing the rate of any of thek flows will either increase the busy probability or the collision probability at link i→ j, making its queue unstable [32]. To keep the rate allocation feasible, the rates of one of the other 85 flows (which have either smaller or equal rates) will have to be reduced. Hence, by definition, allocating the maximum possible equal rate to the k flows sharing the same most congested region is the max-min rate allocation. Idealized WCPCap also allocates equal rates to flows which share the most congested region they traverse because the rate of these flows will be set to the maximum allowable rate in that region. 4.2 SimulationResults In this section we evaluate the performance of WCP and WCPCap in simulation, and in the next we report on results from real-world experiments of WCP. 4.2.1 Methodology We have implemented WCP and WCPCap using the Qualnet simulator [54] version 3.9.5. Our WCP implementation closely follows the description of the protocol in Section 4.1.1. Our WCPCap implementation, on the other hand, does not simulate the exchange of control messages at the end of each epoch; rather, this control infor- mation is made available to the relevant simulated nodes through a central repos- itory. This ignores the control message overhead in WCPCap, so our simulation results overestimate WCPCap performance. This is consistent with our goal, which has been to explore the feasibility of a wireless capacity estimation technique. 86 All our simulations are conducted using an unmodified 802.11b MAC (DCF). We use default parameters for 802.11b in Qualnet unless stated otherwise. Auto- rate adaption at the MAC layer is turned-off and the rate is fixed at 11Mbps. Most of our simulations are conducted with zero channel losses (we report on one set of simulations with non-zero channel losses), although packet losses due to collisions do occur. However, we adjusted the carrier sensing threshold to reduce interference range to equal transmission range in order to generate interesting topologies to study. On this set of topologies (described below), we run bulk transfer flows for 200 seconds for WCP, WCPCap, and TCP. Our TCP uses SACK with ECN, but with Nagle’s algorithm and the delayed ACK mechanism turned off; WCP implements this feature set. (We have also evaluated TCP-Reno on our topologies. The results are qualitatively similar.) Congestion detection for WCP and WCPCap uses the av- erage queue size thresholding technique discussed in Section 4.1.1. Other parame- ters used during the runs are given in Table 4.1. We discuss the choice of parameter U later, but our choice ofα is conservative, ensuring small rate increases over the range of timescales we see in our topologies. This choice of α also works in our real-world experiments, but more experimentation is necessary to determine a ro- bust choice ofα. For each topology we show results averaged over 10 runs. We measure the goodput achieved by each flow in a given topology by TCP, WCP, and WCPCap, and compare these goodputs with the optimal max-min rate allocations for each topology. To compute these allocations, we observe that the 87 Parameter Value Congestion Threshold(K) 4 EWMA Weight (w q ) 0.02 Router Buffer size 64 packets Packet Size 512 bytes Additive Increase Factor (α) 0.1 Utilization Factor (U) 0.7 WCPCap epoch duration constant (k) 10 WCPCap scale factors (ρ andβ) 0.1 Table 4.1: Parameters used in simulations methodology in Section 4.1.2 can be applied to a complete topology to character- ize the achievable rate region for a collection of flows. Intuitively, we can view the service times and arrival rates on links, together with flow conservation constraints, as implicitly defining the achievable rate region for the topology [32]. (Essentially, this is how we derive the achievable rate region in Figure 4.2). The max-min allo- cations can then be found by searching along the boundary of the achievable rate region. Using this methodology, we are also able to identify the links in a given topology that tend to be congested: we simply simulate the optimal max-min rate allocations, and identify congested links as those whose queues are nearly fully utilized. Note that we use this information in our intuitive discussion about the dynamics of each topology; this information is obviously not used in any way by neither WCP nor WCPCap. To understand the performance of WCP and WCPCap, we examine four topolo- gies, with associated flows, as shown in Figures 4.1, 4.5, 4.6, and 4.7. Nodes con- nected by a solid line can hear each others’ transmissions. (Since, in our simula- tions, we equalize interference range and transmission range, only nodes that can 88 hear each others’ transmissions share channel capacity with each other.) Arrows represent flows in the network. Congested links (determined using the methodol- ogy described above) are indicated with a symbol depicting a queue. Each of these four topologies has qualitatively different congestion characteristics, as we discuss below. 1 2 3 4 5 6 8 7 8 9 Figure 4.5: Diamond topology 1 2 3 4 5 6 8 7 8 9 Figure 4.6: Half-Diamond topology 89 9 8 4 1 2 3 10 5 6 7 11 10 Figure 4.7: Chain-Cross topology Stack (Figure 4.1) consists of a single congested region. 4→ 5 is the congested link, and all other links in the topology belong toL 4→5 .Diamond (Figure 4.5) con- tains two intersecting congested regions. 1→ 2 and 7→ 8 are both congested links. L 1→2 includes all outgoing links from nodes 1 to 6 and L 7→8 includes all outgoing link from node 4 to 9.Half-Diamond (Figure 4.6) contains two overlapping con- gested regions. 4→ 5 and 7→ 8 are congested, and L 7→8 is a subset of L 4→5 . Chain-Cross (Figure 4.7) contains two congested regions, with four flows travers- ing one region, and two flows the other. 1→ 2 is a congested link, but 6→ 7 does not belong to L 1→2 . 4→ 5 and 4→ 3 are also congested, and L 4→5 does include 6→ 7. Finally, since WCP uses a rate-based implementation, it is important to ensure that its baseline performance is comparable to that of TCP. To validate this, we ran TCP and WCP on a chain of 15 nodes. WCP gets 20% less throughput on 90 this topology; it is less aggressive than TCP. We also disabled RTT and congestion sharing in WCP, and ran this on all our topologies. In general, this stripped-down version of WCP gets qualitatively the same performance as TCP. For example, Figure 4.8 shows the goodputs achieved by each flow for the Stack topology. As expected, WCP without congestion and RTT sharing starves the middle flow, just as TCP does, although to a lesser extent since its rate increases are less aggressive than that of TCP. 400 500 600 700 800 Goodput (kbits/sec) 1→3 4→6 7→9 0 100 200 300 TCP WCP-strip Goodput (kbits/sec) Figure 4.8: WCP without congestion and RTT sharing, Stack 4.2.2 PerformanceofWCPandWCPCap We now discuss the performance of WCP and WCPCap for each of our topologies. In what follows, we use the notation f i→j to denote a flow from nodei to node j. 91 Stack (Figure 4.9). The optimal (max-min) achievable rates for this topology are 300 kbps for all the flows. TCP, as described earlier, starves the middle flows (f 4→6 ). Intuitively, in TCP, flows traversing links that experience more congestion (4→ 5) react more aggressively to congestion, leading to lower throughput. WCP identifies the single congestion region in this topology (L 4→5 ) and shares the rate equally among all the flows assigning about 250 kbps to all the flows. WCPCap, with a more precise rate feedback, assigns slightly higher rates to all the flows and these allocated rates for each flow is within 15% of the rate allocated to it by the max-min fair rate allocation. Intuitively, one would expect the performance differ- ence between WCPCap and WCP to be higher than what it is, since the latter’s AIMD mechanism is more conservative than one which can directly estimate ca- pacity. However, as we discuss below, we have decided to be conservative and set the utilization factor U to 0.7, which is a value that guarantees stability in every topology we have studied. Higher values ofU are possible depending on the topol- ogy and yield better performance for WCPCap. Finally, from the plot it is evident that for the parameter values discussed above, WCP is within 20% of the optimal achievable rate for this topology. Diamond (Figure 4.10). The optimal achievable rates for this topology are 325 kbps for all the flows. TCP starves flows traversing the congested links in this topol- ogy. By contrast, WCPCap, assigns 300 kbps to all the flows. Hence, it achieves rates within 10% of the max-min optimal rates. WCP, however, assigns f 4→6 ap- proximately half the rate assigned to the other flows. This topology consists of 92 400 500 600 700 800 Goodput (kbits/sec) 1→3 4→6 7→9 0 100 200 300 TCP WCP WCPCap Optimal Goodput (kbits/sec) Figure 4.9: WCP and WCPCap, Stack two congested regions (L 1→2 andL 7→8 ) and f 4→6 traverses both congested regions while the other two flows traverse only one. Roughly speaking, f 4→6 receives con- gestion notification twice as often as the other flows, and therefore reacts more ag- gressively. Thus, WCP is not max-min fair. WCP appears to assign rates to flows in inverse proportion to the number of congested regions traversed. Half-Diamond (Figure 4.11). The optimal max-min rates for this topology are 315 kbps for f 4→6 and f 7→9 , and 335 kbps for f 1→3 ; the asymmetry in this topology permits f 1→3 to achieve a slightly higher rate. Relative to other topologies, TCP performs fairly well for this topology. WCPCap achieves rates within 14% of the max-min optimal rates. WCP assigns comparable rates to f 4→6 and f 7→9 as they traverse both congested regions L 4→5 and L 7→8 . f 1→3 achieves a higher rate as it 93 400 500 600 700 800 Goodput (kbits/sec) 1→3 4→6 7→9 0 100 200 300 TCP WCP WCPCap Optimal Goodput (kbits/sec) Figure 4.10: WCP and WCPCap, Diamond traverses only one congested region (L 4→5 ) but its rate is not twice the rate of the other flow. Thus WCP achieves a form of fairness in which the rate allocations depend not only on the number of congested regions a flow passes through, but also the intensity of congestion in those regions. Chain-Cross (Figure 4.12). The optimal rates for this topology are 420 kbps for f 6→7 and 255 kbps for all other flows. TCP starves the flows traversing the most congested link 1→ 2. WCPCap achieves rates within 15% of the optimal max-min rates. WCP achieves rates that depend inversely on the number of congested regions traversed, with f 1→7 achieving lower goodput and f 1→2 , f 10→11 , f 8→9 achieving equal rates. WCP is able to utilize available network capacity efficiently; f 6→7 does not traverseL 1→2 and gets higher goodput. 94 300 400 500 600 Goodput (kbits/sec) 1→3 4→6 7→9 0 100 200 TCP WCP WCPCap Optimal Goodput (kbits/sec) Figure 4.11: WCP and WCPCap, Half-Diamond 800 1000 1200 1400 1600 1800 Goodput (kbits/sec) 1→2 1→7 6→7 8→9 10→11 0 200 400 600 800 TCP WCP WCPCap Optimal Goodput (kbits/sec) Figure 4.12: WCP and WCPCap, Chain-Cross 95 Fairness Achieved with WCP . WCP assigns rates inversely proportional to the number of congested regions a flow passes through and the intensity of congestion in these regions. More the number of congested regions a flow passes through and higher the intensity of congestion in these regions, the more will be the congestion notifications received at the source, and lower will be the throughput. Note that the intensity of congestion depends on the local topology of the region and the number of flows passing through the region. As an example, consider Figure 4.13 where we plot the evolution of flow rates with WCP in the Chain-Cross topology. When the queue at 1→ 2 gets congested, the rate of flows f 1→2 , f 1→7 , f 8→9 and f 10→11 is cut by half (for example, at 10s, the rate of all these flows is reduced); and when the queue at 4→ 5 gets congested, the rate of flows f 1→7 and f 6→7 is cut by half (for example, at 30s, the rate of both the flows is reduced). Flow 1→ 7 receives congestion notifications from both congested regions, hence it receives a lower throughput than others. Since there are more flows passing through L 1→2 , it has a higher congestion intensity. So, we see more notifications from L 1→2 , and hence a lower throughput for f 1→2 , f 8→9 and f 10→11 . 4.2.3 Discussion Impact of Physical Losses. Thus far, we have assumed perfect wireless links in our simulations (losses do occur in our simulations due to collisions, however). Figure 4.14 shows the performance of WCP and WCPCap for the Stack with a loss rate of 10% on each link. The results are qualitatively similar to Figure 4.9. As 96 0 50 100 150 200 250 0 50 100 150 200 Instantaneous Rate of Flows (pkts/sec) Time (sec) 1->2 1->7 6->7 8->9 10->11 Figure 4.13: Evolution of flow rates with WCP in Chain-Cross expected, the goodputs drop by about 10% for WCP and WCPCap, as do the opti- mal rates. We have conducted similar experiments for the other topologies, but omit their results for brevity. We also illustrate the efficacy of goodput correction in deal- ing with packet loss (Section 4.1.1). Figure 4.15 shows the goodputs achieved in the Stack topology with 10% loss on all links, when goodput correction is disabled. Goodputs are no longer fair. In-NetworkRateAdaptation. The WCP AIMD rate control algorithms described in Section 4.1.1 are implemented at the source. These algorithms can also be im- plemented per flow within the network. Although this approach has scaling impli- cations, it is still interesting to consider. It can reduce the feedback delay in control decisions, and a router can avoid reacting to congestion when the rate of a flow 97 300 400 500 600 700 Goodput (kbits/sec) 1→3 4→6 7→9 0 100 200 300 TCP WCP WCPCap Optimal Goodput (kbits/sec) Figure 4.14: WCP and WCPCap over lossy links, Stack 150 200 250 300 350 400 Goodput (kbits/sec) 1→3 4→6 7→9 0 50 100 150 WCP WCP w/o Goodput Correction Goodput (kbits/sec) Figure 4.15: WCP without goodput correction, Stack 98 traversing it is lower than the rates of flows traversing the congested link. Indeed, in our simulations, such a scheme performs uniformly better than WCP imple- mented at the source. For example, for Chain-Cross (Figure 4.16) f 1→7 gets the same rate as other flows in L 1→2 improving overall fairness while preserving the higher goodput allocated to f 6→7 . Similar performance can be observed for Stack (Figure 4.17) Diamond (Figure 4.18), and Half-Diamond (Figure 4.19) in addition to low variation in goodput across various runs. 300 400 500 Goodput (kbits/sec) 1→2 1→7 6→7 8→9 10→11 0 100 200 At-source In-network Goodput (kbits/sec) Figure 4.16: WCP with in-network rate adaptation, Chain-Cross ChoiceofU inWCPCap . Recall thatU denotes the maximum allowed utilization per queue in WCPCap. In simulations, we set its value to 0.7. We now justify this choice. The analysis described in Section 4.1.2 derives the achievable rate region without losses and hence assumes infinite buffer sizes and infinite MAC retransmit 99 300 400 500 Goodput (kbits/sec) 1→3 4→6 7→9 0 100 200 At-source In-network Goodput (kbits/sec) Figure 4.17: WCP with in-network rate adaptation, Stack 300 400 500 Goodput (kbits/sec) 1→3 4→6 7→9 0 100 200 At-source In-network Goodput (kbits/sec) Figure 4.18: WCP with in-network rate adaptation, Diamond 100 300 400 500 Goodput (kbits/sec) 1→3 4→6 7→9 0 100 200 At-source In-network Goodput (kbits/sec) Figure 4.19: WCP with in-network rate adaptation, Half-Diamond limits. Assuming no losses, operating very close to the capacity region will result in large delays. However, in practice both the buffer sizes and MAC retransmit limits are finite. Hence, these large delays can result in significant losses. For this reason, we operate the network well within the capacity region; the parameterU controls how far the network is from the boundary of the capacity region. To understand how to set its value, we plot the end-to-end delay of the middle flow, f 4→6 , in the Stack topology (which passes through the congested link 4→ 5) (Figure 4.20). We varyU from 0 to 1 and assume very large buffer sizes and MAC retransmit limits. Ideally one would setU near the knee of the curve, which is around 0.8. We choose a conservative estimate and set it to 0.7. 101 60 80 100 120 -to-End Delay (ms) 0 20 40 0 0.2 0.4 0.6 0.8 1 Average End- Utilization Factor (U) Figure 4.20: Delay as a function ofU for f 4→6 , Stack RTS/CTS and WCP. Our simulation results have used RTS/CTS so far. In Sec- tion 4.1.1, we asserted that our definition of congestion in a wireless network is insensitive to the use of RTS/CTS. Indeed, WCP without RTS/CTS (Figure 4.21) performs just as well as (and gets higher goodputs than) WCP with it (Figure 4.9). Other topologies show similar results, except for Half-Diamond (Figure 4.22). With- out RTS/CTS, 1→ 2 and 7→ 8 become the most congested links in Half-Diamond, changing the dynamics of congestion in this topology. Qualitatively, this topology starts to resemble the Diamond, with two overlapping congested regions. A lower goodput for f 7→9 than f 1→3 results from additional links 4→ 8 and 6→ 8 inL 7→8 which reduces the capacity in the region. RTS/CTS and WCPCap. Disabling RTS/CTS has deeper implications for WCP- Cap. Consider the chain-cross topology of Figure 4.7. Without RTS/CTS, increas- ing the rate of flow 6→ 7 will cause more DATA collisions on link 4→ 5 as 4 is 102 400 500 600 700 800 900 Goodput (kbits/sec) 1→3 4→6 7→9 0 100 200 300 400 TCP WCP WCPCap Optimal Goodput (kbits/sec) Figure 4.21: WCP and WCPCap with no RTS/CTS, Stack 300 400 500 600 Goodput (kbits/sec) 1→3 4→6 7→9 0 100 200 TCP WCP WCPCap Optimal Goodput (kbits/sec) Figure 4.22: WCP and WCPCap with no RTS/CTS, Half-Diamond 103 unaware of an ongoing transmission on link 6→ 7. As a result, more retransmis- sions occur on the link 4→ 5, and the total data rate (including retransmissions) on this link increases. Note that an increase in data rate on link 4→ 5 will cause more DATA collisions on link 2→ 3 as 2 is unaware of any ongoing transmission on link 4→ 5. This will reduce the capacity of link 2→ 3. Hence, the effective capacity of edge 2→ 3 decreases even though none of the flows passing through its neighborhood has changed its rate. Hence, finding the residual capacity at a link without rate information from non-neighboring links will overestimate the resid- ual capacity. (By contrast, when RTS/CTS is used, the much smaller RTS packets collide, resulting in a less pronounced overestimation.) We can avoid overestimat- ing the residual capacity by using a smaller value of U (Section 4.1.2). For ex- ample, reducing it to 0.6 from 0.7 avoids any overestimation of capacity for the chain-cross topology. We use this value of U for generating results for WCPCap without RTS/CTS (Figure 4.21 and 4.22). However, the choice ofU now depends to a greater extent on topology, and complete topology information is needed to compute it. PerformanceunderNetworkDynamics. In the simulations above, all flows start and end at the same time. Figures 4.23 and 4.24 show the instantaneous sending rate r f (Section 4.1) of all the flows in the Chain-Cross topology for WCP and WCPCap respectively, when flows do not start and stop at the same time. Specifically, f 1→7 starts at 25s and ends at 100s, while the rest of the flows start at 0s and end at 200s. 104 It is evident from the plot that both WCP and WCPCap are fair not only when all flows are active, but also before the arrival and after the departure of f 1→7 . 0 100 200 300 400 500 600 0 50 100 150 200 Instantaneous Rate of Flows (pkts/sec) Time (sec) 1->2 1->7 6->7 8->9 10->11 Figure 4.23: WCP with delayed flow arrival WCPPerformancewithLargerInterferenceRange. In all the simulations above, the sense signal threshold (PROPAGATION-LIMIT in Qualnet) is set equal to the receive signal threshold (RX-SENSITIVITY in Qualnet) to make the interference range equal to the transmission range for all the nodes. Figure 4.25 shows the per- formance of WCP for Chain-Cross where the interference range in greater than the transmission range and nodes 10 and 8 are moved farther away from node 2 such that they are within interference range of node 2 but not in its transmission range. While node 10 and 8 can sense any transmission from node 2 and vice versa they cannot decode any packet transmission from node 2. This illustrates a case where 105 0 100 200 300 400 500 600 0 50 100 150 200 Instantaneous Rate of Flows (pkts/sec) Time (sec) 1->2 1->7 6->7 8->9 10->11 Figure 4.24: WCPCap with delayed flow arrival nodes sharing a wireless channel (and thus the capacity) are unable to communi- cate control information (like congestion status) among themselves. Thus, when congestion occurs at node 2, node 10 and 8 are not able to receive any congestion information from node 2. In the absence of this information flows traversing node 8 and 10 do not receive any congestion notification leading to unfair throughput to flows in the network. However, the effect of a larger interference range is not equally detrimental to WCP’s performance for the Stack, the Diamond, and the Half-Diamond as shown in Figures 4.26, 4.27, and 4.28 respectively. With larger interference range more nodes can sense each other’s transmission causing Stack and Half-Diamond to exhibit characteristics (in terms of the most congested links) 106 and performance similar to Diamond but with reduced overall goodput (due to re- duction in capacity due to larger sensing range). Figure 4.25: WCP with larger interference range, Chain-Cross We believe that current advancements in radio technologies can help mitigate the effect of interference by increasing the co-channel interference tolerance (or capture effect), i.e., increasing the ability of a receiver to successfully decode the stronger of the signals from two transmitting node. With such advancements, re- ception at a given node would not be affected by transmissions from other nodes that are not within the transmission range of the given node. It would then elim- inate the need to consider such nodes i.e., nodes that are outside the transmission range of a node, as part of the neighborhood of a given node. (The above results are without any capture effects as the current version of Qualnet does not simulate 107 Figure 4.26: WCP with larger interference range, Stack Figure 4.27: WCP with larger interference range, Diamond 108 Figure 4.28: WCP with larger interference range, Half-Diamond it.) In addition, techniques like transmitting congestion information at the base rate (rate with a larger transmission range) could further help mitigate the problem. WCPPerformancewithAuto-RateAdaptation. Auto-rate adaptation affects two characteristics of a topology: Link rates of individual links, and the transmission range of each link. With nodes continuously adapting their link rates, the snapshot of the network at any given instance consists of links with different rate. Figure 4.30 shows WCP’s performance when the rate of each link is fixed for the complete simulation run to the rates shown in Figure 4.29. Since WCP implements fairness at the transport layer, it is able to adapt to different physical rates at the link layer. The lower rates for all the flows compared to Figure 4.9 is due to reduction in the network capacity because of lower link rates compared to 11Mbps in Figure 4.9. 109 Figure 4.29: Stack topology with different link rates Figure 4.30: WCP over links with different link rates, Stack However, with change in the link layer rate, the transmission range of a node changes leading to change in the number of nodes that can successfully receive a 110 packet from a node. This causes the topology of the network and, therefore, the neighborhood of each node to change. Table 4.2 shows the rates achieved by WCP during different simulation runs for the Stack topology with auto-rate adaptation. In these simulation 802.11 MAC, incorrectly, adapts the link layer rates when losses occur due to congestion. Since the resulting changes in topology occur at a time scale faster than the convergence time of WCP, WCP is unable to converge to a fair rate. SimulationRun 1→3 4→6 7→9 1 120kbps 102kbps 117kbps 2 118kbps 29kbps 74kbps 3 112kbps 12kbps 116kbps 4 66kbps 39kbps 70kbps Table 4.2: Stack with auto-rate adaptation WCPPerformanceintheabsenceofLinkLayerRetransmissions. Figures 4.31, 4.32, 4.33, and 4.34 show the performance of WCP with no link layer retrans- missions for data packets. WCP is still able to achieve fairness without link layer retransmissions since it implements fairness at the transport layer. A lossy link with no retransmissions increases the packet loss rate p (section 4.1.1) for all flows traversing the lossy link. Since sources in WCP send at a rate r f /p, increase in the path loss rate causes faster packet transmissions from the source of flows traversing the lossy link. This causes queues to build up on the lossy link, initiating congestion control in the neighborhood which in turn leads to convergence to a fair rate. DelayandConvergence. Since WCPCap keeps the network within the achievable rate region, it is able to maintain smaller queues than WCP. Hence, WCPCap has 111 Figure 4.31: WCP with no link layer retransmissions, Stack Figure 4.32: WCP with no link layer retransmissions, Diamond 112 Figure 4.33: WCP with no link layer retransmissions, Half-Diamond Figure 4.34: WCP with no link layer retransmissions, Chain-Cross 113 smaller average end-to-end delay than WCP (Figure 4.35). The one exception is the Chain-Cross: since the throughput of flows 1→ 7 and 6→ 7 is much higher in WCPCap than WCP, the total traffic over 6→ 7 is much higher for WCPCap (Figure 4.12). This results in a higher delay for these two flows. Finally, WCPCap converges quickly as can be seen in Figure 4.24; for all our topologies, it converges to within 10% of the final rate in less than 10s. Also, as is evident from Figure 4.23, WCP’s convergence is slower. This is expected as WCP is an AIMD scheme. 10 15 20 25 30 35 40 45 Delay (ms) WCP WCPCap 0 5 10 1→3 4→6 7→9 1→3 4→6 7→9 1→3 4→6 7→9 1→2 1→7 6→7 8→9 10→11 Stack Diamond Half- Diamond Chain-Cross Figure 4.35: Average end-to-end delay with WCP and WCPCap 4.2.4 Summary We summarize the throughput properties of WCP and WCPCap observed via simu- lations in this section. (i) WCP allocates rates that depend inversely on the number 114 of congested neighborhoods traversed by a flow and the intensity of congestion in those regions. (ii) WCPCap allocates to each flow a rate within 15% of the rate allocated to it by the max-min fair rate allocation. The main reason for this loss in throughput is the use of a conservative value for U(= 0.7). WCP is less efficient that WCPCap because its an AIMD algorithm. For the Stack topology where the rate vector achieved by WCP has fairness properties similar to the max-min rate al- location, it is within 20% of the optimal. (iii) Finally, WCPCap exhibits low delays and fast convergence. However, while WCP is implementable (we describe results from an imple- mentation in the next section), some challenges need to be addressed before the same can be said of WCPCap: the potentially high overhead of control informa- tion exchange, the sensitivity of the choice of U to the topology, and the ability to estimate the amount of interference from external wireless networks so that the collision probabilities can be correctly computed. None of these challenges are in- surmountable, and we plan to address these as part of future work. 4.3 Experiments We have implemented WCP, and, in this section, report its performance on a real- world testbed. We first validate our simulations by recreating the Stack topology and showing that our experimental results are qualitatively similar to those obtained in simulation. We then demonstrate that WCP performs as expected on two 14 node topology with five flows and 13 node topology with six flows in a real-world setting. 115 Our experiments use an ICOP eBox-3854, a mini-PC running Click [48] and Linux 2.6.20. Each node is equipped with a Senao NMP-8602 wireless card run- ning the madwifi driver [45] and an omni-directional antenna. Wireless cards are operated in 802.11b monitor (promiscuous) mode at a fixed transmission rate of 11Mbps with 18dBm transmission power. RTS/CTS is disabled for the experi- ments. We empirically determined, at the beginning of each experiment, that the packet loss rate on each link was less than 10%. On these nodes, we runexactly the same code as in our simulator by wrapping it within appropriate user-level elements in Click. Furthermore, all experimental pa- rameters are exactly the same as in the simulation (Table 4.1), with one exception: we use receiver buffer sizes of 2048 packets so that flows are not receiver-limited. For repeatability, all experiments were performed between midnight and 8am. All our experiments ran for 500 seconds and we show results averaged over five runs. We re-created the Stack topology by carefully placing nine nodes across three floors, and by removing the antennas on some nodes. Figure 4.36 shows that the experimental results are similar to the simulation results (Figure 4.9). Further- more, WCP achieves goodputs within 20% of an empirically determined maximum achievable rate. (We do not use our theory for determining optimal rates, because we cannot accurately estimate the amount of interference from external wireless networks.) We determine this by sending CBR flows at increasing rate till the goodput of flow f 4→6 is less than 90% of the goodput of the other two flows. 116 300 400 500 600 700 800 Goodput (kbits/sec) 1→3 4→6 7→9 0 100 200 TCP WCP Max-Min Achievable Rate Goodput (kbits/sec) Figure 4.36: Results from Stack experimental topology Finally, to examine the performance of WCP in a real-world setting, we created two arbitrary topologies of 14 nodes and 13 nodes by placing them on one floor of our office building (Figure 4.37 and 4.39). To create a multi-hop topology, we covered antennas of nodes with aluminium foil. On these topologies, we ran five and six flows respectively as shown. Figure 4.38 and 4.40 show the end-to-end goodput achieved by the flows. TCP either starves some of the flows ( f 15→26 , f 22→20 or f 18→11 in Figure 4.38) during different runs or shows high variance in the end-to-end goodput of the flows ( Figure 4.40). By contrast, WCP is able to consistently assign fair goodput to all the flows in each run of the experiment. 117 10 26 12 13 15 22 24 23 16 18 10 26 12 13 15 22 24 23 16 18 26 14 13 15 11 20 19 26 14 13 15 11 20 19 Figure 4.37: Arbitrary experimental topology of 14 nodes 300 400 500 600 700 Goodput (kbits/sec) 10→14 12→23 15→26 22→20 18→11 0 100 200 300 TCP WCP Goodput (kbits/sec) Figure 4.38: Results from arbitrary topology of 14 nodes 4.4 Summary In this chapter, we investigate the design of a congestion control scheme for gen- eral class of 802.11-based wireless networks with many-to-many traffic pattern. 118 Figure 4.39: Arbitrary experimental topology of 13 nodes Figure 4.40: Results from arbitrary topology of 13 nodes WCP, a distributed AIMD-based rate control scheme for such networks recognizes congestion in wireless network as a neighborhood phenomenon and accordingly signals all the flows in the neighborhood. WCPCap, in contrast, estimates the avail- able capacity in the network and apportions it fairly among contending flows in a 119 wireless networks achieving max-min fairness in an idealized version. Thus, it is essential for wireless congestion control to consider congestion at the granularity of a wireless neighborhood rather than at any finer granularity . Finally, there has been a growing interest in industry [3] and academia [38] in using multiple radios per node, in an effort to mitigate or nullify the complex interference found in multi-hop wireless networks. We believe that in dense de- ployments our work will be relevant even if multiple radios are used, since the large number of channels required to completely avoid interference, as well as the complexity associated with their scheduling, would be prohibitively expensive. 120 Chapter5 ConclusionandFutureDirections We start with a summary of the contributions of this dissertation followed by brief discussion of limitations of our proposed scheme. Finally, before concluding, we discuss some of the open problems we observed in the area of wireless congestion control. 5.1 Contributions In this dissertation, we make the following contributions: IFRC an open-loop rate-based fully distributed congestion control protocol for wireless sensor networks. Exploiting the characteristic tree-based traffic pattern of wireless sensor networks IFRC derives two simple, locally enforceable rules to achieve fair and efficient rate allocation with low overhead. IFRC, to the best of our knowledge, is the first practical neighborhood-centric rate control mechanism for wireless sensor networks. 121 WCP a practical AIMD-based, fully distributed end-to-end congestion control protocol for multi-hop wireless mesh networks. By sharing congestion status and average RTTs of flows among nodes in a neighborhood, WCP ensures correct con- gestion signaling and rate clocking of all the flows. Simulations and experiments shows that WCP achieves better fairness than TCP. WCPCap a explicit rate control protocol that estimates spare capacity in a neigh- borhood and apportions it fairly among flows traversing the neighborhood. WCP- Cap achieves max-min fairness in simulations for all our topologies. To our knowl- edge, WCPCap is the first plausibly implementable, neighborhood-centric imple- mentation for fair and efficient rate control for 802.11-based multi-hop wireless networks. 5.2 Limitations We believe that our protocols, IFRC, WCP, and WCPCap, take a significant step toward understanding congestion control in multi-hop wireless networks. However, the following limitations need to be overcome for realization of a widely deployable congestion protocol for multi-hop wireless network. Reducingoverhead In our current design the overhead of WCPCap is prohibitively large, yet not insurmountable, for a deployable implementation. These overheads could be reduced by quantization of shared information among the neighbors. This 122 quantization, however, could affect the performance of WCPCap. Understanding this trade-off between performance and overhead is vital to WCPCap’s design. Mobility All our protocols are designed for static multi-hop wireless networks and have not been tested under node mobility. Mobility changes the routing tree and the neighborhood graph of a node. A scheme for static networks can be adapted for mobile networks if these changes occur at the much larger time scale than the time to converge to a sustainable rate. Understanding the fundamental limits to convergence time that are intrinsic to our design choices and their implication on the degree of mobility that can be supported is important, yet unanswered, questions for our protocols External Interference Wireless networks suffer from internal and external in- terference. Internal interference represents wireless interference from other nodes operating with the same MAC while external interference is caused from devices operating in the same frequency band but not using the same MAC. Our designs implicitly accounts for internal interference only from the communicating neigh- bors. Appropriate extension to the neighborhood definition can help address inter- nal interference, However, addressing external interference is still an open and hard problem. Control Channel To consider wireless neighborhood as a single indivisible en- tity in wireless networks, as this thesis argues, nodes that share a wireless channel ( 123 alternatively, can affect each other’s transmissions) should be able to communicate their congestion status among themselves. The existence of a control channel to disseminate congestion information is critical to the design of a congestion control protocol for wireless networks. Limitations in WCP’s performance due to internal interference, asymmetric links, and auto-rate adaptation as listed in section 4.2.3 can be attributed to shortcomings of the design of the control channel among the nodes, e.g., the lack of a control channel between two nodes than can interfere but are not within each other’s communication range in case of internal interfer- ence and asymmetric links, or an unreliable control channel in the case of auto-rate adaptation. Revisiting the design of the control channel (which currently consist of piggybacking control information on data packets) is the most significant limitation requiring attention in WCP. In addition, other issues requiring attention are coexistence, and gradual de- ployment with current protocols, support for short lived flows, among others. 5.3 OpenProblemsandFutureDirections Fairness Our protocols exhibit comparably better performance in simulations and experiments, yet more investigation is required in understanding the nature of fairness achieved by our protocols. A more fundamental problem is understanding whether it is possible to achieve max-min fairness in a distributed fashion and if 124 neighborhood information is sufficient to accomplish it. In addition, it is also desir- able to have the capability to choose the trade-offs between fairness and aggregated throughput in the design of congestion control. Other Contention-Based MAC Our protocols are designed for two contention- based MACs, namely, 802.11 and 802.15.4. But with plethora of contention-MAC protocols in use today it is more desirable to have a congestion control protocol invariant of the underlying MAC. It is an important problem to investigate if our design can be generalized to other contention-based MAC protocols having dif- ferent resource (wireless channel) contention among links [68, 23] than 802.11 or 802.15.4. Unified Congestion Control protocol We believe that Internet’s edge networks are increasingly going to be multi-hop wireless with the core primarily being wired in nature. A flow in such a network would routinely traverse multiple wired and wireless links. With the understanding gained from our work and from the wealth of research in congestion control we believe that it is plausible to build a single unified congestion control protocol for such a network. 5.4 FinalRemarks Congestion control has vexed networking researchers for nearly three decades. Congestion control in wireless mesh networks is, if anything, harder than in wired 125 networks. In this dissertation, we take significant steps toward understanding con- gestion control for wireless networks. Our experience with the design of three con- gestion control protocols show that, as against links in wired networks, congestion control for wireless networks should be consider collectively for a wireless neigh- borhood. 126 Bibliography [1] F. Abrantes and M. Ricardo. A simulation study of xcp-b performance in wireless multi-hop networks. InProc.ofQ2SWinet, 2007. [2] U. Akyol, M. Andrews, P. Gupta, J. Hobby, I. Saniee, and A. Stolyar. Joint scheduling and congestion control in mobile ad hoc networks. In Proc. of IEEEINFOCOM, 2008. [3] P. Bahl, A. Adya, J. Padhye, and A. Wolman. Reconsidering Wireless Sys- tems with Multiple Radios. ACM SIGCOMM Computer Communications Review, 2004. [4] A. Bakre and B. Badrinath. I-TCP: indirect TCP for mobile hosts. In Proc. ofIEEEICDCS, 1995. [5] A. Bakre and B. R. Badrinath. I-tcp: indirect tcp for mobile hosts. In Pro- ceedings-InternationalConferenceonDistributedComputingSystems, page 136, Vancouver, Can, 1995. IEEE, Piscataway, NJ, USA. [6] H. Balakrishnan, H. Rahul, and S. Seshan. An integrated congestion manage- ment architecture for internet hosts. In Proceedings of the ACM SIGCOMM 1999, 1999. [7] H. Balakrishnan, S. Seshan, and R. H. Katz. Improving reliable transport and handoff performance in cellular wireless networks. WirelessNetworks, 1995. [8] P. Bergamo, S. Asgari, H. Wang, D. Maniezzo, L. Yip, R. E. Hudson, K. Yao, and D. Estrin. Collaborative sensor networking towards real-time acousti- cal beamforming in free-space and limited reverberance. IEEE Trans. Mob. Comput., 3(3):211–224, 2004. [9] V . Bharghavan, A. J. Demers, S. Shenker, and L. Zhang. MACAW: A Media Access Protocol for Wireless LAN’s. InSIGCOMM, pages 212–225, 1994. [10] G. Bianchi. Performance Analysis of the IEEE 802.11 Distributed Coordina- tion Function. IEEEJournalonSelectedAreasinCommunications, 2000. [11] S. Biaz, M. Mehta, S. West, and N. H. Vaidya. TCP over wireless networks using multiple acknowledgements. Technical Report 97-001, Computer Sci- ence,TexasA&MUniversity, 1997. 127 [12] K. Chandran, S. Raghunathan, S. Venkatesan, and R. Prakash. A feedback based scheme for improving tcp performance in ad-hoc wireless networks. In Proc.ofIEEEICDCS, 1998. [13] H. Chang, V . Misra, and D. Rubenstein. A general model and analysis of physical layer capture in 802.11 networks. In Proceedings of IEEE INFO- COM, 2006. [14] Cheng Tien Ee and Ruzena Bajcsy. Congestion control and fairness for many- to-one routing in sensor networks. In SenSys ’04: Proceedings of the 2nd internationalconferenceonEmbeddednetworkedsensorsystems, pages 148– 161, New York, NY , USA, 2004. ACM Press. [15] C. Cordeiro, S. Das, and D. Agrawal. Copas: dynamic contention-balancing to enhance the performance of tcp over multi-hop wireless networks. InProc. ofIEEEICCCN, 2002. [16] D. S. J. D. Couto, D. Aguayo, J. Bicket, and R. Morris. A high-throughput path metric for multi-hop wireless routing. InProc.ofACMMobiCom, 2003. [17] S. M. Das, D. Koutsonikolas, Y . C. Hu, and D. Peroulis. Characterizing multi-way interference in wireless mesh networks. In Proceedings of ACM WinTECHWorkshop, 2006. [18] N. Dukkipati, M. Kobayashi, R. Zhang-Shen, and N. McKeown. Processor Sharing Flows in the Internet. InProc.ofIWQoS, 2005. [19] A. Eryilmaz and R. Srikant. Fair Resource Allocation in Wireless Networks using Queue-length based Scheduling and Congestion Control. In Proc. of IEEEINFOCOM, 2005. [20] S. Floyd and V . Jacobson. Random Early Detection gateways for Congestion Avoidance. IEEE/ACMTransactionsonNetworking, 1993. [21] Z. Fu, H. Luo, P. Zerfos, S. Lu, L. Zhang, and M. Gerla. The impact of mul- tihop wireless channel on tcp performance. In IEEE Transactions on Mobile Computing, 2005. [22] Z. Fu, P. Zerfos, H. Luo, S. Lu, L. Zhang, and M. Gerla. The impact of multi- hop wireless channel on TCP throughput and loss. Proc.ofIEEEINFOCOM, 2003. [23] S. Gollakota and D. Katabi. Zigzag decoding: combating hidden terminals in wireless networks. SIGCOMM Comput. Commun. Rev., 38(4):159–170, 2008. [24] R. Govindan, E. Kohler, D. Estrin, F. Bian, K. Chintalapudi, O. Gnawali, S. Rangwala, R. Gummadi, and T. Stathopoulos. Tenet: An architecture for tiered embedded networks. CENSTechnicalReport56, 2005. 128 [25] B. Greenstein, A. Pesterev, C. Mar, E. Kohler, J. Judy, S. Farschi, and D. Es- trin. Collecting high-rate data over low-rate sensor network radios. CENS TechnicalReport55, 2005. [26] G. Holland and N. Vaidya. Analysis of TCP performance over mobile ad hoc networks. InProc.ofACMMobiCom, 1999. [27] B. Hull, K. Jamieson, and H. Balakrishnan. Mitigating congestion in wireless sensor networks. In SenSys ’04: Proceedings of the 2nd international con- ference on Embedded networked sensor systems, pages 134–147, New York, NY , USA, 2004. ACM Press. [28] B. Hull, K. Jamieson, and H. Balakrishnan. Mitigating congestion in wireless sensor networks. InProc.ofACMSenSys, 2004. [29] C. Intanagonwiwat, R. Govindan, and D. Estrin. Directed Diffusion: A Scal- able and Robust Communication Paradigm for Sensor Networks. InProceed- ingsofSixthInternationalConferenceonMobileComputingandNetworking (MobiCom’00), 2000. [30] K. Jain, J. Padhye, V . N. Padmanabhan, and L. Qiu. Impact of interference on multi-hop wireless network performance. InProc.ofACMMobiCom, 2003. [31] K. Jain, J. Padhye, V . N. Padmanabhan, and L. Qiu. Impact of interference on multi-hop wireless network performance. In MobiCom ’03: Proceedings of the 9th annual international conference on Mobile computing and network- ing, pages 66–80, New York, NY , USA, 2003. ACM Press. [32] A. Jindal and K. Psounis. Characterizing the Achievable Rate Region of Wireless Multi-hop Networks with 802.11 Scheduling. USC Technical Re- port CENG-2007-12, submitted to IEEE/ACM Transactions on Networking, 2007. http://tinyurl.com/5heujd. [33] A. Jindal and K. Psounis. Achievable Rate Region and Optimality of Multi- hop Wireless 802.11-Scheduled Networks. InProc.oftheInformationTheory andApplicationsWorkshop(ITA), 2008. [34] D. Katabi. Decoupling Congestion Control from the Bandwidth Allocation Policy and its Application to High Bandwidth-Delay Product Networks. PhD thesis, MIT, 2003. [35] D. Katabi, M. Handley, and C. Rohrs. Congestion control for high bandwidth- delay product networks. InProc.ofACMSIGCOMM, 2002. [36] F. Kelly, A. Maulloo, and D. Tan. Rate control in communication networks: shadow prices, proportional fairness and stability. In Journal of the Opera- tionalResearchSociety, volume 49, 1998. 129 [37] D. Kim, C.-K. Toh, and Y . Choi. TCP-BuS: improving TCP performance in wireless ad hoc networks. IEEE International Conference on Communica- tions, 2000. [38] M. Kodialam and T. Nandagopal. Characterizing the capacity region in multi- radio multi-channel wireless mesh networks. In Proc. of ACM MobiCom, 2005. [39] V . Kottapalli, A. Kiremidjian, J. P. Lynch, E. Carryer, T. Kenny, K. Law, and Y . Lei. A Two-Tier Wireless Sensor Network Architecture for Structural Health Monitoring. In Proc. of SPIE’s 10th Annual Symposium on Smart StructuresandMaterials, San Diego, CA, March 2003. [40] V . Kumar, M. M.V , S. Parthasarathy, and A. Srinivasan. Algorithmic Aspects of Capacity in Wireless Networks. InProc.ofACMSIGMETRICS, 2005. [41] Y . Li, L. Qiu, Y . Zhang, R. Mahajan, and E. Rozner. Predictable Performance Optimization for Wireless Networks. InProc.ofACMSIGCOMM, 2008. [42] X. Lin and N. B. Shroff. Joint Rate Control and Scheduling in Multihop Wireless Networks. In Proc. of IEEE Conference on Decision and Control, 2004. [43] J. Liu and S. Singh. Atcp: Tcp for mobile ad hoc networks. IEEEJournalon SelectedAreasinCommunications, 2001. [44] C. Lochert, B. Scheuermann, and M. Mauve. A survey on congestion con- trol for mobile ad hoc networks: Research Articles. Wirel. Commun. Mob. Comput., 2007. [45] MadWifi. http://madwifi.org/. [46] K. Mechitov, W. Y . Kim, G. Agha, and T. Nagayama. High-Frequency Dis- tributed Sensing for Structure Monitoring. In Proc. First Intl. Workshop on NetworkedSensingSystems(INSS04), 2004. [47] MIT Roofnet. http://pdos.csail.mit.edu/roofnet/. [48] R. Morris, E. Kohler, J. Jannotti, and M. F. Kaashoek. The Click modular router. SIGOPSOper.Syst.Rev., 1999. [49] M. Neely and E. Modiano. Capacity and Delay Tradeoffs for Ad-Hoc Mobile Networks. IEEETransactionsonInformationTheory, 2005. [50] G. Nychis, Sardesai, and S. Seshan. Analysis of XCP in a Wireless Environ- ment. CarnegieMellonUniversity, 2006. [51] J. Padhye, J. Kurose, D. Towsley, and R. Koodli. A model based TCP-friendly rate control protocol. InProc.ofNOSSDAV, 1999. 130 [52] J. Paek, K. Chintalapudi, J. Cafferey, R. Govindan, and S. Masri. A wireless sensor network for structural health monitoring: Performance and experience. In Proceedings of the Second IEEE Workshop on Embedded Networked Sen- sors(EmNetS-II), May 2005. [53] J. Paek and R. Govindan. RCRT: rate-controlled reliable transport for wire- less sensor networks. InProc.ofACMSenSys, 2007. [54] Qualnet. http://www.scalable-networks.com/products/. [55] B. Radunovi´ c, C. Gkantsidis, D. Gunawardena, and P. Key. Horizon: balanc- ing TCP over multiple paths in wireless mesh network. In MobiCom ’08: Proceedings of the 14th ACM international conference on Mobile computing andnetworking, 2008. [56] M. Rahimi, R. Baer, J. Warrior, D. Estrin, and M. B. Srivastava. Cyclops: In situ image sensing and interpretation in wireless sensor networks. In Pro- ceedings of the 3rd ACM Conference on Embedded Networked Sensor Sys- tems(SenSys2005), 2005. [57] S. Rangwala, R. Gummadi, R. Govindan, and K. Psounis. Interference-aware fair rate control in wireless sensor networks. In Proc. of ACM SIGCOMM, 2006. [58] Y . Sankarasubramaniam, O. Akan, and I. Akyildiz. ESRT: Event-to-Sink Re- liable Transport in Wireless Sensor Networks. In MobiHoc ’03 In Proceed- ings of 4th ACM Symposium on Mobile Ad Hoc Networking & Computing, June 2003. [59] G. Sharma, A. Ganesh, and P. Key. Performance Analysis of Contention Based Medium Access Control Protocols. InProc.ofIEEEINFOCOM, 2006. [60] P. Sinha, N. Venkitaraman, R. Sivakumar, and V . Bharghavan. WTCP: A Re- liable Transport Protocol for Wireless Wide-Area Networks. In Proceedings oftheACMMobicom1999, 1999. [61] A. L. Stolyar. Maximizing queueing network utility subject to stability: greedy primal-dual algorithm. QueueingSystems, 2005. [62] Y . Su and T. Gross. WXCP: Explicit Congestion Control for Wireless Multi- Hop Networks. InProc.ofIWQoS, 2005. [63] K. Sundaresan, V . Anantharaman, H.-Y . Hsieh, and R. Sivakumar. ATP: A Reliable Transport Protocol for Ad Hoc Networks. IEEE Transactions on MobileComputing, 2005. [64] R. Szewczyk, A. Mainwaring, J. Polastre, and D. Culler. An analysis of a large scale habitat monitoring. In Proceedings of the 2nd ACM Conference onEmbeddedNetworkedSensorSystems(SenSys2004), 2004. 131 [65] K. Tan, F. Jiang, Q. Zhang, and X. Shen. Congestion Control in Multihop Wireless Networks. IEEETransactionsonVehicularTechnology, 2006. [66] L. Tassiulas and A. Ephremides. Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks. AutomaticControl,IEEETransactionson, 1992. [67] L. Tassiulas and A. Ephremides. Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks. IEEETransactionsonAutomaticControl, 1992. [68] M. Vutukuru, K. Jamieson, and H. Balakrishnan. Harnessing Exposed Termi- nals in Wireless Networks. In5thUSENIXSymposiumonNetworkedSystems DesignandImplementation, San Francisco, CA, April 2008. [69] C.-Y . Wan, S. B. Eisenman, and A. T. Campbell. CODA: Congestion Detec- tion and Avoidance in Sensor Networks. In Proceedings of the First ACM Conference on Embedded Networked Sensor Systems (SenSys 2003)., pages 266–279, Los Angeles, November 2003. [70] A. Warrier, S. Janakiraman, S. Ha, and I. Rhee. DiffQ: Differential Back- log Congestion Control for Multi-hop Wireless Networks. In Proceedings of INFOCOM, 2009. [71] Wireless Internet Access. http://www.pewinternet.org/pdfs/PIP\ Wireless.Use.pdf. [72] A. Woo and D. Culler. A Transmission Control Scheme for Media Access In Sensor Networks. InProceedingsoftheSeventhAnnualACM/IEEEInter- nationalConferenceonMobileComputingandNetworking(Mobicom2001), 2001. [73] A. Woo, T. Tong, and D. Culler. Taming the Underlying Challenges of Reli- able Multihop Routing in Sensor Networks. InProceedingsoftheFirstACM ConferenceonEmbeddedNetworkedSensorSystems(SenSys2003)., Los An- geles, CA, November 2003. [74] K. Xu, M. Gerla, L. Qi, and Y . Shu. Enhancing TCP fairness in ad hoc wire- less networks using neighborhood RED. In MobiCom ’03: Proceedings of the 9th annual international conference on Mobile computing and network- ing, pages 16–28, New York, NY , USA, 2003. ACM Press. [75] W. Ye, J. Heidemann, and D. Estrin. An energy-efficient mac protocol for wireless sensor networks. In Proceedings of the IEEE Infocom, pages 1567– 1576, New York, NY , USA, June 2002. USC/Information Sciences Institute, IEEE. [76] X. Yu. Improving TCP performance over mobile ad hoc networks by exploit- ing cross-layer information awareness. InProc.ofACMMobiCom, 2004. 132

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

IEEE 802.11 is good enough to build wireless multi-hop networks

PDF

A protocol framework for attacker traceback in wireless multi-hop networks

PDF

Rate adaptation in networks of wireless sensors

PDF

Transport layer rate control protocols for wireless sensor networks: from theory to practice

PDF

USC Computer Science Technical Reports, no. 910 (2009)

PDF

Gradient-based active query routing in wireless sensor networks

PDF

Cooperation in wireless networks with selfish users

PDF

Dynamic routing and rate control in stochastic network optimization: from theory to practice

PDF

On location support and one-hop data collection in wireless sensor networks

PDF

Robust routing and energy management in wireless sensor networks

PDF

Optimal distributed algorithms for scheduling and load balancing in wireless networks

PDF

Efficient and accurate in-network processing for monitoring applications in wireless sensor networks

PDF

Scaling-out traffic management in the cloud

PDF

Design of cost-efficient multi-sensor collaboration in wireless sensor networks

PDF

Domical: a new cooperative caching framework for streaming media in wireless home networks

PDF

Enabling virtual and augmented reality over dense wireless networks

PDF

Towards interference-aware protocol design in low-power wireless networks

PDF

Anycast stability, security and latency in the Domain Name System (DNS) and Content Deliver Networks (CDNs)

PDF

Techniques for efficient information transfer in sensor networks

PDF

Multichannel data collection for throughput maximization in wireless sensor networks

Asset Metadata

Creator Rangwala, Sumit (author)

Core Title Congestion control in multi-hop wireless networks

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Computer Science (Computer Networks)

Publication Date 10/12/2009

Defense Date 12/16/2008

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag congestion control,OAI-PMH Harvest,wireless networks

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Govindan, Ramesh (committee chair), Heidemann, John (committee member), Psounis, Konstantinos (committee member)

Creator Email srangwal@usc.edu,sumitrangwala@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-m2660

Unique identifier UC1464258

Identifier etd-Rangwala-3227 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-267543 (legacy record id),usctheses-m2660 (legacy record id)

Legacy Identifier etd-Rangwala-3227.pdf

Dmrecord 267543

Document Type Dissertation

Rights Rangwala, Sumit

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Repository Name Libraries, University of Southern California

Repository Location Los Angeles, California

Repository Email cisadmin@lib.usc.edu