Computer Science Technical Report Archive

USC Computer Science Technical Reports, no. 681 (1998)

(USC DC Other)

USC Computer Science Technical Reports, no. 681 (1998)

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content An End-to-End TCP-friendly Architecture for Realtime
Playback Applications over the Internet
Ph.D. Dissertation Proposal
Reza Rejaie
Computer Science Department
University of Southern California
Los Angeles, CA 90089
reza@isi.edu
August 19, 1998
This work was supported by DARPA under contract No. DABT63-95-C0095 and DABT63-96-C-0054 as part of SPT
and VINT projects
Abstract
Lack of support for QoS in the Internet has not prevented rapid growth of realtime streaming ap-
plications during the last few years and this trend is expected to continue. Many of these applications
playback stored media (i.e. video, audio) for a client over the network.
In a shared network such as the Internet, all end-systems are expected to be “good network
citizens” and react to congestion by adapting their transmission rates properly and promptly.Since a
dominant portion of today’s Internet traffic is TCP-based, it is crucial that realtime streams perform
TCP-friendly congestion control. Currently, the majority of realtime applications over the Internet
are either open-loop or not TCP-friendly. In the absence of reservations or differentiated service,
wide deployment of these applications will have severe negative impact, ranging from unfairness to
competing TCP traffic to the potential for congestion collapse.
We present an end-to-end TCP-friendly architecture suited for playback of pre-encoded stored
realtime streams over the Internet. Our goal is to make realtime playback applications good net-
work citizens. We separate congestion control from error (quality) control. The Rate Adaptation
Protocol(RAP) performs rate-based congestion control in a TCP-friendly fashion. RAP employs an
additive-increase, multiplicative-decrease(AIMD) algorithm.
We evaluate RAP through extensive simulation, and conclude that bandwidth is usually evenly
shared between TCP and RAP traffic. Unfairness to TCP traffic is directly determined by how TCP
diverges from the AIMD algorithm. We also devised a fine-grain rate-adaptation mechanism to
extend TCP-friendliness over a wider range. We show that deploying RED queue management can
result in more ideal fairness between TCP and RAP traffic. Finally, we sketch a framework based on
layered transmission for quality adaptation as our future work.
Contents
1 Introduction 5
2 Background 8
2.1 Target Applications .... ... .... ... .... .... ... .... ... .... . 8
2.2 Design Space . . . .... ... .... ... .... .... ... .... ... .... . 9
2.2.1 End-to-End vs Hop-by-Hop Congestion Control . . . . . .... ... .... . 9
2.2.2 Congestion Signal . . . .... ... .... .... ... .... ... .... . 9
2.2.3 TCP-friendly Rate Adaptation Algorithm . . .... ... .... ... .... . 10
2.2.4 Separating Congestion Control from Error Control . . . . .... ... .... . 11
2.2.5 Rate-based vs Window-based Congestion Control . . . . . .... ... .... . 11
2.2.6 Error Control and Quality Adaptation .... .... ... .... ... .... . 12
3 Related Work 14
3.1 Congestion Control .... ... .... ... .... .... ... .... ... .... . 14
3.2 Adaptive Encoding .... ... .... ... .... .... ... .... ... .... . 14
3.3 TCP-friendly Congestion Control.... ... .... .... ... .... ... .... . 14
4 The RAP Protocol 16
4.1 Overview . . . . . .... ... .... ... .... .... ... .... ... .... . 16
4.2 Clustered Losses . .... ... .... ... .... .... ... .... ... .... . 20
4.3 Fine grain rate adaptation . . . . .... ... .... .... ... .... ... .... . 20
4.4 Self-limiting issues in RAP . . . .... ... .... .... ... .... ... .... . 23
4.5 Random Early Discard Gateways.... ... .... .... ... .... ... .... . 24
4.6 Startup Phase . . . .... ... .... ... .... .... ... .... ... .... . 24
5 Simulation-based Evaluation 26
5.1 Evaluation Methodology . . . . .... ... .... .... ... .... ... .... . 28
5.2 Experiments and Results . . . . .... ... .... .... ... .... ... .... . 28
5.2.1 TCP-friendliness . . . . .... ... .... .... ... .... ... .... . 29
5.2.2 Fine-grain rate adaptation . . . . . . .... .... ... .... ... .... . 33
5.2.3 Burstiness . .... ... .... ... .... .... ... .... ... .... . 35
5.2.4 RED Gateways . . . . . .... ... .... .... ... .... ... .... . 37
5.2.5 Intra-protocol Behavior . .... ... .... .... ... .... ... .... . 39
3
5.2.6 Smoothness of RAP transmission rate .... .... ... .... ... .... . 39
6 A Framework for Layered Transmission 41
6.1 Introduction . . . . .... ... .... ... .... .... ... .... ... .... . 41
6.1.1 A Simple Approach . . . .... ... .... .... ... .... ... .... . 41
6.1.2 Proposed Approach: Quality Adaptation . . . .... ... .... ... .... . 41
6.1.3 Architecture Overview . .... ... .... .... ... .... ... .... . 42
6.2 Error Control . . . .... ... .... ... .... .... ... .... ... .... . 44
6.3 Quality Adaptation .... ... .... ... .... .... ... .... ... .... . 45
6.4 The Layered Approach . . . . . .... ... .... .... ... .... ... .... . 46
6.4.1 Desired Properties . . . .... ... .... .... ... .... ... .... . 49
6.4.2 Strategies . .... ... .... ... .... .... ... .... ... .... . 51
6.5 Open issues . . . . .... ... .... ... .... .... ... .... ... .... . 57
7 Contribution and Future Work 59
7.1 Contributions . . . .... ... .... ... .... .... ... .... ... .... . 59
7.2 Remaining Thesis . .... ... .... ... .... .... ... .... ... .... . 60
7.2.1 Quality Adaptation . . . .... ... .... .... ... .... ... .... . 60
7.2.2 Error Control . . . . . . .... ... .... .... ... .... ... .... . 61
7.2.3 Buffering Scheme . . . .... ... .... .... ... .... ... .... . 61
7.3 Expected Contribution . . . . . .... ... .... .... ... .... ... .... . 61
4
1 Introduction
The Internet has recently been experiencing an explosive growth in the use of audio and video stream-
ing. Such real-time applications are delay-sensitive, semi-reliable and rate-based. Thus they require
isochronous processing and quality-of-service(QoS) from the end-to-end point of view [CSZ92]. How-
ever, today’s Internet does not attempt to guarantee an upper bound on end-to-end delay or a lower
bound on available bandwidth. As a result, the quality of delivered service to realtime applications over
a best-effort network is neither controllable nor predictable. However, lack of support for QoS in the
Internet has not prevented rapid growth of realtime streaming applications during the last few years and
this trend is expected to continue. Many of these applications [Net, Incb, Inca] playback stored media
(i.e. video, audio) for a client over the network. Examples include continuous media servers, digital
libraries, distant learning and collaboration applications, shopping and entertainment services.
In a shared network such as the Internet, all end-systems (both realtime and non-realtime) are expected
to react to congestion by adapting their transmission rates[FF98], not only to avoid congestion collapse
but also to keep the network utilization high. Another important issue is inter-protocol fairness. The
rate adjustment should result in a fair share of bandwidth for all the flows that coexist along the same
path. Those applications that adapt their transmission rates properly and promptly are known as “good
network citizens”. Since a dominant portion of today’s Internet traffic is TCP-based (e.g. e-mail, FTP,
Web-traffic), it is crucial that realtime streams perform TCP-friendly congestion control. By “TCP-
friendly”, we mean that a realtime flow should obtain approximately the same average bandwidth over
the time scale of a session as a TCP [Jac88] flow along the same path under the same conditions of delay
and packet loss.
Currently, the majority of realtime applications over the Internet are either open-loop, (i.e. without an
end-to-end congestion control mechanism) or not TCP-friendly in their response to congestion. Wide
deployment of these applications will have severe negative impact, ranging from unfairness to compet-
ing TCP traffic, to the potential for congestion collapse. One possible solution would be to some how
force realtime flows to use reservations or differentiated service. However, even if such services become
widely available in the future, there will remain a significant group of users who are interested in using
realtime applications over the best-effort service because reservation will cost them. Even in a network
that supports reservation, different users that fall into the same class of service or share the same reser-
vation still interact as in best effort networks and therefore require a TCP-like mechanism to respond to
congestion of the shared network. Thus we believe that deployment of realtime applications over best-
effort networks deserves more attention. Particularly study of congestion control for these applications
is critical for the future health of the Internet.
5
Buffer Manager
RAP
Source
Error
Control
Receiver
Internet
Server
Client
Quality
Adaptation
Storage
Transmission
Buffer
RAP
Sink
Buffer
Manager
Decoder
Playout
Buffer
Archive
Figure 1: End-to-end architecture for realtime playback applications in the Internet
This paper presents an architecture for playback of pre-encoded stored realtime streams over the In-
ternet. Our goal is to make realtime playback applications good network citizens. A typical target
application for this architecture could be a web-server or a video-on-demand server that provides ac-
cess to a variety of multimedia streams for large numbers of heterogeneous users. Figure 1 shows the
proposed architecture for server and client. We separate congestion control from error (quality) con-
trol [CLZ88] because the former depends on the state of the network while the latter is application
specific. The server’s transmission rate is continuously adjusted by the Rate Adaptation Protocol (RAP)
in a TCP-friendly fashion. The RAP module is in charge of congestion control and loss detection. The
Error control module deals with packet losses through selective retransmission. The Quality adaptation
module adapts the quality of transmitted streams based on the rate specified by the RAP module.
There are many ways to adjust the quality, but the one we are investigating is hierarchical (i.e. layered)
encoding[HS96]. This approach is able to cope with bandwidth heterogeneity among clients without re-
coding the stream for each different client. In this context, the Quality adaptation module tries to deliver
the maximum number of layers that can be fit in the available bandwidth. Rate adaptation happens on a
time scale of round-trip times (RTT) but layers are added and dropped on a longer time scale by using
receiver buffering to accommodate temporary mismatches between transmission and consumption rates.
These playback clients can afford to delay the playback point slightly and buffer some data to partially
absorb variation of the network bandwidth and the end-to-end delay. Buffering at the client side also
provides the opportunity for selective retransmission as determined by the quality control module. Other
types of repair approaches can be also deployed. Note that the aggregate bandwidth used by the server
including retransmission, should not exceed the bandwidth that is specified by the RAP protocol.
6
We presents the design of the RAP protocol and evaluate its performance through extensive sim-
ulations
1
. RAP is an end-to-end rate-based congestion control mechanism that is suited for unicast
playback of realtime streams such as the architecture in figure 1 as well as other semi-reliable or rate-
based applications over the Internet. The goals of RAP are to be well-behaved and TCP-friendly. Our
simulations show that: 1. RAP is TCP-friendly as long as TCP’s congestion control is dominated by
the AIMD algorithm. The more TCP’s congestion control diverges from AIMD, the less bandwidth is
obtained by the TCP traffic. 2. We identified the contribution of TCP’s inherent limitations to unfairness
of RAP against TCP. Our observations lead us to conclude that RAP behaves in a TCP-friendly fash-
ion over a wide range of scenarios. 3. To further improve RAP, we have also devised a fine grain rate
adaptation mechanism that enables it to exhibit TCP-friendly behavior over an even wider range. 4. We
assessed the impact of TCP’s burstiness on inter-protocol fairness in the context of small simulations.
Our results show that deploying RED [FJ93] queue management results in an ideal fairness between
TCP and RAP traffic. 5. We investigated self-limiting issues in RAP and did not observe any evidence
that implies inherent instability in RAP. 6. Finally, we present a framework for layered transmission of
media streams over best effort networks and identify some of the open issues for our future research.
The rest of this paper is organized as follows. First we specify our target applications, identify the
design space, and justify our design choices in section 2. We review some of the related work in section
3. In section 4, we present various aspects of the RAP protocol. Detailed description of the RAP
simulation results are presented in section 5. Section 6 describes our motivation for layer transmission
and sketches a framework for layered transmission. A summary of our contributions and our plans for
future work are listed in section 7.
1
Using the ns-2 simulator developed by the VINT project
7
2 Background
2.1 Target Applications
There are different categorizations for realtime applications. We address some of these categories that
are relevant to our work and finally specify a class of realtime applications that is our target.
Realtime streams could be generated by a live source(e.g. output of a microphone) or be playbacked
from stored data(e.g. a video clip on disk). The main difference between these two cases is that in the
live scenario the entire stream is not available at any moment of time, thus the source can not send faster
than the generation rate. In playback applications, the source can transmit data at any rate. However,
if realtime data is buffered before transmission at the sender side, more data will be available for rate
adjustment. The more data that are buffered at the source, the greater adjustment that can be made in the
transmission rate, the more the live scenario resembles the playback scenario.
This argument implies that there is a spectrum for liveliness. At one end of the spectrum, data is
transmitted as soon as it is generated(i.e. absolutely-live). At the other end, the first packet is transmitted
after the entire stream is generated(i.e. playback). Note that in the absolutely-live case, no congestion
control is applied and transmission rate depends on generation rate. Since the average generation rate
of realtime sources is relatively constant, live transmission will not result in a TCP-friendly behavior.
A common practice is to buffer some data at the source to smooth out the transition rate even for live
sessions.
Another issue in live sessions is the level of interactivity. The more tightly a receiver and the source
interact, the less delivery delay can be tolerated. Buffering data at any of the end-points reduces the
level-of-interactivity of the session. Thus, interactive session can not afford to buffer data at the source
or client. As a result, interactive session are neither congestion controlled nor TCP-friendly. However, a
session with a low level of interactivity can buffer some data and smooth its transmission rate to perform
some degree of congestion control. Finally, non-interactive or lecture-mode sessions can be delayed for
a sufficient amount of time to apply rate adjustment. This scenario is similar to playback applications.
Realtime streams generated by human conversations, both live to pre-recorded, have silence peri-
ods [Mon83]. This means that data is transmitted in an on/off fashion. These silence periods provide an
opportunity for the receiver to partially absorb the variation of inter-packet-gap by adaptive readjustment
of the playback point for each interval(i.e. talk-spurt). This approach introduces a minor distortion that
have been shown to be tolerated in uni-directional scenario. In the context of playback streams, interac-
tivity between client and server would be in the form of VCR-functionalities. This is much looser than
the interaction among humans and in the worst case it will result in redundant transmission or a minor
8
increase in latency.
We focus on a class of realtime applications that playback a pre-recorded stream to a single receiver.
This covers a large group of current applications on the Internet such as web-based realtime streams
(e.g. real audio/video), Video/Audio/News-on-demand systems. Moreover, our approach should be
applicable to lecture-mode sessions as long as the source can afford to buffer enough data to effectively
adapt its rate and perform RAP.
2.2 Design Space
In this section we identify the design space for TCP-friendly congestion and error control mechanisms
that are applicable to realtime playback applications in the Internet and justify our design choices.
2.2.1 End-to-End vs Hop-by-Hop Congestion Control
As we mentioned earlier, today’s Internet does not implement any machinery for congestion control
within the network. Thus, the only feasible congestion control approach at this time, is end-to-end. The
end-to-end approach has played a key role in stability of the Internet[FF98]. It is worth noting that end-
to-end congestion control schemes will remain an essential complementary mechanism for router-based
ones in the future[FF98].
2.2.2 Congestion Signal
Different implicit congestion signals have been proposed: variation of RTT in delay-based conges-
tion avoidance [Jai89, Dab92], variation of throughput in Vegas-like congestion avoidance [BP95], and
packet loss [Jac88].
Delay-based congestion avoidance schemes usually obtain a smaller bandwidth share when they com-
pete with loss-based schemes like TCP. TCP probes the network for available bandwidth by linearly
increasing its transmission rates until it detects a packet loss. Thus TCP periodically drives the network
into a state where packets are dropped. Many congestion avoidance schemes react to “early” conges-
tion signals by decreasing their transmission rates to avoid congestion. This allows TCP flows to utilize
some extra bandwidth and causes non-loss-based flows to back off even further. As a result, a flow that
performs congestion avoidance does not obtain its fair share when it coexists with TCP traffic.
In Vegas-like schemes, a source monitors the difference between the average measured throughput and
the average expected throughput during one RTT. The difference is compared with two thresholds to rate
is adjusted accordingly. The main problem with this approach is choosing the right values of threshold
9
and the interval. The protocol’s behavior directly depends on the value of the thresholds, which in turn
closely depend on the characteristics of the connection.
Packet loss seems to be the only feasible implicit feedback signal in the Internet due to the presence
of fluctuating TCP traffic.
2.2.3 TCP-friendly Rate Adaptation Algorithm
The rate adaptation algorithm is a core component of any end-to-end congestion control mechanism
that increases or decreases its load (i.e. transmission rate) as congestion is detected or more bandwidth
becomes available. The main goal of the rate adaptation algorithm is not only to manage congestion but
also to achieve inter-protocol fairness. Note that these two issues are closely related. When a source
reacts to congestion events, all other sources that observe the same congestion are expected to behave
well and drop their transmission rate. Since the network does not play any active role in monitoring the
behavior of active flows, obtained bandwidth by each flow is determined by the aggregate behavior of
all the coexisting flows. For example, a mis-behaved flow can achieve a higher share of bandwidth and
force all other well-behaved flows to achieve less. In a best-effort network that is shared by a large group
of independent users, it is extremely hard to achieve fairness by independent adjustment of transmission
rates. It has been shown that the Additive Increase and Multiplicative Decrease (AIMD) algorithm
efficiently converges to a fair state [CJ89] in a reasonable time scale. AIMD is the most promising and
probably the only algorithm for rate adaptation to achieve inter-protocol fairness. RAP adopts an AIMD
algorithm.
Note that AIMD presents a class of algorithms, i.e. it can be implemented in different ways that result
in different behaviors. Our goal is to design a TCP-friendly AIMD algorithm. A rate-based congestion
control mechanism is considered TCP-friendly if its average transmission rate is not higher (and hope-
fully not much lower) than that achieved by a TCP connection along the same path [MF97]. Although
TCP uses loss-based congestion control, the “self-clocking” mechanism means that TCP dynamically
adjusts its transmission rate in two time scales simultaneously: 1. Fine-grain rate ack-clocked adjust-
ment, 2. Coarse-grain congestion window adjustment.
The self-clocking mechanism adjusts the instantaneous transmission rate of a TCP source on a per
packet basis. During the congestion avoidance phase, if the queue length suddenly increases at a bottle-
neck (because of a burst of incoming packets or start of a new flow), TCP packets in the queue experience
longer delays and the RTT increases. If no timeout occurs, the TCP source clocks more slowly, adjust-
ing the inter-packet-gap and the instantaneous transmission rate immediately. As soon as the bottleneck
queue is drained, the subsequent TCP packets experience shorter RTT, the source clocks faster and the
10
instantaneous transmission rate increases. Due to random changes in RTT, TCP’s instantaneous trans-
mission rate can change significantly.
The coarse-grain rate adjustment of TCP is achieved with the AIMD window adjustment mechanism.
A TCP source opens up its congestion window proportionally to the ACK rate, i.e. it increases cwnd
by (1/cwnd) for each ACK. If the ACK rate suddenly drops (most probably due to congestion), the
congestion window opens up more slowly and the slope of the average transmission rate is reduced.
In essence, TCP does not simply perform AIMD window (or rate) adjustment. The strength of TCP
congestion control comes from the self-clocking property that adjusts the transmission rate to avoid
congestion with the best possible granularity. A truly TCP-friendly rate-adaptation scheme must emulate
both the fine-grain and the coarse-grain rate-adjustment mechanism in TCP. In other words, a congestion
control scheme only based on AIMD with no additional mechanism can not be TCP-friendly.
TCP has some performance problems that are being investigated and improved. It is not clear how
much of TCP’s behavior must be emulated in order to reach a decent fairness against TCP. However,
AIMD is a crucial requirement for inter-protocol fairness.
2.2.4 Separating Congestion Control from Error Control
In the context of traditional applications, error control schemes usually provide “reliability” by retrans-
mission of a lost packet which increases end-to-end latency. Realtime applications are “semi-reliable”,
i.e. able to tolerate some level of loss as long as variation of delay is limited. In the context of realtime
applications, the goal of error control mechanisms is to achieve “quality” instead of reliability. Since
the definition of quality is application specific, each application should adopt a different error control
scheme based on the impact of loss on the delivered quality. Nevertheless in a best-effort network, all
applications, including realtime applications, are expected to react to congestion properly in an appro-
priate time scale. In other words, error control has application-specific semantics whereas congestion
control in best effort networks only depends on the congestion state of the network regardless of the
applications. As a result, error control should be decoupled from flow control.
2.2.5 Rate-based vs Window-based Congestion Control
End-to-end congestion control schemes are divided into two main categories: window-based and rate-
based schemes. Window-based schemes usually work based on “conservation of packets” or “self-
clocking” [Jac88] and couple congestion control with error control. As a result of the self-clocking,
a source can not inject another packet beyond the current window until arrival of an acknowledgment
11
(ACK). Since the arrival of an ACK directly depends on the load of the network, the instantaneous
transmission rate of a source varies with random changes of the network load.
In Van Jacobson’s seminal TCP congestion control paper[Jac88], the idea of using a rate-based ap-
proach based on the same mechanisms is mooted. However, for normal reliable transport, window-based
solutions offer a number of advantages, including data-driven clocking (avoiding the need for timer-
driven transmission) and inherent stability because ACK-clocking means that the number of packets
each flow has in the network is limited. In practice, this stability property is weakly compromised so
that flows can discover the available bandwidth, and because the number of new flows is not limited, but
the property is still attractive.
In contrast, rate-based protocols are harder to implement because the correct use of timers is non-
trivial and they lack this inherent stability property. However, if we can demonstrate that a rate-based
protocol is well behaved, TCP-friendly, and goes some way towards addressing this stability issue,
then there are a number of applications where a rate-based protocol is a more natural fit than a window-
based protocol. For applications that are inherently rate-based, having the congestion control mechanism
give them an explicit rate is advantageous. Separating the reliability mechanism from the congestion
control mechanism is a requirement for such semi-reliable applications, and is naturally achieved for
rate-based schemes. This can also be achieved with a separate count of “packets-in-flight” for window-
based congestion control. However, the issue of timer-based clocking for new packets when old ones are
missing long enough to be considered lost can result in more bursty behavior which makes predicting
performance of the window-based schemes more difficult.
No one of these reasons is sufficiently strong to rule out window-based mechanisms for these ap-
plications, but in combination they were sufficient to challenge us to investigate whether a rate-based
approach could be made to co-exist well with TCP.
2.2.6 Error Control and Quality Adaptation
The goal of error control mechanisms in realtime applications is to control delivered quality instead
of reliability as in traditional applications. Furthermore, quality has an application-specific semantics.
Since realtime applications are semi-reliable, some degree of packet loss can be tolerated, as long as the
desired quality is met.
In a typical constant-bit-rate streaming application, the error control module is in charge of repair for
packet losses by deploying mechanisms such as FEC or striping. The essence of these mechanisms is to
add some redundancy to the data stream to recover from occasional packet loss. If a playback receiver
can buffer sufficient data to feed the receiver for at least one RTT, retransmission is a feasible option
12
as long as the aggregate transmission rate(including retransmission) is regulated by RAP. We call this
selective error control since a missing packet can be selectively repaired.
Another dimension for error control is to map bandwidth variability into the delivered quality. This
is called Quality adaptation. The quality adaptation is orthogonal to the repair approaches that we
described above. The presence of TCP background traffic in the Internet implies that the available
bandwidth could vary with time. If extra bandwidth becomes available for a sufficiently long period
of time, The source can utilize this bandwidth and increase its transmission rate by sending higher
resolution data. We call this quality control since it manages the delivered quality of the stream.
13
3 Related Work
We classified related works in several areas and addressed them in this section.
3.1 Congestion Control
Congestion control is not a new topic and a large body of work has accumulated describing various
congestion control mechanisms. Unfortunately, many of the well known rate-based congestion con-
trol mechanisms can not be deployed over an unmodified Internet. The hop-by-hop congestion control
scheme proposed by Mishra et. al. [MK92] require machinery to be deployed in the intermediate routers.
Comer et. al propose another rate-based scheme that requires cooperation of the switches in the network
to monitor the traffic [CY90]. The packet-pair scheme proposed by Keshav [Kes91] assumes weighted
fair queueing service discipline that is not currently supported in the Internet.
NETBLT[CLZ88] is the first widely known rate-based congestion control mechanism. NETBLT’s
main contribution is in its sepration of congestion and error control. Unlike TCP, NETBLT does not
decrease its rate as soon as packet loss occurs. However, it adjusts its transmission rate based on AIMD
algorithm once per RTT.
3.2 Adaptive Encoding
A common approach for rate adaptation is adaptive encoding through the adjustment of codec quan-
tization parameters based on state of the network[OK95]. Many of these studies have not addressed
inter-protocol fairness; instead they strive to improve the perceptual quality. However, work in [TZ98]
proposes an adaptive coding scheme, using the formula presented in [MF97, MSMO97] that captures
the macroscopic behavior of TCP. This approach shows promise, but it has yet to be shown that this
formula or the more detailed variant of it in [PFTK98] can be used in a wide range of situations with-
out introducing possible large-scale oscillatory behavior. Moreover, it is CPU-intensive for a server to
adaptively encode a potentially large number of streams simultaneously for all active clients.
3.3 TCP-friendly Congestion Control
The critical work for protocols that wish to co-exist with TCP in the best-effort Internet is somewhat lim-
ited. Jacob et. al. [JE97] propose an architecture for Internet video applications that uses a TCP variant
modified so as not to perform retransmission. However, no details of these modifications are given, so it
is difficult to tell how these changes affect performance. Normally TCP retransmission control and TCP
14
flow control are closely related - separating these functions probably changes the performance subtly.
We discuss in 2.2.5 why we have chosen a rate based approach rather than a window based approach.
Cen et. al[CPW98] present the SCP protocol for media streaming in the Internet. The SCP protocol
is a modified version of TCP that performs TCP Vegas-like rate adjustment in steady state. Their results
show that SCP is not TCP-friendly. This may be due to using the shortest RTT that has been measured
for rate adjustment since it may widely vary for different flows.
There are many commercial media streaming players that are currently deployed over the Internet such
as Realplayer[Net], Vxtreme[Incb] and Microsoft Netshow[Inca]. Although they claim to be adaptive,
no analysis is available to verify any claims. Work in [CTCL95] describes the VDP protocol that is
deployed in V osaic. Their proposed adaptation algorithm is clearly not TCP-friendly.
Our study differs from previous studies of realtime streaming over best-effort networks. We develop
a rate adaptation mechanism that will result in inter-protocol fairness and in particular TCP-friendly
behavior. The majority of the previous works either did not address fairness at all, or have not examined
sufficient cases to attempt to find the bounds where their protocols cease to be fair.
15
4 The RAP Protocol
4.1 Overview
The RAP protocol machinery is mainly implemented at the source. A RAP source sends data packets
with sequence numbers, and a RAP sink acknowledges each packet, providing end-to-end feedback.
Each acknowledgment(ACK) packet contains the sequence number of the corresponding delivered data
packet. Using the feedback, the RAP source can detect losses and sample the round-trip-time (RTT).
To design a rate-adaptation mechanism, three issues must be addressed [Jai89]. These are the decision
function , the increase/decrease algorithm, and the decision frequency.
Decision Function
A high level description of the rate-adaptation scheme could be summarized by its decision function as
follow:
If no congestion is detected, periodically increase the transmission rate;
If congestion is detected, immediately decrease the transmission rate.
Similar to the congestion avoidance algorithm in TCP [Jac88], the RAP source searches for available
bandwidth on the bottleneck link by periodically increasing its transmission rate.
RAP considers losses to be congestion signals, and uses two mechanisms to detect loss: timeouts, and
gaps in the sequence space(i.e. ACK-based). These two mechanisms work in parallel.
Similar to TCP, RAP maintains an estimate of RTT, called SRT T , and calculates the timeout based
on the Jacobson/Karel’s algorithm
2
. However, it detects the timeout losses differently. Because of
the ack-clocking property, a TCP source always receives an ACK and updates the RTT estimate before
transmitting a new packet. Thus, the TCP source always has an updated estimate of RTT to calculate
the timeout. This enables TCP to deploy a timer-driven approach to detect timeout losses. RAP is not
ack-clocked. A RAP source may send several packets before receiving any new ACK to update the RTT
estimate. Therefore, a TCP-like timeout mechanism is not appropriate and results in detecting frequent
incorrect losses that are in fact late ACKs.
RAP couples the timer-based loss detection to the packet transmission. The source maintains a record
for each transmitted packet, containing the sequence number, departure time, transmission rate and sta-
tus flag for each packet. The collection of records for outstanding packets is called the transmission
2
SRT T
i
SRT T
i
SampleRT T
T imeout SRT T VarRT T
16
history. Before sending a new packet, the source checks for a potential timeout among the packets in the
history using the updated value of the SRTT estimate. Then it traverses through the transmission history
and detects all the timeout losses using the following algorithm:
WH I LE D epar tT ime
i
T imeout CurrT ime IF Flag
i
Ack ed TH EN
Seq
i
is l ost
This mechanism may detect a burst of loss at once. Moreover because of the absence of ack-clocking,
the RAP source may still receive a late ACK.
The ACK-based loss detection mechanism in RAP is based on the same intuition as the fast-recovery
mechanism in TCP. To limit the amount of overshoot during the increase phase, the RAP source needs
to detect congestion (i.e. packet loss) as early as possible. If RAP source receives an ACK that implies
delivery of three packets after the missing one, the packet is considered as lost. RAP requires a way to
differentiate the loss of an ACK from the loss of the equivalent packet. We have added some redundancy
in the ack stream to specify the last hole in delivered sequence number space and provide robustness
against single ack losses
3
. An ACK packet contains the following information, as shown in figure 2:
1. The sequence number, A
cur r
, of the packet being acknowledged,
2. The sequence number, N, of the last packet before A
cur r
that was still missing, or 0 if no packet
was missing,
3. The sequence number, A
last
, of the last packet before N that was received, or 0 if A
cur r
was the
first packet.
1 __
1 __ __ 4
1 __ __ 4 __ 6
1 __ __ 4 __ 6 __ __ 9
__ 6 __ __ 9 10
6 __ __ 9 10 11
A
last
A
curr
N
0 0 1
1 3 4
4 5 6
6 8 9
6 8 10
6 8 11
Packet Loss Pattern
ACK Information
A
6
Figure 2: ACK-based loss detection in RAP
3
To achieve resilience to multiple ACK-loss, more information must be carried in ACK packets. We have not studied this
issue any further
17
Using this information, a RAP sender can mark packets as having arrived even when the ACK is dropped
because subsequent ACKs arrive in which their sequence number is greater than N and less than A
cur r
.
Adding A
last
provides sufficient redundancy in the ACK stream to recover loss of AC K
i when data
packet with sequence number i, is lost. After each ACK arrives, the RAP source performs the following
algorithm:
F or each S eq
i
in T r ans H istor y
DO
IF A
cur r
Seq
i
AN D Seq
i
N OR
Seq
i
A
l ast
TH EN
Seq
i
w as r eceiv ed
ELSE I F A
cur r
Seq
i
TH EN
Seq
i
is l ost
WH I LE Seq
i
A
cur r
Note that the timeout mechanism is still required as a back up for critical scenarios such as when a burst
of packets is lost.
Increase/decrease Algorithm
For its increase/decrease algorithm RAP uses AIMD. In the absence of packet loss, the transmission rate,
S, is increased in a step-like fashion. In other words, at each adjusting point we have S
i S
i
.
We refer to the value of as step height. The transmission rate is controlled by adjusting the inter-
packet-gap(IP G). To increase the rate additively, IPG must be iteratively updated based on equation
(1) [Jac88]:
S
i
P ack etS iz e
IP G
i
IP G
i IP G
i
C
IP G
i
C
(1) S
i S
i
P ack etS iz e
C
(2)
Note that in the equation (1), C has the dimension of time and it determines the value of . Upon
detecting congestion, the transmission rate is decreased multiplicatively, by increasing the value of IPG
4
:
S
i
S
i
IP G
i IP G
i
Decision Frequency
Decision frequency specifies how often to change the rate. Based on system control theory, optimal
adjustment frequency depends on the feedback delay. The feedback delay is the time between changing
the rate and detecting the network’s reaction to that change. Feedback delay in ACK-based schemes
(e.g. RAP) is equal to one RTT. It is suggested that rate-based schemes adjust their rates not more than
4
We use a value of which is a conservative choice as it is chosen by TCP (see appendix C in [Jac88]).
18
once per RTT [MF97]. Changing the rate too often results in oscillation whereas infrequent change of
rate leads to an unresponsive behavior.
RAP adjusts the IP G once every round-trip time. The time between two subsequent adjusting points
is called a step. Because of the random nature of the RTT signal[Bol93], using the recent sample RTT as
the step length is likely to result in a poor behavior. We need a smoothed version of RTT that represents
low frequency variation of RTT and filters out the transient (i.e. high frequency) changes. RAP uses the
most recent value of SRTT as the step length.
At the beginning of each step, a timer, called step-timer, is set to the recent value of SRTT and the
IP G is decreased based on equation (1). The value of IP G remains unchanged until the step-timer
expires or a packet loss occurs. If no loss is detected, IP G is decreased and a new step is started.
Adjusting the IP G once every SRTT has a nice property; packets sent during one step are likely to be
acknowledged during the next step. This allows the RAP source to observe the reaction of the network
to the previous adjustment before making a new adjustment.
As we mentioned earlier, C in equation 1 has the dimension of time and is the only parameter that
controls the rate of increase of the transmission rate. One immediate question is what is the right value
for C?. Since our chief goal is to be TCP-friendly, C must be adjusted so that in the steady state, the
number of packets transmitted per step is increased by one. Equation 1 allows us to achieve this goal. If
the value of IP G is updated once every T seconds and we choose the value of C to be equal to Tk , the
number of packets sent during each step is increased by k every step.
RAP uses value of one for k in order to emulate the TCP window adjustment mechanism in the steady
state. At each adjusting point, first the step length (i.e. the step timer) and the value of C are set to the
recent value of SRTT, and then equation 1 is used to update the value of IP G.
Since the length of each step is SRTT and the height of each step is inversely dependent on SRTT
(equation 2), the slope of the transmission rate is inversely related to SRT T
.
S l ope S tepH eig ht
S tepLeng th
SRT T
P ack etS iz e
C SRT T
C SRT T Slope P ack etS iz e
SRT T
(3)
TCP’s rate of increase of transmission rate is related to RTT in the same way in the steady state, and so
a RAP source can exploit RTT variations and adaptively adjust its transmission rate in the same manner
as TCP. The adaptive rate adjustment in RAP is meant to emulate the coarse-grain rate adjustment in
TCP. The step length in RAP is analogous to the time it takes for TCP to send a full window worth of
packets(i.e. one RTT).
RAP suffers from unfairness against TCP flows with longer RTT in the same way that inter-TCP
unfairness has frequently been reported[Flo91]. RAP connections with shorter RTT are more aggressive
and achieve a larger share of the bottleneck bandwidth. However, there are two issues to notice: 1. in
19
general, other measures of fairness can only be achieved by implementing the required machinery in
the network[She90]; 2. As long as the unfairness problem is not resolved along TCP flows, being TCP-
friendly implies accepting this unfairness. Due to lack of space, we have not discussed startup behavior
of the RAP protocol. Note that since RAP is designed for relatively long-lived sessions, its behavior
during the startup phase is not crucial.
4.2 Clustered Losses
In a shared best-effort networks with a high level of statistical multiplexing, the observed loss pattern has
a near random behavior[Bol93] that is determined by the aggregate traffic pattern. Thus it is generally
hard for an end system to predict or control the loss rate by adjusting the transmission rate. End systems
are expected to react to congestion on suitable time scales by dropping their rate exponentially. It takes
one RTT for end systems to detect and react to congestion. Thus an end-system only needs to react at
most once per RTT as long as it reacts sufficiently[MF97].
To achieve this, end systems require a mechanism by which they can identify a cluster of losses that
are potentially related to the same congestion. A simple approach is to ignore all losses that are detected
during the first RTT after a back-off.
RAP employs a slightly different approach. If a back-off occurs because packet Seq
F ir stLoss
was
lost, the RAP source looks at the sequence number of the last packet that has been transmitted, called
Seq
LastS ent
. The outstanding packets in the pipe right after the back-off have a sequence number, Seq,
in the range:
Seq
LastS ent
S eq S eq
F ir stLoss
This is one RTT of packets and we call this a cluster. Any packet in the cluster can be potentially
dropped due to the recent congestion event that was detected by the loss of Seq
F ir stLoss
. As the source
has already reacted to the congestion, loss of other packets from the cluster are silently ignored. This
cluster-loss-mode is triggered by a back-off and terminated as soon as an ACK with sequence number
greater or equal to Seq
LastS ent
is received. This mechanism is similar to that employed in TCP-Sack.
4.3 Fine grain rate adaptation
The AIMD rate adaptation algorithm does not necessarily result in TCP-friendly behavior scenarios
when TCP’s performance is degraded due to heavy load. Figure 3(a) depicts the instantaneous trans-
mission rate(i.e.
P k tS iz e
IP G
) of a single RAP flow without fine grain adaptation. Note that the rate is
monotonically increases until a loss occurs.
20
The motivation for fine grain rate adaptation is to make RAP more stable and responsive to tran-
sient congestion while still performing the AIMD algorithm at a coarser granularity. A fine-grain rate
adaptation has two components:
1. Fine-grain feedback: the feedback must only contain a recent fine-grain signal,
2. Fine-grain adjustment: the feedback must only cause minor adjustment in the transmission rate.
The long-term mean effect of such adjustment should be zero.
Therefore, we need a fine-grain signal that only captures short-term trends in congestion not captured
by packet losses. A short-term exponential moving average of the RTT signal exhibits such a behavior.
However, we require a dimension-less zero-mean feedback signal since it is independent of the con-
nection parameters and has a wider applicability. The ratio of the short-term to the long-term exponential
moving average
5
of the RTT signal has these desired properties. We have exploited the RTT signal and
devised a continuous
6
feedback function that is defined as, F eedback
i
FRTT
i
XRT T
i
, where FRTT
i
and
XRT T
i
are the value of short and long term exponential moving average of RTT samples at the ith
adjusting point respectively.
0
2
4
6
8
10
12
14
20 25 30 35 40
Transmission Rate (KB/s)
Time
Transmission Rate of A RAP source without Fine Grain Adaptaion
RAP Transmission Rate
Link Bandwidth
(a) Transmission rate of a single RAP flow without
fine grain rate adaptation(IP G
i
IP G
i
)
0
2
4
6
8
10
12
14
20 25 30 35 40
Transmission Rate (KB/s)
Time
Transmission Rate of A RAP source with per-ack Fine Grain Adaptaion
RAP Transmission Rate
Link Bandwidth
(b) Transmission rate of a single RAP flow with
per-ack fine grain rate adaptation(IP G
i
IP G
i
F eedback
i
)
Figure 3: Impact of fine grain rate adjustment on RAP’s instantaneous transmission rate (
P k tS iz e
IP G
)
The fine-grain rate adjustment deploys the fine-grain feedback to fine tune the transmission rate. At
each tuning point, the value of IP G
i
is modulated by the feedback signal and the resulting value, IP G
i
,
5
RT T
i
K RT T
i
K SampleRT T
6
Since our proposed feedback function is continuous, we do not need to deal with specifying thresholds which can be
problematic.
21
is used for the transmission timer. IP G
i
IP G
i
F eedback
i
(4)
The value of IP G is adjusted iteratively, as we explained earlier, and acts as a base transmission rate.
Thus, during one step the base transmission rate remains unchanged. However, the actual transmission
rate, IP G
, adaptively varies with the short-term congestion state. Note that the fine-grain feedback does
not have an accumulative effect.
Fine-grain rate adaptation could be performed at several granularities, although the feedback signal
and the rate adjustment must have the same granularity. We have simulated fine-grain adaptation for
two extreme granularities: adapting once per step , and adapting once per ACK. Although both schemes
result in improvements, we focused our attention on the per-ack scheme since it has finer granularity,
performs slightly better, and is intuitively closer to TCP. We plan to study coarser adaptation schemes
by relaxing some restrictions of the per-ack scheme in our future work.
Per-step fine-grain Rate Adaptation
This scheme has the granularity of one step. The fine-grain and coarse grain rate adjustment occur both
at the same time. At the beginning of the ith step, first IP G
i
is calculated based on the equation (1).
Then the updated value of the feedback is applied (using the equation 4) to obtain the value of IP G
i.
The rate remains unchanged for the entire step. The feedback must capture the trend in congestion that
was observed during the last step. Since number of sample RTTs (i.e. received ACK) during one step
varies, weight of exponential filters, K
XRT T
and K
FRTT
, must be accordingly adjusted for each step.
We have adjusted the weights based on the following observation; The initial value of the FRTT at the
beginning of each step contributes only %10 in the value of the FRTT at the end of that step whereas for
XRTT the initial value contributes %90 in the final value. It means:
K
FRTT
n
K
XRT T
n
Where n presents number of outstanding packets in the pipe that are expected to receive during the
current step. The idea in K
FRTT
adjustment is to give a higher weight to the recent RTT samples since
they are more correlated with recent congestion state.
Per-ack fine-grain Rate Adaptation
With per-ack adaptation, the arrival of a new ACK causes FRTT and XRTT to be updated, and so hence
a new value of IP G
is calculated and applied to the timer. Note that the transmission timer is already
running for the next transmission, and so the new IP G
will be used as the next inter-packet gap. Since
multiple ACKs may arrive before expiration of the transmission timer, the most recent value of IP G
is
22
used.
The weights for XRTT and FRTT exponential average filters must be chosen so that the feedback
signal captures the short-term congestion state since the last ACK. We have set the value of K
XRT T
and
K
FRTT
to 0.01 and 0.9 respectively. These weights are constant.
Figure 3(b) shows a single RAP flow with per-ack fine grain adaptation. It clearly illustrates that RAP
fine tunes its transmission while it follows the AIMD algorithm for coarse grain adaptation.
4.4 Self-limiting issues in RAP
There is a consensus about the inherent instability of rate-based control schemes. Although it is hard to
disprove, it has not been theoretically or experimentally proven[MK92].
Self-limiting behavior is the classic problem with rate-based schemes. In window-based schemes
the source stops once it has a full window worth of data on the fly. This property makes the window-
based schemes intrinsically stable. However, if the source allows retransmission beyond the current
window, the stability is lost, and so the number of retransmitted packets must also be limited. Rate-
based schemes need to find some variant analogous to the window to bound the volume of outstanding
data in the network[Jac97]. One way to achieve this goal is use of correctly implemented timers. In the
absence of any feedback, the expired timer forces a source to drop its rate.
RAP achieves self-limiting by using the timeout mechanism for loss detection as a variant of window.
In an extreme case when no ACK is received, the transmission rate drops to half once every RTT, until
the rate falls below the minimum short-term average rate that the application can tolerate. This worst
case scenario only happens if a connection fails (e.g. network is partitioned).
The fine-grain rate adaptation mechanism in RAP effectively strengthens the self-limiting property
in RAP and prevents the source from over running the network. During normal operation of a RAP
connection in a network with a large number of well-behaved flows, the departure rate of data packets and
arrival rate of ACKs are in balance. If the RTT suddenly increases, the arrival rate of ACKs decreases.
therefore the number of out-standing packets grows and the balance is lost. If a loss is detected, the rate
is dropped to half. Otherwise since the values of new RTT samples is growing, the fine-grain feedback
increases
7
the value of IP G
effectively and controls the transmission rate.
7
specially per-ack fine-grain feedback
23
4.5 Random Early Discard Gateways
There seems to be general agreement in the community on deploying Random Early Discard (RED)[FJ93]
gateways to improve both fairness and performance of TCP traffic. Among other benefits, RED queue
management attempts to achieve two goals; it tries to keep the average queue size low, and also by pre-
venting the buffer from filling, it accommodates bursts of packets. One of the main problems for TCP’s
congestion control is to recover from multiple losses within a window [FF96]. Multiple losses occurs
mainly due to buffer overflow in drop-tail queues. Ideally, RED should be configured such that each
flow experiences at most one single loss per RTT. Under these circumstances, TCP flows can efficiently
recover from a single loss without experiencing a transmission timeout or going through slow-start. In-
tuitively, as long as a RED gateway operates in its ideal region, RAP and TCP obtain an even share
of bandwidth since both use the AIMD algorithm. Nevertheless, if the average queue length exceeds
the maximum threshold, RED starts to drop packets with a very high probability. At this point, RAP
and TCP start to behave differently. When regular TCP experiences multiple losses within a window, it
undergoes a retransmission timeout and its congestion control diverges from the AIMD algorithm. RAP,
however, follows the AIMD algorithm as a basis for rate adjustment except for minor tunings due to the
fine grain rate adaptation. Regardless of the number of losses within one RTT, RAP reacts only once to
the first loss. We expect to observe substantial improvement in fairness by deploying RED even if it only
prevents the queue from overflowing. This behavior at least, limits the divergence of TCP’s congestion
control from the AIMD algorithm in bigger simulations.
Since RED parameters are closely dependent on the behavior of aggregate traffic, it is hard to keep
a RED gateway in its ideal region as the behavior of the aggregate traffic changes with time. Thus,
configuration of RED is still a research issue.
4.6 Startup Phase
The startup phase is the time between the transmission of the first packet until the first loss is experienced.
During the startup phase, a RAP source should increase its transmission rate rapidly to explore the
network for available bandwidth. RAP increases its transmission rate linearly but with higher slope than
normal linear increase phase. During the startup phase, RAP uses the following parameter:
C SRT T
N
N N is a configuration parameter and controls the rate of increase for number of transmitted packet per
step. As soon as the first loss is detected, the source switches to the normal mode of operation and N is
set to 1. One side effect of increasing the slope is overshooting beyond the available bandwidth which
24
may result in a burst of packet loss. During the startup phase, the fine-grain feedback mechanism is not
effective.
In the context of live applications, the source may switch to the startup phase during the life of the con-
nection whenever it remains idle for a relatively long period of time and loses the state of the connection.
It is crucial to optimized the startup phase for these applications.
RAP is meant for long-lived playback sessions. The source usually does not go ideal in these applica-
tions. Therefore, the duration of the startup phase does not have a major impact since it is experienced
only once and it is short in compare to the length of the session. Thus we have not investigated this
aspect of RAP’s behavior.
25
5 Simulation-based Evaluation
In this section we present our simulation results. Our main goal is to explore the properties of RAP,
namely TCP-friendliness, ability to cope with background TCP traffic, interaction with RED gateways
and the behavior of the fine-grain rate adaptation over a reasonable parameter space. We are particularly
interested in RAP’s large-scale behavior, i.e. the average performance during a session. Our simulations
demonstrate that RAP is TCP-friendly. However in a real network, we would have a larger number of
short and long-lived TCP flows with some bursty traffic. Currently, we are in the process of implement-
ing a RAP prototype for actual experiments over the Internet.
We have simulated RAP using the ns2 simulator [MF95], and compared it to TCP Tahoe, Reno,
NewReno [FF96] and Sack [MMFR96].
R3
R2
R1
T1
T2
T3
Tn
.
.
Rm
P1
P2
P3
Pm
S1
S2
S3
Sn
SW1 SW2
L
.
.
.
.
.
.
Figure 4: Simulation Topology
Figure 4 shows the topology of our simulations. The link between SW
and SW
is always the
bottleneck and SW
is the bottleneck point. All the other links have higher bandwidth and shorter
delay than the bottleneck. The switches implement FIFO scheduling and DropTail queuing except in
RED simulations. m RAP connections from sources R
i
...R
m
to receivers P
...P
m
share the bottleneck
bandwidth with n TCP flows from sources T
...T
n
to receivers S
...S
n
. Data and ACK packet sizes are
similar for RAP and TCP flows. For a fair comparison, all connections have equal end-to-end delay.
The total delay of the side links for each flow is fixed, but is randomly split between R
i
SW
and
SW
P
i
so that the flows are out of phase. The buffer size at SW
is four times the RTT-bandwidth
product of the bottleneck link, except where otherwise stated.
All simulations were run until they exhibited steady state behavior. All TCP flows are “FTP” sessions
with an infinite amount of data. The TCP receiver window is large enough that it is not a limitation. TCP
and RAP sources start in a random order with a uniform random delay between their start times. This
random delay lessens the resonation between sources and reduces the duration of the initial transition
26
phase. Simulation parameters are summarized in table 1.
Packet Size 100 Byte Tot. Delay of Side-Links 6ms
ACK Size 40 Byte TCP Timeout granularity 100 ms
Bottleneck Delay 20 ms TCP Maximum Window 1000
Bottleneck Buffers 4 * bottleneck B/W * RTT B/W per flow 5 KByte/s
B/W of Side Links 1.25 MByte/s Simulation Length 120 sec
Table 1
The average goodput for each flow is measured by the number of delivered packets
8
during the last
three quarters of the simulation time to ignore transient startup behavior. Since loss only occurs at the
bottleneck switch, the average goodput represents the average bottleneck bandwidth share for each flow.
Typically we will graph the mean, min and max bandwidths of the flows of each protocol to examine
fairness and identify phase effects.
We initially observed severe phase effect phenomena among in our simulations. Phase effects becomes
more pronounced as the number of flows and the amount of resources (i.e. buffer size and bandwidth)
increase. This occurs mainly because of Drop Tail gateways in our small deterministic network as was
reported in [FJ92]. Moreover, in our simulations all flows have the same packet size and observe similar
RTT, which increases the probability of phase effects.
To eliminate this problem without changing our parameters, we have added a small uniform random
delay before transmission of each TCP packet
9
. This delay ranges from zero to the bottleneck service
time and emulates the random packet-processing time of intermediate gateways [FJ92]. Obviously,
adding this random delay slightly decreases the transmission rate of TCP because it always delayed the
transmission. We have added similar randomness to RAP, not only to resolve the phase effect problem
between RAP flows, but also compensate for the added delay to TCP flows. Each RAP source adds a
random delay ranges from zero to the bottleneck service time, to the value of IP G
before scheduling
transmission of the next packet
10
. In a real network, adding randomness is more crucial for RAP than
TCP. Because of the ack-clocking and random change in RTT, TCP experiences some randomness.
Since RAP is not ack-clocked, the RAP source needs to slightly randomize the IP G
to resolve the
phase effect problem. Our fine-grain feedback seems to achieve this goal. Another solution to the phase
8
Here we implicitly assumed that number of duplicate packets a TCP flow receives is negligible and our experiments
confirm this assumption.
9
This is possible by using overhead configuration parameter of TCP agent in ns.
10
Although adding the randomness substantially lessens the phase effect problem, the problem still occurs occasionally in
big simulations.
27
effect problem is to replace the DropTail gateway with a RED gateway. We have explored this in our
simulations as we report later. We believe that the phase effect problem deserves more attention. We
plan to investigate the phase effect among RAP flows as well as phase effect between TCP and RAP
flows in our future work.
5.1 Evaluation Methodology
In an environment with large numbers of parameters, it is generally hard to isolate a particular variable
and study its relation with a particular parameter because of existing inter-dependency among variables.
In particular, TCP’s behavior changes drastically with configuration parameters and it has some inter-
nal constraints. Therefore, it is crucial to distinguish an effect that is caused by TCP’s performance
constraints from those phenomena that are due to coexisting RAP flows. During our simulations, we
attempted to minimize this problem by considering the following guidelines:
1. To identify the impact of TCP’s constraints from the inter-protocol dynamics on our results, we
have compared RAP with different flavors of TCP.
2. We limited the side-effect of resource (i.e. bottleneck bandwidth and buffer space) contention
by scaling up resources proportional to the number of flows so that the amount of resource share
per flow remains fixed across simulations. Since the bandwidth and the buffer size of the bottle-
neck link are scaled up with the same rate, the maximum queuing delay does not change across
simulations. The impact of resource contention is also studied separately.
3. We chose configuration parameters so that the TCP congestion window tends to be sufficiently
large that TCP remains in its well-behaved mode.
4. We have explored a reasonable portion of the parameter space to examine RAP’s behavior over a
wide range of circumstances.
5. As a baseline for comparison, we occasionally replaced all the RAP flows with TCP and ran the
same scenario without any RAP flow. We call this TCP base-case. The TCP base case may help
us to separate phenomenon that are related to TCP traffic.
5.2 Experiments and Results
We have conducted a large number of simulations to evaluate different aspects of the RAP protocol.
Each of these groups of simulations are presented in this section.
28
5.2.1 TCP-friendliness
The first set of simulations examines the TCP-friendliness of RAP without fine-grain rate adaptation.
Figure 5(a) shows the average bandwidth share of n RAP and n TCP Tahoe flows coexisting over the
0
2
4
6
8
10
12
14
0 20 40 60 80 100 120 140 160
Avg. BW Share (KB/s)
Total no of flows
Avg. BW share for TCP/Tahoe and RAP flows w/o f.g. feedback
Range of BW for RAP flows
Range of BW for TCP flows
RAP Avg. BW
TCP Avg. BW
(a) RAP coexisting with Tahoe
0
2
4
6
8
10
12
14
0 20 40 60 80 100 120 140 160
Avg. BW Share (KB/s)
Tot no of flows
Avg. BW share for TCP/Reno and RAP flows w/o f.g. feedback
Range of BW for RAP flows
Range of BW for TCP flows
RAP Avg. BW
TCP Avg. BW
(b) RAP coexisting with Reno
Figure 5: Comparison of RAP with TCP(Tahoe and Reno)
topology depicted in figure 4. The resources (i.e. the bottleneck bandwidth and the buffer size) are
scaled up linearly with the total number of flows. The range of the bandwidth share among RAP and
TCP flows are represented by vertical bars around the average value. This result implies that RAP is
not terribly TCP-friendly across all the simulations. The observed unfairness against TCP traffic can
be due to one of the following reasons: 1. TCP’s inherent performance limitations, 2. An artifact of
configuration parameters and 3. Unfairness imposed by coexisting RAP flows.
TCP suffers from some performance limitations[FF96, Mor97]. In particular, when TCP experiences
multiple losses within a window or the window is smaller than 4, it is constrained to either wait for
retransmission timeout or go through slow-start. As a result, TCP may temporarily lose its ack-clocking
and its congestion control mechanism diverges from the AIMD algorithm. The severity of the prob-
lem varies among different flavors of TCP and mainly depends on window size and loss pattern. TCP
Sack is able to recover from the multiple loss scenarios easier than other flavors of TCP whereas Reno’s
performance is substantially degraded [FF96]. Generally, TCP’s ability to efficiently recover from mul-
tiple loss increases with its window size. The more TCP diverges from the AIMD algorithm, the less
bandwidth it obtains and the lower performance it exhibits in large scale.
We exploited the difference among various TCP flavors to assess the impact of TCP’s performance
problem on the observed unfairness. We have repeated the same experiment with RAP against Reno,
29
NewReno
11
and Sack TCP. Results are shown in Figure 5(b), 6(a), 6(b) respectively. These results
confirm that the large-scale behavior of TCP traffic is in agreement with the behavior reported in[FF96]
. These experiments also reveal that TCP’s inherent performance problems partially contribute to un-
fairness to the TCP traffic. It is interesting to notice that the inter-protocol fairness remains unchanged
across all simulations
12
.
0
2
4
6
8
10
12
14
0 20 40 60 80 100 120 140 160
Avg. BW Share
Tot no of flows (KB/s)
Avg. BW share for TCP/NewReno and RAP flows w/o f.g. feedback
Range of BW for RAP flows
Range of BW for TCP flows
RAP Avg. BW
TCP Avg. BW
(a) RAP coexisting with NewReno
0
2
4
6
8
10
12
14
0 20 40 60 80 100 120 140 160
Avg. BW Share (KB/s)
Tot no of flows
Avg. BW share for TCP/Sack and RAP flows w/o f.g. feedback
Range of BW for RAP flows
Range of BW for TCP flows
RAP Avg. BW
TCP Avg. BW
(b) RAP coexisting with Sack
Figure 6: Comparison of RAP with TCP(NewReno and Sack)
We would like to limit the impact of the TCP’s performance problem and focus on the interaction
between RAP and TCP traffic. Therefore, we choose Sack as an ideal representative for TCP flows. For
the rest of this paper whenever we refer to TCP, we mean Sack TCP otherwise it is explicitly stated. To
attempt to ensure that we have not chosen an unrepresentative set of parameters, we have explored a wide
range of different values. Since we are unable to exhaustively examine the parameter space, we focus
our attention to those parameters that play key roles in protocols’ behavior. RTT and TCP’s congestion
window are particularly important. RTT is a crucial parameter because it affects rate-adjustment in both
RAP and TCP. TCP’s congestion window is a primary factor in the performance of the TCP protocol. We
introduce the term inter-protocol fairness ratio that is the ratio of the average RAP bandwidth calculated
across all the RAP flows over the average TCP bandwidth calculated across all the TCP flows. We
changed the delay of the bottleneck link to control the value of RTT, and the buffering was adjusted
accordingly. Figures 7(a) and 7(b) depict the fairness ratio as a function of the bottleneck link delay and
the total number of flows. Figure 7(b) provides the side view (from the delay axis) of figure 7(a) for
11
NewReno is a modified version of Reno TCP that avoids some of the Reno’s performance problems. For more details,
refer to [FF96]
12
Notice that some of our results seem to have minor phase effect, e.g. for 100 flows in figure 5(b).
30
easier comparison. Each data point is obtained from an experiment where half of the flows are RAP and
the other half are Sack TCP. The bottleneck resources (i.e. the buffer size and the bandwidth) are linearly
scaled up with the total number of flows and all other parameters are the same as previous simulations.
Fairness Ratio across the parameter space
Fairness Ratio
0
20
40
60
80
100
120
140
160
180
200
0
20
40
60
80
100
120
140
160
180
0
0.5
1
1.5
2
2.5
3
Bottleneck Delay (ms)
Total number of flows
Fairness Ratio
(a) Fairness Ratio across the parameter space
0
0.5
1
1.5
2
2.5
3
3.5
4
0 20 40 60 80 100 120 140 160 180
Fairness Ratio (RAP BW / Sack BW)
Total number of flows
Fairness Ratio for RAP without F.G Adaptaion against Sack
Delay 20ms
Delay 40ms
Delay 60ms
Delay 80ms
Delay 100ms
Delay 120ms
Delay 140ms
Delay 160ms
Delay 180ms
Delay 200ms
(b) Side view of the previous graph
Figure 7: Exploring the inter-protocol fairness across the parameter space
Figure 7(a) reveals several interesting trends in the fairness ratio as we traverse the parameter space.
Some of these trends are the following; 1. For a particular value of the bottleneck delay, increasing the
number of flows improves the fairness ratio except for the smallest value of delay(i.e. delay = 20ms) in
which the ratio never converges to one. This special case is in fact, the problem that we have observed
in the previous section. These figures illustrate that except for small simulations, RAP exhibits TCP-
friendly behavior. The different behavior in small simulations has to do with TCP’ burstiness and loss
pattern in these scenarios. We will address these problems shortly. 2. Excluding the simulations with
small bottleneck delay (i.e. Del ay ) as well as small simulations, the fairness ratio is mostly close
to one and is not a function of the bottleneck delay (i.e. RTT). The problem with short bottleneck delay
in small simulations has to do with the small size of TCP’s congestion window. In these scenarios, TCP
has a smaller congestion window and frequently experiences multiple loss due to the lack of statistical
multiplexing. As the bottleneck delay increases, both the bottleneck pipe size and the buffer size increase
13
. This allows TCP flows to have a larger number of packets on-the-fly and easily recover from multiple
losses.
13
Note that we keep the ratio of the buffer size to the pipe size fix for the bottleneck link across the simulations. Thus the
maximum queuing delay increases with the bottleneck delay. This in turn, could further increase the average RTT depends
on the behavior of the aggregate traffic.
31
Impact of TCP’s Congestion Window on Fairness
Fairness Ratio
0
20
40
60
80
100
0
5
10
15
20
25
30
0
2
4
6
Total number of flows
Mean No of outstanding TCP pkts
Fairness Ratio
Figure 8: Variation of the Fairness ratio with TCP’s congestion window
We conducted another set of simulations to observe the primary effect of TCP’s congestion window
on the fairness ratio. The congestion window is dependent on several parameters such as available
bandwidth per flow, buffer size, mean queue size, queue management scheme and number of flows. We
change the bottleneck bandwidth as a primary factor to control the value of congestion window. We
decided to measure the number of outstanding TCP packets per flow instead of congestion window for
two reasons. Firstly, TCP’s congestion window may not be full during the fast-recovery period. In
those cases, TCP’s behavior depends on the number of outstanding packets. Secondly, since RAP is not
a window-based mechanism, the number of packets on-the-fly seems to be the only common base of
comparison from the network’s point of view.
Figure 8 shows the variation of the fairness ratio as a function of the number of flows and the amount
of allocated bandwidth per flow. Since the number of outstanding packets is dependent on both variables,
we have used the mean number of outstanding packets (averaged across all the TCP flows in a simulation)
as the x coordinate for the corresponding data point instead of the amount of allocated bandwidth per
flow
14
. This graph clearly confirms our hypothesis that TCP’s performance is directly influenced by the
number of outstanding packets in transit. It shows that as the number of outstanding packet grows, the
fairness ratio enhances except for simulations with a small number of flows (n=1). Therefore, under a
heavy load, if the number of outstanding packets for a TCP flow drops below a threshold, its performance
is degraded substantially. Under these circumstances, RAP can easily utilize the available bandwidth
because it decouples congestion control from error control and only perform the former.
Figure 8 also implies that the number of coexisting flows does not have a visible impact on TCP’s
performance when resources scale appropriately, except for very small numbers of flows.
14
This slightly changes alignment of the graph.
32
After exploring the parameter space in several directions, our evidence implies that “the RAP protocol
behaves in a TCP-friendly fashion over a large portion of the space”. The observed unfairness against
TCP is mainly due to TCP’s inability to recover from multiple losses with small congestion window. In
other words, as long as TCP’s congestion control does not diverge from the AIMD algorithm, TCP and
RAP traffic evenly share the bandwidth. The critical case is mainly observed when 1. load is high and
TCP has only a small number of outstanding packets or 2. the number of flows is small and each one
experience burst of loss due to lack of sufficient statistical multiplexing. While TCP is forced to slow
down and perform error recovery, RAP simply abides by AIMD rules and is not sensitive to number
of losses per RTT due to its cluster loss mechanism. Therefore, RAP obtains a larger share of the
bandwidth.
For the rest of this section, first we examine the impact of the proposed fine-grain rate adaptation on
TCP-friendliness. Then we revisit the problem that we frequently observed with small simulations.
5.2.2 Fine-grain rate adaptation
As we theorized that one reason TCP performed poorly was that ACK-clocking provides a degree of con-
gestion avoidance which RAP did not duplicate, we investigated the effect of fine-grain rate adaptation
in RAP.
We simulated fine-grain rate adaptation with two granularities; adapting once per-step and adapting
once per-ack based on the RTT signal. Both performed acceptably, but per-ack adaptation is slightly
better, and so those results are presented here.
0
2
4
6
8
10
12
14
0 20 40 60 80 100 120 140 160
Avg. BW Share (KB/s)
Tot no of flows
Avg. BW share for TCP/Sack and RAP flows w per-ack f.g. feedback
Range of BW for RAP flows
Range of BW for TCP flows
RAP Avg. BW
TCP Avg. BW
Figure 9: RAP with per-ack fine grain rate adaptation against Sack
Figure 9 shows the average bottleneck bandwidth share for n Sack flows against n RAP flows with
per-step fine-grain rate adaptation and 20ms bottleneck link delay. This demonstrates consistent im-
33
provement in RAP’s behavior. Although the impact is marginal in small simulations, it becomes more
visible as the simulation size grows. We have conducted the same experiment with Tahoe, Reno and
NewReno. Our results show that fine-grain rate-adaptation improves the fairness particularly in large
simulations.
It is interesting to note that fine-grain rate-adaptation also enhances the intra-protocol fairness among
the RAP flows, i.e. vertical bars become smaller. This is probably because there are some vestiges of
phase effect, and fine-grain feedback helps to eliminate these.
Fairness Ratio across the parameter space with F.G. adaptation
Fairness Ratio
0
20
40
60
80
100
120
140
160
180
200
0
20
40
60
80
100
120
140
160
180
0
0.5
1
1.5
2
2.5
3
Bottleneck Delay (ms)
Total number of flows
Fairness Ratio
(a) Fairness ratio across the parameter space
0
0.5
1
1.5
2
2.5
3
3.5
4
0 20 40 60 80 100 120 140 160 180
Fairness Ratio (RAP BW/Sack BW)
Total number of flows
Fairness Ratio for RAP with F.G. Adaptaion against Sack
Delay 20ms
Delay 40ms
Delay 60ms
Delay 80ms
Delay 100ms
Delay 120ms
Delay 140ms
Delay 160ms
Delay 180ms
Delay 200ms
(b) Side view of the previous graph
Figure 10: Exploring the impact of fine-grain rate adaptation on fairness ratio
To increase our confidence in these results and evaluate the impact of the fine-grain rate-adaptation
mechanism over a wide range of parameters, we explored the parameter space further. Figure 10(a)
shows the fairness ratio as a function of the bottleneck link delay and the total number of coexisting
flows where half of the traffic consists of RAP flows. Comparison of figure 10(a) with figure 7(a) reveals
that the fine grain rate-adaptation only improves the fairness among connections with small RTT while
it does not affect other areas. Figure 10(b) provides the side view of figure 10(a) for easier comparison.
This result implies that as long as TCP flows do not diverge from the AIMD algorithm, the fairness
ratio is primarily determined by TCP’s behavior and the large-scale behavior remains intact. This is
indeed a desired property. However, for those scenarios where TCP traffic is vulnerable to multiple
losses and achieves a smaller share of the bandwidth, the fine grain rate adaptation enhances resolution
of rate adaptation for RAP flows by preventing them from overshooting the available bandwidth share.
This in turn, reduces the probability of experiencing multiple losses across all flows. Consequently, TCP
traffic obtains a fairer share of bandwidth.
34
5.2.3 Burstiness
We have observed two special cases where inter-protocol fairness was not achieved that have not been
addressed yet. These cases are discussed in this section separately.
The first special case occurs in simulations with a relatively small number of flows(i.e. 10 to 50) over
a bottleneck link with a small delay value(Figure 10(a)). Although this scenario is not usually exercised
over the Internet the problem still deserves attention since RAP might be deployed over an ISDN line
where it coexists with a small number of TCP flows.
0
2
4
6
8
10
12
14
0 20 40 60 80 100 120 140 160
Avg. BW Share
Tot no of flows
Avg. BW share for TCP/Sack and Bursty RAP flows w per-ack f.g. adaptation, BurstSize = 2
Range of BW for RAP flows
Range of BW for TCP flows
RAP Avg. BW
TCP Avg. BW
Figure 11: Bursty RAP with per-ack fine grain adaptation against Sack
The chief reason for the unfairness in these scenarios is TCP’s burstiness. Since TCP’s behavior
is more bursty than RAP, TCP has a greater probability of experiencing loss due to buffer overflow.
These losses tend to be bursty in small simulations because of the DropTail queues and lack of sufficient
statistical multiplexing. As we mentioned earlier, TCP’s congestion control algorithm can diverge from
AIMD as a result of a bursty loss, while RAP reacts to different loss patterns similarly because of the
cluster loss mode. Thus on long time-scales, RAP obtains more bandwidth than TCP.
To observe the effect of burstiness, we changed RAP to send a burst of size b packets every b IP G
seconds. Figure 11 shows the average bandwidth share of n TCP Sack coexisting with n bursty
RAP flows and 20ms bottleneck delay. The RAP flows perform fine-grain rate adaptation and the burst
size is 2 packets. The resources are linearly scaled up as the number of flows increases. This graph
demonstrates two points. Firstly, adding burstiness to RAP’s behavior increases the probability of loss
and experiencing back off. Secondly, as the simulation size grows and the level of statistical multiplexing
increases, TCP’s burstiness gradually disappears and TCP’s performance improves. However, RAP’s
burstiness does not depend on the observed level of statistical multiplexing, and so as we move toward
bigger simulations, TCP gradually outperforms this bursty RAP.
35
We have repeated the previous simulation for one TCP flow against one bursty RAP for different
burst sizes to assess the impact of burst size on performance. Figure 12(a) illustrates the average RAP
and TCP bandwidth for these scenarios as the burst size increases. This figure clearly illustrates that
burstiness is harmful to performance.
The second special case is observed when the fairness ratio among a very small number of flows
monotonically decreases as the bottleneck delay increases. This phenomenon was observed in figure 7(a)
and 10(a) in simulation with two flows(n = 1) as the bottleneck delay changes. We do not have enough
evidence to provide a solid explanation for this case. However, we speculate that this has to do with
impact of increasing buffer size on TCP’s behavior. In the context of small simulations, if the buffer size
at the bottleneck increases, TCP’s burst can be absorbed at the bottleneck. This leads to a higher share
of bandwidth for TCP.
2
3
4
5
6
7
8
1 1.5 2 2.5 3 3.5 4 4.5 5
Avg. BW Share (KB/s)
Burst Size
BW share for 1 TCP and 1 Bursty RAP flow
Avg. RAP BW
Avg. TCP BW
(a) Average bandwidth share for bursty RAP
against TCP
0
20
40
60
80
100
120
140
20 40 60 80 100 120 140 160 180 200
Mean no of outstanding packets
Bottleneck Delay (ms)
Impact of the Bottleneck Delay on no of outstanding packet
Mean no of outstanding pkt for TCP
Mean no of outstanding pkt for RAP
(b) Mean number of outstanding packets for one
RAP and one TCP
Figure 12: Impact of burstiness on RAP’s behavior against TCP
Figure 12(b) depicts the number of outstanding packets for RAP and TCP flows in the same set of
simulations. Notice that the buffer size linearly increases with the bottleneck delay
15
. This figure re-
veals that the number of outstanding packets for TCP rapidly increases with buffer size while it remains
unchanged for RAP. As the buffer size increases, TCP manages to rapidly inflate its window and obtain
a bigger share of the buffer. RAP is not as sensitive to buffer size as TCP because of its smooth trans-
mission. Thus it simply operates with the left over portion of the bottleneck buffer. This special case
requires further investigation.
15
Because we scale up the buffer size proportionally. It remains four times of the bottleneck pipe across our simulations
36
5.2.4 RED Gateways
The main challenge here was to configure the RED gateway so that it behaves uniformly across all sim-
ulations. As we mentioned earlier, RED’s performance closely depends on behavior of the aggregate
traffic. Since this behavior could change with the number of flows, it is hard to obtain the same per-
formance over a wide range without reconfiguring the gateway. Table 2 summarizes our configuration
parameters:
Bottleneck BW 5KByte/s * No. of flows Min. Threshold 5 packets
Bottleneck Delay 20 ms Max. Threshold 0.5 * Buffer
Buffer Size 12 * RTT * bottleneck b/w q weight 0.002
Table 2
Half of the traffic consists of RAP flows with fine grain adaptation. We have provided sufficient buffer
at the bottleneck point to eliminate any buffer overflow.
0.5
1
1.5
2
2.5
3
0 20 40 60 80 100 120 140
Fairness Ratio
Total number of flows
Variation of fairness Ratio with max. drop probability
max_p is 0.005
max_p is 0.01
max_p is 0.02
max_p is 0.04
max_p is 0.08
max_p is 0.16
Figure 13: Impact of RED on the fairness
Figure 13 shows the fairness ratio for different value of max
p
(maximum probability of loss) as the
numbers of flows changes. This graph clearly illustrates three interesting points:
1. The behavior of the aggregate traffic is substantially different in small simulations.
2. Except for small simulations, the fairness ratio does not change with simulation size.
3. There exists a range for max
p
where RAP and TCP evenly share the bottleneck bandwidth.
Figure 13 demonstrates that RED is able to evenly distribute the losses across the flows and avoid buffer
overflow over a wide range. Thus RED has eliminated the unfairness caused by TCPs burstiness. The
higher the value of max p, the more likely RED is to drop a packet when the buffer is less full, and so
37
the lower the mean buffer utilization is. Figure 8 has already shown that TCP performs poorly with small
values of cwnd, and higher values of max p tend to reduce TCP’s mean cwnd. RAP takes advantage of
this, and a degree of unfairness results.
0
0.1
0.2
0.3
0.4
0.5
0.6
0 20 40 60 80 100 120
RTT (sec)
Time (sec)
RTT for 1 RAP and 1 TCP, max_p = 0.005
Sample RTT
Max_thresh
(a) RTT for 1 RAP, 1 TCP and max
p
=
0.005
0
0.1
0.2
0.3
0.4
0.5
0.6
0 20 40 60 80 100 120
RTT (sec)
Time (sec)
RTT for 1 RAP and 1 TCP, max_p = 0.16
Sample RTT
Max_thresh
(b) RTT for 1 RAP, 1 TCP and max
p
=
0.16
Figure 14: Impact of max
p
on variation of queue length
As long as the average queue size remains in RED’s operating region (below max
th
), the bandwidth
share between RED and TCP is quite fair. However, if the value of max
p
is too small, the average queue
size reaches max
th
, and RED then starts dropping all packets until the average queue size decreases
below max
th
again. This process repeats and oscillations occur, with the loss probability alternating
between max
p
and one. RED should not be operated in this region, and the curve in figure 13 shows
this effect when max
p
. The differences between RAP and TCP are due to TCP’s burstiness
interacting with periodic oscillations of the average queue size about max
th
. With small simulations, the
oscillation period is long, and both TCP and RAP lose whole RTT worth of packets, but TCP takes a very
long time to recover, while RAP recovers comparatively easily. With large simulations, the period of
these oscillations is much shorter, and although a few TCP’s may lose, on average a TCP is less likely to
be hit by one of the loss periods than a RAP flow which spaces its packets out evenly. Hence, on average
TCP performs better than RAP. It should be emphasized that this RED regime will impose terrible loss
bursts on real-time flows, and should be avoided at all costs. Figures 14(a), 14(b) graph the measured
RTT for small simulations, and demonstrate these oscillations in figure 14(a) with max
p
versus
normal RED behavior in figure 14(b) with max
p
.
We conclude that with appropriate tuning, RED can help significantly with the fairness between RAP
and TCP, but that aggressively pushing for very low buffer utilization is counter-productive when RAP
and TCP share a link because TCP then diverges from AIMD.
38
5.2.5 Intra-protocol Behavior
Figure 15 shows the average bandwidth share among n RAP flows as we increase the number of flows
while the amount of resources remain fixed. The bottleneck bandwidth is 250 KByte/s and the bottleneck
buffer size is four times the bandwidth-bottleneck RTT product. The rest of the parameters are similar
to table 1.
0
20
40
60
80
100
120
0 20 40 60 80 100 120 140 160
Avg. BW Share (KB/s)
Total no of flows
Avg. BW share among RAP flows
Range of BW for RAP flows
RAP Avg. BW
Figure 15: Intra-protocol fairness among RAP flows
We performed a number of such simulations to show that RAP flows divide the bandwidth fairly for
a wide range of network loads. Note that we have already covered the impact of load on inter-protocol
fairness in Figure 8.
5.2.6 Smoothness of RAP transmission rate
Figures 16(a) and 16(b) show the variation of goodput for a sample TCP Tahoe
16
and RAP flow
respectively from a simulation with 32 RAP and 32 TCP. Although the transmission rate of TCP flows
has higher variance than that of RAP flows, both have the similar mean values.
Thus RAP satisfies our design goal of providing a smoother, more predictable congestion-control
16
TCP Sack is smoother than Tahoe, so this illustrates near worst case behavior by TCP
39
0
50000
100000
150000
200000
250000
300000
0 10 20 30 40 50 60
Transmission Rate(Bytes/sec)
Time
Transmitted Rate of a TCP flow
"504.txr-tcp1"
(a) Transmission rate of a sample TCP flow
0
50000
100000
150000
200000
250000
300000
0 10 20 30 40 50 60
Transmission Rate(Bytes/sec)
Time
Transmitted Rate of a RAP flow
"504.txr-rap1"
(b) Transmission rate of a sample RAP flow
Figure 16: Comparing the smoothness of transmission rate between RAP and TCP
signal to our real-time applications than TCP does.
40
6 A Framework for Layered Transmission
6.1 Introduction
Our proposed architecture has three major components: RAP, Error control, and Quality adaptation. We
have discussed the RAP module in previous sections. This section addresses some of the design issues in
Error control and Quality adaptation(figure1) to identify some of the open issues and serve as a guideline
for our future work.
The majority of realtime applications have a constant generation(or consumption) rate on average.
Thus, there is a mis-match between rate variation that is caused by RAP and the constant-bit-rate nature
of realtime applications. To absorb the variation in transmission rate the buffer manager at the client
side buffers a limited amount of data. We focus on long-lived sessions since one can always pre-fetch a
short-lived stream, using TCP, and then play it back from a local file.
We assume that the client has a limited amount of buffer space. Thus using RAP for delivery of
a long stream may lead to accumulation of data and buffer overflow at the client side, if the average
transmission rate(i.e. available bandwidth) is higher than average consumption rate.
6.1.1 A Simple Approach
Figure 17 shows how the RAP source and client’s buffer manager can interact to resolve the prob-
lem. The RAP source adjusts its transmission rate in the AIMD fashion until the client’s buffer space
is exhausted. Then the client’s buffer manager signals the RAP source to drop its rate to aggregate
consumption rate(t
t
)
17
. The RAP source continues transmission with the same rate until a loss is
detected(t
) or the client’s buffer manager asks the RAP source to speed up(t
). In the former case, loss
is considered a signal of congestion. Thus RAP source backs off and continues in the AIMD mode until
the client’s buffer manager signals to slow down(t
). The client’s buffer manager may signal the RAP
source to return to the AIMD mode to speed up. This scenario could occur if the client’s clock is faster
than the server resulting in a higher consumption rate. This scheme does not utilize the fair share of
bandwidth.
6.1.2 Proposed Approach: Quality Adaptation
Another way to use RAP is to adjust the quality of the delivered stream with long-term variation of
transmission rate. We call this Quality adaptation. In the context of realtime applications, Quality
17
Aggregate consumption rate consists of consumption rate plus retransmission rate
41
t
2
t
3
t
4
t
5
t
6
t
1
R
c
R
r
+
time
Transmission rate
R
min
R
c
t
7
t
8
Buffered data
Consumed data
Figure 17: Rate adjustment in RAP for receiver with limited buffer
adaptation is in fact another dimension of error control. To further explain this distinction, figure 18
illustrates dimensions for quality adaptation and error control. Quality adaptation occurs along the
transmission rate(i.e. bandwidth) axes whereas any type of error control occurs along the time axes.
In other words, quality can be adjusted by the volume of transmitted data at any point of time, whereas
time is required to detect a loss and recover (retransmission or added redundancy).
Error control has been studied by many researchers[Per98]. It addresses different ways to deal with
packet losses, for example adding some redundancy to the stream(e.g. FEC) or retransmission of a lost
packet. In contrast quality control deals with adjusting quality of the transmitted stream. It is important
to notice that aggregate transmission rate for both error control and quality adaptation must be fit within
the regulated rate by RAP. Furthermore, both mechanisms closely interact with each other as well as
with the client’s buffer manager. Their overall behavior determines the quality of the delivered stream
to the client.
6.1.3 Architecture Overview
Figure 19 illustrates the proposed architecture. The client buffers some data by delaying the playback
point of the first packet. The buffered data is used to absorb both the variation in RAP’s transmission
rate and the jitter that is introduced by the network. As the client learns more about the connection
throughout the session, it may slowly adjust the amount of buffered data to cope with variation of rate
more efficiently. The client always tries to keep data packets in the memory but once allocated memory
42
Retransmission
Time
Retransmission
Error Control
Quality
Adaptation
Lost
Lost
Lost
Figure 18: Error Control vs Quality Control
Buffer Manager
RAP
Source
Error
Control
Receiver
Internet
Server
Client
Quality
Adaptation
Storage
Transmission
Buffer
RAP
Sink
Buffer
Manager
Decoder
Playout
Buffer
Archive
Figure 19: End-to-end architecture for playback applications
43
is exhausted, it can also store packets on disk and pre-fetch them before their playback times.
At the server side, the buffer manager pre-fetch the data before its transmission time and caches the
data in memory until there is no need for its retransmission. Retransmission is controlled by the error
control module that is informed about losses by the RAP module. Furthermore, the quality adaptation
module monitors transmission rate of the RAP module and adjust the quality of the streams.
For the rest of this section, we briefly describe some implications of retransmission-based error re-
covery on the client’s buffer manager. Then focus on quality adaptation and try to identify some of the
design issues and establish some guidelines for our future work.
6.2 Error Control
We assume a retransmission-based error recovery mechanism. However other repair schemes can be
deployed as well. If the time that a packet waits at the client’s buffer is longer than one RTT and extra
bandwidth is available, retransmission is a feasible option. Retransmission can be initiated by either
server or client. The server knows about the importance of a particular packet(data) (e.g. I frame vs
B frame in a MPEG encoded video) and this information may not be available at the client. However
only the client has precise information about the actual amount of buffered data. Thus the server has
no knowledge about volume of the buffered data except if it is continuously reported by the client. As
a result, the server and the client should both participate in error recovery. The client could piggyback
the current playback time with ACK packets. This scheme should work well in high transmission rate.
In low transmission rate, the client may send a separated request for retransmission. In this case, the
request packets must perform congestion control as well. Since the server knows about the playout time
for each sample, it can easily estimate the amount of buffered data at the client.
Buf t
p
l ast sent
t
p
l ast pl ay ed
RT T
C
t
p
last sent
: Playout time of the last transmitted packet
t
p
last pl ay ed
: Playout time of the last played packet from the last ACK
C: Consumption rate
When the server detects a packet loss and there is enough time to retransmit that lost packet (know-
ing the current playout time), the server can decide to retransmit the lost packet.
44
6.3 Quality Adaptation
There are three ways to adjust the quality of a stored stream.
1. Adaptive encoding.
2. Switching between multiple encodings.
3. Using a layered encoded stream
We discuss each one of these schemes separately. In adaptive encoding approach, the server encodes
each stream on-the-fly and adjust its quantization parameters based on the feedback that is provided
by RAP. This approach is not desirable since a server with large number of clients can not afford the
encoding load because encoding is computationally expensive. Moreover, it is hard to do adaptive
encoding for a large range of transmission rates which is a requirement for a server in the Internet.
In the second approach, the server maintains several encoded versions of a single stream with differ-
ent qualities(i.e. resolutions). The higher the quality of encoding is, the higher the transmission rate
becomes. For example, a single audio clip can be encoded with four different qualities and the output
file with the best quality is the largest in size.
The idea is that as the available bandwidth changes, source switches from one stream to another
to adapt the delivered quality with available bandwidth. As the available bandwidth increases, source
switches to streams with higher quality and vice versa. In the worst case, if the bandwidth is not even
enough to deliver the stream with the lowest quality, the session is terminated. For smooth transition at
the switching points, a cross-index among streams is required. Moreover, in the context of video, since
a block of frames are encoded together, the switch point must be placed only at the boundary of blocks.
This implicitly means that granularity of switching is at most one block. The main issue is to design
a switching mechanism that jumps from one stream to another as bandwidth changes. We address this
problem in the layer approach.
The larger the number of files with closer resolution, the smoother transition as bandwidth changes
and the higher storage requirement. The main disadvantage of this approach is that the server must
maintain multiple copies of each stream. This may consume large amount of storage space specially
when a server is expected to maintain large number of streams that are large in size. This motivates the
layered approach.
The third approach is to playback layered-encoded stream. The idea of the layered approach is to add
new layers as more bandwidth becomes available and drop some of the higher layers when congestion
occurs. There is a duality between the layered approach and multiple encoded stream approach, i.e. add
45
and drop mechanism in the former scheme is analogous to switching to higher or lower quality in the
latter. Thus, our add and drop mechanism should be equally applicable to the multiple encoded stream
approach as well.
First we focus on quality adaptation mechanism and discuss its interaction with other modules later.
6.4 The Layered Approach
We have adopted the layered approach for quality adaptation. For the rest of this paper we assume
linearly layered encoded streams just to simplify the problem, i.e. all layers have the same bandwidth(C).
Applying our result to exponentially layered encoded streams only adds another level of complexity.
Since all layers have the same bandwidth, the visual improvement caused by each layer decreases as we
go to higher layer. Intuitively, there is a minimum amount of continuous buffered data, called a unit of
display, that is worth displaying. Any amount of buffered data that is less than a unit is not displayed to
ensure that resulting improvement in quality would last for a reasonable period of time. This prevents
rapid variation in quality of the delivered stream. Size of the unit varies among layers. We also assume
that the maximum number of layers is known a priori.
The main issue in the layered approach is the add and drop mechanism. Figure 20 illustrates a naive
approach for add and drop. Server starts transmitting the base layer (i.e. l
) and transmission rate is
regulated by RAP. Once transmission rate reaches the consumption rate, transmission of the second
layer is initiated and so on. Upon detecting a congestion, half of the layers are dropped. Display of a
layer can be started whenever there is enough data buffered at the client and average arrival rate is higher
or equal to the consumption rate. For example, the client can start displaying l
, l
, l
and l
at time t
,
t
, t
and t
respectively.
This approach has several problems; the first problem has to do with the dropping policy. Dropping
half of the layers after each back off can cause oscillation in the delivered quality, which is annoying
for the user and must be avoided. Furthermore, this is the most conservative mechanism for dropping,
i.e. it does not take advantage of the buffered data at all. The buffered data can be used to cope with
variation in transmission rate. Figure 21 shows another approach for inter-layer bandwidth allocations
after the back off(i.e. from t
to t
) that evenly shares the available bandwidth among all layers. Notice
that the cumulative amount of buffered data for all layers was enough to feed the client after the back off.
The second problem is related to the adding policy. Since congestion may occur at a random time,
the transmission rate may back off shortly after adding a new layer. If the volume of received data for
a layer is less than a unit, the client does not initiate display of that layer and the utilized bandwidth for
46
L
4
L
3
L
2
L
1
L
0
Time
Transmission Rate
C
2C
3C
4C
t
1
t
2
t
3
t
4
Time
Time
Time
Time
Trans. Rate Trans. Rate Trans. Rate Trans. Rate
L
0
L
1
L
2
L
3
t
1
t
2
t
3
t
4
t
5 t 6
t
5 t 6
Figure 20: A naive approach to Add & Drop
47
L
4
L
3
L
2
L
1
L
0
Time
Transmission Rate
C
2C
3C
4C
t
1
t
2
t
3
t
4
Time
Time
Time
Time
Trans. Rate Trans. Rate Trans. Rate Trans. Rate
L
0
L
1
L
2
L
3
t
5
t
6
t
1
t
2
t
3
t
4
t
5
t
6
Figure 21: Even share of bandwidth among layers after back off
48
delivery of that layer is wasted. This scenario is experienced by l
in figure 20 if the amount of stored
data between t
and t
does not suffice to initiate l
. The amount of minimum data for display depends
on the coding scheme and consumption rate of a layer. To reduce the probability of transmitting data
that will not be displayed, the server can probe the bandwidth before initiating a new layer. This idea is
further described in following sections.
6.4.1 Desired Properties
We are looking for an Add and Drop policy with the following properties:
1. Adding policy: Avoid adding a layer when it can not be delivered for a reasonable period of
time. The quality adaptation module should ensure the availability of sufficient bandwidth before
initiating a new layer. Moreover, adding a new layer should not endanger the lower layers.
2. Dropping policy: The server should try to deliver the maximum number of layers that fits in the
average bandwidth.
3. Buffering policy: Generally, layer i is more important than layer j, thus the server should buffer
more data for layer i than j to achieve greater protection against rate variations. In particular, the
base layer must be treated differently since the quality of stream is most heavily dependent on the
base layer.
The basic idea is to buffer some data for each layer and use that to recover from congestion. When
congestion occurs (i.e. loss detected), transmission rate is reduced exponentially(i.e. back off). Since
the bandwidth does not suffice to feed all the layers, the buffered data for each layer is used to until
transmission(or arrival) rate of each layer reaches its consumption rate hopefully before the buffer goes
dry. Moreover, when the buffered data for a layer is used, the server must allocate some bandwidth
(besides the consumption rate) for that layer to refill its buffer in preparation of some back offs that will
occur in the future. The time period between a back off and the time when buffers are refilled after
the back off, is called recovery period(figure 22). Those layers that are kept after a backoff are called
surviving layers.
For a simple periodic loss pattern, we can calculate the buffer requirement for each layer assuming
an even inter-layer bandwidth allocation. Figure 22 illustrates the variation of transmission rate for each
layer in the parallel recovery with even bandwidth allocation among layers. For a periodic loss pattern,
transmission oscillates around the average value. The average slope of increase is measured throughout
the session and averaged out using an exponential moving average filter. Note that the average slope of
49
increase for the aggregate transmission rate must be evenly divided among layers. Given this informa-
tion, the server can simply estimates the minimum required amount of data per layer as followings:
C: Consumption/Generation rate for each layer
K: Back off factor
R: Average slope of linear increase
n: Number of surviving layers after a back off
Buf
c
: Minimum amount of required buffer data for recovery
Buf
c
n C R K nC
R
K K (1)
2KC/(K+1)
2C/(K+1)
C
Buf
c
R/n
RecoveryPeriod
Bufferd
Data
Used
Buffer
Refilled
Figure 22: Calculation of a lower bound for buffer requirement
Equation (1) reveals that the minimum buffer requirement for parallel scheme is a linear function of
the number of surviving layers.
The more bandwidth is allocated for a layer, the faster it recovers from a back off. If a layer runs out
of buffer before it finishes the recovery, the display will be terminated and it is automatically dropped.
The server should ensure that layers are dropped in a reverse order because of coding requirements (i.e.
decoder can only decode n consecutive layers starting from the base layer).
There is a tradeoff here. The server can keep more layers, assuming that the available bandwidth
(after a back off) and the buffered data for all the remaining layers are enough to recover before another
congestion event. This implicitly means that the server does not expect another back off before the
end of recovery period. If another back off occurs before the end of recovery period, those layers that
do not have enough data will be dropped. As a result, allocated bandwidth for the recovery of the
50
dropped layers is wasted. A more conservative approach is to drop some layers as soon as back off
occurs to allocate more bandwidth for recovery of the remaining layers, reducing the length of recovery
period and increasing the probability of successful recovery. Clearly, dropping layers frequently is not
appropriate since it causes oscillation in the quality, and consequently does not result in the optimal
deliverable quality.
To summarize, we have to answer the following questions
1. How should the bandwidth be allocated across surviving layers after a back off to ensure smooth
and successful recovery? (i.e. Buffering policy)
2. When should the server add a new layer? (i.e. Adding policy)
3. How many layers should be dropped after a back off? (i.e. Dropping policy)
These questions are inter-related, meaning that answer to one depends on the answer to others; Not only
add and drop policies are related, but also they are both related to the buffer management scheme and
inter-layer bandwidth allocation. We present some strategies for our future work in the next subsection.
6.4.2 Strategies
Adaptive mechanisms
As we pointed out earlier, the most challenging aspect of quality adaptation is predicting upcoming
congestion which has a random nature. In a shared network with large number of flows, behavior of the
aggregate traffic changes fairly smoothly. Thus the recent past could be a reasonable prediction of near
future. This is the philosophy in the majority of measurement-based or adaptive approaches.
The server can measure the average behavior of the connection throughout the session. As it learns
more about the connection, it can adaptively adjust its parameters. These adjustments occur in a longer
time scale than rate adaptation and are based on long term trend rather than transient changes in network
traffic. The server will always consider a moving average history of the network’s variation to capture a
long-term trend in the network’s behavior. The major issue is to find the appropriate weight for averaging
that result in a smooth behavior without oscillation.
One may study the behavior of the network traffic to extract some patterns and design its add and
drop schemes based on the observed patterns. There are couple of problems with this approach. Many
of the studies on the Internet traffic are out dated. Since the emergence of the Web, the behavior of the
network traffic has change substantially. Web applications have a combination of short and long term
51
transmissions(known as mice and elephants respectively). This impacts the behavior of the aggregate
traffic because the large portion of today’s Internet load consist of the Web traffic.
Inter-layer Bandwidth Allocation
We have described the impact of inter-layer rate adaptation on the duration of recovery period. Here we
try to discuss some of the extreme cases to identify border lines of the design space.
Assuming the number of surviving layers are known after a back off, the following questions must be
answered in turn:
1. How many of the surviving layers will participate in recovery?
Two extreme cases are possible; 1. the serve may drop half of the layers and the transmission of the
surviving layers remain intact and no recovery is needed. This approach does not take advantage
of buffered data at all. This scheme is shown in figure 20. 2. All the surviving layers participate
in recovery. Since all layers buffer some data at the client side, this scheme results in an optimal
use of buffering if back off does not occur earlier than it is expected. Thus we follow the second
scheme and assume all surviving layers will participate in the recovery.
2. How much bandwidth should be allocated for each one of the surviving layers?
As we mentioned earlier, one of our goal is to have more buffered data for lower layers. Given
that, if the server allocates more bandwidth for lower layers, higher layers should consume more
from buffered data while they have less amount of buffering. Allocating more bandwidth for
higher layers also seems inappropriate because they consume their buffers faster and lose their
protections(i.e. buffering) against an unexpected back off. Thus, even allocation of the aggregate
bandwidth across the surviving flow seems the most reasonable scheme. The even allocation of
bandwidth across layers during recovery period can be performed within boundary of two extreme
cases:
(a) Sequential recovery
(b) Parallel recovery
In the sequential recovery, the server allocates all the extra bandwidth for recovery of a single
layer, starting from the base layer, until its recovery is completed, i.e. transmission of the layer
reaches to the consumption rate and goes beyond that to refill the buffers for next back off. Once
the recovery period of layer i is completed, its transmission rate goes back to its consumption rate
and the extra bandwidth is exclusively utilized for recovery of layer i . This approach minimizes
52
Aggregate Tx Rate. Tx Rate, L0 Tx Rate, L1 Tx Rate, L2 Tx Rate Li (Parallel)
C
C
C
C
3C
Sequential Recovery
Parallel Recovery
Consumed Data From Buffer
Refilling Buffer
Recovery Period
Time
Time
Time
Time
Time
Figure 23: Comparison of sequential and parallel recovery
53
the amount of wasted bandwidth for delivery of small chunks of data for the highest layer that are
delivered but are not large enough to be displayed, i.e. less than a unit. Since we recover a lower
layer before start a higher one, if an unexpected congestion occurs, only the layer that is being
recovered might be affected. All the higher layers that have not been recovered yet are dropped
automatically since we didn’t get a chance to send any data for them. The main draw back of this
scheme is that amount of buffered data for participating layers in recovery has a skewed pattern.
This implies that larger amount of storage is required at the client.
In the parallel recovery, all layers go through recovery simultaneously. Thus the server allocates
an even portion of the available bandwidth for each layer. The last graph at the bottom of figure 23
shows rate adaptation for a single layer in the parallel recovery. Note that transmission of all layers
have the same variation. The advantage of this approach is that the amount of buffered data for all
layers remain close to each other. Thus the volume of aggregate buffered data for all layers is less
than the sequential scheme.
In summary, a parallel recovery scheme with even share of bandwidth among all the surviving layers
seems the most reasonable approach.
Adding Mechanism
Intuitively, a conservative adding scheme should minimize the oscillation in quality. Assuming a peri-
odic loss pattern with n layers, this implies:
n C BW
av g
n C, BW
extr a
= BW
av g
n C
This means that BW
extr a
is not enough to add a new layer(i.e. layer n ). Instead, the extra bandwidth
is used for retransmission of potential losses after refilling buffers for all layers. Since required amount
of buffered data increases linearly with the number of layers(i.e. equation 1), the server must ensure
that current n layers have enough data to recover from a back off with n layers before initiating
transmission of a new layer. Based on equation (1), if the number of layers is increased by one, the
minimum required buffering is increased as followings:
Buf
c
n Buf
c
n C
R
K K (2)
Where Buf
c
n is defined in equation(1). During linear increase, the server allocates an even share
54
of the available bandwidth for each layer. If no back off occurs, first all layers will reach to their con-
sumption rates. Then all buffers will be filled up to the level that is required for parallel recovery with n
layers (i.e. the level they had before the back off). At this point if available bandwidth is still increasing,
this extra bandwidth is evenly used by all layers to further increase their buffers as preparation for adding
a new layer.
To add a new layer, two conditions must be met:
1. All the current layers should evenly increase their buffers until each layer has enough data to
recover with n layers, i.e. Buf
c
n CKR .
2. The average bandwidth satisfy the following condition:
BW
av g
n C
Once both conditions are met, the server can initiate transmission of a new layer. If the new layer can
buffer enough data to recover from a back off, it will be kept. This approach increases the probability of
successful adding.
Dropping Mechanism
Dropping usually occurs after a back off. In a simple scenario, when average bandwidth remains un-
changed, the server backs off periodically. Having an estimate for the time when next back off occurs
and the average slope of increase based on the measurement of the recent history as well as amount of
buffering for each layer, the server will run the following algorithm as a primary dropping scheme to
decide how many layers can survive.
i = No of lay er
While (Buf
i
(R eq uir ed R ecov er y B uf f er W ith i S ur v iv ing Lay er s))
D r op Lay er i
i i In this algorithm, the server checks the amount of buffered data for each layer, starting from the highest
layer. If layer i does not have enough buffered data to consume until its transmission rate reaches to its
consumption rate(i.e. half way through the next estimated back off), it i is dropped. The next iteration
of the algorithm is done with a smaller i. Note that as the number of surviving layers decreases in subse-
quent iterations , the minimum buffering requirement decreases (since each layer obtains higher portion
55
of bandwidth). Once layer i satisfies the condition, all the lower layers below will be automatically
qualify.
This scheme relies on estimation of the time of the next back off. If the back off occurs earlier than it
is expected or the average bandwidth decreases layers can not fill up their buffers in time. Under these
circumstances, the server needs to invoke a secondary dropping mechanism. This is a subject of our
future work.
Buffering Mechanism
The Adding and Dropping mechanism assume that the following condition is always held for every i
and j:
Buf
i
Buf
j
,iff i j (3)
Moreover, the adding mechanism ensures that each layer would have enough buffer to recover with
an extra layer before the new layer is added.
All layers require the same amount of buffered data during the recovery period. The amount of
buffered data is required for the recovery of a single layer is called recovery buffer(Buf
rec
) which is a
linear function of the number of surviving layers. To ensure that lower layers have higher protections,
we assume that the amount of the required buffering per layer can be estimated as following:
Buf
i
= n Buf
rec
+ Buf
pr otect
i
where Buf
pr otect
i
is the protection buffer for layer i such that it satisfies equation (3). Thus we need:
Buf
pr otect
i
Buf
pr otect
j
iff i j
During normal operation, the recovery buffer is consumed and refilled periodically. If there is a mi-
nor variation in average bandwidth, the client may use a little bit of the protection buffer too. The
protection buffer will be used in critical situation when an unexpected back off occurs and provide some
room for the server to adjust the number of layers to the new situation. The server requires to refill the
protection buffers of current layers before adding a new layer.
The larger the protection buffer is, the less interactive the session becomes, the bigger buffer (at the
client) is required. At the other end, small protection buffers are not able to protect against a critical
56
event. Amount of the protection buffer for each layer can be fixed or adaptive. For the fixed case,
it can be enough data to feed the display for t seconds and t is a configuration parameter. Whereas
for the adaptive case, the amount of the protection buffer is adaptively adjusted with the behavior of
the connection so that it would be able to handle large portion of potential unexpected events. These
adjustments have longer time scale than rate variations.
Retransmission
Retransmission seems a feasible approach for repair because of the sufficient buffering at the client
side. Retransmission has a time window and must be completed before playback time of the lost packet
reaches. The server relies on the moving average measured bandwidth and moving average retrans-
mission rate to allocate a portion of the bandwidth for selective retransmission. Thus the server can
retransmit a lost packet even during the recovery period. If there is not sufficient bandwidth for retrans-
mission, the server needs to be selective and set up some priorities. We consider the following rules for
(re)transmission:
1. It is generally prefered to use extra bandwidth to retransmit losses rather than adding a new layer.
2. Retransmission of some packets is more important than others (e.g. I frame in MPEG coded
stream), thus the server should assign them a higher priority.
3. Retransmission of a packet from layer i has a higher priority than a packet from higher layers and
a lower priority than a packet from lower layers.
6.5 Open issues
In this section, we only identified some of the design issues for a layered framework and sketched some
guidelines for add and drop schemes that works with a simple periodic loss pattern. In a real network,
loss pattern varies with time. Thus, the server should monitor the pattern and predict the near future
based on the recent past. Once the time for the upcoming congestion is estimated, the server invokes the
primary mechanism that we described in the previous section to drop those layers that can not recover.
If an unexpected back off occurs, the server should invoke a secondary mechanism for add and drop and
try to adjust its behavior promptly.
Toward this end, the following issues are still open and needs to be studies:
1. What is the time scale for averaging and adaptation?
57
2. What is the appropriate moving average algorithm that captures long-term trend of network’s
behavior without oscillating?
3. How does the server predict network’s behavior from the moving averaged history?
4. When does a server invokes a secondary mechanism for dropping?
5. What is the secondary mechanism for dropping?
These issues are part of our future work as we describe in the next section.
58
7 Contribution and Future Work
The goal of our proposed architecture is to make realtime playback applications good network citizens.
We attempt to provide a practical solution for a large group of playback applications to enable large scale
deployment of these applications in the Internet in the absence of reservation or differentiated services.
This document presents our initial work to achieve this goal. In this section we present our contribu-
tions and describe our proposed future work
7.1 Contributions
As part of our initial attempt to design and evaluate an architecture for realtime playback applications,
we have:
Designed and developed the rate adaptation protocol(RAP) to be well-behaved and achieve inter-
protocol fairness in general and exhibit TCP-friendly behavior in particular. We have only em-
ulated those mechanisms from TCP’s congestion control that are known as strength of TCP and
avoid from those issues that might cause performance problems.
Devised and evaluated a fine grain rate adaptation mechanism to emulate TCP’s ack-clocking
property and improve stability of the protocol. The fine grain adaptation mechanism only fine
tune the rate that is coarsely adjusted by the RAP protocol.
Presented a methodology for simulations to limit the inter-dependency among different variables.
This methodology allows us to recognize an effect that is caused by TCP’s performance problem
from those phenomena that are due to coexisting with RAP flows.
Evaluated various aspects of the RAP protocol to examine its interaction with TCP through exten-
sive simulations. Although achieving TCP-friendliness over a wide range of network parameters
is extremely challenging, RAP reasonably fulfill this goal. We have conducted detailed simulation
and explore a large portion of parameter space to ensure that an observed behavior is not an artifact
of simulation parameters. Our results show that the fine grain adaptation extends inter-protocol
fairness to a wider range. Divergence of TCP’s congestion control from the AIMD algorithm is
often the main cause for the imposed unfairness against TCP in special cases. This problem is
pronounced more clearly with Reno and Tahoe while it has a more limited impact on Sack. Reno
and Tahoe while it has a more limited impact on Sack. Toward this end, we observed that the big-
ger TCP’s congestion window is, the closer it follows the AIMD algorithm. We have studied the
59
impact of burstiness on fairness. Our results reveals that protocol’s burstiness affects the obtained
bandwidth but the effect varies among different simulations. We have examined the interaction
between RAP and RED gateways in the presence of TCP traffic. We found out that RED gateways
with a proper configuration, can result in an ideal inter-protocol sharing.
Introduced Quality adaptation as a new dimension for error control in the context of realtime
applications. We also sketched a layered frame work for quality adaptation and identified some of
the design issues as a foundation for our future work.
7.2 Remaining Thesis
Our future work focuses on three remaining components of our proposed architecture; 1. Quality adap-
tation, 2. Error control, 3. Buffering Mechanism
7.2.1 Quality Adaptation
We plan to study different aspects of quality adaptation through simulations using ns. We will particu-
larly study the following issues:
Adding scheme: We plan to further investigate on a probing approach that not only refills buffers
of the lower layers but also explores the availability of bandwidth before adding a new layer. We
also plan to quantify the buffer requirement for the probing approach.
Dropping scheme: We plan to further study the primary mechanism for dropping and its inter-
action with inter-layer bandwidth allocation. Furthermore, we will investigate on the necessity
of a secondary mechanism for dropping to be invoked after an unexpected congestion event and
potential adjustments on inter-layer bandwidth allocations in these scenarios.
In addition, we will examine different moving average algorithms for measurement of network
behavior to capture long-term trend in aggregate traffic and predict the time for an upcoming
congestion.
Another task is to specify the appropriate time scale for measurements and adjustments. We
also plan to study the behavior of the network traffic to extract some statistics and pattern. This
information will lead us to identify the appropriate time scale for the adjustments as well as typical
unexpected congestion events as the worst case scenario to target for recovery. Furthermore, we
will use this information to specify when the secondary mechanism must be invoked and how it
60
must react. In this context, we will address the stability and smoothness of add and drop scheme
with different averaging technics.
As a starting point, we use a random function as a predictor for next congestion. The result with
the random predictor will serve as a lower bound for efficiency. We also will calculate an optimal
add and drop plan for a given loss pattern by post processing. The optimal scenario specifies an
upper bound for efficiency. We can evaluate our add and drop schemes from its distance to the
optimal solution.
Inter-layer Bandwidth Allocation: Our investigations on drop scheme would affect the inter-layer
bandwidth allocation scheme. Particularly, a hybrid approach of parallel and sequential recovery
is investigated. For example, the base layer may utilize the entire bandwidth to recover as in
sequential recovery and then the rest of the layers recover in a parallel fashion. This approach
provides a higher level of protection for the base layer. We will also investigate buffer requirement
for different recovery mechanisms.
7.2.2 Error Control
Our future work on error control will mainly focus on retransmission. We will quantify buffer require-
ment for retransmission as well as bandwidth allocation and timing issues for retransmission in our sim-
ulations. We will study other repair approach for error recovery as a potential complementary schemes
for retransmission.
7.2.3 Buffering Scheme
The buffering scheme is mainly influenced by the add scheme. We plan to evaluate buffer requirement
for protection of lower layers and its implications on add and drop schemes. We will also investigate
adaptive buffering schemes where the amount of protection buffers will adaptively adjusted with network
behavior.
7.3 Expected Contribution
Evaluation of the overall behavior of the proposed end-to-end architecture for realtime applications
over the best effort network.
Design and evaluation of Quality adaptation. More specifically, design and evaluation of add and
drop schemes that leads to a smooth display of realtime stream.
61
– Interaction between Error control and Quality adaptation mechanisms within the regulated
rate by the RAP module.
– Quantifying buffer requirements for add, drop schemes as well as protections
References
[Bol93] J. C. Bolot. Characterizing end-to-end packet delay and loss in the internet. Journal of
High Speed Networks, 2(3):289–298, September 1993.
[BP95] L. Brakmo and L. Peterson. TCP Vegas: End to end congestion avoidance on a global
internet. IEEE Journal of Selected Areas in Communication, 13(8):1465–1480, October
1995.
[CJ89] D. Chiu and R. Jain. Analysis of the increase and decrease algorithm for congestion
avoidance in computer networks. Journal of Computer Networks and ISDN, 17(1):1–14,
June 1989.
[CLZ88] D. D. Clark, M. L. Lambert, and L. Zhang. NETBLT: A high throughput transport protocol.
ACM SIGCOMM, August 1988.
[CPW98] S. Cen, C. Pu, and J. Walpole. Flow and congestion control for internet streaming applica-
tions. Proceedings Multimedia Computing and Networking, January 1998.
[CSZ92] D. D. Clark, S. Shenker, and L. Zhang. Supportng realtime applications in an integrated
service packet network: Architecture and mechanism. ACM SIGCOMM, pages 14–26,
August 1992.
[CTCL95] Z. Chen, S-M Tan, R. H. Campbell, and Y . Li. Real time video and audio in the world wide
web. Fourth International World Wide Web Conference, December 1995.
[CY90] D. E. Comer and R. S. Yavatkar. A rate-based congestion avoidance and control scheme
for packet switched networks. Proc. of the ICDCS, IEEE, 1990.
[Dab92] W. Dabbous. Analysis of delayed-based congestion avoidance algorithm. proc. 4th IFIP
Conf. on High Performance Newtworking, December 1992.
[FF96] K. Fall and S. Floyd. Simulation-based comparison of tahoe, reno and sack TCP. Computer
Communication Review, 26(3):5–21, July 1996.
62
[FF98] S. Floyd and K. Fall. Promoting the use of end-to-end congestion control in the internet.
Under submission, February 1998. http://www-nrg.ee.lbl.gov/floyd/papers.html/end2end-
paper.html.
[FJ92] S. Floyd and V . Jacobson. On traffic phase effect in packet-switched gateways. Internet-
working: Research and Experiences, 3(3):115–156, September 1992.
[FJ93] S. Floyd and V . Jacobson. Random early detection gateways for congestion avoidance.
IEEE/ACM Transactions on Networking, 1(4):397–413, August 1993.
[Flo91] S. Floyd. Connections with multiple congested gateways in packet-switched networks.
Computer Communication Review, 21(5):30–47, October 1991.
[HS96] D. Hoffman and M. Speer. Hierarchical video distribution over iternet-style network. Pro-
ceedings of the IEEE International Conference on Image Processing, pages 5–8, Septem-
ber 1996.
[Inca] Microsoft Inc. Netshow service, streaming media for business.
http://www.microsoft.com/NTServer/Basics/NetShowServices.
[Incb] Vxtreme Inc. Vxtreme streaming video player. http://www.vxtreme.com.
[Jac88] V . Jacobson. Congestion avoidance and control. In ACM SIGCOMM, pages 314–329.
ACM, August 1988.
[Jac97] V . Jacobson. Email on the end-to-end email list. February 1997.
[Jai89] R. Jain. A delay-based approach for congestion avoidance in interconnected heterogeneous
computer networks. ACM Computer Communication Review, 19(5):56–71, October 1989.
[JE97] S. Jacobs and A. Eleftheriadis. Real-time dynamic rate shaping and control for internet
video applications. Workshop on Multimedia Signal Processing, pages 23–25, June 1997.
[Kes91] S. Keshav. A control-theoretic approach to congestion control. ACM SIGCOMM, pages
3–16, September 1991.
[MF95] S. McCanne and S. Floyd. Ns (network simulator). 1995. http://www-
mash.cs.berkeley.edu/ns.
63
[MF97] J. Mahdavi and S. Floyd. TCP-friendly unicast rate-based flow con-
trol. Technical note sent to the end2end-interest mailing list, January 1997.
http://www.psc.edu/networking/papers/tcp-friendly.html.
[MK92] P. Mishra and H. Kanakia. A hop-by-hop rate-based congestion control scheme. ACM
SIGCOMM, 1992.
[MMFR96] M . Mathis, S. Mahdavi, S. Floyd, and A. Romanow. TCP selective acknowledgement
options. RFC 2018, April 1996.
[Mon83] W. Montgomery. Techniques for packet voice synchronization. IEEE Journal of Selected
Areas in Communication, (1):1022–1028, December 1983.
[Mor97] R. Morris. TCP behavior with many flows. IEEE Int’l Conf. on Network Protocols, October
1997.
[MSMO97] M. Mathis, J. Semke, J. Mahdavi, and T. Ott. The macroscopic behavior of the TCP con-
gestion avoidance algorithm. Computer Communication Review, 27(3), July 1997.
[Net] Progressive Networks. Http versus realaudio client-server streaming.
http://www.realaudio.com/help/content/http-vs-ra.html.
[OK95] A. Ortega and M. Khansari. Rate control for video coding over variable bit rate channels
with applications to wireless transmission. Proceedings of the 2nd IEEE International
Conference on Image Processing (ICIP), October 1995.
[Per98] C. Perkins. Options for repair of streaming mdeia. Internet Draft, March 1998.
[PFTK98] J. Padhye, V . Firoiu, D. Towsley, and J. Kurose. Modeling TCP throughput: a simple model
and its empirical validation. ACM SIGCOMM, September 1998.
[She90] S. Shenker. A theoretical analysis of feedback flow control. In ACM SIGCOMM, pages
156–165, September 1990.
[TZ98] W. Tan and A. Zakhor. Error resilient packet video for the internet. Submitted to Proceed-
ings of the IEEE International Conference on Image Processing (ICIP), May 1998.
64

Linked assets

Computer Science Technical Report Archive

Conceptually similar

PDF

USC Computer Science Technical Reports, no. 686 (1998)

PDF

USC Computer Science Technical Reports, no. 718 (1999)

PDF

USC Computer Science Technical Reports, no. 679 (1998)

PDF

USC Computer Science Technical Reports, no. 700 (1999)

PDF

USC Computer Science Technical Reports, no. 693 (1999)

PDF

USC Computer Science Technical Reports, no. 709 (1999)

PDF

USC Computer Science Technical Reports, no. 725 (2000)

PDF

USC Computer Science Technical Reports, no. 628 (1996)

PDF

USC Computer Science Technical Reports, no. 672 (1998)

PDF

USC Computer Science Technical Reports, no. 678 (1998)

PDF

USC Computer Science Technical Reports, no. 717 (1999)

PDF

USC Computer Science Technical Reports, no. 704 (1999)

PDF

USC Computer Science Technical Reports, no. 495 (1991)

PDF

USC Computer Science Technical Reports, no. 588 (1994)

PDF

USC Computer Science Technical Reports, no. 661 (1997)

PDF

USC Computer Science Technical Reports, no. 822 (2004)

PDF

USC Computer Science Technical Reports, no. 609 (1995)

PDF

USC Computer Science Technical Reports, no. 594 (1994)

PDF

USC Computer Science Technical Reports, no. 642 (1996)

PDF

USC Computer Science Technical Reports, no. 811 (2003)

Description

Reza Rejaie. "An end-to-end TCP-friendly architecture for realtime playback applications over the internet." Computer Science Technical Reports (Los Angeles, California, USA: University of Southern California. Department of Computer Science) no. 681 (1998).

Asset Metadata

Creator Rejaie, Reza (author)

Core Title USC Computer Science Technical Reports, no. 681 (1998)

Alternative Title An end-to-end TCP-friendly architecture for realtime playback applications over the internet (title)

Publisher Department of Computer Science,USC Viterbi School of Engineering, University of Southern California, 3650 McClintock Avenue, Los Angeles, California, 90089, USA (publisher)

Tag OAI-PMH Harvest

Format 64 pages (extent), technical reports (aat)

Language English

Unique identifier UC16270839

Identifier 98-681 An End-to-End TCP-friendly Architecture for Realtime Playback Applications over the Internet (filename)

Legacy Identifier usc-cstr-98-681

Format 64 pages (extent),technical reports (aat)

Rights Department of Computer Science (University of Southern California) and the author(s).

Internet Media Type application/pdf

Source 20180426-rozan-cstechreports-shoaf (batch), Computer Science Technical Report Archive (collection), University of Southern California. Department of Computer Science. Technical Reports (series)

Access Conditions The author(s) retain rights to their work according to U.S. copyright law. Electronic access is being provided by the USC Libraries, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.

Repository Name USC Viterbi School of Engineering Department of Computer Science

Repository Location Department of Computer Science. USC Viterbi School of Engineering. Los Angeles\, CA\, 90089

Repository Email csdept@usc.edu

Inherited Values

Title Computer Science Technical Report Archive

Description Archive of computer science technical reports published by the USC Department of Computer Science from 1991 - 2017.

Coverage Temporal 1991/2017

Repository Email csdept@usc.edu

Repository Name USC Viterbi School of Engineering Department of Computer Science

Repository Location Department of Computer Science. USC Viterbi School of Engineering. Los Angeles\, CA\, 90089

Publisher Department of Computer Science,USC Viterbi School of Engineering, University of Southern California, 3650 McClintock Avenue, Los Angeles, California, 90089, USA (publisher)