University of Southern California Dissertations and Theses

Backpressure delay enhancement for encounter-based mobile networks while sustaining throughput optimality

(USC Thesis Other)

Backpressure delay enhancement for encounter-based mobile networks while sustaining throughput optimality

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content BACKPRESSURE DELAYENHANCEMENTFORENCOUNTER-BASED MOBILE
NETWORKSWHILESUSTAININGTHROUGHPUTOPTIMALITY
by
MajedAlresaini
ADissertationPresented tothe
FACULTYOFTHEUSCGRADUATESCHOOL
UNIVERSITYOFSOUTHERNCALIFORNIA
InPartialFulfillmentofthe
RequirementsfortheDegree
DOCTOROFPHILOSOPHY
(COMPUTERENGINEERING)
August2012
Copyright 2012 MajedAlresaini
Dedication
To my loving parents, my loving wife Amjad, my sweet daugh-
tersJenanandNorah,andmy sweetsonAbdullah.
ii
Acknowledgments
I would like to express my deepest gratitude and thanks to my advisor, Prof. Bhaskar
Krishnamachari without whose guidance I would not be able to accomplish most of the
work in this dissertation. Bhaskar is truly a great mentor whom I learned from not only
top most helpful research skills but also how I could be a better person in general. His
valuabletimehegivestomefordiscussionsandadvicesaregreatlyappreciatedandhad
inspiredmealottothinkdeeptocomeupwithideasthatsolvegivenresearchproblems.
IwouldlikealsotoexpressmydeepestgratitudeandthankstoProf. MichaelJ.Neely
whoseworkinstochasticnetworkoptimizationinspiredmealottoworkinbackpressure
whichcaughtmymindandheartandfeltgreatdesiretoenhanceinthisfieldthatIreally
love. Thereisagreatjoytoworkonwhatyoulove. Mikeguidedmegreatlyondefining
and solvinginteresting research problems. Ilearned from Mikethat simplicity,ultimate
accuracy, and goingonestep at atimeare thebest toolstoadvancein complexresearch
problems. I appreciate the valuabletime, discussion,and encouraging Mikeprovidesto
mewithoutwhichIwouldnotbeabletocomeupwithmostworkinthisdissertation.
I would like also to thanks Prof. Konstantinos Psounis who guided me through the
interesting subject of routing in intermittently connected mobile networks. My deepest
iii
thanks also dedicated to Prof. Leana Golubchik and Prof. Cauligi S. Raghavendra for
importantfeedbacks theyprovidetometoenhancetheworkinthisdissertation.
ThroughoutmyPh.DstudiesatUniversityofSouthernCalifornia(USC), Iconsider
myselfsoluckytobesurroundedbythebrightestmindsfromallovertheworld. Iwould
like to thank all of my friends at USC for the great life I had during this time. I would
likeespeciallytothankdeeplymycolleaguesinAutonomousNetworksResearchGroup
(ANRG) whom I consider as a family to me. I am going to miss each and every one of
them. Their helps, supports, and discussions have great impact on me to fight on and
succeedduringmyPh.D.studies.
Finally,I wouldliketo thankdeeply mywifeand thegreat motherofmy threekids,
Amjadwhosesacrificeisgreatlyappreciatedandwillneverbeforgotten. Amjad,without
yourendlessloveandsupport,IwouldnotbeabletosucceedinmyPh.D.studies.
iv
TableofContents
Dedication ii
Acknowledgments iii
ListofFigures vii
Abstract x
Chapter1: Introduction 1
Chapter2: Background 7
2.1 StochasticNetworkOptimization . . . . . . . . . . . . . . . . . . . . . 7
2.2 RoutinginIntermittentlyConnectedMobileNetworks(ICMN) . . . . . 11
2.3 StabilityExample . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3.1 PrioritizingTransmissionstoDestinations . . . . . . . . . . . . 15
2.3.2 OptimalStrategy . . . . . . . . . . . . . . . . . . . . . . . . . 16
Chapter3: BackpressurewithAdaptiveRedundancy 17
3.1 TraditionalBackpressureSchedulingandRouting . . . . . . . . . . . . 17
3.1.1 TraditionalBackpressureExample . . . . . . . . . . . . . . . . 20
3.2 BWARSchedulingandRouting . . . . . . . . . . . . . . . . . . . . . 23
3.2.1 DestinationAdvantageExampleinBWAR . . . . . . . . . . . 26
3.2.2 DuplicationExampleinBWAR . . . . . . . . . . . . . . . . . 28
3.2.3 DuplicateTransmissionExampleinBWAR . . . . . . . . . . . 31
3.2.4 FullDuplicateBufferExampleinBWAR . . . . . . . . . . . . 34
3.3 BackpressureRoutinginICMN . . . . . . . . . . . . . . . . . . . . . 36
Chapter4: AnalysisReviewofBasicBackpressure 38
Chapter5: AnalysisofBWAR 42
v
Chapter6: BWAREnhancements 50
6.1 BWAR-ID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
6.1.1 DuplicationExampleinBWAR-ID . . . . . . . . . . . . . . . 51
6.1.2 FullDuplicateBufferatReceiverExampleinBWAR-ID . . . . 54
6.2 BWAR-TD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
6.2.1 DuplicationExampleinBWAR-TD . . . . . . . . . . . . . . . 57
6.2.2 UnacknowledgedFlaggedPacketinBWAR-TD . . . . . . . . . 58
6.2.3 UnacknowledgedFlaggedPacketExampleinBWAR-TD . . . . 59
6.3 BWAR-ID-E andBWAR-TD-E . . . . . . . . . . . . . . . . . . . . . . 61
Chapter7: Model-basedSimulations 65
7.1 TheCell-PartitionedModel . . . . . . . . . . . . . . . . . . . . . . . . 65
7.2 ProtocolVariants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
7.3 I.I.DMobility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
7.4 RandomWalkMobility . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Chapter8: RealTraceSimulations 80
Chapter9: RelatedWork 87
Chapter10:ConclusionsandFutureWork 91
References 93
vi
ListofFigures
1.1 ICMNexamplewherenodesaresparseandmobile. Alinkbetweentwo
nodesappearsiftheyareintheradiorangeofeach other. . . . . . . . . 2
2.1 Generaldiagramofstochasticnetworks. . . . . . . . . . . . . . . . . . 9
2.2 ICMN example in which there is no complete path between node 1 and
node9ataparticulartime. . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 Stabilityexampleofacell-partitionedICMNinwhichprioritizingtrans-
missionstodestinationisnotnecessarilythebestapproach. . . . . . . . 14
3.1 A backpressure example at time t showing how node i maintains N
queues, one for each destination. Here, Q
1
i
(t) = 3, Q
2
i
(t) = 2, and
Q
N
i
(t) = 4. Queueiatnodeiisalwaysempty. . . . . . . . . . . . . . 18
3.2 An illustration of how traditional backpressure works. Here, the green
commodity, which is destined to node 4, will be routed from node 2 to
node3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3 A BWAR example at time t showing how node i maintains N queues
and N duplicate buffers. Here, Q
1
i
(t) = 3, D
1
i
(t) = 1, Q
2
i
(t) = 2,
D
2
i
(t) = 2, Q
N
i
(t) = 4, and D
N
i
(t) = 1. Queue i at node i is always
empty. Similarly,Duplicatebufferiatnodeiisalwaysempty. . . . . . 23
3.4 A BWAR example showing how it gives advantage to packets that en-
countertheirdestinationwhentherearetiesinqueuedifferentials. . . . 26
3.5 ABWARexampleshowinghowBWARcreates duplicates. . . . . . . . 29
3.6 ABWARexampleshowinghowduplicatesaretransmittedinBWAR. . 32
vii
3.7 A BWAR example showing how full duplicate buffers prevent creating
duplicates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
6.1 AnexampleshowinghowBWAR-ID creates duplicates. . . . . . . . . 52
6.2 ABWAR-IDexampleshowinghowfullduplicatebuffersatreceiverpre-
ventcreatingduplicates. . . . . . . . . . . . . . . . . . . . . . . . . . 54
6.3 AnexampleshowinghowBWAR-TDflagsoriginalpacketswhendupli-
cated. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6.4 A BWAR-TD exampleshowinghowaflagged packet movedtooriginal
queueafterthetime-outP whengetsnegativeacknowledgement(NAK)
bydestination. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
6.5 A BWAR-ID-E example showing how an original packet is duplicated
and how duplicates are distributed to guarantee number of duplicates
doesnotexceedconstantLusingcopycount. . . . . . . . . . . . . . . . 62
6.6 ABWAR-ID-Eexampleshowinghowduplicatesaretransmittedanddis-
tributedusingcopycount. . . . . . . . . . . . . . . . . . . . . . . . . . 63
6.7 ABWAR-ID-E exampleshowinghowaduplicatewithcopycount=1 is
notallowedtobeduplicatednortransmitted. . . . . . . . . . . . . . . . 64
7.1 DelayaswevaryN forlowλ = 0.001underi.i.dmobility . . . . . . . 70
7.2 DelayaswevaryλforN = 44ofbackpressurevariantsunderi.i.dmobility 71
7.3 ComparingS&WdelaywithBWAR-IDandBWAR-TDunderi.i.d. mo-
bilityaswevaryλforN = 44 . . . . . . . . . . . . . . . . . . . . . . 72
7.4 Comparing energy consumption as we vary λ for N = 44 under the
cell-partitionedmodelwithi.i.d. mobility. . . . . . . . . . . . . . . . . 72
7.5 Timeout effect of BWAR-TD and comparing it with BWAR-ID for dif-
ferent λ ∈ {0.001,0.016,0.064,0.128} for N = 44 under the cell-
partitionedmodelwithi.i.d. mobility. . . . . . . . . . . . . . . . . . . 73
7.6 Delay as we varyλ forN = 44 of backpressure variants under random
walkmobility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
viii
7.7 Comparing S&W delay with BWAR-ID and BWAR-TD under random
walkmobilityaswevaryλforN = 44 . . . . . . . . . . . . . . . . . 75
7.8 Comparing energy consumption as we vary λ for N = 44 under the
cell-partitionedmodelwithrandomwalkmobility. . . . . . . . . . . . 76
7.9 Showinghowenergy-efficientBWAR-ID-EandBWAR-TD-Ehaveclose
consumptionperformancetoS&Wunderrandomwalkmobility. . . . . 77
7.10 Showinghowenergy-efficientBWAR-ID-EandBWAR-TD-Ehaveclose
delayandthroughputperformancetoBWAR-IDandBWAR-TD,respec-
tively,underrandomwalkmobility. . . . . . . . . . . . . . . . . . . . 78
8.1 Channelstatesshowingaveragecontactdurationbetweeneachtwotaxis
intermsofsecondsperslot. . . . . . . . . . . . . . . . . . . . . . . . 81
8.2 Totalnumberofcontactsforeach timeslottduringtheday. . . . . . . 82
8.3 Delay as we vary loadλ underreal mobilitytraces for RB-DA, BWAR-
ID,andBWAR-TD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
8.4 Comparing S&F delay with BWAR-ID and BWAR-TD under real mo-
bilitytracesaswevaryloadλ. . . . . . . . . . . . . . . . . . . . . . . 85
ix
Abstract
Backpressure scheduling and routing, in which packets are preferentially transmitted
over links with high queue differentials, offers the promise of throughput-optimal op-
eration for a wide range of communication networks. However, when the traffic load
is low, due to the corresponding low queue occupancy, backpressure scheduling/routing
experiences long delays. This is particularly of concern in intermittent encounter-based
mobile networks which are already delay-limited due to the sparse and highly dynamic
network connectivity. While state of the art mechanisms for such networks have pro-
posed theuseof redundanttransmissionsto improvedelay, theydo not work well when
the traffic load is high. We propose in this dissertation a novel hybrid approach that we
refer to as backpressure with adaptive redundancy (BWAR), which provides the best of
bothworlds. Thisapproachishighlyrobustanddistributedanddoesnotrequireanyprior
knowledgeofnetworkloadconditions. WepresentanenhancedvariantofBWARsothat
duplicatesareremovedbasedondistributed,easy-to-implement,time-outmechanismin
ordertoobtainclosedelayperformancecomparedtoidealremovalofdeliveredpackets.
In addition, we introducean energy optimizedvariant of BWAR whileat the same time
maintainingthegreatdelayandthroughputperformanceofBWAR. WeevaluateBWAR
x
through both mathematical analysis and simulations based on a cell-partitioned model
and real traces of taxis in Beijing. We prove theoretically that BWAR does not perform
worse than traditional backpressure in terms of the maximum throughput, while yield-
ing a better delay bound. The simulations confirm that BWAR outperforms traditional
backpressureatlowload,whileoutperformingstateoftheartencounter-routingschemes
(Spray &WaitandSpray &Focus)athighload.
xi
Chapter1
Introduction
Queue-differential backpressure scheduling and routing was shown by Tassiulas and
Ephremides to be throughput optimal in terms of being able to stabilize the network
underany feasible trafficrate vector[55]. Additionalresearch has extended theoriginal
result to show that backpressure techniques can be combined with utility optimization,
resulting in simple, throughput-optimal, cross-layer network protocols [3,8,11,13,15,
21,22,24,25,30,31,37–44,54,57,58]. Recently, some of these techniques have been
translatedtopracticallyimplementedroutingandrate-controlprotocolsforwirelessnet-
works[16,27,28,46,52,53,61].
The basic idea of backpressure mechanisms is to prioritize transmissions over links
that have the highest queue differentials. Backpressure effectively makes packets flow
through the network as though pulled by gravity towards the destination, which has the
smallestqueue occupancy of 0. Underhigh traffic conditions,this works very well, and
backpressure is ableto fully utilizethe availablenetwork resources in a highlydynamic
fashion. Underlowtrafficconditions,however,becausemanyothernodesmayalsohave
1
a small or 0 queue size, there is inefficiency in terms of an increase in delay, as packets
maylooportakealongtimetomaketheirwaytothedestination.
Figure 1.1: ICMN example where nodes are sparse and mobile. A link between two
nodesappearsiftheyareintheradiorangeofeachother.
In thisdissertation,wefocus primarilyon intermittentlyconnected mobilenetworks
(ICMN), such as encounter-based mobilenetworks (sometimesalsoreferred to as delay
or disruptiontolerant networks (DTN)). In such networks, conventionalpath-discovery-
based MANET routing techniques like AODV [45] and DSR [20] are not feasible be-
causethenetwork maynotform asingleconnected partitionat anytime,and thusa full
path may never exist between the source and the destination (see Figure 1.1). Instead,
it is necessary to use store-and-forward type protocols that can handle the underlying
2
mobility. An important application of ICMN/DTN is a disasterrecovery network to de-
liverimportantmessagestosavelivesandrespondtoemergences. Abackpressurebased
routing schemecan be easily implemented in such a network, with the decision ofwhat
information to exchange being made between each pair of nodes based on their queue
differentials whenever they encounter each other. However, the above-mentioned delay
inefficiencyofthebackpressuremechanismatlowtraffic loadsisfurtherexacerbated in
such networks, because they are already delay-limiteddueto sparse network connectiv-
ity.
In the literature on intermittently connected networks, there are several proposed
schemes for store-and-forward based routing, such as [2,5,9,48–50]. Some of these,
suchasSprayandWait,advocatetheuseofredundanttransmissions,tomakeadditional
copies of the communicated information in the network. The replication of the content
makesitfasterforthedestinationtoaccessacopy. However,astheadditionalreplication
alwaysincreases thenetworkload,theseprotocols,whicharenotthroughput-optimalto
beginwith,sufferadditionalcongestion.
In order to resolve delay inefficiency of backpressure, we propose a novel hybrid
approach, an adaptive redundancy technique for backpressure routing, that yields the
benefitsofreplicationtoreducedelayunderlowloadconditions,whileatthesametime
preserving the performance and benefits of traditional backpressure routing under high
trafficconditions. Thistechnique,whichwerefertoasbackpressurewithadaptiveredun-
dancy (BWAR) [1], essentially creates copies of packets in a new duplicatebuffer upon
3
an encounter, when the transmitter’s queue occupancy is low. These duplicate packets
aretransmittedonlywhentheoriginalqueueisempty. Thismechanismcandramatically
improvedelayofbackpressureduringlowloadconditionsduetotworeasons: (1)dueto
theexistenceofmultiplecopiesofthesamepacketsatmultiplenodes,thedestinationis
morelikelytoencounteramassageintendedforit. (2)thisway,thealgorithmbuildsup
gradientstowardsthedestinationsfasterandreduces packetlooping.
The additional transmissions incurred by BWAR due to the duplicates utilize avail-
ableslotswhichwouldotherwisegoidle,inordertoreducethedelay. Thisoffersamore
efficient way to utilize the available bandwidth during low load conditions. In order to
minimize the storage resource utilization of duplicate packets, ideally, these duplicate
packets should be removed from the network whenever a copy is delivered to the desti-
nation. Since thismay bedifficultto implement(exceptinsomekindsofnetworkswith
a separate control plane), we also propose and evaluate a practical timeout mechanism
forautomaticduplicateremoval. Wealsointroduceanenergy-efficientvariantofBWAR
inwhichnumberofcopiesofeach packetisbounded.
Under high load conditions, because queues are rarely empty, duplicates are rarely
created,andBWAReffectivelyrevertstotraditionalbackpressureandinheritsitsthrough-
putoptimalityproperty. Bydesign,BWARishighlyrobustand distributedand doesnot
requirepriorknowledgeoflocations,mobilitypatterns,andloadconditions.
4
Thefollowingarethekeycontributionsofthiswork,
• WeproposeBWAR,anewadaptiveredundancytechniqueforbackpressureschedul-
ing/routinginintermittentlyconnectednetworks. Wealsopresentatimeoutmech-
anism for duplicate removal, which allows BWAR to be easily implemented in
practiceandanenergy-efficientvariantofBWARbylimitingnumberofcopiesfor
eachpacket.
• We developan analytical modelof BWAR, and provetheoretically that it yields a
smallerupperboundontheaveragequeuesize(andhencetheaveragedelay)than
traditionalbackpressure,whileretainingthroughputoptimality.
• Through simulations using an idealized cell-partition network with i.i.d. and ran-
domwalkmobilitymodelsandsimulationsusingrealtracesoftaxisinBeijing,we
quantifythebenefitsfromusingBWAR. Specifically,weshowthatitoutperforms
both traditional backpressure and state of the art DTN/ICN routing mechanisms
(Spray&Wait[50]andSpray&Focus[49]).
The work in this dissertation has been published in part in the paper, M. Alresaini,
M. Sathiamoorthy, B. Krishnamachari, M. J. Neely. “Backpressure with Adaptive Re-
dundancy(BWAR),” IEEEINFOCOM,2012.
The rest ofthedissertationis organizedas follows. In chapter2, we providea back-
ground on traditional backpressure and existing work on routing in intermittently con-
nected mobile networks (ICMN). In chapter 3, we introduce and describe BWAR. In
5
chapter 4, we review thetheory behind traditional backpressurescheduling and routing.
We show in chapter 5 the queue dynamics for BWAR and how it can improvethe delay
theoretically. We show in chapter 6 some enhancements of BWAR to provide practical
distributedtimeout mechanismto removedelivered packets and to optimizepowercon-
sumption of BWAR. In chapter 7, we present our model-based simulation results over
i.i.d. and random walk mobility. We present in chapter 8 simulation results over real
tracesoftaxisinBeijing,China. Inchapter9,wedescriberelatedworkinthissubjectto
placeourcontributionsincontext. Weconcludeinchapter10anddiscussfuturework.
6
Chapter2
Background
In thischapter, wefirst, providegeneral overviewonstochasticnetwork optimizationin
whichbackpressureisresultedfrom. Thenweprovidebasicbackgroundonhowexisting
routingtechniqueswouldworkinintermittentlyconnectedmobilenetworks.
2.1 StochasticNetworkOptimization
Stochastic network optimization provides powerful tools to model, design, analyze and
optimizecross-layer network protocols that are suitable for wireless, wired, mobileand
static networks with random arrival and channel states processes [34]. Based on these
mathematical tools, several research studies were succeeded to introduce and enhance
low-cost, easy-to-implement, efficient algorithms for resource allocation, scheduling,
routing, and admission control that are proven mathematically to be throughput opti-
mal. Maximumweight scheduling[26,56],inwhichlinkswithhigherqueuedifferential
aregivenhigherprioritytobescheduled,andbackpressurerouting[8,55],inwhichflows
7
with higher queue differential are givenhigher priority to be routed, are considered two
topmostcommonexamplesofthesealgorithms.
A general example of how to model stochastic networks, which are networks with
random events like random arrival process and random channel state process (see Fig-
ure 2.1), is to define first the queue dynamics as time evolves mathematically for each
queueinthenetwork. Forexample,IfwedefineQ
i
(t) tobenumberofpacketsinqueue
i at time slott, queue dynamics essentially state Q
i
(t + 1) in terms of Q
i
(t). Then, a
Lyapunov function is defined to map queue occupancy vector Q(t) = (Q
i
(t))
i
in the
network to a real numberL(Q(t)) that gives an indication of how much the network is
congested. After that, Lyapunov drift Δ(Q(t)) is defined to capture the expected in-
crease or decrease on the Lyapunov value of how much the network is congested in
the next time slot t + 1 given the queue lengths vector at the current time slot t; i.e.,
Δ(Q(t)) =E{L(Q(t+1))−L(Q(t))|Q(t)}. ThemainideaisoptimizetheLyapunov
drifteach timeslotbychoosingrightcontrolactionsforschedulingandrouting[8,34].
Networkiscalledstablewhennoqueueinthenetworkblowuptoinfinityandalmost
all packets in the network are successfully delivered to intended destination with finite
time. Clearly whenthearrivalrates tothenetworkistoohighforthenetworktohandle,
the network will be unstable. Lyapunov drift optimization helps in designing schedul-
ing and routing algorithms that can stabilize largest possible set of arrival rates vectors
which is defined as network capacity region [55]. Those algorithms are considered to
be throughput optimal in a sense that as long as there exits at least one algorithm that
8
Figure2.1: Generaldiagramofstochasticnetworks.
can stabilize the network under given arrival rate vector then those throughput optimal
algorithmsstabilizethenetworkaswell.
In addition, drift optimization can be used to optimize power consumption or any
other utility as a cost or reward defined in the network [3,15,30,31]. Backpressure
algorithm, for example, is resulted from optimizing the Lyapunov drift each time slot.
You can view backpressure as packets are transferred by maximumgravity which is the
queuedeferential. Sincedestinationshavealwaysemptyqueueforpacketsthatdestined
to themselves, packets are more likely to traverse towards destinations when the arrival
ratevectorisfeasible. However,whenthearrivalvectoristoosmall,manyotherqueues
would be empty as well making backpressure to suffer large packet looping problem,
andhence,inefficientintermsofdelayforlowloadcasescenarios. Aswestatedbefore,
9
backpressure stabilizes the network whenever the arrival rates are inside the capacity
region.
TheOpenSystemsInterconnection(OSI)modelemphasizestheabstractionandiso-
lationbetweenphysical,datalink,network,transport,session,presentation,andapplica-
tion layers. This isolationare praised in many computerscience and software engineer-
ing aspects for simplifying design, development, update, and maintenance. However,
if boundaries between layers are remove and layers can share information and collabo-
rate to make smart optimized decisions, we would be able to come up with much more
efficientprotocols.
These ideas of broken boundaries between layers, sharing information, and collab-
oration for smart decisions are the main concepts and key advantages for cross-layers-
protocols. Cross-layers protocols utilize information from all layers to design more ef-
ficient and optimizedprotocols. One common exampleof how isolationbetween layers
canleadtopoorperformanceisthatpreviousversionsofTCPcanfalselyinterpretpacket
loose due wireless channel problems as network congestion. The mathematical models
in stochastic network optimizationaddress this issue and are suitableto design efficient
cross-layerprotocols.
10
2.2 RoutinginIntermittentlyConnectedMobileNetworks
(ICMN)
Intermittently Connected Mobile Networks (ICMN) are wireless networks with sparse
mobile nodes in which complete connected end-to-end path from source to destination
rarely exists (see Figure 2.2). As a result of this, traditionalrouting methods which rely
on establishing paths would fail to deliver any packets. Several studies were conducted
toovercomethisproblembyintroducingmobility-assistedroutingmethodsthatusemo-
bilityofthenodestoroutepackets. Inthesemethods,thesourcestoresthepacket while
moving until it encounters another node that has better chance to deliver the packet to
thedestination. When that happens,thesourcehands thepacket tothat nodewhich will
be in charge to deliver the packet. This node also can encounter another one and so on.
Themomentoftimethatanodecarriesaparticularpacketiscalledthecustodianofthat
packet.
ICMNsometimesalsocalledDelayTolerantNetworks(DTN)becausewhenwerely
on nodes mobility to deliver packets, delay becomes large. In this case, applications
that required immediate packet delivery cannot be supported. There are two kinds of
mobility-assisted routings: 1) Single Copy [48]: Only one copy of a particular packet
in the network. This results in high delay to deliver the packet but it has an advantage
of increasing throughput and reducing congestion in the network. 2) Multi Copy [49]:
Packetsduplicatesarecreatedtoenhancethedelaybytakingtheadvantageoffollowing
11
Figure2.2: ICMNexampleinwhichthereisnocompletepathbetweennode1andnode
9ataparticulartime.
multiple paths in parallel. Duplicates creation needs to be carefully performed. High
amountofredundancyincreasescontentioninthenetworkandlimitsthroughput. Asthe
networkiscongestedbymuchredundancy,delayisgettingmuchworse.
Epidemicroutingorsometimescalledfloodinginwhichnodesduplicateandtransmit
so bothtransmitterand receiverhavecopiesofthesamepacket is guaranteed tosupport
optimal delay in the case of having unrealistic infinite bandwidth. When bandwidth is
limitedhowever,epidemicroutingsuffersfromlargedelayduetocongestionresultedby
large number of duplicates. Another method called spraying [49] that limits number of
copiesforeachindividualpacket. Rightamountofsprayingincombinationwithefficient
single-copyroutingforeachcopycanminimizedelaysignificantly[17].
Studiesshowanefficientsingle-copyroutingisperformedbyusingtimersofthelast
timetwo nodesmeet each otherso packet is forwarded to nodethat has minimumtimer
12
valueto destination[48]. Timers are used in this case to givean indicationabout which
nodeisclosertothedestinationandhencethenodethatmightbeclosertothedestination
is handed the packet. The simplest single-copy routing is called direct transmission in
which the source keeps the packet until it meets the destination directly. While direct
transmissionsuffersfromhighdelay,ithasoptimalpowerconsumption. Itisconsidered
to be an upper bound of all smart routing methods in term of delay. Other approach in
single-copyroutingiscalledrandomizedforwardinginwhichifthecustodianmeetsany
other node it forwards the packet with probabilityp. Another approach that combines
the advantages of both randomized forwarding and timer-based routing is called seek
andfocus[48].
Direct transmissiondelay can be significantly improvedby using spraying and rout-
ingeachcopyasregulardirecttransmissioninwhichcalledSprayandWait(S&W)[50].
Similarly, seek and focus can be combined with spraying and called Spray and Focus
(S&F) [49] which has superior performance in terms of delay and energy consumption;
however,noneoftheaboveguaranteesoptimumthroughput.
Analyticalframeworkswereproposedtocomputetheoreticallytheexpectationdelay
of these routing methods mentioned above in the both cases of having contention or
not [17,18,48,49]. When contention is taken into consideration, it is required to take
intoaccount finitebandwidth,collisions,scheduling,interference, and queuing. In [19],
threequantitieshavebeendrivencalled,meetingtime,encountertime,andinter-meeting
time. Meetingtimeisthetimeittakesfortwonodestomeetstartingfromthestationary
13
distribution. Encountertimerefers totheamountoftimethat twonodesremaininradio
range ofeach other. Theinter-meetingtimeis theamountoftimeittakes for two nodes
to remeet again after they left each other. These three quantities play important role to
analyzethenetwork.
2.3 StabilityExample
Figure2.3: Stabilityexampleofacell-partitionedICMNinwhichprioritizingtransmis-
sionstodestinationisnotnecessarilythebestapproach.
Consider a network with four mobilenodes: Star denoted byS,Circle denoted by
C, Triangle denoted by T, and sQuare denoted by Q. Assume the network consists
of cells that are isolated from each other in which a transmission in one cell can not be
14
heard in another cell. Moreover, a transmission in one cell can not interfere or collide
withanothertransmissioninanothercell. Assumetimeisdiscretizedandnodesmobility
is a periodic process that repeats every 4 time slots. Assume at t = 0, all nodes are
outsidetheradiorangeofeach other. Att = 1,nodesS,C,andT becomeclosetoeach
other and can communicate; however, only one node can transmit to another to avoid
collisions. Att = 2, nodeC encounters nodeQ. Att = 3, nodeS encounters nodeC.
Assumepackets arriveto the network according to a periodic process that repeats every
every 4 time slots as well. Att = 0, two packets arrive, one arrives at nodeS destined
to node C and the other arrives at node T destined to node Q. Assume no arrivals at
t∈{1,2,3}(seeFigure2.3).
Consider in thisexamplethefollowingtwo periodicschedulingand routingpolicies
inwhichoneprioritizetransmissionstodestinationsandtheotherisoptimalstrategy.
2.3.1 PrioritizingTransmissionstoDestinations
In this strategy, att = 1, nodeS, which has a packet needs to be delivered to nodeC,
is given priority to transmit to nodeC. By doing this, however, no packets from node
T will be delivered to node Q. Packets will keep accumulating at node T, hence, the
networkbecomesunstable.
15
2.3.2 OptimalStrategy
In this strategy, att = 1, nodeT is scheduled to transit to nodeC as a relay node. At
t = 2,nodeC transmitsthepackettoitsdestinationnodeQ. Att = 3,nodeS transmits
the packet it has to the destination nodeC. By doing this, all packets will be delivered
successfully and the network becomes stable. Backpressure is guaranteed to stabilize
anynetworkwithgeneralergodicorperiodicarrivalsandchannelstatesprocesses[60].
16
Chapter3
BackpressurewithAdaptiveRedundancy
In this chapter, we first describe traditional backpressure scheduling and routing and
then our new proposal for backpressure scheduling/routing with adaptive redundancy
(BWAR). In both cases, we assume that there are N nodes in the network, and time is
discretized. We assumeamulti-commodityflowsysteminwhich everynodecould bea
potentialdestination(correspondingtoaparticularcommodity).
3.1 TraditionalBackpressureSchedulingandRouting
We assume that each node maintainsN queues, one for each commodity, with the j
th
queue at each node containing packets that are destined for nodej. LetQ
c
i
(t) indicate
thenumberofpacketsofcommodityc(i.e.,destinedtonodec)queuedatnodeiattime
t. Naturally, Q
i
i
(t) = 0 ∀t (see Figure 3.1). Let μ
c
ij
(t) be the scheduling and routing
variable that indicates the number of packets of commodity c to be scheduled on link
17
Figure3.1: A backpressureexampleat timet showinghownodei maintainsN queues,
oneforeachdestination. Here,Q
1
i
(t) = 3,Q
2
i
(t) = 2,andQ
N
i
(t) = 4. Queueiatnodei
isalwaysempty.
18
(i,j). Traditional backpressure scheduling/routing [8,55] selects the μ
c
ij
(t) ≥ 0 that
solvethefollowingproblem(aformofmaximumweightindependentsetselection),
max
X
i,j,c
Δ
c
ij
(t)·μ
c
ij
(t)
subjectto,
P
c
μ
c
ij
(t)≤θ
ij
(t), ∀i,∀j
μ
c
ij
(t)·μ
d
km
(t) = 0, ((i,j),(k,m))∈ Ω(t),∀c,∀d
μ
c
ij
(t)≥ 0, ∀i,∀j,∀c (3.1)
where Δ
c
ij
(t) = Q
c
i
(t)−Q
c
j
(t) is the link weight, which denotes the queue differential
for commodityc on link (i,j) at slott. θ
ij
(t) is the channel state in terms of number of
packetsthatcan betransmittedoverlink(i,j) duringslott. Ω(t) isthelinkinterference
setatslottsuchthatiflink(i,j)interfereswithlink(i
′
,j
′
)atslottthen((i,j),(i
′
,j
′
))∈
Ω(t) and hence, those two links can not be both scheduled at slott. The maximization
problem in (3.1) can be solved by finding the maximumcommodityc
∗
ij
(t) for each link
(i,j) at slott that maximizes Δ
c
ij
(t) and assign μ
c
ij
(t) = 0 for allc 6= c
∗
ij
(t) and then
solve,
19
max
X
i,j
Δ
c
∗
ij
(t)
ij
(t)·μ
c
∗
ij
(t)
ij
(t)
subjectto,
μ
c
∗
ij
(t)
ij
(t)≤θ
ij
(t), ∀i,∀j
μ
c
∗
ij
(t)
ij
(t)·μ
c
∗
km
(t)
km
(t) = 0, ((i,j),(k,m))∈ Ω(t)
μ
c
∗
ij
(t)
ij
(t)≥ 0, ∀i,∀j (3.2)
3.1.1 TraditionalBackpressureExample
Consider a network with 5 nodes as shown in Figure 3.2. Assumeat timet, nodes 1, 2,
and 3areintheradiorangeofeachothersotheycancommunicate;suchthat,
θ
12
(t) =θ
21
(t) =θ
13
(t) =θ
31
(t) =θ
23
(t) =θ
32
(t) = 1
Assumethequeueoccupancyattimetisasfollows,
Q
1
1
(t) = 0,Q
2
1
(t) = 2,Q
3
1
(t) = 1,Q
4
1
(t) = 3,Q
5
1
(t) = 2,
Q
1
2
(t) = 3,Q
2
2
(t) = 0,Q
3
2
(t) = 2,Q
4
2
(t) = 5,Q
5
2
(t) = 1,
Q
1
3
(t) = 0,Q
2
3
(t) = 2,Q
3
3
(t) = 0,Q
4
3
(t) = 1,Q
5
3
(t) = 4
20
Figure3.2: An illustrationofhow traditionalbackpressureworks. Here, thegreen com-
modity,whichisdestinedtonode4,willberoutedfromnode2tonode3.
21
Queuedifferentialsarecomputedandfoundtobe,
Δ
1
12
(t) =−3, Δ
2
12
(t) = 2, Δ
3
12
(t) =−1, Δ
4
12
(t) =−2, Δ
5
12
(t) = 1,
Δ
1
21
(t) = 3, Δ
2
21
(t) =−2, Δ
3
21
(t) = 1, Δ
4
21
(t) = 2, Δ
5
21
(t) =−1,
Δ
1
13
(t) = 0, Δ
2
13
(t) = 0, Δ
3
13
(t) = 1, Δ
4
13
(t) = 2, Δ
5
13
(t) =−2,
Δ
1
31
(t) = 0, Δ
2
31
(t) = 0, Δ
3
31
(t) =−1, Δ
4
31
(t) =−2, Δ
5
31
(t) = 2,
Δ
1
23
(t) = 3, Δ
2
23
(t) =−2, Δ
3
23
(t) = 2, Δ
4
23
(t) = 4, Δ
5
23
(t) =−3,
Δ
1
32
(t) =−3, Δ
2
32
(t) = 2, Δ
3
32
(t) =−2, Δ
4
32
(t) =−4, Δ
5
32
(t) = 3
In order to prevent collisions, only one link is scheduled for transmission out of the
six links, namely, (1,2), (2,1), (1,3), (3,1), (2,3), and (3,2). After computing the
maximumcommodityforeachlink,theresultwouldbe,
c
∗
12
(t) = 2, c
∗
21
(t) = 1,
c
∗
13
(t) = 4, c
∗
31
(t) = 5,
c
∗
23
(t) = 4, c
∗
32
(t) = 5
By solvingthemaximizationproblemin(3.2), thetransmissionrateμ
4
23
(t) = 1, and
allothertransmissionratesμ
c
ij
(t) = 0,suchthat, (i,j,c)6= (2,3,4).
Thus by backpressure in this example, a packet will be transmitted from queue 4 at
node 2toqueue4atnode 3.
22
3.2 BWARSchedulingandRouting
Figure3.3: ABWARexampleattimetshowinghownodeimaintainsN queuesandN
duplicatebuffers. Here,Q
1
i
(t) = 3,D
1
i
(t) = 1,Q
2
i
(t) = 2,D
2
i
(t) = 2,Q
N
i
(t) = 4, and
D
N
i
(t) = 1. Queueiat nodeiisalways empty. Similarly,Duplicatebufferiatnodeiis
alwaysempty.
Our proposed enhancement of backpressure with adaptiveredundancy works as fol-
lows [1]. We have an additional set ofN duplicate buffers of sizeD
max
at each node.
Besides the original queue occupancy Q
c
i
(t) which has the same meaning as in tradi-
tional backpressure, the duplicate buffer occupancy is denoted byD
c
i
(t), that indicates
the number of duplicate packets at nodei that are destined to nodec at timet. Again,
23
Q
i
i
(t) =D
i
i
(t) = 0∀t sincedestinationsneed not bufferany packets intended for them-
selves (see Figure 3.3 where the top queues are regular queues and the bottom ones are
theduplicatebuffers). Theduplicatequeuesaremaintainedandutilizedasfollows:
• Original packets when transmitted are removed from the main queue; however,
if the queue occupancy is lower than a certain thresholdq
th
, then the transmitted
packetisduplicatedandkeptintheduplicatebufferassociatedwithitsdestination
if it is not full (otherwise no duplicate is created). We found that setting both
q
th
andD
max
to the value of the maximum link service rate is enough and gives
superiordelayresults.
• Duplicate packets are not removed from the duplicate buffer when transmitted.
Theyareonlyremovedwhentheyarenotifiedtobereceivedbythedestination,or
apre-defined timeouthasoccurred.
• Whenacertainlinkisscheduledfortransmission,theoriginalpacketsinthemain
queuearetransmittedfirst. Ifnomoreoriginalpacketsareleft,onlythenduplicates
aretransmitted. Thustheduplicatequeuehasastrictlylowerpriority.
• Ideally, all copies of a deliveredpacket in the network should bedeleted instanta-
neouslywhenthefirstcopyisdeliveredtotheintendeddestination.
Similar to original backpressure scheduling/routing, the BWAR scheduling/routing
alsorequiresthesolutionofasimilarmaximumweightindependentsetproblem:
24
max
X
i,j,c
Δ
c
BWAR,ij
(t)·μ
c
ij
(t)
subjectto,
P
c
μ
c
ij
(t)≤θ
ij
(t), ∀i,∀j
μ
c
ij
(t)·μ
d
km
(t) = 0, ((i,j),(k,m))∈ Ω(t),∀c,∀d
μ
c
ij
(t)≥ 0, ∀i,∀j,∀c (3.3)
We define an enhanced link weight for BWAR, Δ
c
BWAR,ij
(t) as follows, to take into
accounttheoccupancyoftheduplicatebuffer.
Δ
c
BWAR,ij
(t) =

Q
c
i
(t)−Q
c
j
(t)

+
1
2

1
j=cAndQ
c
i
(t)+D
c
i
(t)>0

+
1
4
1
D
max

D
c
i
(t)−D
c
j
(t)

(3.4)
Here the indicator function1
j=cAndQ
c
i
(t)+D
c
i
(t)>0
denotes that node j is the final desti-
nation for the considered commodityc. This gives higher weight to commodities that
encounter their destinations. We show later how this effectively results in dramatic de-
lay improvement. Similarly, the maximization problem in (3.3) can be solved first by
findingthemaximumcommodityc
∗
BWAR,ij
(t) foreach link (i,j) at slottthatmaximizes
25
Δ
c
BWAR,ij
(t) followed by the same approach discussed earlier in 3.1. It is important to
noticethatanysolutionto(3.3)isalsoasolutionto(3.1)(butnotnecessarilyviceversa)
assuming thatQ
c
i
(t) andμ
c
ij
(t) are integers. The small weight added in (3.4) gives ad-
vantage first to links/commodities which encounter the destination and then to higher
duplicatebufferdeferential toincreasethechanceofservingduplicates. Thesmallfrac-
tionsin(3.4)assuresthisprioritywhentherearetiesin(3.1)toboostdelayperformance.
3.2.1 DestinationAdvantageExampleinBWAR
Figure3.4: ABWARexampleshowinghowitgivesadvantagetopacketsthatencounter
theirdestinationwhentherearetiesinqueuedifferentials.
26
Consider a network with 5 nodes as shown in Figure 3.4. Assume at timet, node 1
encountersnode 2sotheycancommunicate;suchthat,
θ
12
(t) =θ
21
(t) = 1
Assumethequeueoccupancyattimetisasfollows,
Q
1
1
(t) = 0,Q
2
1
(t) = 0,Q
3
1
(t) = 0,Q
4
1
(t) = 4,Q
5
1
(t) = 0,
Q
1
2
(t) = 3,Q
2
2
(t) = 0,Q
3
2
(t) = 0,Q
4
2
(t) = 1,Q
5
2
(t) = 0
Thequeuethresholdforduplicationq
th
= 1andtheduplicatebuffershavemaximum
sizeofD
max
= 2andhaveoccupancyattimetasfollows,
D
1
1
(t) = 0,D
2
1
(t) = 0,D
3
1
(t) = 0,D
4
1
(t) = 2,D
5
1
(t) = 0,
D
1
2
(t) = 0,D
2
2
(t) = 0,D
3
2
(t) = 0,D
4
2
(t) = 0,D
5
2
(t) = 0
BWARdifferentialsarecomputedandfoundtobe,
Δ
1
BWAR,12
(t) =−3, Δ
2
BWAR,12
(t) = 0, Δ
3
BWAR,12
(t) = 0,
Δ
4
BWAR,12
(t) = 3.25, Δ
5
BWAR,12
(t) = 0,
Δ
1
BWAR,21
(t) = 3.5, Δ
2
BWAR,21
(t) = 0, Δ
3
BWAR,21
(t) = 0,
Δ
4
BWAR,21
(t) =−3.25, Δ
5
BWAR,21
(t) = 0
27
Weassumenodescannottransmitandreceiveatthesametime,hence,onlyonelink
is scheduled for transmissionout of the two links (1,2) and (2,1). After computingthe
maximumcommodityforeachlink,theresultwouldbe,
c
∗
BWAR,12
(t) = 4, c
∗
BWAR,21
(t) = 1
By solvingthemaximizationproblemin(3.3), thetransmissionrateμ
1
21
(t) = 1, and
allothertransmissionratesμ
c
ij
(t) = 0,suchthat, (i,j,c)6= (2,1,1).
Thus by BWAR in this example, a packet will be transmitted from queue 1 at node
2 to the destination node 1. No duplicate will be created because queue 1 occupancy at
node 2 does not get lower thanq
th
= 1 and even if it does no duplicate will be created
because the transmission is for a packet to its destination. Note that using traditional
backpressure, there would be a tie between settingμ
4
12
(t) = 1 orμ
1
21
(t) = 1, since both
havequeuedifferentialof 3.
3.2.2 DuplicationExampleinBWAR
Consider a network with 5 nodes. Assume at time t, node 1 encounters node 2 (see
Figure3.5)sotheycancommunicate;suchthat,
θ
12
(t) =θ
21
(t) = 1
28
Figure3.5: ABWARexampleshowinghowBWARcreatesduplicates.
29
Assumethequeueoccupancyattimetisasfollows,
Q
1
1
(t) = 0,Q
2
1
(t) = 0,Q
3
1
(t) = 1,Q
4
1
(t) = 0,Q
5
1
(t) = 0,
Q
1
2
(t) = 0,Q
2
2
(t) = 0,Q
3
2
(t) = 0,Q
4
2
(t) = 0,Q
5
2
(t) = 0
Assume that there are no arrivals at time t and the queue threshold for duplication
q
th
= 1and theduplicatebuffers havemaximumsizeofD
max
= 2 and haveoccupancy
attimetasfollows,
D
1
1
(t) = 0,D
2
1
(t) = 0,D
3
1
(t) = 0,D
4
1
(t) = 0,D
5
1
(t) = 1,
D
1
2
(t) = 0,D
2
2
(t) = 0,D
3
2
(t) = 0,D
4
2
(t) = 2,D
5
2
(t) = 0
BWARdifferentialsarecomputedandfoundtobe,
Δ
1
BWAR,12
(t) = 0, Δ
2
BWAR,12
(t) = 0, Δ
3
BWAR,12
(t) = 1,
Δ
4
BWAR,12
(t) =−0.25, Δ
5
BWAR,12
(t) = 0.125,
Δ
1
BWAR,21
(t) = 0, Δ
2
BWAR,21
(t) = 0, Δ
3
BWAR,21
(t) =−1,
Δ
4
BWAR,21
(t) = 0.25, Δ
5
BWAR,21
(t) =−0.125
30
Weassumenodescannottransmitandreceiveatthesametime,hence,onlyonelink
is scheduled for transmissionout of the two links (1,2) and (2,1). After computingthe
maximumcommodityforeachlink,theresultwouldbe,
c
∗
BWAR,12
(t) = 3, c
∗
BWAR,21
(t) = 4
By solvingthemaximizationproblemin(3.3), thetransmissionrateμ
3
12
(t) = 1, and
allothertransmissionratesμ
c
ij
(t) = 0,suchthat, (i,j,c)6= (1,2,3).
Thus by BWAR in this example, a packet will be transmitted from queue 3 at node
1 to queue 3 at node 2. Since queue 3 occupancy at node 1 gets lower thanq
th
= 1 and
since the the transmission is not for a packet to its destination and duplicate buffer 3 at
node 1 is not full, a duplicate for the transmitted packet is created and kept in duplicate
buffer 3atnode 1(seehowduplicateiscreated att+1inFigure3.5).
3.2.3 DuplicateTransmissionExampleinBWAR
Consider a network with 5 nodes. Assume at time t, node 1 encounters node 2 (see
Figure3.6)sotheycancommunicate;suchthat,
θ
12
(t) =θ
21
(t) = 1
31
Figure3.6: ABWARexampleshowinghowduplicatesaretransmittedinBWAR.
32
Assumealloriginalqueuesareemptyattimetnamely,
Q
1
1
(t) =Q
2
1
(t) =Q
3
1
(t) =Q
4
1
(t) =Q
5
1
(t) = 0,
Q
1
2
(t) =Q
2
2
(t) =Q
3
2
(t) =Q
4
2
(t) =Q
5
2
(t) = 0
Assume that there are no arrivals at time t and the queue threshold for duplication
q
th
= 1and theduplicatebuffers havemaximumsizeofD
max
= 2 and haveoccupancy
attimetasfollows,
D
1
1
(t) = 0,D
2
1
(t) = 0,D
3
1
(t) = 1,D
4
1
(t) = 0,D
5
1
(t) = 0,
D
1
2
(t) = 0,D
2
2
(t) = 0,D
3
2
(t) = 0,D
4
2
(t) = 0,D
5
2
(t) = 0
BWARdifferentialsarecomputedandfoundtobe,
Δ
1
BWAR,12
(t) = 0, Δ
2
BWAR,12
(t) = 0, Δ
3
BWAR,12
(t) = 0.125,
Δ
4
BWAR,12
(t) = 0, Δ
5
BWAR,12
(t) = 0,
Δ
1
BWAR,21
(t) = 0, Δ
2
BWAR,21
(t) = 0, Δ
3
BWAR,21
(t) =−0.125,
Δ
4
BWAR,21
(t) = 0, Δ
5
BWAR,21
(t) = 0
Aftercomputingthemaximumcommodityforlink(1,2),theresultwouldbe,
c
∗
BWAR,12
(t) = 3
33
By solvingthemaximizationproblemin(3.3), thetransmissionrateμ
3
12
(t) = 1, and
allothertransmissionratesμ
c
ij
(t) = 0,suchthat, (i,j,c)6= (1,2,3).
ThusbyBWARinthisexampleandsincethatqueue3atnode1isempty,aduplicate
will be transmitted from duplicate buffer 3 at node 1 to duplicate buffer 3 at node 2. In
BWAR,transmittedduplicatesarenotremoved(seehowduplicatebuffer3atnode1still
hastheduplicateaftertransmissionatt+1inFigure3.6).
3.2.4 FullDuplicateBufferExampleinBWAR
Figure 3.7: A BWAR example showing how full duplicate buffers prevent creating du-
plicates.
34
Consider a network with 5 nodes. Assume at timet, node 1 encounters node 2 (see
Figure3.7)sotheycancommunicate;suchthat,
θ
12
(t) =θ
21
(t) = 1
Assumethequeueoccupancyattimetisasfollows,
Q
1
1
(t) = 0,Q
2
1
(t) = 0,Q
3
1
(t) = 1,Q
4
1
(t) = 0,Q
5
1
(t) = 0,
Q
1
2
(t) = 0,Q
2
2
(t) = 0,Q
3
2
(t) = 0,Q
4
2
(t) = 0,Q
5
2
(t) = 0
Assume that there are no arrivals at time t and the queue threshold for duplication
q
th
= 1and theduplicatebuffers havemaximumsizeofD
max
= 2 and haveoccupancy
attimetasfollows,
D
1
1
(t) = 0,D
2
1
(t) = 0,D
3
1
(t) = 2,D
4
1
(t) = 0,D
5
1
(t) = 0,
D
1
2
(t) = 0,D
2
2
(t) = 0,D
3
2
(t) = 0,D
4
2
(t) = 0,D
5
2
(t) = 0
35
BWARdifferentialsarecomputedandfoundtobe,
Δ
1
BWAR,12
(t) = 0, Δ
2
BWAR,12
(t) = 0, Δ
3
BWAR,12
(t) = 1.25,
Δ
4
BWAR,12
(t) = 0, Δ
5
BWAR,12
(t) = 0,
Δ
1
BWAR,21
(t) = 0, Δ
2
BWAR,21
(t) = 0, Δ
3
BWAR,21
(t) =−1.25,
Δ
4
BWAR,21
(t) = 0, Δ
5
BWAR,21
(t) = 0
Aftercomputingthemaximumcommodityforlink(1,2),theresultwouldbe,
c
∗
BWAR,12
(t) = 3
By solvingthemaximizationproblemin(3.3), thetransmissionrateμ
3
12
(t) = 1, and
allothertransmissionratesμ
c
ij
(t) = 0,suchthat, (i,j,c)6= (1,2,3).
ThusbyBWARinthisexample,apacketwillbetransmittedfromqueue 3atnode1
toqueue 3atnode2. Sinceduplicatebuffer3atnode 1isfull,therewillbenoduplicate
createdeventhoughqueue3occupancyatnode1getslowerthanq
th
= 1(seeFigure3.7
att+1).
3.3 BackpressureRoutinginICMN
In general backpressure scheduling is NP-hard, owing to the maximum weighted inde-
pendent set (MWIS) problem that needs to be solved at each time slot [32]. However,
36
inthisdissertation,wefocusonintermittentlyconnectednetworks,thatconsistofsparse
encounters between pairs of nodes. Therefore, at any given time, the size of any con-
nected component of the network is very small. In this case, the scheduling problem is
dramaticallysimplified.
In the next chapter, Chapter 4, we provide an overview analysis of traditional back-
pressure. AfterthatinChapter5,weundertakeananalysisoftheperformanceofBWAR
and compareitwiththeknownresultsfortraditionalbackpressurerouting. Specifically,
we prove that any feasible rate vector is also stabilized by BWAR, and the bound that
we can give on the expected queue occupancy for BWAR is better than that for regular
backpressure.
37
Chapter4
AnalysisReviewofBasicBackpressure
We consider a timeslotted network with N nodes that communicate with each other.
Packets arriveto each node, and each packet mustbe deliveredto a specific destination,
possiblyviaa multi-hoppath. Each nodemaintainsseveralqueues, oneper destination,
tostorepackets. Eachqueuehasthefollowingdynamics,
Q(t+1) = max[Q(t)−μ(t),0]+A(t) (4.1)
whereQ(t) is the queue size at timet,μ(t) is the transmission rate out of the queue at
timet,andA(t)isthetotalpacketarrivalstothequeueattimet.
Eachtimeslot,weobservethequeuestatesandthechannelstatesandmakeschedul-
ing and routing decisions based on this information. Let Q
c
n
(t) be the queue backlog
(number of packets) in noden ∈ {1,2,...,N} that are destined for nodec ∈ {1,...,N}
at slott. LetA
c
n
(t) be the exogenous packet arrivals that come to noden and destined
to nodec at timet with rateλ
c
n
. Exogenous arrivals are thepackets that just entered the
38
network. Endogenous arrivals, however, are arrivals from other nodes and were already
insidethenetwork. Packetsmaybeforwardedtoseveralnodesbeforereachingthedesti-
nation. LetusdefinethecapacityregionΛtobethesetofallpossiblearrivalratevectors
(λ
c
n
)
n,c
that can be stablysupported by somescheduling and routingstrategy. Letθ
ab
(t)
bethechannelstatefromnodeatonodebattimetintermsofhowmanypacketscanbe
transmitted. Letμ
ab
(t) bethescheduledservicerate fromnodeatonodebat slott. Let
μ
c
ab
(t)betheservicerateforcommoditycroutedfromnodeatonodebattimet, which
mustsatisfy,
X
c
μ
c
ab
(t)≤μ
ab
(t)≤θ
ab
(t) (4.2)
Thequeuedynamicsforeachtimeslotandforeachqueueisthefollows,
Q
c
n
(t+1) = max[Q
c
n
(t)−
X
b
μ
c
nb
(t),0]
+A
c
n
(t)+
X
a
˜ μ
c
an
(t) (4.3)
where ˜ μ istheactual transferrate dueto insufficientpackets in thequeue. Forexample,
onsomeslotswemaybeabletosend5packets,butweonlysend3,becauseonly3were
availableinthequeue. Inequation(4.3),A
c
n
(t)aretheexogenousarrivalsand
P
a
˜ μ
c
an
(t)
aretheendogenousarrivalstonoden.
39
Define the vectorQ(t) = (Q
c
n
(t))
n,c
to be the vector of all queues in the network at
timet. TheLyapunovfunctionL(Q(t))canbedefinedasfollows,
L(Q(t)) =
X
n,c
Q
c
n
(t)
2
(4.4)
TheLyapunovdrift Δ(Q(t))isdefinedasfollows,
Δ(Q(t)) =E{L(Q(t+1))−L(Q(t))|Q(t)} (4.5)
Ithasbeenalreadyprovenby[8,55]that,
Δ(Q(t))≤
X
n,c
E{β
c
n
(t)}−2
X
n,c
Q
c
n
(t)E{ψ
c
n
(t)|Q(t)} (4.6)
suchthat,
β
c
n
(t) =

X
b
μ
c
nb
(t)
!
2
+

A
c
n
(t)+
X
a
μ
c
an
(t)
!
2
(4.7)
and,
ψ
c
n
(t) =
X
b
μ
c
nb
(t)−
X
a
μ
c
an
(t)−A
c
n
(t) (4.8)
40
Maximizing
P
n,c
Q
c
n
(t)E{ψ
c
n
(t)|Q(t)}in(4.6)whichisequivalenttothemaximiza-
tionproblemdefined in(3.1)yieldsthebackpressurealgorithmforschedulingandrout-
ing and it has been proven by [8,55] that it supports the maximum capacity Λ. The
averagequeueoccupancyboundforbackpressureschedulingandroutingis,
¯
Q≤
¯
β
2ǫ
(4.9)
suchthat,
¯
Q = lim
T→∞
1
T
T
X
τ=0
X
n,c
E{Q
c
n
(τ)} (4.10)
¯
β = lim
T→∞
1
T
T
X
τ=0
X
n,c
E{β
c
n
(τ)} (4.11)
ǫ = argmax
x≥0
(λ
c
n
+x)
n,c
∈ Λ (4.12)
where,
¯
Q is the average of total queue backlog occupancy.
¯
β is the sum of the second
moment of the scheduled transmission rate out of each queue plus the second moment
of the sum of the arrivals and scheduled transmissionrate into each queue and summed
overallqueues.ǫisthemaximumpositivenumbersuchthataddingǫtoeacharrivalrate
stillmakestheminsidethecapacityregionΛ.
41
Chapter5
AnalysisofBWAR
Here is a formal mathematical description of backpressure with adaptive redundancy
(BWAR). As before, let Q
c
n
(t) to be queue backlog in node n of commodityc at time
slott. We defineD
c
n
(t) to benumberofredundant packets in noden ofcommodityc at
timet. Redundantpacketsarestoredseparatelyinredundantbuffers. Redundantpackets
have lower priority in such a way that no redundant packet is served unless the queue
of original packets is empty. For all timeslotst,A
c
n
(t),θ
ab
(t),μ
ab
(t),μ
c
ab
(t) and ˜ μ
c
ab
(t)
are defined exactly as before. Arrival rates λ
c
n
are also defined as before. The queue
dynamicsinequation(4.3)isupdatedinBWARtobe,
Q
c
n
(t+1) = max[Q
c
n
(t)−γ
c
n
(t)−
X
b
μ
c
nb
(t),0]
+A
c
n
(t)+
X
a
˜ μ
c
an
(t) (5.1)
42
whereγ
c
n
(t) isthenumberoforiginalpackets insidenodenofcommoditycat timeslot
t that are known to be delivered by some duplicates to the destination using our BWAR
strategy. One ideal model is that we find out which packets are delivered immediately,
anotheristhatwefindoutaftersomedelay. Ouranalysisallowsforanysuchknowledge
of delivered packets. We show later a practical timeout-based strategy for duplicate re-
movals. Those γ
c
n
(t) packets are needed to be removed from the queue since they are
alreadyknowntobedelivered. Weassumethatthedeletionhappensduringthetimeslot
thenceatthebeginningoftimeslottnoneofthosepacketsaredeletedyetbutareknown
to be deleted. The queue dynamics in (5.1) consider only original packets and does not
take into account the duplicate packets. We define the redundant buffer dynamics that
areisolatedfromtheoriginalqueuedynamicsasfollows,
D
c
n
(t+1) =D
c
n
(t)− ˜ γ
c
n
(t)+δ
c
n
(t)+
X
a
ω
c
an
(t) (5.2)
where ˜ γ
c
n
(t) denotes the number of duplicates in noden of commodityc at timet that
are known to be already delivered to the destination and hence they must be removed.
δ
c
n
(t) is number of duplicates created at noden during slott according to the adaptive
redundancy criteria. ω
c
ab
(t) is the actual duplicate transmissions from nodea to nodeb
of commodityc at timet. No duplicates are created in nor transmitted to full duplicate
buffers. Therefore,D
c
n
(t+1)≤D
max
∀t.
43
Asbefore,Q(t) = (Q
c
n
(t))
n,c
isthevectorofallqueuebacklogsattimet. LetU
c
n
(t)
tobetheundeliveredqueuebackloginnodenofcommoditycattimet. Hence,
U
c
n
(t) =Q
c
n
(t)−γ
c
n
(t) (5.3)
LetU(t) = (U
c
n
(t))
n,c
be the vector of all queue backlogs of undelivered packets at
timet. LetΓ(t) = (γ
c
n
(t))
n,c
be the vector of all removed duplicates at timet. Define
the Lyapunov functionL(X) =
P
(X
i
)
2
. Assumethat
¯
Q,
¯
β andǫ are defined as before
in(4.10),(4.11)and(4.12)respectively.
Letalsodefine,
¯
U = lim
T→∞
1
T
T
X
τ=0
X
n,c
E{U
c
n
(τ)} (5.4)
Γ
2
= lim
T→∞
1
T
T
X
τ=0
X
n,c
E

(γ
c
n
(τ))
2

(5.5)
Q·Γ = lim
T→∞
1
T
T
X
τ=0
X
n,c
E{Q
c
n
(τ).γ
c
n
(τ)} (5.6)
U ·Γ = lim
T→∞
1
T
T
X
τ=0
X
n,c
E{U
c
n
(τ).γ
c
n
(τ)} (5.7)
where,
¯
U istheaverageoftotalqueuebacklogoccupancyforundeliveredpacketsinthe
main queues. Γ
2
is the second moment of number of removed original packets that has
been already delivered by duplicatessummed overall original queues. Q·Γ is thejoint
second moment of number of removed packets and the queue backlog summed over all
44
queues. U ·Γ is the joint second moment of number of removed packets and the queue
backlogofundeliveredpacketssummedoverallqueues.
Forsimplicityofexposition,weprovetheresultinthesimplecasewhenarrivalrates
A
c
n
(t) and the channel statesθ
ab
(t) are i.i.d. overslots. This can be extended to general
ergodic(possiblynon-i.i.d.) processesusingaT-slotdriftargumentasin[32].
Theorem 1. If the channel states θ
ab
(t) are i.i.d. and the arrival processes A
c
n
(t) are
i.i.d. with ratesλ
c
n
that are inside the capacity region Λ such that (λ
c
n
+ǫ)
n,c
∈ Λ for
someǫ> 0,thenBWARstabilizesallqueueswiththefollowingboundon theaverageof
totalqueueoccupancyof undeliveredpackets
¯
U,
¯
U ≤
¯
β−Γ
2
−2U ·Γ
2ǫ
(5.8)
Proof. Squaringbothsidesof(5.1),
Q
c
n
(t+1)
2
≤ (Q
c
n
(t)−γ
c
n
(t))
2
+β
c
n
(t)
−2(Q
c
n
(t)−γ
c
n
(t))ψ
c
n
(t) (5.9)
whereβ
c
n
(t)andψ
c
n
(t) aredefinedasbeforein(4.7)and(4.8)respectively.
45
Summingoverallnandc,
X
n,c
Q
c
n
(t+1)
2
≤
X
n,c
(Q
c
n
(t)−γ
c
n
(t))
2
+
X
n,c
β
c
n
(t)
−2
X
n,c
(Q
c
n
(t)−γ
c
n
(t))ψ
c
n
(t) (5.10)
TakingtheconditionalexpectationE{.|Q(t)−Γ(t)},
E{L(Q(t+1))−L(Q(t)−Γ(t))|Q(t)−Γ(t)}≤
E
(
X
n,c
β
c
n
(t)−2
X
n,c
(Q
c
n
(t)−γ
c
n
(t))ψ
c
n
(t)

Q(t)−Γ(t)
)
(5.11)
Since our BWAR policy maximizes (3.3) and hence (3.1) taking into account the
undeliveredpacketsU(t)only,itwillalsomaximize,
E
(
X
n,c
(Q
c
n
(t)−γ
c
n
(t))ψ
c
n
(t)

Q(t)−Γ(t)
)
(5.12)
However, because (λ
c
n
+ǫ)
n,c
are inside the capacity region Λ, we know from [32]
that there exists a stationary and randomized algorithm alg
∗
, which makes decisions
independentofQ(t)−Γ(t),yieldingψ
∗c
n
(t)thatsatisfy,
E{ψ
∗c
n
(t)}≥ǫ ∀n,c
46
BecauseBWARmaximizes(5.12),itfollowsthat,
E
(
X
n,c
(Q
c
n
(t)−γ
c
n
(t))ψ
c
n
(t)

Q(t)−Γ(t)
)
≥
E
(
X
n,c
(Q
c
n
(t)−γ
c
n
(t))ψ
∗c
n
(t)

Q(t)−Γ(t)
)
≥
X
n,c
(Q
c
n
(t)−γ
c
n
(t))ǫ (5.13)
Usingthisin(27)yields,
E{L(Q(t+1))−L(Q(t)−Γ(t))|Q(t)−Γ(t)}≤
X
n,c
E{β
c
n
(t)|Q(t)−Γ(t)}−2ǫ
X
n,c
(Q
c
n
(t)−γ
c
n
(t)) (5.14)
Takingiterativeexpectation,
E{L(Q(t+1))}−E{L(Q(t)−Γ(t))}≤
X
n,c
E{β
c
n
(t)}−2ǫ
X
n,c
E{(Q
c
n
(t)−γ
c
n
(t))} (5.15)
Noticethat,
E{L(Q(t)−Γ(t))} =E{L(Q(t))}+E{L(Γ(t))}
−2E{Q(t)·Γ(t)} (5.16)
Hencebysummingovertimeslotsτ ∈{0,...,T}andbytelescoping,
47
E{L(Q(T))}−E{L(Q(0))}−
T
X
τ=0
E{L(Γ(τ))}
+2
T
X
τ=0
E{Q(τ)·Γ(τ)}≤
T
X
τ=0
X
n,c
E{β
c
n
(τ)}−2ǫ
T
X
τ=0
X
n,c
E{(Q
c
n
(τ)−γ
c
n
(τ))} (5.17)
DividingbyT andtakingthe limforT →∞implies,
¯
Q−
¯
Γ≤
¯
β +Γ
2
−2Q·Γ
2ǫ
(5.18)
Nowforundeliveredpackets
¯
U,wehaveby(5.3)and(5.18),
¯
U ≤
¯
β−Γ
2
−2U ·Γ
2ǫ
Remark: Note that the computation of Γ
2
andU ·Γ is determined by the duplicate
removal strategies. Depending on these terms, the queue bound in this above theorem
couldbemuchlowerthanthequeueoccupancyboundforregularbackpressurein(4.9).
Thus we have a formal guarantee that BWAR is no worse in terms of throughput than
backpressure, and potentially much better in terms of delay, since by Little’s theorem
48
average delay is proportional to the average number of undelivered packets. We will
validatethisfindingwithmodelinchapter7andrealtracesinchapter8.
49
Chapter6
BWAREnhancements
In this chapter, we present three enhancements of BWAR. The first enhancement is that
originalpackets are movedto duplicatebuffers upon copy toreduce delay furthermore.
The second enhancement is that duplicates are removed based on distributed, easy-to-
implement, time-out mechanism in order to obtain close delay performance of ideal re-
moval of delivered packets. The third enhancement is to limit number of duplicates for
eachpackettooptimizeenergyconsumptionwhileatthesametimemaintainsamedelay
andthroughputperformance.
6.1 BWAR-ID
In the first enhancement of BWAR, we designBWAR with Ideal packet removaland
original packets moved to Duplicate buffer upon copy (BWAR-ID) which is very
similarto BWAR presented in chapter 3. The only difference is that wheneveran origi-
nalpacketisduplicated,boththeoriginalpacketandtheduplicatearestoredinduplicate
50
buffers (original packet is stored in the duplicatebuffer at the receiver and the duplicate
packet is stored in the the duplicate buffer at the transmitter). In BWAR-ID, whenever
theduplicatebufferat thetransmitteror receiverisfull, no duplicatesare created. From
nowone,wecallBWARpresentedinchapter3BWARwithIdealpacketremovaland
original packets retained in the Main queue (BWAR-IM). When an original packet
gets duplicated in BWAR-IM, the original packet is stored in the original queue at the
receiver and the duplicate is stored in the duplicate buffer at the transmitter (see Fig-
ure 3.5). In BWAR-IM, it requires only for duplication that the duplicate buffer at the
transmitternotbefull.
6.1.1 DuplicationExampleinBWAR-ID
Similar to example 3.2.2, consider a network with 5 nodes. Assume at time t, node 1
encountersnode 2(seeFigure6.1)sotheycancommunicate;suchthat,
θ
12
(t) =θ
21
(t) = 1
Assumethequeueoccupancyattimetisasfollows,
Q
1
1
(t) = 0,Q
2
1
(t) = 0,Q
3
1
(t) = 1,Q
4
1
(t) = 0,Q
5
1
(t) = 0,
Q
1
2
(t) = 0,Q
2
2
(t) = 0,Q
3
2
(t) = 0,Q
4
2
(t) = 0,Q
5
2
(t) = 0
51
Figure6.1: AnexampleshowinghowBWAR-ID createsduplicates.
52
Assume that there are no arrivals at time t and the queue threshold for duplication
q
th
= 1and theduplicatebuffers havemaximumsizeofD
max
= 2 and haveoccupancy
attimetasfollows,
D
1
1
(t) = 0,D
2
1
(t) = 0,D
3
1
(t) = 0,D
4
1
(t) = 0,D
5
1
(t) = 1,
D
1
2
(t) = 0,D
2
2
(t) = 0,D
3
2
(t) = 0,D
4
2
(t) = 2,D
5
2
(t) = 0
BWARdifferentialsarecomputedandfoundtobe,
Δ
1
BWAR,12
(t) = 0, Δ
2
BWAR,12
(t) = 0, Δ
3
BWAR,12
(t) = 1,
Δ
4
BWAR,12
(t) =−0.25, Δ
5
BWAR,12
(t) = 0.125,
Δ
1
BWAR,21
(t) = 0, Δ
2
BWAR,21
(t) = 0, Δ
3
BWAR,21
(t) =−1,
Δ
4
BWAR,21
(t) = 0.25, Δ
5
BWAR,21
(t) =−0.125
Weassumenodescannottransmitandreceiveatthesametime,hence,onlyonelink
is scheduled for transmissionout of the two links (1,2) and (2,1). After computingthe
maximumcommodityforeachlink,theresultwouldbe,
c
∗
BWAR,12
(t) = 3, c
∗
BWAR,21
(t) = 4
By solvingthemaximizationproblemin(3.3), thetransmissionrateμ
3
12
(t) = 1, and
allothertransmissionratesμ
c
ij
(t) = 0,suchthat, (i,j,c)6= (1,2,3).
53
ThusbyBWAR-IDinthisexample,apacketwillbetransmittedfromqueue3atnode
1 to node 2. Since queue 3 occupancy at node 1 gets lower thanq
th
= 1 and since the
thetransmissionis not for apacket to itsdestinationand duplicatebuffer 3 at bothnode
1 and node 2 are not full, a duplicate for the transmitted packet is created and kept in
duplicatebuffer 3 at node 1 and thetransmittedpacket willbe stored in duplicatebuffer
3atnode2(seehowduplicateiscreatedandhowthetransmittedpacketisalsostoredin
theduplicatebufferatt+1inFigure6.1).
6.1.2 FullDuplicateBufferatReceiverExampleinBWAR-ID
Figure6.2: ABWAR-IDexampleshowinghowfullduplicatebuffersatreceiverprevent
creatingduplicates.
54
In BWAR-IM, itrequires onlytheduplicatebufferat thetransmitternot tobe fullin
ordertocreateduplicatesatlowqueueoccupancy. InBWAR-IDhowever,itrequiresthe
duplicate buffers in both the transmitterand the receiver not to be full in order to create
duplicatesatlowqueueoccupancy.
Consider a network with 5 nodes. Assume at timet, node 1 encounters node 2 (see
Figure6.2)sotheycancommunicate;suchthat,
θ
12
(t) =θ
21
(t) = 1
Assumethequeueoccupancyattimetisasfollows,
Q
1
1
(t) = 0,Q
2
1
(t) = 0,Q
3
1
(t) = 1,Q
4
1
(t) = 0,Q
5
1
(t) = 0,
Q
1
2
(t) = 0,Q
2
2
(t) = 0,Q
3
2
(t) = 0,Q
4
2
(t) = 0,Q
5
2
(t) = 0
Assume that there are no arrivals at time t and the queue threshold for duplication
q
th
= 1and theduplicatebuffers havemaximumsizeofD
max
= 2 and haveoccupancy
attimetasfollows,
D
1
1
(t) = 0,D
2
1
(t) = 0,D
3
1
(t) = 0,D
4
1
(t) = 0,D
5
1
(t) = 0,
D
1
2
(t) = 0,D
2
2
(t) = 0,D
3
2
(t) = 2,D
4
2
(t) = 0,D
5
2
(t) = 0
55
BWARdifferentialsarecomputedandfoundtobe,
Δ
1
BWAR,12
(t) = 0, Δ
2
BWAR,12
(t) = 0, Δ
3
BWAR,12
(t) = 0.75,
Δ
4
BWAR,12
(t) = 0, Δ
5
BWAR,12
(t) = 0,
Δ
1
BWAR,21
(t) = 0, Δ
2
BWAR,21
(t) = 0, Δ
3
BWAR,21
(t) =−0.75,
Δ
4
BWAR,21
(t) = 0, Δ
5
BWAR,21
(t) = 0
Aftercomputingthemaximumcommodityforlink(1,2),theresultwouldbe,
c
∗
BWAR,12
(t) = 3
By solvingthemaximizationproblemin(3.3), thetransmissionrateμ
3
12
(t) = 1, and
allothertransmissionratesμ
c
ij
(t) = 0,suchthat, (i,j,c)6= (1,2,3).
Thus by BWAR-ID in this example, a packet will be transmitted from queue 3 at
node 1 to node 2. Since duplicatebuffer 3 at the receivernode 2 is full, there will beno
duplicatecreated eventhoughqueue 3 occupancyatnode 1getslowerthanq
th
= 1 (see
Figure6.2att+1).
6.2 BWAR-TD
As a second enhancement of BWAR, we design BWAR with Time-out based packet
removal and original packets moved to Duplicate buffer upon copy (BWAR-TD)
56
which is a practical implementation of BWAR in which duplicates are deleted from the
duplicate buffer after a predefined timeout value P has passed since the first time the
original packet is admitted to the network. However, the original packet that is kept in
duplicate buffer is flagged and will not be deleted when a timeout occurred. A flagged
packets is only deleted if it gets acknowledged directly by the destination if it has al-
ready received or otherwise it is moved back to the main queue when it encounters the
destination.
6.2.1 DuplicationExampleinBWAR-TD
Figure6.3: AnexampleshowinghowBWAR-TDflagsoriginalpacketswhenduplicated.
57
This is a very similar example to Example 6.1.1. The difference is that the origi-
nal packet is flagged after it has been duplicated and stored in the duplicate buffer (see
Figure 6.3). Flagged packets are not removed by time-out removal. This is useful to
prevent packet loss for the case when none of the packet duplicates are delivered to the
destinationwithinthetime-outperiodP.
6.2.2 UnacknowledgedFlaggedPacketinBWAR-TD
In BWAR-TD, if time-outP has already passed for one packet m, all duplicates of m
that are stored in duplicate buffers are removed except for the flagged copy m
∗
which
correspond to the original copy ofm. Whenm
∗
encounters the destination at timet af-
ter the time-outP (see Figure 6.4), it either gets an acknowledgement that packetm is
already received, and hence,m
∗
will be removed, or it gets a negativeacknowledgment
thatm is not received yet, hence,m
∗
will be movedto original queue at timet+1 (see
Figure6.4forthecasewhenm
∗
getsnegativeacknowledgment). Thisapproachguaran-
tees that no packet will be lost in the case of any two nodes in the network encounters
eachotherinfinitelyoften. However,forthecasethatthisisnottrueandforthecasethat
theflaggedpacketnevergetsachancetobeservedfromtheduplicatebuffer,theflagged
packet might be stuck for ever in the duplicate buffer without delivery. However, it is
guaranteedthatthenumberofthosestuckpacketsisboundedandforthelongrun100%
throughputisguaranteed.
58
6.2.3 UnacknowledgedFlaggedPacketExampleinBWAR-TD
Figure 6.4: A BWAR-TD example showing how a flagged packet moved to original
queueafterthetime-outP whengetsnegativeacknowledgement(NAK)bydestination.
Consider a network with 5 nodes. Assume at timet, node 1 encounters node 2 (see
Figure6.4)sotheycancommunicate;suchthat,
θ
12
(t) =θ
21
(t) = 1
59
Assumethequeueoccupancyattimetisasfollows,
Q
1
1
(t) = 0,Q
2
1
(t) = 0,Q
3
1
(t) = 3,Q
4
1
(t) = 0,Q
5
1
(t) = 0,
Q
1
2
(t) = 0,Q
2
2
(t) = 0,Q
3
2
(t) = 0,Q
4
2
(t) = 0,Q
5
2
(t) = 0
Assume that there are no arrivals at time t and the queue threshold for duplication
q
th
= 1and theduplicatebuffers havemaximumsizeofD
max
= 2 and haveoccupancy
attimetasfollows,
D
1
1
(t) = 0,D
2
1
(t) = 1,D
3
1
(t) = 0,D
4
1
(t) = 0,D
5
1
(t) = 0,
D
1
2
(t) = 0,D
2
2
(t) = 0,D
3
2
(t) = 0,D
4
2
(t) = 0,D
5
2
(t) = 0
BWARdifferentialsarecomputedandfoundtobe,
Δ
1
BWAR,12
(t) = 0, Δ
2
BWAR,12
(t) = 0.625, Δ
3
BWAR,12
(t) = 3,
Δ
4
BWAR,12
(t) = 0, Δ
5
BWAR,12
(t) = 0,
Δ
1
BWAR,21
(t) = 0, Δ
2
BWAR,21
(t) =−0.125, Δ
3
BWAR,21
(t) =−3,
Δ
4
BWAR,21
(t) = 0, Δ
5
BWAR,21
(t) = 0
Aftercomputingthemaximumcommodityforlink(1,2),theresultwouldbe,
c
∗
BWAR,12
(t) = 3
60
By solvingthemaximizationproblemin(3.3), thetransmissionrateμ
3
12
(t) = 1, and
allothertransmissionratesμ
c
ij
(t) = 0,suchthat, (i,j,c)6= (1,2,3).
Thus by BWAR-TD in this example, a packet will be transmitted from queue 3 at
node 1 to queue 3 at node 2. There will be no duplicate created because queue 3 oc-
cupancy at node 1 does not get lower thanq
th
= 1 after transmission. Assume that the
flagged packet in duplicate buffer 2 at node 1 has already passed its time-outP and as-
sume that node 2 (the destination of the flagged packet) informed node 1 at timet that
thepacket is not yet delivered. Hence, theflagged packet is movedto queue 2 at node 1
(seeFigure6.4att+1).
6.3 BWAR-ID-EandBWAR-TD-E
As a third enhancement of BWAR, we design BWAR with Ideal packet removal and
original packets moved to Duplicate buffer upon copy with Energy enhancement
(BWAR-ID-E)andBWARwithTime-outbasedpacketremovalandoriginalpackets
moved to Duplicate buffer upon copy with Energy enhancement (BWAR-TD-E).
BWAR-ID-E is very similar to BWAR-ID except for that number of copies for each
packet is bounded to be less than or equal constantL. Similarly, BWAR-TD-E is very
similarto BWAR-TD except for thatnumberof copies for each packet is bounded to be
lessthanorequalconstantL.
TolimitthenumberofcopiesforeachpacketbyconstantL,weborrowthespraying
idea presented in Spray and Wait (S&W) and Spray and Focus (S&F) [49,50]. Each
61
packet in the network has a field called copycount in its header to specify number of
copiesallowedforthispacket. Whenapacketisadmittedtothenetworkthevalueofits
copycountissettoL. InBWAR-ID-E andBWAR-TD-E,onlypacketswithcopycount>
1 are allowed to be duplicated. When a duplicate m
′
for packet m (that has previous
copycount value equals to L
m
) is created, the copycount for m is set to ⌈
Lm
2
⌉ and the
copycount for m
′
is set to L
m
−⌈
Lm
2
⌉. This method assures fast way of distributing
duplicates to different nodes (since copycount is splittedhalf and halfwhen duplicateis
created) and it also guarantees that number of duplicates for any packet can not exceed
L.
Figure 6.5: A BWAR-ID-E example showing how an original packet is duplicated and
how duplicates are distributed to guarantee number of duplicates does not exceed con-
stantLusingcopycount.
62
Figure 6.5 shows an example of BWAR-ID-E in which copycount is used to govern
number of duplicates. In the figure, you can see that the original packet is allowed to
be duplicated 16 times. By BWAR-ID-E in the figure, the packet is transmitted from
node 1 to node 2 and duplicated. Both copies are stored in the duplicate buffer and the
copycountisreducedtohalfforeach one.
Figure 6.6: A BWAR-ID-E example showing how duplicates are transmitted and dis-
tributedusingcopycount.
In Figure 6.6, an example shows how a packet in a duplicate buffer in BWAR-ID-
E gets a chance to be transmitted and duplicated in which copycount is used to govern
number of duplicates. In the figure, you can see that the duplicate packet is allowed to
beduplicated8times. ByBWAR-ID-Einthisfigure,thisduplicatepacketistransmitted
63
fromnode 1tonode 2andduplicated. Bothcopiesarestoredintheduplicatebufferand
thecopycountisreduced tohalfforeachone.
Figure6.7: A BWAR-ID-E exampleshowinghowa duplicatewith copycount= 1is not
allowedtobeduplicatednortransmitted.
Figure6.7showsanexampleofhowapacketinaduplicatebufferinBWAR-ID-E is
notallowedtobeduplicatednortransmittedsinceithas copycount= 1.
64
Chapter7
Model-basedSimulations
Inthischapter,wepresentoursimulationresultsovercell-partitionednetworkconsider-
ingbothi.i.dandrandomwalkmobility.
7.1 TheCell-PartitionedModel
Themodelheresimplifiesthecontrolvariablestobethewholetransmissionratesμ
ab
(t)
forschedulingandthecommoditytransmissionratesμ
c
ab
(t)forrouting.
We simulate BWAR in the context of encounter-based scheduling and routing for a
simplemodel(cell-partitionednetwork),whichyieldsusefulinsightsonitsperformance.
Inthisidealizedmodelthenetworkdeploymentareaisseparatedintodisjointcells [33]
as follows. We have N nodes and C cells. For collision and interference simplicity,
only one transmission (one packet) is allowed in each cell in each time slot. Because
of this we set q
th
= D
max
= 1. Another simplifying assumption is that the nodes in
thenetworkareorganized intopairs,actingas destinationstoeach other. Each nodehas
65
Bernoulli exogenous arrivals intended for its pair. Depending on the number of cellsC
in the network we can choose the right number of the nodesN ≈ 1.79·C in order to
maximize throughput as shown in [33]. Our simulation results show that by optimizing
numberofnodesbasedonthenumberofcellstomaximizethroughput,thedelayalsois
improved. FortimeoutduplicateremovalswesetthetimeoutvalueP =C.
HereweshowhowBWARworksinthecell-partitionednetworkwiththesimplifying
assumptionthat onlyonetransmissionis allowedper cell pertimeslot. Each timeslott
andforeachcelll wechoosetwonodesa
∗
andb
∗
andcommodityc
∗
suchthat:
• a
∗
andb
∗
areincelll.
• Q
c
∗
a
∗(t)−Q
c
∗
b
∗(t) ≥ Q
c
a
(t)−Q
c
b
(t); for allc, for alla andb in celll at time slott.
Thiscapturesthemaximizationofqueuedifferentialsofthemainqueues.
• Ifthereexistsa,bincelll suchthat,
Q
b
a
(t)−Q
b
b
(t) = Q
c
∗
a
∗(t)−Q
c
∗
b
∗(t) then c
∗
= b
∗
. This captures the destination
advantage.
• Ifthereexistsa,bincelll andcsuchthat
Q
c
a
(t)−Q
c
b
(t) =Q
c
∗
a
∗(t)−Q
c
∗
b
∗(t)and
{c
∗
6=b
∗
or [(c =b) and(c
∗
=b
∗
)]}then
(Q
c
a
(t)+D
c
a
(t))−(Q
c
b
(t)+D
c
b
(t))≤ (Q
c
∗
a
∗(t)+D
c
∗
a
∗(t))−(Q
c
∗
b
∗(t)+D
c
∗
b
∗(t)). This
capturesthemaximizationofduplicatebufferdifferentialsiftherearesometiesin
mainqueuedifferentials.
66
The algorithm simply assigns μ
c
∗
a
∗
b
∗(t) a value of 1, and assigns all other μ
c
ab
(t) a
valueof0suchthata,bincelll.
When a transmission is made from nodea to nodeb of commodityc at time slott
andthattransmissionwillmakeQ
c
a
(t+1)+D
c
a
(t+1) = 0thenthistransmittedpacket
is duplicated and stored in the duplicate buffer of nodea makingD
c
n
(t) = 1 instead of
0. Duplicatepacketsareservedonlyiftherearenooriginalpacketstotransmit. Thereis
strictlowerpriorityofduplicatepacketscomparedtooriginalpackets.
7.2 ProtocolVariants
In thesimulations,weimplementand comparedifferent routingprotocolvariants. They
aredescribedasfollows:
• RegularBackpressure (RB): Thisisthebasicbackpressureschedulingandrout-
ingmechanism,wheredecisionsaremadepurelybasedonqueuedifferentials.
• Regular Backpressure with Destination Advantage (RB-DA): This is a slight
modificationinwhichpacketscorrespondingtothedestinationareprioritizedwhen
the destination is encountered. As we show, this already yields significant delay
improvementsoverregularbackpressure.
• BWAR with Ideal packet removal and original packets retained in the Main
queue (BWAR-IM): This is our novel backpressure with adaptive redundancy in
67
which the destination advantage also holds. Here, when an original packet is du-
plicatedtheoriginalpacketremainsinthemainqueuewhiletheduplicateisstored
in the duplicate buffer. We assume here whenever a packet reaches the destina-
tion, all of its duplicates are deleted including the original one in the main queue
instantaneously.
• BWAR with Ideal packet removal and original packets moved to Duplicate
buffer upon copy (BWAR-ID): This is very similar to BWAR-IM. The only dif-
ference is that whenever an original packet is duplicated, both the original packet
and theduplicateare stored in duplicatebuffers (of coursein two different nodes,
oneinthereceiverandtheotherinthesenderrespectively).
• BWAR with Time-out based packet removal and original packets moved to
Duplicate buffer upon copy (BWAR-TD): This is a practical implementationof
BWARinwhichduplicatesaredeletedfromtheduplicatebufferafterapredefined
timeout value P has passed since the first time the original packet is admitted
to the network. However, the original packet that is kept in duplicate buffer is
flagged and will not be deleted when a timeout occurred. It is only deleted if it
gets acknowledged directly by the destination if its already received or otherwise
itmovedbacktothemainqueuewhenitencountersthedestination.
68
• BWAR with Ideal packet removal and original packets moved to Duplicate
buffer upon copy with Energy enhancement (BWAR-ID-E): This is very sim-
ilar to BWAR-ID. The only difference is that number of copies for each packet is
boundedtobelessthanorequalconstantL.
• BWAR with Time-out based packet removal and original packets moved to
DuplicatebufferuponcopywithEnergyenhancement(BWAR-TD-E):Thisis
very similar to BWAR-TD. The only difference is that number of copies for each
packetisboundedtobelessthanorequalconstantL.
• SprayandWait(S&W):Thisisnotabackpressurebasedmechanism. Sprayand
Wait is presented by T. Spyropoulos et al. [49] which is a state of the art routing
scheme in intermittently connected mobile networks. S&W creates a predefined
fixed number of copies (spraying) of the packet when admitted to the network.
Those copies are distributed to distinct nodes and then each copy waits until it
encounters the destination. We implemented S&W for comparison with BWAR.
OurresultsshowthatBWARoutperformsS&Wespeciallyinhighloadscenarios.
TheevaluationsareconductedusingacustomsimulatorwritteninC++(forrepeata-
bility,wemakeourcodeavailableonlineat http://anrg.usc.edu/downloads/)considering
twomobilitycases: 1)I.I.Dmobility. 2)RandomWalkmobility.
69
15 20 25 30 35 40 45
0
10
20
30
40
50
60
70
80
Number of Nodes (N)
Averae Delay

RB
RB−DA
BWAR−IM
BWAR−ID
BWAR−TD
S&W
Figure7.1: DelayaswevaryN forlowλ = 0.001underi.i.dmobility
7.3 I.I.DMobility
In thissection, we presentour simulationresults forthecase when nodes haveindepen-
dent and identical distributed (i.i.d.) mobility, in which at each time slott, noden can
be inside any cell with equal probabilities of
1
C
. In Figure 7.1, we show average delay
under i.i.d mobility of all above protocol variants as number of nodes N vary for low
loadλ = 0.001 out of the per node capacity region Λ
node
= [0,0.14]. Delay is reduced
significantly when BWAR is used. For this low load scenario all BWAR variants have
almostthesameaveragedelayandtheyperformslightlybetterthanSprayandWait. Fig-
ure 7.1 also shows the great dramatic delay improvement of destination advantage only
withoutanyredundancyinRB-DA comparedtoregularbackpressureRB.
70
10
−3
10
−2
10
−1
10
0
10
1
10
2
10
3
Load(λ)
Average Delay

RB
RB−DA
BWAR−IM
BWAR−ID
BWAR−TD
Figure7.2: DelayaswevaryλforN = 44ofbackpressurevariantsunderi.i.dmobility
Figure7.2comparestheaveragedelayunderi.i.dmobilityofallvariantsofbackpressure-
based protocols as we vary the load. As expected, as the load increases the delay im-
provementofBWARdeclinescompared toRB-DA. Figure7.2 alsoshowshowBWAR-
ID performs much better compared to BWAR-IM beyond some threshold of load(λ).
This shows how moving the duplicated original packet to the duplicate buffer has great
delayenhancementforhighloadscenarios.
In Figure 7.3, results show how BWAR mechanism outperforms Spray and Wait
(S&W)delayperformanceunderi.i.d. mobilityforhighload. ItshowsalsohowBWAR
supportsalmosttwicethecapacityregionofS&W.
Surprisingly in Figure 7.4, BWAR-IM has a better total number of transmissions
compared to regular backpressure RB-DA for low load despite the flooding duplicates
71
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14
0
20
40
60
80
100
120
140
Load(λ)
Average Delay
S&W
BWAR−TD
BWAR−ID
Figure7.3: ComparingS&WdelaywithBWAR-IDandBWAR-TDunderi.i.d. mobility
aswevaryλforN = 44
10
−3
10
−2
10
−1
0
2
4
6
8
10
12
14
x 10
6
Load(λ)
Number of Transmissions

RB
RB−DA
BWAR−IM
BWAR−ID
BWAR−TD
S&W
Figure 7.4: Comparing energy consumption as we vary λ for N = 44 under the cell-
partitionedmodelwithi.i.d. mobility.
72
nature of BWAR at low load. However, we find that Spray and Wait has significantly
superior energy consumption performance. This encourages us to develop an energy
enhancementofBWARthathasenergyconsumptionperformanceclosetosprayandwait
whileinthesametimepreservinggreatdelayandthroughputperformanceofBWAR(see
Figure7.9).
10
0
10
1
10
2
10
3
10
4
10
0
10
1
10
2
10
3
Timeout
Average Delay

λ = 0.001 (BWAR−TD)
λ = 0.016 (BWAR−TD)
λ = 0.064 (BWAR−TD)
λ = 0.128 (BWAR−TD)
λ = 0.001 (BWAR−ID)
λ = 0.016 (BWAR−ID)
λ = 0.064 (BWAR−ID)
λ = 0.128 (BWAR−ID)
Figure7.5: Timeouteffect ofBWAR-TD and comparingitwithBWAR-ID fordifferent
λ∈{0.001,0.016,0.064,0.128}forN = 44 underthecell-partitionedmodelwithi.i.d.
mobility.
Figure 7.5 studies the effect of timeout value P of BWAR-TD for removing du-
plicates under different load scenarios and compares its delay performance with ideal
duplicateremovalsinBWAR-ID underi.i.dmobility.
73
7.4 RandomWalkMobility
0.001 0.002 0.004 0.008 0.016 0.032 0.064 0.128
10
0
10
1
10
2
10
3
Load(λ)
Average Delay

RB
RB−DA
BWAR−IM
BWAR−ID
BWAR−TD
Figure7.6: DelayaswevaryλforN = 44ofbackpressurevariantsunderrandomwalk
mobility
Wepresentinthissectionoursimulationresultsforthecasewhennodeshaverandom
walkmobility,inwhicheachnodeforeachtimesloteithermoveup,down,left,right,or
stayatthesamecellwithequalprobabilitiesof
1
5
.
Figure7.6showshowBWARvariantshavemuchbetterdelayperformancecompared
to traditional backpressure under random walk mobility. It also shows the great benefit
of destination advantage when there are ties in queue differentials. As expected, delay
enhancementdecaysasloadgetclosertothecapacityregion.
74
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16
0
100
200
300
400
500
600
700
800
Load(λ)
Average Delay
S&W
BWAR−TD
BWAR−ID
Figure7.7: ComparingS&WdelaywithBWAR-ID andBWAR-TD underrandomwalk
mobilityaswevaryλforN = 44
75
InFigure7.7,resultsshowhowBWARhasmuchmorethroughputperformancecom-
pared to spray and wait under random walk mobility as it shows how BWAR support
almost as twice as spray and wait can support. It shows how as load get higher, delay
performanceofsprayandwaitgetmuchworsecomparedtoBWAR.
0.001 0.002 0.004 0.008 0.016 0.032 0.064 0.128
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Load(λ)
Average #trans per node per slot

RB
RB−DA
BWAR−IM
BWAR−ID
BWAR−TD
S&W
Figure 7.8: Comparing energy consumption as we vary λ for N = 44 under the cell-
partitionedmodelwithrandomwalkmobility.
Figure7.8 showshowspray and waithas superiorenergy consumptionperformance
compared to traditional backpressure, RB-DA, BWAR-IM, BWAR-ID, and BWAR-TD.
This inspired us to design energy efficient variants of BWAR called, BWAR-ID-E and
BWAR-TD-E in which number of copies for each packet governed to be bounded by
76
predefined constantL. Each packet in BWAR-ID-E and BWAR-TD-E has a field in its
headerspecifyinghowmanycopiesareallowedforthispacket.
0.001 0.002 0.004 0.008 0.016 0.032 0.064 0.128
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Load(λ)
Average #trans per node per slot

RB
RB−DA
BWAR−IM
BWAR−ID
BWAR−TD
S&W
BWAR−ID−E
BWAR−TD−E
Figure 7.9: Showing how energy-efficient BWAR-ID-E and BWAR-TD-E have close
consumptionperformancetoS&Wunderrandomwalkmobility.
After simulating BWAR-ID-E and BWAR-TD-E, results are shown in Figure 7.9.
Resultsshowhowenergy-enhancedBWARvariantshaveveryefficientconsumptionper-
formancethatisveryclosetosprayandwait.
Figure7.10showshowthisenergyenhancementofBWARdoesnotaffectandmain-
tains great delay performance. Results show that BWAR-ID-E and BWAR-TD-E have
almostsamedelayresultscomparedtoBWAR-ID andBWAR-TD respectively.
77
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16
0
100
200
300
400
500
600
700
800
Load(λ)
Average Delay
S&W
BWAR−TD & BWAR−TD−E
BWAR−ID & BWAR−ID−E
Figure 7.10: Showing how energy-efficient BWAR-ID-E and BWAR-TD-E have close
delay and throughput performance to BWAR-ID and BWAR-TD, respectively, under
randomwalkmobility.
78
RegardingoverheadenergyconsumptionforBWAR,Iftheoverheadpacketstoshare
queueoccupancybetweennodesarerelativelysmallcomparedtomuchlargerdatapack-
ets, the power needed for this overhead will be relatively small. Moreover, no need to
send the length of empty queues and this will reduce overhead power in low load. In
addition,foranyschedulingalgorithmbesidesBWAR,someoverheadisneededtocom-
pletely prevent collisions. Some overhead is also needed for nodes just to identify who
are in radio range. Our energy consumption results presented earlier take into account
onlydatatransmissionwithoutanyoverheadtransmission.
79
Chapter8
RealTraceSimulations
In addition to our evaluation of BWAR under simplified model-based simulations, We
went one step further to evaluate BWAR under more realistic mobility. We simulated
BWARbasedonrealvehicularGPStracesfor24hoursobtainedfromafleet of95taxis
in Beijing [4]. Time is slotted so that the day has 1440 one-minute-length time slots.
Weassumetaxishaveradiorangesof 100meters. Toobtainsteadystatethroughputand
delayresults,wesimulateforlargetimebyrepeatingthetracestohaveperiodicchannel
states process, where the period is 24 hours. Our simulation time varies from 128 days
to 1024 days depending on the load setting. We design our simulation in such a way it
keeprunninguntilitfoundasteadystateresultbycheckingandcomparingwithprevious
observations. For high load case scenarios, it takes much more time to find the steady
state result compared to low load case scenarios. Our simulation is also designed to
identify the case when results keeps increasing linearly with time in which the network
isunstableandtheloadishigherthanthecapacityregion.
80
10 20 30 40 50 60 70 80 90
10
20
30
40
50
60
70
80
90

0.5
1
1.5
2
2.5
3
3.5
Figure 8.1: Channel states showing average contact duration between each two taxis in
termsofsecondsperslot.
81
We assume Bernoulli batch packet arrivals to each node that are destined to random
destination. Each batch consists of 60 packets. We assume packets have fixed length of
375000 bytes and transmission rate, between two taxis that are in radio range of each
other, is 3 Mbps. We assume taxis have enough transceivers so they can transmit and
receive to/from one or more taxis at the same time without collisions or interference.
Figure 8.1 shows the average contact duration between each pair of taxis in unit of sec-
onds per slot. Figure 8.2 shows total number of contacts for each time slott during the
day.
0 500 1000 1500
0
5
10
15
20
25
t
Total number of contacts
Figure8.2: Totalnumberofcontactsforeach timeslottduringtheday.
82
We compare BWAR with Spray and Focus (S&F) [49], a state of the art DTN/ICN
routingmechanismsimilartoSprayandWait(S&W).S&Fcreatespredefinedfixednum-
ber of copies (spraying) of the packet when admitted to the network. Those copies are
distributed to distinct nodes similar to S&W. The difference is that each copy is routed
tothedestinationbasedonutilitytimersreseteachtimenodeencountersthedestination.
Packets are routed to nodes with less timer value of its destination. The timer can be
a good indicator of how close is a node to a destination. For this reason, we choose to
compareBWARwithS&Fthatcouldutilizethosetimerstoroutepacketsfastertodesti-
nationsundertherealtraces. However,ourresultsshownosignificantdifferenceofS&F
performancecomparedtoS&W.
In our evaluations, we configure BWAR parameters as follows. The time-out for
BWAR-TD P = 1440 which is the period size. The duplicate buffer size D
max
= 60
which is themaximumnumberof packets ataxi can send duringa timeslot. Thequeue
threshold for duplicationq
th
=D
max
= 60. For Spray and Focus, numberof copies for
eachpacketislimitedbyaconstantL = 64.
Figure8.3,asexpected,showshowBWAR-ID andBWAR-TD havegreatdelayper-
formance compared to regular backpressure with destinationadvantage (RB-DA) under
real mobilitytraces of 95 taxis. Delay enhancement decays as load get closer to the ca-
pacity region as you can see in the figure how BWAR-ID and BWAR-TD converges to
regular backpressure because queue occupancies rarely become low, so duplicates are
rarelycreated.
83
10
−5
10
−4
10
−3
10
−2
10
2
10
3
10
4
Load (λ)
Average Delay

RB−DA
BWAR−ID
BWAR−TD
Figure 8.3: Delay as we vary loadλ under real mobility traces for RB-DA, BWAR-ID,
andBWAR-TD.
84
0 0.002 0.004 0.006 0.008 0.01 0.012 0.014 0.016 0.018 0.02
0
500
1000
1500
2000
2500
3000
3500
Load (λ)
Average Delay
BWAR−TD
BWAR−ID
S&F
Figure 8.4: Comparing S&F delay with BWAR-ID and BWAR-TD under real mobility
tracesaswevaryloadλ.
85
InFigure8.4,resultsshowhowBWARhasmuchmorethroughputperformancecom-
paredtosprayandfocusunderrealmobilitytraces. Itshowshowasloadgethigher,delay
performanceofsprayandfocusgetmuchworsecomparedtoBWAR.
86
Chapter9
RelatedWork
The first theoretical work on backpressure scheduling is the classic result by Tassiulas
and Ephremides in 1992, proving that this queue-differential based scheduling mecha-
nismisthroughputoptimal(i.e.,itcanstabilizeanyfeasibleratevectorinanetwork)[55].
Since then, researchers have combined the basic backpressure mechanism with util-
ity optimization to provide a comprehensive approach to stochastic network optimiza-
tion[8,23,34].
Of most relevance to this work are papers on delay enhancements to backpressure.
A number of papers [10,14,35,36] address the utility-delay tradeoff in optimization-
oriented backpressure, to obtain a tradeoff based on aV parameter such that the utility
is improved by a factor ofO(1/V) while the delay is made to be polylogarithmicinV.
SuchatradeoffhasbeenshowntobepracticallyachievableusingLIFOqueueingin[12],
at the cost of a small probability of dropping packets. The first-ever implementation of
dynamicbackpressureroutingaimedforwirelesssensornetworks(BCP) [27]usessuch
87
a LIFO mechanism. As our focus in this work is not on utility optimization, the tech-
niquespresentedintheseworksaresomewhatorthogonaltotheredundancyapproachwe
develophere. Anothersetofpapers [29,30,62]considertheuseofshortestpathrouting
in conjunction with backpressure to improve the delay performance. These techniques
arewellsuitedforstaticnetworksinwhichsuchpathscan becomputed;however,since
ourfocusison encounterbased networkswithlimitedconnectivity,such an approach is
notapplicable.
In [3], the authors present a mechanism whereby only one real queue is maintained
for each neighbor, along with virtual counters/shadow queues for all destinations, and
show that this yields delay improvements. In [15], a novel variant of backpressure
schedulingmechanismisproposedwhichusesheadoflinepacketdelayinsteadofqueue
lengthsasthebasisofthebackpressureweightcalculationforeachlink/commodity,also
yieldingenhanced delay performance. However, theseworks both assumetheexistence
of static fixed routes. It would be interesting to explore in future work whether their
techniquescanbeappliedtointermittentlyconnectedencounter-basedmobilenetworks,
and if so, how these approaches can be further enhanced by the use of the adaptive re-
dundancythatweproposeinthiswork.
Ryu et al. present twoworks on backpressureroutingaimed specifically forcluster-
basedintermittentlyconnectednetworks[46,47,63]. In[47],theauthorsdevelopatwo-
phase routing scheme, combining backpressure routing with source routing for cluster-
based networks, separating intra-cluster routing from inter-cluster routing. They show
88
that this approach results in large queues at only a subset of the nodes, yielding smaller
delaysthanconventionalbackpressure. In[46],theauthorsimplementtheabove-mentioned
algorithminareal experimentalnetworkandshowthedelayimprovementsempirically.
The key difference of these works from ours is that we do not make any assumption
about theintermittentlyconnected network being organizedin acluster-based hierarchy
andwerequirenopreviousknowledgeofnodesmobility.
Dvir and Vasilakos [6] also consider backpressure routing for intermittently con-
nected networks, with link weights similar to that used in BCP [27]. They evaluate
WeightedFairQueueinginadditiontoLIFOandshowthroughsimulationsthatitoffers
energy improvements. Theirworkdoes not explicitlyaddress additionaldelay improve-
mentsneededforthesekindsofnetworks.
There is a rich literature on routing in delay tolerant / intermittently connected en-
counter based mobile networks (see [51] for a comprehensive survey). Although there
existsingle-copyroutingmechanismsforsuchnetworks[48],ithasbeenwell-recognized
that replication is helpful in reducing delay. While basic epidemic routing [59] creates
multiple message replicas for reliable, fast delivery, it incurs too high of a transmission
cost. Smarter multi-copy routing mechanisms have therefore been developed such as
Spray andWait [49],and SARP[7]. Theseworksintroduceredundantpackettransmis-
sions to improve delay. However, all of these approaches are not adaptive to the traffic
and therefore will hurt thethroughputperformance of thenetwork. This has been noted
89
before, bytheauthors of[46], whowritethat “replication-basedalgorithmssuch as epi-
demic routing for DTNs ... result in lower throughput since multiple copies of a piece
ofdataneedtobeforwardedandstored(andthereforenotthroughputoptimal).” Infact,
in [33], it has been theoretically proved that capacity of such schemes that use fixed re-
dundancyis necessarily lower. In this work, we present the first backpressure algorithm
thatusesreplicationinanadaptivemannersoastomaintainthroughputoptimalitywhile
reducingdelay. WeexplicitlycompareourBWARschemewithSpray&WaitandSpray
& Focus, and show through our evaluation that not only does it provide similar, even
better,delayperformance, itdoessowithouthurtingthroughputoptimality;specifically,
we show that BWAR can handle much higher traffic load than Spray & Wait and Spray
&Focus.
To summarize, this dissertation on BWAR is the first work that explicitly combines
the best of both worlds: multi-copy routing for intermittently connected networks and
throughput-optimal backpressure scheduling. This combination yields better delay per-
formance than traditional backpressure, particularly at low loads, and better ability to
handlehightrafficthantraditionalDTN/ICNroutingschemes.
90
Chapter10
ConclusionsandFutureWork
We havepresented in this dissertationBWAR, an enhanced backpressurealgorithm that
introducesadaptiveredundancytoimprovedelayperformance. Wehaveprovedanalyti-
callythatBWARisthroughputoptimalwhileprovidingabetterdelaybound,particularly
atlowloadsettings.
We have presented an enhanced variant of BWAR so that duplicates are removed
based on distributed, easy-to-implement, time-out mechanism in order to obtain close
delayperformancecomparedtoidealremovalofdeliveredpackets. Inaddition,wehave
presented an energy optimized variant of BWAR that has close energy performance to
Spray&WaitandSpray&Focuswhileatthesametimemaintainingthegreatdelayand
throughputperformanceofBWAR.
Through simulationresults we have shownthat BWAR outperforms both traditional
backpressure (at low loads) and conventional DTN-routing mechanisms (at high loads)
inencounter-basedmobile. Oursimulationscoverthreedifferentmobilityscenarios. The
firstscenarioisi.i.dmobilityinwhichnodescanbeinanycelleachtimeslotwithequal
91
probabilities. The second scenario is random walk mobility in which nodes each time
slot can moveup, down,left, right or stay at thesame cell with equal probabilities. The
thirdscenarioisactualGPSmobilitytraces of95taxisinBeijing,China.
Thereareafewopenavenuesforfutureworksuggestedbyourstudy. First,itwould
beusefultoundertakeamorecarefulanalysisofthedelayimprovementsobtained,relat-
ing them more explicitly, for instance, to arrival process parameters and the underlying
mobility model. Second, it would be good to investigate automated self-configuration
ofthetimeoutparameterfor duplicateremovalthroughadistributedmechanism,as this
is currently statically configured in BWAR. Finally, it would be great to perform real
experimentswithactualimplementationofBWARsoitcanbealiveinpracticeandcom-
pare it with other DTN-routing mechanisms and investigate more for further potential
enhancementsofBWAR.
92
References
[1] M. Alresaini, M. Sathiamoorthy, B. Krishnamachari, M. J. Neely. “Backpressure
withAdaptiveRedundancy(BWAR),” IEEEINFOCOM, 2012.
[2] F. Bai and A. Helmy, “Impact of mobility on last encounter routing protocols,”
IEEESECON, 2007.
[3] L. Bui, R. Srikant, and A. Stolyar, “Novel architectures and algorithms for delay
reductioninback-pressureschedulingandrouting,”IEEEINFOCOM 2009.
[4] Y. Ding, T. Miyaki, T. Riedel, M. Beigl, W Zhang, L Zhang. “Smart Beijing:
Correlation of Urban Electrical Energy Consumption With Urban Environmental
Sensing for OptimizingDistributionPlanning,” Smart Grids, Green Communica-
tionsand ITEnergy-aware Technologies,2011.
[5] H.Dubois-Ferriere,M.Grossglauser,andM.Vetterli,“Agematters: efficientroute
discoveryinmobileadhocnetworksusingencounterages,”ACMMobiHoc2003.
[6] A. Dvir and A. V. Vasilakos, “Backpressure-based routing protocol for DTNs.”
ACMSIGCOMM,2010.
[7] A. Elwhishi and P.-H. Ho, “Sarp - a novel multi-copy routing protocol for inter-
mittentlyconnectedmobilenetworks,”IEEEGLOBECOM 2009.
[8] L.Georgiadis,M. J.Neely,and L. Tassiulas,Resource allocationand cross-layer
controlinwirelessnetworks,FoundationsandTrendsinNetworking,vol.1,2006.
[9] M. Grossglauser and M. Vetterli, “Locating mobilenodes with ease: learning ef-
ficient routes from encounter histories alone,” IEEE/ACM Transactions on Net-
working,vol.14,no.3,2006.
[10] L. Huang and M. Neely, “Delay reduction via lagrange multipliers in stochastic
network optimization,” IEEE Transactions on Automatic Control, vol. 56, no. 4,
2011.
[11] L. Huang. “Deterministic mathematical optimization in stochastic network con-
trol,”Ph.D.Thesis,UniversityofSouthernCalifornia,2011.
93
[12] L. Huang, S. Moeller, M. Neely, and B. Krishnamachari, “Lifo-backpressure
achievesnearoptimalutility-delaytradeoff,”WiOpt,2011.
[13] L. Huang and M. J. Neely. “The optimality of two prices: Maximizing revenue
in a stochasticcommunicationsystem,” IEEE/ACM Transactionson Networking,
vol.18,2010.
[14] L.HuangandM.J.Neely.“Delayreductionvialagrangemultipliersinstochastic
networkoptimization,”WiOpt,2009.
[15] B. Ji, C. Joo, and N. Shroff, “Delay-based back-pressure schedulingin multi-hop
wirelessnetworks,”IEEEINFOCOM, 2011.
[16] J.Jin,A.Sridharan,B.Krishnamachari,andM.Palaniswami,“Handlinginelastic
traffic in multi-hopwireless networks,” IEEE Journal on Selected Areas in Com-
munications,vol.28,no.7,2010.
[17] A. Jindal, and K. Psounis, “Optimizing Multi-Copy Routing Schemes for Re-
sourcesConstrainedInternmittentlyConnectedMobileNetworks,”ACSSC 2006.
[18] A.Jindal,andK.Psounis,“Contention-AwarePerformanceAnalysisofMobility-
AssistedRouting,”Mobilecomputing,vol.8,2009.
[19] A. Jindal, and K. Psounis, “Fundamental mobility properties for realistic perfor-
manceanalysisofintermittentlyconnectedmobilenetworks,”ICMAN 2007.
[20] D.JohnsonandD.Maltz,“Dynamicsourceroutinginadhocwirelessnetworks,”
Mobilecomputing,vol.353,1996.
[21] C. Li. “Stochastic optimization over parallel queues: channel-blind scheduling,
restless bandit, and optimaldelay,” Ph.D. Thesis, University of Southern Califor-
nia,2011.
[22] C. Li and M. J. Neely. “Energy-optimal scheduling with dynamic channel acqui-
sitioninwirelessdownlinks,”IEEECDC, 2007.
[23] X.Lin, N. Shroff, and R. Srikant, “A tutorialoncross-layeroptimizationin wire-
lessnetworks,”IEEEJournalonSelectedAreasinCommunications,vol.24,no.8,
2006.
[24] S. Liu, L. Ying,and R. Srikant. “Throughput-optimalopportunisticschedulingin
thepresenceofflow-leveldynamics,”IEEEINFOCOM,2010.
[25] J. Liu, A. Stolyar, M. Chiang, and H. Poor. “Queue back-pressurerandom access
in multi-hop wireless networks: Optimality and stability,” IEEE Transactions on
InformationTheory, vol.55,2009.
94
[26] N.McKeown,V.Anantharam,andJ.Walrand,“Achieving100%throughputinan
input-queuedswitch,”IEEEINFOCOM 1996.
[27] S. Moeller, A. Sridharan, B. Krishnamachari, and O. Gnawali, “Routing without
routes: the backpressure collection protocol,” ACM/IEEE International Confer-
enceon InformationProcessinginSensorNetworks2010.
[28] S.Moeller.“Dynamicroutingandratecontrolinstochasticnetworkoptimization:
fromtheorytopractice,”Ph.D.Thesis,Universityof SouthernCalifornia,2010.
[29] M. Naghshvar and T. Javidi, “Opportunistic routing with congestion diversity in
wirelessmulti-hopnetworks,”IEEEINFOCOM 2010.
[30] M. J. Neely and R. Urgaonkar, “Optimal backpressure routing for wireless net-
workswithmulti-receiverdiversity,”AdHocNetworks,vol.7,no.5,2009.
[31] M. J. Neely, “Optimal backpressure routing for wireless networks with multi-
receiverdiversity,”InformationSciencesand Systems(CISS)2006.
[32] M.J.Neely,E.Modiano,andC.E.Rohrs,“Dynamicpowerallocationandrouting
fortimevarying wireless networks,” IEEE Journal on Selected Areas in Commu-
nications,vol.23,no.1,January2005.
[33] M. J. Neely and E. Modiano, “Capacity and delay tradeoffs for ad hoc mobile
networks,”IEEETransactionsonInformationTheory,vol.51,no.6,2005.
[34] M.J.Neely,“Stochasticnetworkoptimizationwithapplicationtocommunication
and queueing systems,” Synthesis Lectures on Communication Networks, vol. 3,
no.1,2010.
[35] M.Neely,“Super-fastdelaytradeoffsforutilityoptimalfairschedulinginwireless
networks,” IEEE Journal on Selected Areas in Communications, vol. 24, no. 8,
2006.
[36] M.Neely,“Intelligentpacketdroppingforoptimalenergy-delaytradeoffsinwire-
lessdownlinks,”IEEETransactionsonAutomaticControl,vol.54,no.3,2009.
[37] M. J. Neely and R. Urgaonkar. “Opportunism, backpressure, and stochastic opti-
mizationwiththewirelessbroadcastadvantage,”IEEESSC,2008.
[38] M. J. Neely, E. Modiano, and C. E. Rohrs. “Power allocation and routing in
multi-beam satellites with time-varying channels,” IEEE/ACM Transactions on
Networking,vol.11,2003.
[39] M. J. Neely, E. Modiano, and C. Li. “Fairness and optimal stochastic control for
heterogeneousnetworks,”IEEE/ACMTransactionsonNetworking,vol.16,2008.
95
[40] M.J. Neely. “Delay analysisfor maxweight opportunisticschedulingin wireless
systems.”IEEETransactionson AutomaticControl,vol,54,2009.
[41] M.J.Neely.“Orderoptimaldelayforopportunisticschedulinginmulti-userwire-
less uplinks and downlinks,” IEEE/ACM Transactions on Networking, vol. 16,
2008.
[42] M. J. Neely. “Energy optimal control for time varying wireless networks,” IEEE
TransactionsonInformationTheory,vol.52,2006.
[43] M. J. Neely. “Dynamic power allocation and routing for satellite and wireless
networks with time varying channels,” Ph.D. Thesis, Massachusetts Institute of
Technology,2003.
[44] J.NiandR.Srikant.“Q-csma: Queue-lengthbasedcsma/caalgorithmsforachiev-
ingmaximumthroughputandlowdelayinwirelessnetworks,”ITA, 2009.
[45] C.PerkinsandE.Royer,“Ad-hocon-demanddistancevectorrouting,”IEEEWM-
CSA2009.
[46] J.Ryu,V.Bhargava,N.Paine,andS.Shakkottai,“Back-pressureroutingandrate
controlforicns,”ACM Mobicom,2010.
[47] J.Ryu,L.Ying,andS. Shakkottai,“Back-pressureroutingforintermittentlycon-
nectednetworks,”IEEEINFOCOM 2010.
[48] T. Spyropoulos, K. Psounis, and C. S. Raghavendra, “Efficient routing in inter-
mittently connected mobile networks: The single-copy case,” IEEE/ACM Trans-
actionson Networking,vol.16,no.1,2008.
[49] T.Spyropoulos,K.Psounis,andC.S.Raghavendra,“Efficientroutinginintermit-
tently connected mobile networks: the multiple-copycase,” IEEE/ACM Transac-
tionson Networking,vol.16,no.1,2008.
[50] T.Spyropoulos,K.Psounis,andC. S.Raghavendra,“Spray andwait: anefficient
routingschemeforintermittentlyconnectedmobilenetworks,”ACM SIGCOMM,
2005.
[51] T. Spyropoulos, R. N. Rais, T. Turletti, K. Obraczka, and A. Vasilakos, “Rout-
ing for disruption tolerant networks: taxonomy and design,” Wireless Networks,
vol.16,2010.
[52] A. Sridharan, S. Moeller, and B. Krishnamachari, “Implementing backpressure-
basedratecontrolinwirelessnetworks,”ITAWorkshop,2009.
[53] A.Sridharan,S.Moeller,andB.Krishnamachari.“Makingdistributedratecontrol
usinglyapunovdriftsarealityinwirelesssensornetworks,”WiOpt,2008.
96
[54] A. Stolyar. “Maximizing queueing network utility subject to stability: Greedy
primal-dualalgorithm,”QueueingSystems,vol.50,2005.
[55] L. Tassiulas and A. Ephremides, “Stability properties of constrained queueing
systems and scheduling policies for maximum throughput in multihop radio net-
works,”IEEETransactionson AutomaticControl,vol.37,no.12,1992.
[56] L. Tassiulas, and A. Ephremides, “Dynamic server allocation to parallel queues
with randomly varying connectivity,” IEEE Transactions on Information Theory,
vol.39,1993.
[57] R. Urgaonkarand M.J. Neely.“Opportunisticschedulingwithreliabilityguaran-
tees in cognitiveradio networks,” IEEE Transactions on Mobile Computing, vol.
8,2009.
[58] R. Urgaonkar. “optimal resource allocation and cross-layer control in cognitive
andcooperativewirelessnetworks,”Ph.D.Thesis,UniversityofSouthernCalifor-
nia,2011.
[59] A. Vahdat and D. Becker, “Epidemic routing for partially-connected ad hoc net-
works,”DukeUniversity,Tech.Rep.,2000.
[60] P.Ven,S.Borst,andS.Shneer.“Instabilityofmaxweightschedulingalgorithms,”
IEEEINFOCOM 2009.
[61] A.Warrier,S.Janakiraman,S.Ha,andI.Rhee,“Diffq: Practicaldifferentialback-
logcongestioncontrolforwirelessnetworks,”IEEEINFOCOM, 2009.
[62] L. Ying, S. Shakkottai, and A. Reddy, “On combining shortest-path and back-
pressureroutingovermultihopwirelessnetworks,”IEEEINFOCOM 2009.
[63] L. Ying, R. Srikant, and D. Towsley. “Cluster-based back-pressure routing algo-
rithm,”IEEEINFOCOM 2008.
97

Abstract (if available)

Abstract Backpressure scheduling and routing, in which packets are preferentially transmitted over links with high queue differentials, offers the promise of throughput-optimal operation for a wide range of communication networks. However, when the traffic load is low, due to the corresponding low queue occupancy, backpressure scheduling/routing experiences long delays. This is particularly of concern in intermittent encounter-based mobile networks which are already delay-limited due to the sparse and highly dynamic network connectivity. While state of the art mechanisms for such networks have proposed the use of redundant transmissions to improve delay, they do not work well when the traffic load is high. We propose in this dissertation a novel hybrid approach that we refer to as backpressure with adaptive redundancy (BWAR), which provides the best of both worlds. This approach is highly robust and distributed and does not require any prior knowledge of network load conditions. We present an enhanced variant of BWAR so that duplicates are removed based on distributed, easy-to-implement, time-out mechanism in order to obtain close delay performance compared to ideal removal of delivered packets. In addition, we introduce an energy optimized variant of BWAR while at the same time maintaining the great delay and throughput performance of BWAR. We evaluate BWAR through both mathematical analysis and simulations based on a cell-partitioned model and real traces of taxis in Beijing. We prove theoretically that BWAR does not perform worse than traditional backpressure in terms of the maximum throughput, while yielding a better delay bound. The simulations confirm that BWAR outperforms traditional backpressure at low load, while outperforming state of the art encounter-routing schemes (Spray & Wait and Spray & Focus) at high load.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

On practical network optimization: convergence, finite buffers, and load balancing

PDF

Joint routing, scheduling, and resource allocation in multi-hop networks: from wireless ad-hoc networks to distributed computing networks

PDF

Algorithmic aspects of throughput-delay performance for fast data collection in wireless sensor networks

PDF

Dynamic routing and rate control in stochastic network optimization: from theory to practice

PDF

Modeling intermittently connected vehicular networks

PDF

Multichannel data collection for throughput maximization in wireless sensor networks

PDF

Efficient delivery of augmented information services over distributed computing networks

PDF

Strategic and transitory models of queueing systems

PDF

Optimizing task assignment for collaborative computing over heterogeneous network devices

PDF

Understanding and exploiting the acoustic propagation delay in underwater sensor networks

PDF

Optimal resource allocation and cross-layer control in cognitive and cooperative wireless networks

PDF

QoS-aware algorithm design for distributed systems

PDF

Dynamic packet fragmentation for increased virtual channel utilization and fault tolerance in on-chip routers

PDF

Time synchronization and scheduling in underwater wireless networks

PDF

Scheduling and resource allocation with incomplete information in wireless networks

PDF

Data-driven optimization for indoor localization

PDF

Learning, adaptation and control to enhance wireless network performance

PDF

A variation aware resilient framework for post-silicon delay validation of high performance circuits

PDF

Robust routing and energy management in wireless sensor networks

PDF

Relative positioning, network formation, and routing in robotic wireless networks

Asset Metadata

Creator Alresaini, Majed (author)

Core Title Backpressure delay enhancement for encounter-based mobile networks while sustaining throughput optimality

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Computer Engineering

Publication Date 07/24/2012

Defense Date 06/20/2012

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag delay tolerant networks,DTN,duplicates,intermittently connected,Lyapunov,OAI-PMH Harvest,redundancy,routing,scheduling,stability

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Krishnamachari, Bhaskar (committee chair), Golubchik, Leana (committee member), Neely, Michael J. (committee member)

Creator Email alresain@usc.edu,malresaini@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-61078

Unique identifier UC11290086

Identifier usctheses-c3-61078 (legacy record id)

Legacy Identifier etd-AlresainiM-971.pdf

Dmrecord 61078

Document Type Dissertation

Rights Alresaini, Majed

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA