Computer Science Technical Report Archive

USC Computer Science Technical Reports, no. 726 (2000)

(USC DC Other)

USC Computer Science Technical Reports, no. 726 (2000)

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content P erformance Ev aluation of Multip oin t Proto cols Using Systematic Scenario Syn thesis A
Case Study for Timersuppression Mec hanisms
Ahmed Helm y Sandeep Gupta Deb orah Estrin Alb erto Cerpa Y an Y u
Univ ersit y of Southern California Los Angeles CA email helm ycenguscedu sandeepb o oleuscedu festrincerpay an yuguscedu
A bstr act The adv entofm ultip ointm ulticastbased applications
and the gro wth and complexityofthe In ternet has com
plicated net w ork proto col design and ev aluation
In this pap er w e presen t a metho d for automatic syn
thesis of w orst and b est case scenarios for m ultip oin t pro
to col p erformance ev aluation Our metho d uses a fault
orien ted test generation F OTG algorithm for searc hing
the proto col and system state space to syn thesize these
scenarios The algorithm is based on a global nite state
mac hine FSM mo del W e extend the algorithm with
timing seman tics to handle endtoend dela ys and address
p erformance criteria W e in tro duce the notion of a vir
tual LAN to represen t dela ys of the underlying m ulticast
distribution tree
As a case studyw e use our metho d to ev aluate v arian ts
of the timer suppression mec hanism used in v arious m ulti
poin t proto cols with resp ect to t w o p erformance criteria
o v erhead of resp onse messages and resp onse time Sim ula
tion results for reliable m ulticast proto cols sho w that our
metho d pro vides a scalable w a y for syn thesizing w orstcase
scenarios automatically W e exp ect our metho d to serv e
as a mo del for applying systematic scenario generation to
other m ultip oin t proto cols
I Intr oduction
The longevit y and po w er of In ternet tec hnologies de
riv es from its abilit y to op erate under a wide range of
op erating conditions underlying top ologies and trans
mission c haracteristics as w ell as heterogeneous appli
cations generating v aried trac inputs P erhaps more
than an y other tec hnology the range of op erating condi
tions is enormous it is the cross pro duct of the top and
b ottom of the IP proto col stac k
P erhaps it is this enormous set of conditions that has
inhibited the dev elopmen t of systematic approac hes to
analyzing In ternet proto col designs Howcan w e test cor
rectness or c haracterize p erformance of a proto col when
the set of inputs is in tractable Nev ertheless net w orking
infrastructure is increasingly critical and there is enor
mous need to increase the robustness and understanding
of net w ork proto cols It is time to dev elop tec hniques for
systematic testing of proto col b eha vior ev en in the face
of the ab o vec hallenges and obstacles A t the same time
w e do not exp ect that complex adaptiv e proto cols will
b e automatically v eriable under their full range of con
ditions Rather w e are prop osing a framew ork in whic h
a proto col designer can follo w a set of systematic steps
assisted b y automation where p ossible to co v er a sp ecic
part of the design and op erating space
In our prop osed framew ork a proto col designer will
still need to create the initial mec hanisms describ e it
in the form of a nite state mac hine and iden tify the
p erformance criteria or correctness conditions that needs
to b e in v estigated Our automated metho d will pickup
at that p oin t pro viding algorithms that ev en tually result
in scenarios or test suites that stress the proto col with
resp ect to the iden tied criteria
This pap er demonstrates our progress in realizing this
vision as w e presen t our metho d and apply it to the p er
formance ev aluation of m ultip oin t proto cols
A Motivation
The recen t growthofthe In ternet and its increased het
erogeneit y has in tro duced new failure mo des and added
complexit y to proto col design and testing In addition
the adv entof m ultip oin t applications has in tro duced new
c hallenges of qualitativ ely dieren t nature than the tra
ditional p oin ttop oin t proto cols Multip oin t applications
in v olv e a group of receiv ers and one or more senders si
m ultaneously As more complex m ultip oin t applications
and proto cols are coming to life the need for systematic
and automatic metho ds to study and ev aluate suchpro to cols is becoming more apparen t Suc h metho ds aim
to exp edite the proto col dev elopmen t cycle and impro v e
resulting proto col robustness and p erformance
Through our prop osed metho dology for test syn thesis
w e hop e to address the follo wing k ey issues of proto col
design and ev aluation
Scenario dep endentev aluation and the use of v alida
tion test suites Proto cols maybe ev aluated for correct
ness and p erformance In man yev aluation studies of m ul
tip oin t proto cols the results are dep enden tuponsev eral
factors suc h as mem b ership distribution and net w ork
top ology Hence conclusions dra wn from these studies
dep end hea vily up on the ev aluation scenarios
Proto col dev elopmen t usually passes through iterativecy cles of renemen t whic h requires revisiting the ev aluation
scenarios to ensure that no erroneous b eha vior has b een
in tro duced This brings ab out the need for v alidation test
suites Constructing these test suites can b e an onerous
and errorprone task if p erformed man ually Unfortu
natelylittlew ork has b een done to automate the gener
ation of suc h tests for m ultip ointnet w ork proto cols In
this pap er w e prop ose a metho d for syn thesizing test sce
narios automatically for m ultip oin t proto col ev aluation
W orstcase analysis of proto cols It is dicult to de
sign a proto col that w ould p erform w ell in all en viron
men ts Ho w ev er iden tifying breaking p oin ts that violate
correctness or exhibit w orstcase p erformance b eha viors
of a proto col maygiv e insigh t to proto col designers and
help in ev aluating design tradeos In general it is de
sirable to iden tify early on in the proto col dev elopmen t
cycle scenarios under whic h the proto col exhibits w orst
or b est case b eha vior
The metho d presen ted in this pap er automates the gen
eration of scenarios in whichm ultip oin t proto cols exhibit
w orst and b est case b eha viors
P erformance benc hmarking New proto cols ma y pro
pose to rene a mec hanism with resp ect to a particu
lar p erformance metric using for ev aluation those sce
narios that sho w p erformance impro v emen t Ho w ev er
without systematic ev aluation these renemen t studies
often though unin ten tionally o v erlo ok other scenarios
that ma y be relev an t T o alleviate suc h a problem w e
prop ose to in tegrate stress test scenarios that pro vide an
ob jectiv e b enc hmark for p erformance ev aluation
Using our scenario syn thesis metho dology w e hop e to
con tribute to the understanding of b etter p erformance
benc hmarking and the design of more robust proto cols
B Backgr ound
The design of m ultip oin t proto cols has in tro duced new
c hallenges and problems Some of the problems are com
mon to a wide range of proto cols and applications One
suc h problem is the multir esp onder problem where m ul
tiple mem b ers of a group ma y resp ond almost sim ulta
neously to an ev en t whic hma y cause a o o d of messages
throughout the net w ork and in turn ma y lead to syn
c hronized resp onses and ma y cause additional o v erhead
eg the w ellkno wn A ck implosion problem leading to
p erformance degradation
One common tec hnique to alleviate the ab o v e prob
lem is the multic ast damping tec hnique whic h emplo ys a
timer suppr ession mec hanism TSM TSM is emplo y ed
in sev eral m ultip oin t proto cols including the follo wing
IPm ulticast proto cols eg PIM and IGMP use TSM on LANs to reduce JoinPrune con trol o v er
head
Reliable m ulticast sc hemes eg SRM and
MFTP use this mec hanism to alleviate A ck implo
sion V arian ts of the SRM timers are used in registry
replication eg RRM and adaptiv ew eb cac hing Multicast address allo cation sc hemes eg AAP and
SDr use TSM to a v oid an implosion of resp onses dur
ing the collision detection phase
Activ e services use m ulticast damping to launc h
one service agen t serv en t from a p o ol of serv ers
TSM is also used in selforganizing hierarc hies SCAN and transp ort proto cols eg XTP and R TP
W e b eliev e TSM is a go o d building blo c k to analyze
as our rst endtoend case study since it is ric hinm ul
ticast and timing seman tics and can b e ev aluated using
standard p erformance criteria Asacase study w e ex
amine its w orst and b est case b eha viors in a systematic
automatic fashion
In TSM a mem ber of a m ulticast group that has de
tected loss of a data pac k et m ulticasts a request for re
co v ery Other mem b ers of the group that receiv e this re
quest and that ha v e previously receiv ed the data pac k et
Suchbeha vior is not proto col sp ecic and if a proto col is comp osed
of previously c hec k ed building blo c ks these parts of the proto col need
not b e rev alidated in full Ho w ev er in teraction b et w een the building
blo c ks still needs to b e v alidated
sc hedule transmission of a resp onse In general random
ized timers are used in sc heduling the resp onse While
a resp onse timer is running at one no de if a resp onse
is receiv ed from another no de then the resp onse timer is
suppressed to reduce the n um b er of resp onses triggered
Consequen tly the resp onse time ma y b e dela y ed to allo w
for more suppression
Tw o main p erformance ev aluation criteria used in this
case are o v erhead of resp onse messages and time to re
co v er from pac k et loss Dep ending on the relativedela ys
bet w een group mem b ers and the timer settings the mec h
anism ma y exhibit dieren t p erformance In this study our metho d attempts to obtain scenarios of b est case and
w orst case p erformance according to the ab o v e criteria
The rest of the pap er is organized as follo ws Section I I
in tro duces the proto col and top ology mo dels Section I I I
outlines the main algorithm and Section IV presen ts the
mo del for TSM Sections V and VI presen t p erformance
analyses for proto col o v erhead and resp onse time and
Section VI I presen ts sim ulation results Related w ork is
giv en in Section VI I I Issues and future w ork are dis
cussed in Section IX Wepresen t concluding remarks in
Section X Algorithmic details mathematical mo dels and
example case studies are giv en in the app endices
II The model
The mo del is a pro cessable represen tation of the system
under study that enables automation of our metho d Our
o v erall mo del consists of A the proto col mo del B the
top ology mo del and C the fault mo del
A The Pr oto c ol Mo del
W e represen t the proto col b y a nite state mac hine
FSM and the o v erall system b y a global FSM GFSM
I FSM mo del Ev ery instance of the proto col run
ning on a single endsystem is mo deled b y a deterministic
FSM consisting of i a set of states ii a set of stim
uli causing state transitions and iii a state transition
function or table describing the state transition rules
A proto col running on an endsystem i is represen ted b y
the mac hine M i S i i i where S i is a nite set of
state sym bols i is the set of stim uli and i is the state
transition function S i i S i II Glob al FSM mo del The global state is dened as
the comp osition of individual endsystem states The b e
ha vior of a system with n endsystems ma y b e describ ed
b y M G S G G G where S G S S S n is
the global state space G n
S
i i is the set of stim uli and
G is the global state transition function S G G S G B The T op olo gy Mo del
The top ology cannot b e captured simply b y one metric
Indeed its dynamics ma y b e complex to mo del and some
times in tractable W e mo del the dela ys using the dela y
matrix and loss patterns using the fault mo del Weuse a
virtual LAN VLAN mo del to represen t the underlying
net w ork top ology and m ulticast distribution tree The
VLAN captures delayseman tics using a dela y matrix D
see Figure where d ij is the dela y from system i to
system j Q
V. LAN
1
2 3
0 dQ,1 dQ,2 dQ,3
d1,Q 0 d1,2 d1,3
d2,Q d2,1 0 d2,3
d3,Q d3,1 d3,2 0
D =
Fig The virtual LAN and the dela y matrix
C The F ault Mo del
A fault is a lo w lev el eg ph ysical la y er anomalous
beha vior that ma y aect the proto col under test F aults
ma y include pac k et loss system crashes or routing lo ops
F or brevit yw e only consider selectiv e pac k et loss in this
study Selectiv epac k et loss o ccurs when a m ulticast mes
sage is receiv ed b y some group mem b ers but not others
The selectiv e loss of a message prev en ts the transition
that this message triggers at the in tended recipien t
III Algorithm and Objectives
T o apply our metho d the designer sp ecies the proto
col as a global FSM mo del In addition the ev aluation
criteria b e it related to p erformance or correctness are
giv en as input to the metho d In this pap er w e address
p erformance criteria correctness has b een addressed in
previous studies The algorithm op erates on
the sp ecied mo del and syn thesizes a set of test scenar
ios proto col ev en ts and relations b et w een top ology de
la ys and timer v alues that stress the proto col according
to the ev aluation criteria eg exhibit maxim um o v er
head or dela y In this section w e outline the algorithmic
details of our metho d The algorithm is further discussed
in section V and illustrated b y a case study A A lgorithm Outline
Our algorithm is a v arian t of the faultorien ted test gen
eration F OTG algorithm presen tedin It includes
the top ology syn thesis the bac kw ard searc h and the for
w ard searc h stages Here w e describ e those asp ects of
our algorithm that deal with timing and p erformance se
man tics The basic algorithm passes through three main
steps the target ev entiden tication the searc h
and the task sp ecic solution
The target ev en t The algorithm starts from a giv en
ev en t called the target ev en t The target ev en t eg
sending a message is iden tied b y the designer based on
the proto col ev aluation criteria eg o v erhead
The searc h Three steps are tak en in the searc h
a iden tifying conditions b obtaining sequences and
c form ulating inequalities
a Identifying c onditions The algorithm uses the pro
to col transition rules to iden tify transitions necessary to
trigger the target ev en t and those that prev en t it these
transitions are called wante d tr ansitions and unwante d
tr ansitions resp ectiv ely b Obtaining se quenc es Once the ab o v e transitions
are iden tied the algorithm uses bac kw ard and forw ard
searc h to build ev en t sequences leading to these transi
tions and calculates the times of these ev en ts as follo ws
i Bac kw ard searc h is used to iden tify ev en ts preced
ing the w an ted and un w an ted transitions and uses im
plication rules that op erate on the proto cols transition
table Section IVB describ es the implication rules
ii F orw ard searc h is used to v erify the bac kw ard
searc h Ev ery bac kw ard step m ust corresp ond to v alid
forw ard steps Branc hes leading to con tradictions b e t w een forw ard and bac kw ard searc h are rejected F orw ard
searc h is also used to complete ev en t sequences necessary
to main tain system consistency
c F ormulating ine qualities Based on the transitions
and timed sequences obtained in the previous steps the
algorithm form ulates relations b et w een timer v alues and
net w ork dela ys that trigger the w an ted transitions and
a v oid the un w an ted transitions
T ask sp ecic solution The output of the searchis
a set of ev en t sequences and inequalities that satisfy the
ev aluation criteria These inequalities are solv ed mathe
matically to nd a top ology or timer conguration de
p ending on the task denition
B T ask Denition
W e apply our metho d to t w o kinds of tasks
T op ology syn tehsis is p erformed when the timer v al
ues are giv en and the ob jectiv e is to iden tify the dela y
matrix that pro duces the b est or w orst case b eha vior
Timer conguration is p erformed when the top ol
ogy or dela y matrix is giv en and the timer v alues that
cause the b est and w orst case b eha vior are to b e deter
mined
IV The Timer Suppression Mechanism TSM
In this section w epresen t a simple description of TSM
then presen t its mo del used thereafter in the analysis
TSM in v olv es a request q and one or more resp onses p When a system Q detects the loss of a data pac k et it sets
a request timer and m ulticasts a request q When a sys
tem i receiv es q it sets a resp onse timer eg randomly
the expiration of whic h after duration Exp i triggers a
resp onse p If the system i receiv es a resp onse p from
another system j while its timer is running it suppresses
its o wn resp onse
A Performanc e Evaluation Criteria
W e use t w o p erformance criteria to ev aluate TSM
Ov erhead of resp onse messages where the w orst case
pro duces the maxim um n um ber of resp onses per data
pac k et loss As an extreme case this o ccurs when all
p oten tial resp onders do indeed resp ond and no suppres
sion tak es place
The role of forw ard searc h will b e further illustrated in the resp onse
time analysis in Section VI
The resp onse dela y where w orst case scenario pro
duces maxim um loss reco v ery time
B Timer Suppr ession Mo del
F ollo wing is the TSM mo del used in the analysis
B Proto col states S F ollo wing is the state sym b ol table for the TSM mo del
State Meaning
R original state of the requester Q
R
T
requester with the request timer set
D p oten tial resp onder
D
T
resp onder with the resp onse timer set
B Stim uli or Ev en ts Sendingreceiving messages sending resp onse p t and request q t receiving resp onse p r and request q r Timer and other ev en ts the ev en ts of ring the re
quest timer Req and resp onse timer Res and the ev entof
detecting pac k et loss L B Notation
F ollo wing are the notations used in the transition table
An ev en t subscript denotes the system initiating the
ev en t eg p t
i
is resp onse sen t b y system i while the
subscript m denotes m ulticast reception eg p r m
denotes
reception of a resp onse byall mem b ers of the group if no
loss o ccurs When system i receiv es a message sen t b y
system j this is denoted b y the subscript i j eg p r
ij
is system i receiving resp onse from system j The state subscript T denotes the existence of a timer
and is used b y the algorithm to apply the timer implica
tion to re the timer ev en t after the expiration p erio d
A state transition has a start state and an end state
and is expressed in the form star tS tate endS tate eg
D D T It implies the existence of a system in the
star tS tate ie D as a condition for the transition to
the endS tate ie D T
An ee ct in the transition table ma y con tain state
transition and stim ulus in the form star tS tate endS tate stimul us whic h indicates that the condition
for triggering stimul us is the state transition An eect
ma y con tain sev eral transitions eg T rans T rans
whic h indicates that out of these transitions all transi
tions with satised conditions will o ccur
B T ransition T able F ollo wing is the transition table for TSM
Sym bol Ev en t Eect Meaning
loss L R R
T
q t loss detection causes q t
and setting of request timer
tx req q t q r m
transmission of q causes
m ulticast reception of q
after net w ork dela y
rcv req q r D D
T
reception of q causes a
system in D state
to set resp onse timer
res tmr Res D
T
D p t resp onse timer expiration
causes transmission of p
and a c hange to D state
tx res p t p r m
transmission of p causes
m ulticast reception of p
after net w ork dela y
rcv res p r R
T
R reception of p bya
D
T
D system with the timer
set causes suppression
req tmr Req q t expiration of request timer
causes transmission of q
The mo del con tains one requester Q and sev eral p oten
tial resp onders eg i and j Initially the requester
Q exists in state R and all p oten tial resp onders exist in
state D Let t be the time at whic h Q sends the re
quest q The request sen t b y Q is receiv ed b y i and j
at times d Qi and d Qj resp ectiv ely When the request q
is sen t the requester transitions in to state R T b y setting
the request timer Up on receiving a request a p oten tial
resp onder in state D transitions in to state D T b y setting
the resp onse timer The time at whichan ev en t o ccurs is
giv en b y t ev ent eg q r
j
occurs at t q r
j
B Implication Rules
The bac kw ard searc h uses the follo wing causeeect im
plication rules
T ransmissionReception Tx Rcv By the reception
of a message the algorithm implies the transmission of
that message without loss sometime in the past after
applying the net w ork dela ys An example of this impli
cation is p r
ij
p t
j
where t p r
ij
t p t
j
d ji Timer Expiration Tmr Exp When a timer expires
the algorithm infers that it w as set Exp time units in
Since there is only one requester w esimply use q t instead of q t
Q
and q r
i
instead of q r
iQ
The time of a state is when the state w as rst created so t D
T
i
is
the time at whic h i transited in to state D
T

the past and that no ev en t o ccurred during that p erio d
to reset the timer An example of this implication is
Res i D i D T
i
D T
i
where t Res i t D T
i
Exp i and Exp i is the duration of the resp onse timer Res i State Creation St Cr A state is created from an
other b y rev ersing the transition rules and going to
w ards the star tS tate of the transition F or example
D T
i
D T
i
D i
In the follo wing sections w e use the ab o v e mo del to
syn thesize w orst and b est case scenarios according to pro
to col o v erhead and resp onse time
V Pr otocol O verhead Anal ysis
In this section w e conduct w orst and b est case p erfor
mance analyses for TSM with resp ect to the n um ber of
resp onses triggered p er pac k et loss Initiallyw e assume
no loss of request or resp onse messages un til reco v eryand
that the request timer is high enough that the reco v ery
will o ccur within one request round The case of m ultiple
request rounds is discussed in App endix I I I A Worstc ase analysis
W orstcase analysis aims to obtain scenarios with max
im um n um b er of resp onses p er data loss In this section
w e presen t the algorithm to obtain inequalities that lead
to w orstcase scenarios These inequalities are a function
of net w ork dela ys and timer expiration v alues
A T arget ev en t and conditions
Since the o v erhead in this case is measured as the n um
b er of resp onse messages the designer iden ties the ev en t
of triggering a resp onse p t as the target ev en t and the
goal is to maximize the n um b er of resp onse messages
A The searc h
As previously describ ed in section I I IA the main steps
for the searc h algorithm are
Iden tifying the w an ted and un w an ted transitions
Obtaining sequences leading to the ab o v e transitions
and calculating the times for these sequences
F orm ulating the inequalities that ac hiev e the time con
strain ts required to in v okew an ted transitions and a v oid
un w an ted transitions
W e use the notation E v entE f f ect to represen t a transition
F ollo wing w e apply these steps to our case study Iden tifying conditions
The algorithm searc hes for the transitions necessary to
trigger the target ev en t and their conditions recursiv ely These are called wante d tr ansitions and wante d c ondi
tions resp ectiv ely The algorithm also searc hes for tran
sitions that n ullify the target ev entor in v alidate anyof
its conditions These are called unwantedtr ansitions In our case the target ev en t is the transmission of a re
sp onse ie p t F rom the transition table describ ed in
Section IVB the algorithm iden ties transition r es tmr
Res D T D p t as a wante d tr ansition and its con
dition D T as a wante d c ondition T ransition r cv r e q
q r D D T is also iden tied as a wantedtr ansition since
it is necessary to create D T The unwantedtr ansition is
iden tied as transition r cv r es p r D T D since it alters
the D T state without in v oking p t Obtaining sequences
Using bac kw ard searc h the algorithm obtains sequences
and calculates time v alues for the follo wing transitions
w an ted transition r es tmr w an ted transition
r cv r e q and un w an ted transition r cv r es as follo ws
T o obtain the sequence of ev en ts for transition
r es tmr the algorithm applies implication rules see Sec
tion IVB Tmr Exp St Cr Tx Rcv in that order and
w e get
Res i D i D T
i
p t
i
q r
i
D T
i
D i q t
Q
Hence the calculated time for t p t
i
b ecomes
t p t
i
t d Qi Exp i T o obtain the sequence of ev en ts for transition r cv r e q
the algorithm applies implication rule Tx Rcv and w e get
q r
i
D T
i
D i q t
Q
Hence the calculated time for t q r
i
becomes
t q r
i
t d Qi T o obtain sequence of ev en ts for transition r cv r es for
systems i and j the algorithm applies implication rules
Tx RcvTmr Exp St Cr Tx Rcv in that order and w e
get
p r
ij
D i D T
i
Res j D j D T
j
p t
j
q r
j
D T
j
D j q t
Q
Hence the calculated time for t p r
ij
b ecomes
t p r
ij
t d Qj Exp j d ji F orm ulating Inequalities
Based on the ab o vew an ted and un w an ted transitions the
algorithm a v oids transition r cv r es while in v oking transi
tion r es tmr to transit out of D T T o ac hievethis the
algorithm automatically deriv es the follo wing inequalit y
see App endix I for more details
t p t
i
t p r
ij
Substituting expressions for t p t
i
and t p r
ij
previously
deriv ed w e get
d Qi Exp i d Qj Exp j d ji In other w ords V t
i
V t
j
d ji where V t
i
d Qi Exp i V t
i
is the time required for system i to trigger a resp onse
transmission if an y
Alternativ ely w e can a v oid the un w an ted transition
r cv r es if the system did not exist in D T when the re
sp onse is receiv ed Hence the algorithm automatically
deriv es the follo wing inequalit y see App endix I for more
details
t p r
ij
t q r
i
Again substituting expressions deriv ed ab o v e w eget d Qi d Qj Exp j d ji Note that equations and are general for an yn um
b er of resp onders where i and j are anyt w o resp onders
in the system Figure a and b sho w equations and resp ectiv ely A T ask sp ecic solutions
T op ology syn thesis Giv en the timer expiration v al
ues or ranges w e w an t to nd a feasible solution for
the w orstcase dela ys A feasible solution in this con text
means assigning p ositivev alues to the dela ys d ij i j In equation ab o v e if w etak e d Qi d Qj
w eget Exp i Exp j d ji The n um b er of inequalities n
where n is the n um b er of resp onders
is less then the n um b er of the unkno wns d
ij
n
n hence there are
m ultiple solutions W e can obtain a solution b y assigning v alues to n
unkno wns eg d
Qi
s and solving for the others
Timer
suppression
time
Q
2
1
Exp 1
Exp 2
d 1,2
d Q,2
d Q,1
p
r2,1
p
t2
q
r2
p
t1
q
r1
q
t
q
t
Exp 1 d 1,2
d Q,2
d Q,1
p
r2,1
q
r2
p
t1
q
r1
Exp 2 Exp 2
p
t2
q
r2
Exp 1 d 1,2
d Q,2
d Q,1
p
r2,1
p
t1
q
r1
q
t
(a) (b) (c)
t(p
t2
) < t(p
r2,1
) t(p
r2,1
) < t(q
r2
) t(p
t2
) > t(p
r2,1
)
t(p
r2,1
) > t(q
r2
)
Fig Time lines sho wing p ossible ev en t sequencing a and b
sequences do not lead to suppression while c leads to timer
suppression
These inequalities put a lo w er limit on the dela ys d ji hence w e can alw a ys nd a p ositiv e d ji to satisfy the
inequalities
Note that the dela ys used in the dela y matrix reect
dela ys o v er the m ulticast distribution tree In general
these dela ys are aected b y sev eral factors including the
m ulticast and unicast routing proto cols tree t yp e and
dynamics propagation transmission and queuing dela ys
One simple top ology that reects the dela ys of the de
la y matrix is a completely connected net w ork where the
underlying m ulticast distribution tree coincides with the
unicast routing
Timer conguration Giv en the dela y v alues or
ranges ie b ounds w ew an t to obtain timer expiration
v alues that pro duce w orstcase b eha vior
W e can obtain a range for the relativ e timer settings ie
Exp i Exp j using equation ab o v e
The solution for the system of equations giv en by and ab o v e can be solv ed in the general case using
linear programming LP tec hniques see App endix I I for
more details Section VI I uses the ab o v e solutions to
syn thesize sim ulation scenarios
Note ho w ev er that it ma y not be feasible to satisfy
all these constrain ts due to upp er b ounds on the dela ys
for example In this case the problem b ecomes one of
maximization where the w orstcase scenario is one that
triggers maxim um n um b er of resp onses p er pac k et loss
This problem is discussed in App endix I I B Bestc ase analysis
Best case o v erhead analysis constructs constrain ts that
lead to maxim um suppression ie minim um n um ber of
resp onses The follo wing conditions are form ulated using
steps similar to those giv en in the w orstcase analysis
t p t
i
t p r
ij
and
t p r
ij
t q r
i
These are complemen tary conditions to those giv en in
the w orst case analysis Figure c sho ws equations and Refer to the App endix I for more details on the
inequalityderiv ation
In this section w eha v e describ ed the algorithm to con
struct w orst and b estcase dela ytimer relations for o v er
head of resp onse messages Solutions to these relations
representdela ytimer settings for stress scenarios
VI Response Time Anal ysis
In this section w e conduct the p erformance analysis
with resp ect to the resp onse time F or our analysis w e
allo w selectiv e loss of a single resp onse message during
the reco v ery phase
In this case transition rules are
applied to only those systems that receiv e the message
The algorithm obtains p ossible sequences leading to the
target ev en t and calculates the resp onse time for eachse quence Tosyn thesize the w orst case scenario that max
imizes the resp onse time for example the sequence with
maxim um time is c hosen
A T ar get event
The resp onse time is the time tak en b y the mec hanism
to reco v er from the pac k et loss ie un til the requester
receiv es the resp onse p and resets its request timer b y
transitioning out of the R T state In other w ords the
Complete details of the b estcase analysis and the task sp ecic solu
tions w ere conducted and will b e included in a more elab orate tec hnical
rep ort They are remo v ed for brevit y Without loss of resp onse messages this problem b ecomes one of max
imizing the round trip dela y from the requester to the rst resp onder
resp onse in terv al is t p r
Q
t q t
Q
t p r
Q
t The
designer iden ties t p r
Q
as the target time hence p r
Q
is the target ev en t
B The se ar ch
W e presen t in detail the case of single resp onder then
discuss the m ultiple resp onders case
Bac kw ard searc h As sho wn in Figure the bac k
w ard searc h starts from p r
Q
and is p erformed o v er the
transition table in Section IVB using the implication
rules in Section IVB yielding
D j p r
Q
R Q R T
Q
p t
j
D j D T
j
R es j R T
Q
q r
j
D T
j
D j R T
Q
A t whichpoin t the algorithm reac hes a branc hing p oin t
where t w o p ossible preceding states could cause q r
j
The rst is transition loss D j q t
Q
R T
Q
R Q and
since the initial state R Q is reac hed the bac kw ard searc h
ends for this branc h
The second is transition r e q tmr D j R eq Q q t
Q
R T
Q

Note that Req Q indicates the need for a transition to
R T
Q
and the searc h for this last state yields ev en tually
D j q t
Q
R T
Q
R Q p tj .(D j
D Tj ).Res j .R TQ
D j .p rQ .(R Q R TQ )
q rj .(D jT
D j ).R TQ
D j .q tQ .(R TQ R Q )
D j .Req Q .q tQ .R TQ
D j .q tQ .(R TQ R Q )
loss req_tmr
. . .
Fig Bac kw ard searc h for resp onse time analysis
F orw ard searc h The algorithm p erforms a forw ard
searc h and c hec ks for consistency of the GFSM
The forw ard searc h step maylead to con tradiction with
the original bac kw ard searc h causing rejection of that
branc h as a feasible sequence F or example as sho wn in
The GFSM ma y b e represen ted b y comp osition of individual states
eg State
S tate
or tr ansition
S tate

Figure one p ossible forw ard sequence from the initial
state giv es
D j q t
Q
R Q R T
Q
q r
j
D j D T
j
R T
Q
p t
j
D T
j
D j R es j R T
Q
p tj .(D Tj D j ).Res j .R TQ
D j .q tQ .(R Q R TQ )
q rj .(D j
D Tj ).R TQ
D j .Req Q .q tQ .R TQ
p lost by Q p not lost
D j .R Q D j .R TQ
Fig F orw ard searc h for resp onse time analysis
The algorithm then searc hes t w o p ossible next states
If p t
j
is not lost and hence causes p r
Q
then the
next state is D j R Q But the original bac kw ard searc h
started from D j q t
Q
R eq Q R T
Q
whic h cannot b e reac hed
from D j R Q Hence w e get con tradiction and the algo
rithm rejects this sequence
If the resp onse p is lost b y Q w e get D j R T
Q
that
leads to D j R eq Q q t
Q
R T
Q
The algorithm iden ties this
as a feasible sequence
Calculating the time for eac h feasible sequence the algo
rithm iden ties the latter sequence as one of maxim um
resp onse time
F or m ultiple resp onders the algorithm automati
cally explores the dieren t p ossible selectiv e loss patterns
of the resp onse message The searc h iden tied the se
quence with maxim um resp onse as one in whic h only one
resp onder triggers a resp onse that is selectiv ely lost bythe
requester T o construct suc h a sequence the algorithm
creates conditions and inequalities similar to those for
m ulated for the b estcase o v erhead analysis with resp ect
to n um b er of resp onses see Section VB
VI I Simula tion using systema tic scenario
synthesis
T o ev aluate the utilit y of our metho d w e ha v e con
ducted a set of sim ulations for the scalable reliable m ulti
cast SRM based on our w orstcase scenario syn thesis
results for the timersuppression mec hanism W e tied our
metho d to the net w ork sim ulator NS The output
of our metho d in the form of inequalities see Section V is solv ed using a mathematical pac k age LINDO The so
lution in terms of a dela y matrix is then used to generate
the sim ulation top ologies for NS automatically F or our sim ulations w e measured the n um ber of re
sp onses triggered for eac h data pac k et loss W e ha v e
conducted t w o sets of sim ulations eac h using t w o sets
of top ologies The sim ulated top ologies included top olo
gies with up to no des The rst set of top ologies
w as generated according to the o v erhead analysis pre
sen ted in this pap er W e call this set of top ologies the
str ess top ologies An example str ess top ology is sho wn
in Figure The second set of top ologies w as generated
b y the GTITM top ology generator generating b oth
at random and transit stub top ologies
Wecall this
set of top ologies the r andom top ologies

105 ms
5 ms
Q S
S: Source
Q: Requester
Fig An example no de str ess top ology used for the sim ulation
The rst set of sim ulations w as conducted for the SRM
deterministic timers
The results of the sim ulation are
sho wn in Figure The n um b er of resp onses triggered for
The top ology generator is probably represen tativ e of a standard to ol
for top ology generation used in net w orking researc h Using GTITM w e
ha veco v ered most top ologies used in sev eral SRM studies W e faced diculties when c ho osing the lossy link for the r andom
top ologies in order to maximize the n um b er of resp onses This is an
example of the diculties net w orking researc hers face when trying to
stress net w orking proto cols in an adho c w a y SRM resp onse timer v alues are selected randomly from the in ter
v al D
d r D
D
d r where d r is the estimated distance to the
requester and D
D
dep end on the timer t yp e F or deterministic
timers D
and D

all the str ess top ologies w as n where n is the n um ber
of no des in the top ology ie no suppression o ccurred
F or the r andom top ologies the n um b er of resp onses trig
gered w as almost resp onses in the w orst case
Using the same t w o sets of top ologies the second
setofsim ulations w as conducted for the SRM adaptiv e
timers

The results are giv en in Figure F or the
str ess topologies almost of the nodes in the topology
triggered resp onses Whereas random top ologies sim ula
tion generated almost resp onses in the w orst case
These sim ulations illustrate ho w our metho d maybe
used to generate consistentw orstcase scenarios in a scal
able fashion It is in teresting to notice that w orstcase
top ologies generated for simple timers also exp erienced
substan tial o v erhead p erhaps not the w orst though for
more complicated timers suc h as the adaptiv e timers It
is also ob vious from the sim ulations that str ess scenarios
are more consisten t than the other scenarios when used to
compare dieren tmec hanisms in this case deterministic
and adaptiv e timers the p erformance gain for adaptiv e
timers is v ery clear under str ess scenarios
So in addition to exp eriencing the w orstcase b eha v
ior of a mec hanism our stress metho dology ma ybe used
to compare proto cols in the ab o v e fashion and to aid in
making design tradeos It is a useful to ol for generating
meaningful sim ulation scenarios that w e b eliev e should b e
considered in p erformance ev aluation of proto cols in ad
dition to the a v erage case p erformance and random sim
ulations W e plan to apply our metho d to test a wider
range of proto cols through sim ulation
VI I I Rela ted w ork
Related w ork falls mainly in the areas of proto col v er
ication VLSI test generation and net w ork sim ulation
There is a large b o dy of literature dealing with v erica
tion of proto cols V erication systems t ypically address
w elldened prop erties suc has safety livenessand r e
sp onsiveness and aim to detect violations of these
prop erties In general the t w o main approac hes for pro
to col v erication are theorem pro ving and reac habilit y
analysis Theorem pro ving systems dene a set of ax
ioms and relations to pro v e prop erties and include mo del
Adaptiv e timers adjust their in terv al based on the n um b er of dupli
cate resp onses receiv ed and the estimated distance to the requester
0
20
40
60
80
100
120
140
160
180
200
0 50 100 150 200
Number of Nodes
Number of Responses
Stress Scenarios
Random Scenarios
Fig Sim ulation results for deterministic timers o v er str ess
top ologies and r andom top ologies
0
20
40
60
80
100
120
140
160
180
200
0 50 100 150 200
Number of Nodes
Number of Responses
Stress Scenarios
Random Scenarios
Fig Sim ulation results for adaptiv e timers o v er str ess top ologies
and r andom top ologies
b ase d and lo gicb ase d formalisms These systems
are useful in man y applications Ho w ev er these systems
tend to abstract out some net w ork dynamics that w e will
study eg selectiv e pac k et loss Moreo v er they do not
syn thesize net w ork top ologies and do not address p erfor
mance issues p er se
Reac habilit y analysis algorithms on the other
hand try toinspectreac hable proto col states and suf
fer from the state space explosion problem T o circum
v en t this problem state reduction tec hniques could be
used These algorithms ho w ev er do not syn thesize
net w ork top ologies Reduced reac habilit y analysis has
been used in the v erication of cac he coherence proto
cols using a global FSM mo del W e adopt a similar
FSM mo del and extend it for our approac h in this study Ho w ev er our approac h diers in that w e address end
toend proto cols that encompass ric h timing dela yand
loss seman tics and w e address p erformance issues suc h
as o v erhead or resp onse dela ys
There is a go o d n um ber of publications dealing with
conformance testing Ho w ev er confor
mance testing v eries that an implemen tation as a blac k
bo x adheres to a giv en sp ecication of the proto col b y
constructing inputoutput sequences Conformance test
ing is useful during the implemen tation testing phase whichw e do not address in this pap er but do es not ad
dress p erformance issues nor top ology syn thesis for design
testing By con trast our metho d syn thesizes test scenar
ios for proto col design according to ev aluation criteria
Automatic test generation tec hniques ha v e b een used in
sev eral elds VLSI c hip testing uses test v ector gen
eration to detect target faults T est v ectors maybegen erated based on circuit and fault mo dels using the fault
orien ted tec hnique that utilizes implic ation tec hniques
These tec hniques w ere adopted in to dev elop fault
orien ted test generation F OTG for m ulticast routing
In F OTG w as used to study correctness of am ul
ticast routing proto col on a LAN W e extend F OTG to
study p erformance of endtoend m ultip ointmec hanisms
Wein tro duce the concept of a virtual LAN to represen t
the underlying net w ork in tegrate timing and dela y se
man tics in to our mo del and use p erformance criteria to
driveour syn thesis algorithm
In a sim ulationbased stress testing framew ork
based on heuristics w as prop osed Ho w ev er that metho d
do es not pro vide automatic top ology generation nor do es
it address p erformance issues The VINT to ols pro
vide a framew ork for In ternet proto cols sim ulation Based
on the net w ork sim ulator NS and the net w ork ani
mator NAM VINT pro vides a library of proto cols
andasetofv alidation test suites Ho w everit doesnot
pro vide a generic to ol for generating these tests automat
ically W ork in this pap er is complemen tary to suc h stud
ies and ma ybe in tegrated with net w ork sim ulation to ols
as w e do in Section VI I
IX Issues and Future W ork
In this pap er weha v e presen ted our rst endea v or to
automate the test syn thesis as applies to p erformance
ev aluation of m ultip oin t proto cols Our case studies
w ere b y no means exhaustiv e ho w ev er they ga v e us
insigh ts in to the researc h issues in v olv ed F uture w ork
should explore p oten tial extensions and applications of
our metho d
Automated generation of sim ulation test suites
Sim ulation is a v aluable to ol for designing and ev aluating
net w ork proto cols Researc hers usually use their insigh t
and exp ertise to dev elop sim ulation inputs and test suites
Our metho d ma y be used to assist in automating the
processofc ho osing sim ulation inputs and scenarios
The inputs to the sim ulation ma y include the top ology host ev en ts suc h as trac mo dels net w ork dynamics
suc h as link failures or pac k et loss and mem b ership dis
tribution and dynamics
Our future w ork includes implemen ting a more complete
to ol to automate our metho d including searc h algorithms
and mo deling seman tics and tie it to a net w ork sim ulator
to b e applied to a wider range of m ultip oin t proto cols
V alidating proto col building blo c ks
The design of new proto cols and applications often b or
ro ws from existing proto cols or mec hanisms Hence there
is a go o d c hance of reusing established mec hanisms as
appropriate in the proto col design pro cess Iden tify
ing v erifying and understanding building blo c ks for suc h
mec hanisms is necessary to increase their reusabilit y Our metho d ma y b e used as a to ol to impro v e that un
derstanding in a systematic and automatic manner
Ultimately one ma yen vision that a library of these build
ing blo c ks will b e a v ailable from whic h proto cols or parts
thereof will b e readily comp osable and v eriable using
CAD to ols similar to the w a y circuit and c hip design is
carried out to da y using VLSI design to ols In this w ork
and earlier w orks some mec hanistic building
blo c ks for m ultip oin t proto cols w ere iden tied namely the timersuppression mec hanism and the JoinPrune
mec hanism for m ulticast routing More w ork is needed
to iden tify more building blo c ks to co v er a wider range of
proto cols and mec hanisms
Generalization to p erformance b ound analysis
An approac h similar to the one w e ha v e tak en in this
pap er ma y b e based on some p erformance b ounds instead
of w orst or best case analyses W e call suc h approac h
conditionorien ted test generation
F or example a target ev entmaybe dened as the re
sp onse time exceeding certain dela y b ounds either ab
solute or parametrized b ounds If suc h a scenario is not
feasible that indicates that the proto col giv es absolute
guaran tees under the assumptions of the study This
ma y b e used to design and analyze qualit yofservice or
realtime proto cols
Applicabilit y to other problem domains
So far our metho d has b een applied to case studies on
m ultip oin t proto col p erformance ev aluation in the con
text of the In ternet
Other problem and application domains ma y in tro duce
new mec hanistic seman tics or assumptions ab out the sys
tem or en vironmen t One example of suc h domains in
cludes sensor net w orks These net w orks similar to ad
ho c net w orks assume dynamic top ologies lossy c hannels
and deal with stringentpo w er constrain ts whichdier en tiates their proto cols from In ternet proto cols P ossible researc h directions in this resp ect include
Extending the top ology represen tation or mo del to
capture dynamics where dela ys v ary with time
Dening new ev aluation criteria that apply to the sp e cic problem domain suc has po w er usage
In v estigating the algorithms and searc h tec hniques
that b est t the new mo del or ev aluation criteria
X Conclusion
Weha v e presen ted a metho dology for scenario syn the
sis for p erformance ev aluation of m ultip oin t proto cols
W e used a virtual LAN mo del to represen t the underly
ing net w ork top ology and an extended global FSM mo del
to represen t the proto col mec hanism W e adopted the
faultorien ted test generation algorithm for searc h and
extended it to capture timingdela y seman tics and p er
formance issues for endtoend m ultip oin t proto cols
Our metho d w as applied to p erformance ev aluation of
the timer suppression mec hanism a common building
blo c k for v arious m ultip oin t proto cols Tw o p erformance
criteria w ere used for ev aluation of the w orst and best
case scenarios the n um b er of resp onses p er pac k et loss
and the resp onse dela y Sim ulation results illustrate ho w
our metho d can b e used in a scalable fashion to test and
compare reliable m ulticast proto cols
W e do not claim to ha v e a generalized algorithm that
applies to an y arbitrary proto col Ho w ev er w e hop e that
similar approac hes ma y b e used to iden tify and analyze
other proto col building blo c ks W e b eliev e that suchsys tematic analysis to ols will b e essen tial in designing and
testing proto cols of the future
References
D Estrin D F arinacci A Helm y D Thaler S Deering M Hand
ley V Jacobson C Liu P Sharma and L W ei Proto col Indep en
den t Multicast Sparse Mo de PIMSM Proto col Sp ecication
RF C URL httpnetwebusce dupimpimsmPIMSMvExp
RF C ftxtps ggz Marc h
D Estrin D F arinacci A Helm y V Jacobson and L W ei
Proto col Indep enden t Multicast Dense Mo de PIMDM
Proto col Sp ecication Pr op ose d Exp erimental RF C URL
httpnetwebusce dupimpimdmPIMDM ftxtps ggz Septem ber
W F enner In ternet Group Managemen t Proto col V ersion
IDMR InternetDr aft pr op ose d standar dNo v em b er
S Flo yd V Jacobson C Liu S McCanne and L Zhang A
Reliable Multicast F ramew ork for Ligh tw eigh t Sessions and Ap
plication Lev el F raming IEEEA CM T r ansactions on Networking No v em b er
K Miller K Rob ertson A Tw eedly and M White StarBurst
Multicast File T ransfer Proto col MFTP Sp ecication Internet
Dr aft R Go vindan H Y u and D Estrin Largescale w eakly consisten t
replication using m ulticast T ec hnical Rep ort USC sep
h ttpwwwisiedu haob o ylesrrmpsgz
L Zhang S Mic hel K Nguy en A Rosenstein S Flo yd and
V Jacobson AdaptiveW eb Cac hing T o w ards a New Global
Cac hing Arc hitecture r d International WWW Caching Workshop June M Handley The Address Allo cation Proto col InternetDr aftAu gust M Handley The sdr Session Directory An Mb one
Conference Sc heduling and Bo oking System URL
httpugwwwe dacukmic ear chivesdrhtml Elan Amir Stev e McCanne and Randy Katz An activ e service
framew ork and its application to realtime m ultimedia transco ding
A CM SIGCOMM Septem b er
R Go vindan C Alaettinoglu and D Estrin SCAN LargeScale
F ault Isolation work in pr o gr ess J A t w o o d O Catrina J F en ton and W Stra y er Reliable Mul
ticasting in the Xpress T ransp ort Proto col Pr o c e e dings of the st
L o c al Computer Networks Confer enc e Octob er
H Sc h ulzrinne S Casner R F rederic k and V Jacobson R TP A
T ransp ort Proto col for RealTime Applications RFC Jan uary A Helm y and D Estrin Sim ulationbased STRESS T esting Case
Study A Multicast Routing Proto col Sixth International Sym
p osium on Mo deling A nalysis and Simulation of Computer and
T ele c ommunic ation Systems MASCOTS July
Ahmed Helm y Deb orah Estrin and Sandeep Gupta F ault
orien ted test generation for m ulticast routing proto col design F or
mal Description T e chniques F OR TE XI Pr oto c ol Sp e cic ation
T esting and V eric ation PSTV XVIII IFIP TCWG
Joint International Confer enc e Paris F r anc e No v em b er S Ba ja j L Breslau D Estrin K F all S Flo yd P Haldar
M Handley A Helm y J Heidemann P Huang S Kumar S Mc
Canne R Rejaie P Sharma K V aradhan H Y u Y Xu and
D Zappala Impro ving Sim ulation for Net w ork Researc h ToAp p e ar in IEEE Computer Magazine Septem b er K Calv ert M Doar and E Zegura Mo deling In ternet T op ology
K V aradhan D Estrin and S Flo yd Impact of Net w ork Dynam
ics on EndtoEnd Proto cols Case Studies in Reliable Multicast
ISCC P Huang D Estrin and J Heidemann Enabling LargeScale Sim
ulations Selectiv e Abstraction Approac h to The Study of Multi
cast Proto cols Sixth International Symp osium on Mo deling A nalysis
and Simulation of Computer and T ele c ommunic ation Systems MAS
COTS July K Saleh I Ahmed K AlSaqabi and A Agarw al A reco v ery ap
proac h to the design of stabilizing comm unication proto cols Jour
nal of Computer Communic ation V ol No pages April E Clark e and J Wing F ormal Metho ds State of the Art and
F uture Directions A CM Workshop on Str ate gic Dir e ctions in Com
puting R ese ar ch V ol No pages Decem b er R Bo y er and J Mo ore A Computational Logic Handb o ok A c a
demic Pr ess Boston J Spiv ey Understanding Z a Sp ecication Language and its F or
mal Seman tics Cambridge University Pr ess F Lin P Ch u and M Liu Proto col V erication using Reac ha
bilit y Analysis Computer Communic ation R eview V ol No
P Go defroid Using partial orders to impro veautomaticv erica
tion metho ds Pr o c nd Workshop on ComputerA idedV eric ation
Springer V erlag New Y ork F P ong and M Dub ois V erication T ec hniques for Cac he Coher
ence Proto cols A CM Computing Surveys V olume No pages
Marc h Mihalis Y annak akis and Da vid Lee T esting Finite State Mac hines
F ault Detection Journal of Computer and Systems Scienc es
D Ra yner OSI conformance testing Computer Networks and ISDN
Systems Sp e cial issue on ConformanceT esting V ol No pages
K Sabnani and A Dah bura A new tec hnique for generating pro
to col tests A CM Computer Communic ation R eview V ol No Septem b er K Sabnani and A Dah bura A proto col test generation pro cedure
Computer Networks and ISDN Systems V ol
M Abramo vici M Breuer and A F riedman Digital Systems
T esting and T estable Design AT TL abs S Ba ja j L Breslau D Estrin K F all S Flo yd P Haldar
M Handley A Helm y J Heidemann P Huang S Kumar S Mc
Canne R Rejaie P Sharma S Shenk er K V aradhan H Y u
Y Xu and D Zappala Virtual In terNet w ork T estb ed Status and
researc h agenda USCCSTR July D Estrin M Handley J Heidemann S McCanne Y Xu and
H Y u Net w ork Visualization with the VINT Net w ork Animator
Nam ToApp e ar in IEEE Computer MagazineNo v em b er
Deb orah Estrin Ramesh Go vindan and John Heidemann Scalable
co ordination in sensor net w orks T e chnic al R ep ort Univer
sity of Southern California Jan uary N Karmark ar A new p olynomialtime algorithm for linear pro
gramming Combinatoric a pages G Dan tzig Simplex Metho d for Solving Linear Programs The
Macmil lian Pr ess Ltd L ondon B Borc hers and J Mitc hell An Impro v ed Branc h and Bound Al
gorithm for Mixed In teger Nonlinear Programs Computers and
Op er ations R ese ar ch
D Estrin D F arinacci A Helm y D Thaler S Deer
ing M Handley V Jacobson C Liu P Sharma and
L W ei Proto col Indep enden t Multicast Sparse Mo de PIM
SM Motiv ation and Arc hitecture Pr op osedExp erimental RF C
URL httpnetwebusce dupimpimsmPIMA r ch ftxtps ggz Oc
tob er Appendix
In this app endix w e presen t details of inequalit y form u
lation for the endtoend p erformance ev aluation In ad
dition w e presen t the mathematical mo del to solvethese
inequalities W e also discuss the case of m ultiple request
rounds for the timer suppression mec hanism and presen t
sev eral example case studies
I Conditions and Inequalities f or O verhead
Anal ysis
Giv en the target ev en t transitions are iden tied as ei
ther w an ted or un w an ted transitions according to the
maximization or minimization ob jectiv e F or maximiza
tion w an ted transitions are those that establish condi
tions to trigger the target ev en t while un w an ted transi
tions are those that n ullify these conditions
Let W b e the w an ted transition and t W b e the time
of its o ccurrence Let C b e the condition for the w an ted
transition and t C is the time at whic h it is satised
and let U b e the un w an ted transition o ccurring at t U Wew an t to establish and main tain C un til W o ccurs
ie in the duration t C t W Hence U ma y only
o ccur outside b efore or after that in terv al In Figure this means that U can only o ccur in region or region Hence the inequalities m ust satisfy the follo wing
the condition for the w an ted transition C m ust b e
established b efore the ev en t for the w an ted transition
W triggers ie t C t W and
one of the follo wing t w o conditions m ust b e satised
a the un w an ted transition U m ust o ccur b efore C ie t U t C or
time
(1) (2) (3)
U U
t(C) t(W)
Fig The timeline for transition ordering
b the un w an ted transition U m ust o ccur after the
w an ted transition W ie t W t U These conditions m ust b e satised for all systems In
addition the algorithm needs to v erify using bac kw ard
searc h and implication rules that no con tradiction exists
bet w een the ab o v e conditions and the nature of the ev en ts
of the giv en problem
A Worstc ase Overhe ad A nalysis
The target ev entfor theo v erhead analysis is p t The ob jectiv e for the w orst case analysis is to max
imize the n um ber of resp onses p t The w an ted transi
tion is transition r es tmr Res D T D p t see Sec
tion IV Hence t W t p t The condition for the
w an ted transition is D T and its time from transition
tx r e q q r D D T is t C t q r The un w an ted transition is one that n ullies the con
dition D T T ransition r cv r es p r D T D is iden ti
ed b y the algorithm as the un w an ted transition hence
t U t p r
F or a giv en system i the inequalities b ecome
t q r
i
t p t
i
and either
t p r
ij
t q r
i
or
t p t
i
t p r
ij
The ab o v e automated pro cess is sho wn in Figure F rom the timer expiration implication rule ho w ev er w e
t(qr
i
)‘Cond’ t(pt
i
)‘Wanted’
time
rcv_req
rcv_res
(D
Ti
→D
i
).pt
i res_tmr
D
i
→D
Ti
Event Effect Time
Res
qr
i
pr
i,j D
Ti
→D
i
t(pt
i
)
t(qr
i
)
t(pr
i,j
)
t(Cond) < t(Wanted) → t(qr
i
) < t(pt
i
)
and
[t(Unwanted) < t(Cond) → t(pr
i,j
) < t(qr
i
)
or
t(Wanted) < t(Unwanted) → t(pt
i
) < t(pr
i,j
)]
pr
i,j
‘Unwanted’
pr
i,j
Symbol
Fig F orm ulating the inequalities automatically
get that the resp onse time m ust ha v e b een set earlier b y
the request reception ie
Res i D i D T
i
p t
i
q r
i
D T
i
D i and t p t
i
t q r
i
Exp i Hence
t q r
i
t p t
i
is readily satised and w e need not add
an y constrain ts on the expiration timers or dela ys to sat
isfy this condition
Th us the inequalities form ulated b y the algorithm to
pro duce w orstcase b eha vior are
t p r
ij
t q r
i
or
t p t
i
t p r
ij
B Bestc ase A nalysis
Using a similar approac h to the ab o v e analysis the
algorithm iden ties transition r cv r es p r D T D as
the w an ted transition Hence t W t p r and t C
t q r The un w an ted transition is transition r es tmr and
t U t p t
F or system i the inequalities b ecome
t q r
i
t p r
ij
and either
t p t
i
t q r
i
or
t p r
ij
t p t
i

But from the bac kw ard implication w e ha v e t q r
i
t p t
i
Hence the algorithm encoun ters con tradiction and
the inequalit y t p t
i
t q r
i
cannot b e satised
Th us the inequalities form ulated b y the algorithm to
pro duce w orstcase b eha vior are
t q r
i
t p r
ij
and
t p r
ij
t p t
i
II Ma thema tical Model f or Sol ving the System
of Inequalities
In this section w e presen t the general mo del of the con
strain ts or inequalities generated b y our metho d As a
rst step w e form a linear programming problem and at
tempt to nd a solution If a solution is not found then
w e form a mixed nonlinear programming problem to get
the maxim um n um b er of feasible constrain ts
In general the system of inequalities generated b y our
metho d to obtain w orst or best case scenarios can be
form ulated as a linear programming problem
In our case satisfying all the constrain ts regardless of
the ob jectiv e function leads to obtaining the absolute
w orstb est case F or example in the case of w orst case
o v erhead analysis this means obtaining the scenario lead
ing to nosuppression
The form ulated inequalities b y our metho d as giv en in
Section V are as follo ws
for the w orst case b eha vior
d Qi Exp i d Qj Exp j d ji or
d Qi d Qj Exp j d ji for the b est case b eha vior
d Qi Exp i d Qj Exp j d ji and
d Qi d Qj Exp j d ji The ab o v e systems of inequalities can b e nicely repre
sen ted b y a linear programming mo del The general form
of a linear programming LP problem is
M aximiz eZ C
T
X X
i n
c i x i
sub ject to
AX B
X where Z is the ob jectiv e function C is a v ector of n
constan ts c i X is a v ector of n v ariables x i A is m n
matrix and B is a v ector of m elemen ts
The ab o v e problem can b e solv ed practically in p olyno
mial time using Karmark ar or simplex metho d if a feasible solution exists
In some cases ho w ev er the absolute w orstb est case
ma y not b e attainable and it ma y not b e p ossible to nd
a feasible solution to the ab o v e problem In suc h cases w e
w an t to obtain the maxim um feasible set of constrain ts
in order to get the w orstb est case scenario Toac hiev e
this w e dene the problem as follo ws
M aximiz e
X
i m
y i
sub ject to
y i f i x i
y i f g
or
y i y i
where f i x is the original constrain tfromthe previous
problem
This problem is a mixed in teger nonlinear program
ming MINLP problem that can b e solv ed using branc h
and b ound metho ds III Mul tiple request r ounds
In Section V w e conducted the proto col o v erhead anal
ysis with the assumption that reco v ery will o ccur in one
round of request In general ho w ev er loss reco v ery ma y
require m ultiple rounds of request and w e need to con
sider the request timer as w ell as the resp onse timers
Considering m ultiple timers or stim uli adds to the branc h
ing factor of the searc h Some of these branc hes ma y
not satisfy the timing and dela y constrain ts It w ould b e
more ecien t then to incorp orate timing seman tics in to
the searc h tec hnique to prune o infeasible branc hes
Let us consider forw ard searchrst F or example con
sider the global state q t
i
R T
i
ha ving a transmitted request
message and a request timer running Dep ending on the
timer expiration v alue Exp i and the dela y exp erienced b y
the message d ij w ema y get dieren t successor states If
d ij Exp i then the request timer res rst triggering
the ev en t Req i and w e get q t
i
R eq i as the successor state
Otherwise the request message will b e receiv ed rst and
the successor state will b e q r
j
R T
i
Note that in this case
the timer v alue m ust be decremen ted b y d ij This is
illustrated in gure The condition for branc hing is
giv en on the arro w of the branc h and the timer v alue of
i is giv en b y T i q ti.R Ti
q ti.Req i
q rj.R Ti
d i,j > Exp i
d i,j < Exp i
T i = Exp i - d i,j
Fig F orw ard searc h for m ultiple sim ultaneous ev en ts
F or bac kw ard searc h instead of decreasing timer v al
ues as is done with forw ard searc h timer v alues are in
creased and the starting p oin t of the searc h is arbitrary
in time as opp osed to time for forw ard searc h
T o illustrate consider the global state ha ving D i D T
i
R T
j
with the request timer running at j and the
resp onse timer ring at i Res i . p ti .(D i D Ti ).R TQ
T Q = x T i = 0
Exp i < Exp Q - x
q ri .(D Ti D i ).R TQ
T Q = Exp i + x T i = Exp i
Exp i > Exp Q - x
D Ti .(R TQ R Q ).q tQ
T Q = Exp Q T i = Exp Q - x
q tQ .D i .(R TQ R Q )
q rm .D Ti .(R TQ R Q ).q tQ
d Q,i < Exp Q - (x + Exp i )
d Q,i > Exp Q - (x + Exp i )
Fig Bac kw ard searchfor m ultiple sim ultaneous ev en ts
Figure sho ws the bac kw ard branc hing searc h with
the timer v alues at eac h step and the condition for eac h
branc h In the rst state the timer T Q starts at an arbi
trary p ointintime x and the timer T i is set to ie
the timer expired triggering a resp onse p t
i
One step
bac kw ard either the timer at i m ust ha v e b een started
Exp Q x units in the past or the resp onse timer m ust
ha v e been started Exp i units in the past Dep ending
on the relativev alues of these times some branc hes b e
come v alid The timer v alues at eac h step are up dated
accordingly Note that if a timer expires while a message
is in igh tie transmitted but not y et receiv ed w e use
the m subscript to denote it is still m ulticast as in q r m
in the gure
Sometimes the v alues of the timers and the dela ys are
giv en as ranges or in terv als F ollo wing w e presen t ho w
branc hing decision are made when comparing in terv als
Branc hing decision for in terv als
In order to conduct the searchfor m ultiple stim uli w e
need to c hec k the constrain ts for eac h branc h T o decide
on the branc hes v alid for searc h w e compare v alues of
timers and dela ys These v alues are often giv en as in ter
v als eg a b Comparison of t woin terv als Int a b and Int a b is done according to the follo wing rules
Branc h Int Int b ecomes v alid if there exists a
v alue in a b that is greater than a v alue in a b ie
if there is o v erlap of more than one n um ber b et w een the
in terv als W e dene the and relations similarly ie if there are an yn um b ers in the in terv al that satisfy
the relation then the branc h b ecomes v alid
F or example if w e ha v e the follo wing branc h condi
tions i Exp i Exp j ii Exp i Exp j and iii
Exp i Exp j If Exp i and Exp j
then according to our ab o v e denitions all the branc h
conditions are v alid Ho w ev er if Exp i and
Exp j then only branc hes i and ii are v alid
The ab o v e denitions are sucien ttoco v er the forw ard
searc h branc hing Ho w ev er for bac kw ard searc h branc h
ing w emayha v e an arbitrary v alue x as noted ab o v e
F or example tak e the state D i D T
i
R T
Q
Con
sider the timer at Q the expiration duration of whic h
is Exp Q and the v alue of whic h is x and the timer
at i the expiration duration of whic h is Exp i and the
v alue of whic h is as giv en in gure Dep end
ing on the relev an t v alues of Exp i and Exp Q x the
searc h follo ws some branc hes If Exp Q a b then
x b and Exp Q x b Hence w e can apply
the forw ard branc hing rules describ ed earlier b y taking
Exp Q x b as follo ws Since Exp i a b
where a and b hence the branc h condi
tion Exp i Exp Q x is alw a ys true The condition
Exp i Exp Q x is v alid when i Exp i Exp Q or ii
Exp i Exp Q The last condition Exp i Exp Q xis
v alid only if Exp i Exp Q These rules are in tegrated in to the searc h algorithm
for our metho d to deal with m ultiple stim uli and timers
sim ultaneously IV Example Case Studies
In this section w e presen t sev eral case studies that
sho w ho w to apply the previous analysis results to ex
amples in reliable m ulticast and related proto col design
problems
A T op olo gy Synthesis
In this subsection weapply thetestsyn thesis metho d
to the task where the timer v alues are kno wn and the
top ology ie D matrix is to b e syn thesized according
to the w orstcase b eha vior W e explore v arious timer set
tings W e use the virtual LAN in Figure to lo ok at t w o
examples of top ology syn thesis one uses a timers with
xed randomization in terv als and the other uses timers
that are function of distance
Q
V. LAN
1
2 3
Fig The virtual LAN with p oten tial resp onders
Let Q be the requester and and be poten tial
resp onders Attime t Q sends the request
F or simplicit y w e assume without loss of generalit y that the systems are ordered suchthat V t
i
V t
j
for ij
eg system has the least d Q Exp then and
then Th us the inequalities V t
i
V t
j
d ji are readily
satised for i j and w e need only satisfy it for i j F rom equation for the w orstcase ab o veweget V t
V t
d V t
V t
d V t
V t
d
By satisfying these inequalities w e obtain the dela y set
tings of the w orst case top ologyas willbe sho wn in the
rest of this section
A Timers with xed randomization in terv als
Some m ulticast applications and proto cols suc h as
wb IGMP or PIM emplo y xed randomiza
tion in terv als to set the suppression timers F or instance
for the shared white b oard wb the resp onse timer is
assigned a random v alue from the uniformly distributed
in terv al t!t where t msec for the source sr c and
msec for other resp onders
Assume Q is a receiv er with a lost pac k et Using wb
parameters weget Exp sr c msec and Exp i msec for all other no des
Toderivew orstcase top ologies from inequalities w e
ma y use a standard mathematical to ol for linear or non
linear programming for more details see App endix II Ho w ev er in the follo wing w e illustrate general tec hniques
that ma y b e used to obtain the solution
F rom inequalities w eget d Q Exp V t
V t
d d Q Exp d This can b e rewritten as
d Q d Q d Exp Exp dif f
where
dif f
if is src if is src Otherwise
Q 1
2 3
d Q,1
d Q,2
d Q,3
d 1,3
d 2,3
d 1,2
Fig The virtual LAN with dela y assignmen t and lab els
Similarlyw e deriv e the follo wing from inequalities for
V t
d Q d Q d diff and
d Q d Q d diff If w e assume system to b e the source and for a con
serv ativ e solution wec ho ose the minim um v alue of dif f w e get
min dif f min dif f min dif f W e then substitute these v alues in the ab o v e inequali
ties and assign the v alues of some of the dela ys to com
pute the others
Example if w e assign d Q d Q d Q msec
w e get d d and d Figure sho ws one possible top ology to whic h the
ab o v e assigned dela ys can b e applied These dela ys ex
hibit w orstcase b eha vior for the timer suppr ession me ch
anism A Timers as function of distance
In con trast to xed timers this section uses timers
that are function of an estimated distance The expi
ration timer ma y b e set as a function of the distance to
the requester F or example system i ma y set its timer
to rep ond to a request from system Q in the in terv al
C E iQ C C E iQ where E iQ is the estimated
distancedela y from i to Qwhic h is calculated using mes
sage exc hange eg SRM session messages and is equal
to d iQ d Qi Note that this estimate assumes sym
metry whic h sometimes is not v alid
suggests v alues for C and C as or log Gwhere
G is the n um ber of mem b ers in the group
W e tak e C C to syn thesize the w orstcase
top ology W e get the expression
Exp Exp d Q d Q d Q d Q d Q d Q d Q d Q Example If w e assume that d Q d Q d Q d Q msec w e can rewrite the ab o v e relation as
Exp Exp msec
Substituting in equation ab o v e w e get d msec Under similar assumptions w e can obtain
d msec and d msec
T op ologies with the ab o vedela y settings will exp erience
the w orst case o v erhead b eha vior as dened ab o v e for
the timer suppr ession mec hanism
As w as sho wn the inequalities form ulated automati
cally b y our metho d in section V can b e used with v arious
timer strategies eg xed timers or timers as function
of distance Although the top ologies weha v e presen ted
are limited a mathematical to ol suc hasLINDO can be
used to obtain solutions for larger top ologies
B Timer c ongur ation
In this subsection wegiv e simple examples of the timer
conguration task solution where the dela y b ounds ie
D matrix are giv en and the timer v alues are adjusted to
ac hiev e the required b eha vior
In these examples the delayis giv en as an in terv al xy
msec W e sho w an example for w orstcase analysis
B W orstcase analysis
If the giv en ranges for the dela ys are msec for
all dela ys then the term d Qj d Qi d ji ev aluates to
F rom equation ab o v e w e get
Exp i Exp j to guaran tee that a resp onse is
triggered
If the dela ys are msec w e get
Exp i Exp j ie is expiration timer m ust be less than j s b y at
least msecs Note that weha v e an implied inequalit y
that Exp i for all i These timer expiration settings w ould exhibit w orst
case b eha vior for the giv en dela y b ounds

Abstract (if available)

Linked assets

Computer Science Technical Report Archive

Conceptually similar

PDF

USC Computer Science Technical Reports, no. 696 (1999)

PDF

USC Computer Science Technical Reports, no. 727 (2000)

PDF

USC Computer Science Technical Reports, no. 690 (1998)

PDF

USC Computer Science Technical Reports, no. 673 (1998)

PDF

USC Computer Science Technical Reports, no. 860 (2005)

PDF

USC Computer Science Technical Reports, no. 730 (2000)

PDF

USC Computer Science Technical Reports, no. 755 (2002)

PDF

USC Computer Science Technical Reports, no. 716 (1999)

PDF

USC Computer Science Technical Reports, no. 801 (2003)

PDF

USC Computer Science Technical Reports, no. 674 (1998)

PDF

USC Computer Science Technical Reports, no. 657 (1997)

PDF

USC Computer Science Technical Reports, no. 809 (2003)

PDF

USC Computer Science Technical Reports, no. 811 (2003)

PDF

USC Computer Science Technical Reports, no. 678 (1998)

PDF

USC Computer Science Technical Reports, no. 765 (2002)

PDF

USC Computer Science Technical Reports, no. 816 (2004)

PDF

USC Computer Science Technical Reports, no. 663 (1998)

PDF

USC Computer Science Technical Reports, no. 856 (2005)

PDF

USC Computer Science Technical Reports, no. 702 (1999)

PDF

USC Computer Science Technical Reports, no. 644 (1997)

Description

Ahmed Helmy, Sandeep Gupta, Deborah Estrin, Alberto Cerpa, Yan Yu. "Performance evaluation of multipoint protocols using systematic scenario synthesis: A case study for timer-suppression mechanisms." Computer Science Technical Reports (Los Angeles, California, USA: University of Southern California. Department of Computer Science) no. 726 (2000).

Asset Metadata

Creator Cerpa, Alberto (author), Estrin, Deborah (author), Gupta, Sandeep (author), Helmy, Ahmed (author), Yu, Yan (author)

Core Title USC Computer Science Technical Reports, no. 726 (2000)

Alternative Title Performance evaluation of multipoint protocols using systematic scenario synthesis: A case study for timer-suppression mechanisms (title)

Publisher Department of Computer Science,USC Viterbi School of Engineering, University of Southern California, 3650 McClintock Avenue, Los Angeles, California, 90089, USA (publisher)

Tag OAI-PMH Harvest

Format 18 pages (extent), technical reports (aat)

Language English

Unique identifier UC16269760

Identifier 00-726 Performance Evaluation of Multipoint Protocols Using Systematic Scenario Synthesis A Case Study for Timer-suppression Mechanisms (filename)

Legacy Identifier usc-cstr-00-726

Format 18 pages (extent),technical reports (aat)

Rights Department of Computer Science (University of Southern California) and the author(s).

Internet Media Type application/pdf

Source 20180426-rozan-cstechreports-shoaf (batch), Computer Science Technical Report Archive (collection), University of Southern California. Department of Computer Science. Technical Reports (series)

Access Conditions The author(s) retain rights to their work according to U.S. copyright law. Electronic access is being provided by the USC Libraries, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.

Repository Name USC Viterbi School of Engineering Department of Computer Science

Repository Location Department of Computer Science. USC Viterbi School of Engineering. Los Angeles\, CA\, 90089

Repository Email csdept@usc.edu

Inherited Values

Title Computer Science Technical Report Archive

Description Archive of computer science technical reports published by the USC Department of Computer Science from 1991 - 2017.

Coverage Temporal 1991/2017

Repository Email csdept@usc.edu

Repository Name USC Viterbi School of Engineering Department of Computer Science

Repository Location Department of Computer Science. USC Viterbi School of Engineering. Los Angeles\, CA\, 90089

Publisher Department of Computer Science,USC Viterbi School of Engineering, University of Southern California, 3650 McClintock Avenue, Los Angeles, California, 90089, USA (publisher)