Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
Computer Science Technical Report Archive
/
USC Computer Science Technical Reports, no. 674 (1998)
(USC DC Other)
USC Computer Science Technical Reports, no. 674 (1998)
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
Sim ulationbased STRESS T esting Case Study
A Multicast Routing Proto col
Ahmed Helm y Deb orah Estrin
Computer Science Departmen tISI
Univ ersit y of Southern California
Los Angeles CA
email fahelm y estrin guscedu
April Abstract
Multipart y proto cols supp ort an imp ortan t class of
applications ranging from m ultimedia teleconferenc
ing to net w ork games Designing widearea m ulti
part y proto cols is b ecoming more complex with the
gro wth of the In ternet and the in tro duction of new
service mo dels Unexp ected com binations of ev en ts
can driv e proto cols in to undesirable states and ma y
lead to errors An ticipating all suc h cases is often
imp ossible and at b est ma y require extensivesim ula
tion and testing In large systems the cost of testing
all p ossible scenarios exhaustiv ely is prohibitiv e and
man y unexp ected cases are not observ ed un til deplo y
men t
Protot yping in testb eds or individual sim ulations
t ypically fo cuses on p erformance under a limited set
of randomized proto col transitions F ormal and ana
lytical mo dels represen ting suc h proto cols tend to b e
complex sometimes rendering the mo del in tractable
In this w ork w e prop ose a metho d for using sim
ulation to analyze the robustness of m ultipart y
m ulticastbased proto cols in a systematic fashion
W e call our metho d Systematic T esting of R obustness
by Examination of Sele ctedSc enarios STRESS STRESS aims to cut the time and eort needed to ex
This material is based up on w ork supp orted b y the Defense
Adv anced Researc h Pro jects Agency D ARP A under Con tract
No D ABTC An y opinions ndings and conclu
sions or recommendations expressed in this material are those
of the authors and do not necessarily reect the views of the
D ARP A
plore the pathological cases of a proto col during its
design This pap er has t w o goals to describ e the
metho d and to serveas a case study of robust
ness analysis of m ulticast routing proto cols W e do
not pro v e correctness but aim to oer design metho d
to ols similar to those used in CAD and VLSI design
W e also demonstrate howpo w erful and eectiv esys tematic sim ulation can be in studying proto col ro
bustness
Keyw ords Simulation metho d pr oto c ol design
and testing r obustness evaluation multic ast r outing
In tro duction
In this pap er w e describ e a metho d for Systematic
Testing of Robustness by Examination of Sele cte d
Sc enarios STRESS It is based on a sim ulation frame
w ork supp orted b y a set of to ols and is designed for
studying proto col beha vior in the con text of patho
logical cases and scenarios Some of the general con
cepts for STRESS dra w from sim ulationbased v eri
cation tec hniques and reac habilit y analysis W e
apply these tec hniques to supp ort the design analy
sis and testing of m ultipart y proto cols
In particular w e in tro duce tec hniques for state
and top ology reduction and in v estigate v arious pac k et
loss scenarios to capture robustness c haracteristics
The denition of error conditions enables us to cap
ture the fault y errorprone cases automatically Multipart y proto cols mayin v olvem ultiple receiv ers
and one or more senders These proto cols include
m ulticast routing proto cols eg D VMRP MO
SPF PIMDM CBT and PIMSM
m ulticast transp ort proto cols eg SRM R TP and R TCP and m ultipart y applications eg
WB v at vic n te and sdr This
pap er fo cuses on m ulticast routing proto cols whic h
deliv er pac k ets ecien tly to group mem bers b y es
tablishing distribution trees Figure sho ws a v ery
simple exampleofa source S sending to a group of
receiv ers R
i
As a case studyw e apply our metho d to
the m ulticast routing proto col Proto col Indep enden t
MulticastSparse Mo de PIMSM
S
R1
R2
R3
R4 R5
S: sender to the group
Ri: receiver i of the group
Figure Establishing m ulticast deliv ery tree
Our study rev ealed sev eral pathological errors in
PIMSM through the use of STRESS to ols and ev al
uated solutions to eliminate these errors The sug
gested solutions ha vesince b een added to the PIM
SM sp ecication The rest of the pap er is organized as follo ws Sec
tionpro vides an o v erview of the STRESS metho d The case study for PIMSM is presen ted in section
Results are giv en in section Sections and
address related w ork summary and future w ork
resp ectiv ely The Metho d Ov erview
The r obustness of a proto col is its abilit y to resp ond
correctly in the face of net w ork failures and pac k et
loss The goal of the STRESS metho d is to pro vide a
framew ork for systematic testing of proto col robust
ness through the examination of selected scenarios
F or a giv en proto col w e rst capture a set of error
prone scenarios This is ac hiev ed b y a in v estigat
ing a r epr esentative subset of the proto col state space
and b dening error conditions W e use these sce
narios to ev aluate design tradeos analyze b eha vior
and as a test suite to examine v arious implemen ta
tions of the proto col
Our basic approac h consists of three stages sc e
nario gener ation prepro cessing tr acing sim ulation
and output analysis p ostpro cessing Figure illus
trates these stages The building blo c ks in the gure
are explained in detail throughout the rest of this
section
Note that the engineering design pro cess is usually
iterativ e where an in v estigator ma y cycle and feed
backin to previous stages based on hisher in tuition
and insigh t sometimes gained b y the analysis of ear
lier sim ulations Our metho dology do es not con tra
dict suc h pro cess In fact w e will sho w in the results
section howw e iterated through the stages to guide
our sim ulations This section ho w ev er only discusses
the mo dules supp orting the dieren t stages
Scenario Generation
Scenarios are comp osed of routed top ologies and se
quences of ev en ts input stim uli and state transitions
and describ e the sim ulation con text that ma y cause
proto col transitions Scenario parameters include the
r outedtop olo gy host sc enarios and loss sc enarios Routed top ology
The routed top ology is the net w ork infrastructure
up on whic h the proto col op erates eg net w ork no des
and links and the unicast routing that determines
howpac k ets are forw arded
W e try to iden tify simple top ologies that capture
a large p ercen tage of the proto cols state space and
to whic h other more complex top ologies ma ybe re duced
F or m uc h of our study w e c ho ose a LAN
with four connected routers as the basic top ology W e
showho w other top ologies are reducible to the four
r outer LAN top ologyand discuss the limitations of
suc h a top ology in section W e further extend the
top ology to capture particular c haracteristics of the
proto col under study PIMSM
As a comp onen t of the routed top ology unicast
route inconsistencies ma y b e a common source of er
ror Unicast routing ma y exist in one of the follo wing
three states a consisten t routing b transien tin consisten t routing and c long liv ed inconsistency Case a requires no c hanges The study of case b
is con v ergence analysis whic h has b een addressed
Tw o top ologies are said to b e reducible or equiv alen t if
they driv e the proto col according to the host scenarios ap
plied in to the same states exp eriencing the same set of state
transitions
Scenario
Generation
Host
Scenarios
Routed
Topology
Loss &
Failures
End Point
Tracing
Protocol
Tracing
Code
Annotation
Simulation & Tracing
Output Analysis
Identifying
End Point
Errors
Relating Errors
to Protocol
Code
Profiling
Simulation
Set-up
Link
Tracing
Simulation
Engine
Figure STRESS metho d blo c k diagram
elsewhere
W e are particularly in terested in case
c
W e add an inconsisten t unicast routing com
p onen t to force the m ulticast routing proto col in to
states encoun tered in suc h pathology and analyze
those states
Host scenarios
Host scenarios are com binations of p ossible host ac
tions In our case study these are dened b y the
m ulticast service mo del Host actions include joining
or lea ving groups or sending pac k ets to groups F or
large n um b ers of hosts and groups it is prohibitiv ely
costly to explore all p ossible com binations exhaus
tiv ely The heuristics used in this study do not guaran tee
that all fault y scenarios for a proto col will be co v
ered Our more practical and ac hiev able ob jectiv e
is to study m ulticast proto col b eha vior for scenarios
that include the primary host ev en ts in this case
joining a group lea ving a group and sending to a
group F or these scenarios w e generate all possible
message loss cases and extract the fault y scenarios
automatically F or illustrativ e purp oses w ec ho ose the simplest
m ulticast host scenario that has a single source S
and t w o receiv ers R and R for the same group
W e shall address this simple scenario in this section
and sho w in section ho w it can be utilized and
extended for our case study W e estimate all the p ossible com binations of our
host mo del and try to reduce the n um ber to those
F or con v ergence analysis of PIMSM mec hanisms refer
to This ma y b e caused bya m ulticast region spanning more
than one unicast routing AS
scenarios that ma y aect the proto col state transi
tions W e call suc h scenarios r epr esentative sc enar
ios T o obtain the represen tativ e scenarios w e apply
the sc enario lter sho wn in gure The use of the lter sho wn in the gure is illus
trated b y the follo wing example F or one source and
t w o receiv ers the v e p ossible host ev en ts are source
S sending to a group or S for short receiv er join
ing a group or J and J for receiv ers R and
R resp ectiv ely and receiv er lea ving a group or
L and L for receiv ers R and R resp ectiv ely
Host Events
Protocol Constraints
Practical Input
Symmetry &
Equivalence
Rep.
Scenarios
Figure The scenario lter
F or all p ossible p erm utations there exists scenarios considering that eac h host ev en t oc curs once Then as sho wn b y gure w e apply pro
to col constrain ts eg ar e c eiver c annot le ave b efor e
it joins the gr oup to reduce the n um b er of p ossible
com binations to scenarios F urther
as a practical input w e assume without loss of gen
eralit ythat the sour c e sends p ackets thr oughout the
simulation to reduce the n um b er of p ossible scenar
ios to scenarios These six scenarios are
JJLL JJLL JLJL
JJLL JJLL JLJL
The n um ber of represen tativ e scenarios can be
ev en reduced further if the host distribution is sym
metric with resp ect to the top ology since the follo w
ing scenarios will b e equiv alen t i equiv alentto ii equiv alen t to and iii equiv alenttoie w e need only in v estigate dieren t host scenarios for
the giv en top ology These scenarios maybe generated automatically
b y the metho d Ho w ev er generalizing the pro cess of
obtaining represen tativ e scenarios for v arious m ulti
part y proto cols is curren tly under study
Loss and F ailures
The loss and failure scenarios considered include the
loss and corruption of pac k ets during transp ort rout
ing or forw arding within the net w ork or loss of state
in router no des due to proto col daemon failures ma
c hine crashes or insucien t resources eg memory
In this studyw e address single fault mo dels ie
those that address the o ccurrence of a single fault p er
scenario eg single pac k et loss
Loss of pac k ets P ac k et loss ma y o ccur in v arious
parts of the net w ork due to congestion or link no de
or in terface failures W e classify these ev en ts as sim
ply pac k et loss regardless of cause and create ex
haustiv e loss scenarios to capture all the p ossible pro
to col transitions and pathologies due to pac k et loss
Here the fault mo del used for pac k et loss is the
loss of a single proto col message b et w een the sender
of that message and anyofthe in tended receiv ers
F or most m ulticast proto cols when routers are
connected via a m ultiaccess net w ork or LAN
hop
b yhop messages are m ulticast on the LAN and ma y
exp erience selectiv e loss ie ma y b e receiv ed bysome
no des but not others The lik eliho o d of selectiveloss
is increased b y the fact that LANs often con tain h ubs
bridges switc hes and other net w ork devices Selec
tiv e loss ma y aect proto col robustness Similarly m ultipart y proto cols and applications m ust deal with
situations of selectiv e loss This dieren tiates these
applications most clearly from their unicast coun ter
parts and raises in teresting robustness questions
Our case study illustrates wh y selectiv e loss should
be considered when ev aluating proto col robustness
This do cumen t do es not attempt to address automatic sce
nario generation whic h is the sub ject of our ongoing w ork W e
are in v estigating t w o approac hes based on forw ard and bac k
w ard searchtec hniques to extract errorprone scenarios and
top ologies automatically from a nitestatemac hine mo del of
the proto col
W e use the term LAN to designate a connected net w ork
with resp ect to IPm ulticast This includes shared media suc h
as Ethernet or FDDI h ubs switc hes etc
This lesson is lik ely to extend to the design of higher
la y er proto cols that op erate on top of m ulticast and
can ha v e similar selectiveloss The input to the loss failures substage sho wn
in gure is obtained from initial traces of sim u
lations without proto col message loss These traces
guide further sim ulations to co v er all p ossible proto
col message loss scenarios
Loss of state State loss in no des ma y o ccur due
to crashes loss of m ulticast unicast or all forw arding
en tries Wein v estigate howsuc h loss aects the pro
to cols o v erall correctness esp ecially from the end
systems p ersp ectiv e
Sim ulation and T racing
During this stage the proto col mec hanisms are sim u
lated and traces are collected
Sim ulation
One desirable approachfor sim ulating complex pro
to cols is to include detailed mec hanisms of parts of
the proto col while abstracting out others w e call this
approac h subsetting Subsetting refers to selecting subsets of the proto
col functions while abstracting or remo ving others
This allo ws us to fo cus on sp ecic parts of the proto
col state space Subsetting can b e based on
Pr oto c ol functions Subsetting proto col func
tions or mec hanisms refers to the abstraction
of these functions This maybeac hiev ed byre placing a complex mec hanism b y a simpler one
exhibiting similar external b eha vior under re
laxed assumptions F or example one mayuse
static conguration instead of sim ulating a de
tailed b o otstrap algorithm This w a y one ma y
study other proto col mec hanisms assuming cor
rectness of the b o otstrap mec hanism
Pr oto c ol states Astudy ma y fo cus on sp ecic
proto col states This allo ws for example the
studyofm ulticast group state without dealing
with sourcesp ecic state
Messages typ es This allo ws the examination of
sp ecic proto col message t yp es in the absence
of others
Note that subsetting do es not permit all com bi
nations of the ab o v e items T o main tain proto col
correctness an abstracted part has to b e replaced b y
its equiv alen t that exhibits similar external b eha vior
F or example one ma y not simply remo vethe boot strap mec hanism and run the sim ulations
T racing
T racing is the pro cess of logging information ab out
ev en ts or pac k ets during the sim ulation run Logged
information is analyzed during the p ostpro cessing
ie the output analysis stage In addition some
traces are used as feedbac k to the scenario genera
tor to guide further sim ulations W e consider sev eral
kinds of tracing
Endp oin t tracing T racing endp oin ts includes log
ging information p ertaining to hosts sending or re
ceiving pac k ets and joining or lea ving m ulticast groups
A detailed description of the traces used in the case
study is giv en in section T o iden tify errors and pathologies in the proto col
itself w e fo cus on the eect of the m ulticast routing
proto col transitions on the endp oin tpac k et deliv ery
as explained in section Proto col state transition tracing A proto col can
b e represen ted b y a nite state mac hine automaton
consisting of states transitions and stim uli inputs
outputs and timer actions Based on kno wledge of
initial proto col states w e obtain the sequence of pro
to col transitions b y tracing all stim uli
W e use proto col traces to diagnose and v erify pro
to col b eha vior and to analyze errors
Link tracing Wek eep trackof pac k ets tra v ersing
links or LANs b et w een no des as w ell as the ev en ts
of pac k et loss on links or LANs This information is
used in sev eral w a ys in output visualization output
analysis or as feedbac k for scenario generation Links
carrying message t yp es of in terest are targeted for in
ten tional pac k et loss in further sim ulations This re
duces the n um b er of loss scenarios examined to those
directly aecting the proto col b eha vior under in v es
tigation
Co de annotation When placed in k ey p oin ts suc h
as b eginning of proto col pro cedures or co de mo d
ifying the state of the proto col co de annotations
capture in ternal execution of the proto col mac hinery W e use co de annotation to estimate what part of the
co de and subsequen tly the proto col has b een exe
cuted and stressed co de co v erage
Output Analysis
One ma jor concern of STRESS is to iden tify patho
logical cases and indicate when and if an error o c curred and wh y This is ac hiev ed in the output anal
ysis stage whic h consists of
Iden tifying endp oin t errors Error conditions ma y
b e sp ecied with resp ect to endp oin t traces Exam
ples of endp oin t error conditions are blac k holes
and pac k et duplication where more than one cop yof
the same pac k et is receiv ed b y a group mem b er
If the factors during one sim ulation run are rela
tiv ely static ie static unicast routing static top ol
ogy and con trolled loss the error ma y b e attributed
to an error in the m ulticast routing proto col
Once the sp ecied error is iden tied b y the out
put analyzer the trace log is rolled backin time to
in v estigate the proto col traces as explained next
Relating errors to proto col After detecting an
endp oin t error the output analyzer isolates the p os
sible causes of suc h errors in the form of proto col
traces The output analyzer in this case is similar
to a logic analyzer allo wing the designer to na vigate
bac kw ard in time and in v estigate the causes of the
error
As will be sho wn in section the pro cess of
iden tifying a proto col error ma y suggest xes to the
problem
Co de proling The proler captures information
ab out the annotated co de suc h as whic h pro cedures
w ere or w ere not in v ok ed and the order and fre
quency of in v oking proto col pro cedures This infor
mation indicates the p ortion of the proto col stressed
b y the examined scenarios
Case Study of PIMSM
Toev aluate the utilit y of STRESS w e applied it to a
complex m ulticast routing proto col PIMSM Before
going in to details of the case study w e rst giv ean
o v erview of m ulticast routing and PIMSM
Multicast Routing Ov erview
Multicast distribution trees ma y be established b y
either broadcastandprune or explicit join proto cols
In the former suc has D VMRP or PIMDM a m ulti
cast pac k et is broadcast to all leaf subnet w orks Sub
net w orks with no lo cal mem bers for the group send
prune messages to w ards the sources of the pac k ets
to stop further broadcasts Link state proto cols suc h
as MOSPF broadcast mem b ership information to all
no des In con trast in explicit join proto cols suc h
as CBT or PIMSM routers send hopb yhop join
messages for the groups and sources for whic h they
ha v e lo cal mem b ers When receiv ed these messages
A blackhole isanobserv able amoun t of consecutivepac k et
loss b et w een p erio ds of pac k et deliv ery
build routing state in routers and cause further mes
sages to b e sen t upstream un til the distribution tree
is established Up on arriving at a router a m ulticast
pac k et is forw arded according to the routing state
In this pap er w e study PIMSMs mec hanisms for
building shared m ulticast trees F or simplicit yw edo
not address sourcesp ecic trees in this description
AB C
D
1. Receiver sends a PIM join toward the RP
RP
Sender
Receiver
2. Sender sends a PIM register to the RP
3. RP sends data packets
down the established path
establishing a path from RP back to the receiver.
Figure Ho w senders rendezv ous with receiv ers
As sho wn in gure when a receiv ers lo cal router
A disco v ers it has lo cal receiv ers it starts send
ing periodic join messages to w ard a groupsp ecic
Rendezv ousP oin t RP The join messages are m ulti
cast hopb yhop Eac h router along the path to w ard
the RP builds a wildcard an ysource r oute entry for
the group and sends the join messages on to w ard the
RP A route en try is the state held in a router to
main tain the distribution tree T ypically it includes
the source address group address the in terface from
whichpac k ets are accepted inc oming interfac e and
the list of in terfaces to whic h pac k ets are sent out
going list This state forms a shared RPro oted
distribution tree that reac hes all group mem b ers
When a source rst sends to a group its lo cal
router D unicasts r e gister messages to the RP with
the sources data pac k ets encapsulated within Data
pac k ets reac hing the RP are forw arded nativ ely do wn
the shared tree to w ard group mem b ers
Similarly when a mem ber lea v es the group a
prune message is sen t b y the lo cal router to stop
the m ulticast trac from o wing do wn the branc h
leading to the pruned mem ber Being robust to at least a single message loss
w as a design goal for PIMSM
W e poin t out t w o PIM mec hanisms relev an t to
this study Assert and pruneoverride The PIM As
sert mec hanism is the pro cess b y whic h at most one
forw arder for a LAN is selected to a v oid duplicates
in case of m ultiple p oten tial forw arders due to par
allel paths to the source or RP The pruneoverride
enables a do wnstream router ie with do wnstream
mem b ers to retain its established branc h of the tree
in case another router on the same LAN tries to prune
that branc h
The rest of this section is outlined as follo ws Sec
tion establishes the equiv alence relationship for
the top ology used for the case study Section describ es the sim ulation test suites An example of
applying the metho d is explained in section and
the scenario and proto col co v erage ac hiev ed for the
case study is giv en in section T op ology Equiv alence
Tw o top ologies are equiv alen t if they driv e the proto
col transitions in to the same states under the same
set of ev en t sequences A top ology is reducible to an
other top ology with few er connections and routers
if the t w o top ologies are equiv alen t
W e sho w in this section that for single message
loss scenarios the fourr outer LAN top ology adopted
in this study exp eriences the same proto col errors
that an Nr outer LAN top ology exp eriences where
N and hence they are equiv alen t Wesho w this
relationship for the messages under study for PIM
SM namely joins prunes and asserts F or brevit y w e only pro v e equiv alence in the case of prune mes
sages and hin t to the pro of approac h in the other
cases W e also iden tify assumptions and limitations
under whic h this equiv alence relationship holds
A
B
C
A B
C
Topology I
A B
C D
D
[3-router LAN]
Topology II
[4-router LAN;
Topology III
[4-router LAN;
downstream addition] upstream addition]
downstream
upstream
Figure The equiv alen t top ologies
Prunes
First w e consider Nr outer LAN top ologies where
N and resp ectiv ely It is trivial to pro v e
Toac hievethis a do wnstream router receiving a prune on
its incoming in terface triggers a join upstream
that these top ologies are not equiv alen t for hopb y
hop messages
Assumption Nr outer LAN top olo gy wher e N is r e ducible to the thr e er outer LAN top olo gy for
prunes wrt single message loss sc enarios
T o justify our assumption w e rst pro v e that a
fourr outer LAN top ology is reducible to a thr e er outer
LAN top ology Correctness condition Ifar outer on the LAN
has the LAN as its inc oming interfac e ther emust b e
one other r outer with the LAN in its outgoing list Once this condition is satised violating it is consid
ered a proto col error
Next w e examine the thr e er outer LAN top ology In gure top ology I assume that A and B are
do wnstream routers and C is an upstream router
In gure top ology I router C has the LAN
in its outgoing list router A has the LAN as
its incoming in terface and router B is lea ving
the group and so sends a prune to w ards CThe
prune is m ulticast on the LAN
The only case where the correctness condition
ma y be violated is when C receiv es the prune
while A do es not In the other cases either the
prune is not receiv ed b y C or is receiv ed b y A
whic h triggers a pruneoverride to reestablish
the LAN in Cs outgoing list This is illustrated
b y the follo wing selectiv e loss pattern table for
the prune message sentb y B A C
error
where a indicates noloss and indicates
loss The error o ccurs where the upstream router
C receiv ed the prune but the router with
do wnstream mem bers A did not receiv eit In gure top ology I I w e add another do wn
stream router D The selectiv e loss pattern ta
ble follo ws
This is to dieren tiate b et w een join latency whic hisnot
considered a proto col error and a blac k hole whic h is a proto col
error
A D C
error
The only error o ccurs when the upstream router
C receiv es the prune but neither of the do wn
stream routers receiv es it If the prune is re
ceiv ed byan y of the do wnstream routers a prune
override w ould reestablish the LAN in Cs out
going list
F rom the symmetry of the loss patterns and top ol
ogy w e see that all errors are triggered b y the same
transitions exp erienced b y router A in top ology I
Hence the extended top ology I I do es not in tro duce
an y new errors and exhibits the same external b eha v
ior as do es top ology I W e conclude that top ology
I and top ology I I are equiv alen t for prunes Wenowsho w that the Nr outer LAN top ology
is reducible to the N case where N
With the addition of an upstream router gure top ology I I I no added error cases are encoun tered
The addition of a do wnstream router ho w ev er ma y
in tro duce new error scenarios Similar to the four
r outer LAN case w e establish the follo wing asser
tion the only err or c ase o c curs when al l downstr e am
r outers lose the prune and the upstr e am r outer r e
c eives it If the prune w as receiv ed b y an y of the
do wnstream routers the correctness condition w ould
b e retained using pruneoverrides
The assertion holds in b oth top ologies Hence w e
conclude that the Nr outer top ology exp eriences the
same errors as the Nr outer top ology F rom the ab o v e w e see that b y sim ulating the
thr e er outer LAN top ology w e capture all the errors
with resp ect to selectiv e loss for the prune mec ha
nism that ma y b e exp erienced byan y Nr outer LAN
top ology where N
Joins
F or lossfree scenarios the equiv alence pro of is straigh t
forw ard F or lossy scenarios the loss of a join mes
pruneo v errides are actually join messages The eect of
join message loss is describ ed in section
sage sen t to an upstream router ma y lead to join la
tency but do es not cause blac kholes
Joins lead
ingtopac k et duplication lead to asserts that are dis
cussed next
Asserts
In most cases pro ofs similar to those presen ted can
b e applied to Asserts Ho w ev er since asserts ma ybe
triggered due to parallel paths the base case is estab
lished for the fourr outer LAN top ology Figure top ology I I I represen ts the fourr outer LAN top ol
ogywhere A and B are do wnstream routers and C
and D are upstream routers
RP
S1
S2, R2 R1
AB
CD
RP
S1
R2 R1
AB
CD
RP
S2, R2 R1
AB
CD
unicast route
to RP
Topology 1 Topology 2 Overall topology
Figure The top ology used for the case study
F or our case study w e use a fourr outer LAN
top ology with an added Rendezv ousP oin t RP to
capture shared tree c haracteristics The o v erall ph ys
ical top ology consists of v e routers four of whic hare
connected via a LAN as sho wn in gure T est suites
In this section w e elab orate on the routed top ology host scenarios and loss pattern generation used for
our case study W e also describ e the simplications
and subsettings applied
Note that if join suppression as describ ed b y the PIMSM
sp ecication is implemen ted the equiv alence relation can
b e established similar to the prunes case ab o v e
A limitation to the fourr outer LAN top ology is giv en for
the esoteric case of three upstream routers and three do wn
stream routers with inconsisten t unicast routing tables This
case creates one extra transition that can only b e captured b y
at least a sixr outer LAN top ology W e do not consider this a
practically signican t scenario and w e consider its analysis as
a sp ecial case not captured b ythe fourr outer LAN top ology Ho w ev er aside from this exception the Nr outer LAN top ol
ogy where N is equiv alenttoa fourr outer LAN top ology
wrt asserts Ph ysical and routed top ologies The o v erall top ol
ogy used is that sho wn in gure F or the unicast
routing proto col w e use a cen tralized v ersion of Dijk
stras Shortest P ath First SPF algorithm PIMSM uses the underlying unicast routing ta
bles for building m ulticast trees Therefore unicast
routing inconsistencies aect the op eration of PIM
SM T oin v estigate suchin teraction w e add a comp o
nen t to force inconsisten t m ulticast routes bet w een
PIM routers as sho wn in gure top ology Incon
sisten t unicast routing cases arise for example when
am ulticast region spans m ultiple unicast domains or
ASs
Host scenarios Since proto col states for dieren t
groups do not in teract w e consider only one group
Also since proto col states for dieren t sources do
not in teract it suces to consider only one source
S per sim ulation run
The source is mo deled as
a constan tbit rate CBR stream with xed pac k et
size The source mo del do es not aect the correctness
of the metho d Ho w ev er to assure full con trollabilit y
o v er the selectivelossmodel w e set the data rate to
ensure that no loss o ccurs due to queue o v ero w
While w e consider only a single source w e con
sider t w o receiv ers R and R for the same group
to accoun t for shared tree state in teractions Weuse
the host scenarios describ ed in section Loss patterns Wein v estigate all p ossible selectiv e
loss scenarios for m ulticast hopb yhop PIMSM mes
sages in this represen tativ e top ology Loss mo dels are applied exhaustiv ely to those links
that carry the proto col messages under in v estigation
The tracing stage iden ties these links during the rst
sim ulation run without pac k et loss and feeds bac k
the link information to the loss generation mo dule as
sho wn in gure As wewillsho w in section the
n um b er of represen tativ e scenarios is quite small and
hence the n umberofo v erall lossy scenarios explored
is manageable
W e do not address state loss or no de crashes in
this do cumen t Ho w ev er crash scenarios ma ybe im plemen ted in a w a y similar to loss scenarios
A similar in teraction with unicast routing ma y b e created
easily when switc hing to the shortest paths in PIMSM
W e do not consider aggregated source or group en tries in
this study
F or this w e use pac k et size of b ytes and a send in terv al
of ms ie source rate of kbs this ensures no queue
drops on the Mbs links used with pac k et queue limit
T racing T race information includes the ev entt yp e
send or receiv e the no de exp eriencing the ev en t
the t yp e of message sen t or receiv ed and the time
at whic h the ev en t o ccurred Ev ery data pac k et is
assigned a unique sequence n um ber F or example a trace ma y tak e the follo wing for
mat R No de A Rcv t meaning that
receiv er R in no de A receiv ed a data pac k et with
sequence n um ber at time ms from the b egin
ning of the sim ulation run
Subsetting F or brevit yw e do not consider source
sp ecic trees and switc hing to the shortest paths in
this pap er This is an example of state subsetting since w e consider shared group states while disregard
ing sourcesp ecic states
The messages considered in the study are join prune assert and r e gister messages Tostudy joins prunes and asserts without the eect of r e gistersw e
consider a top ology where the source and the RP are
colo cated see S in gure top ology This is an
example of message subsetting When studying r e gisters joins and prunes w e con
sider top ology in gure where a S is the source
hence no de A sends registers to the RP and b the
routed top ology has consisten t unicast routing to
eliminate the eect of the assert mec hanism This
represen ts function or me chanism subsetting Only triggered actions are in v estigated for sim
plicit y Timer action analysis is not considered in
this study and is part of our w orkinprogress
Applying the Metho d
This section pro vides an illustrativ e example sho w
ing ho w STRESS may beusedto iden tify and analyze
errors encoun tered during the sim ulation of the rep
resen tativ e scenarios
The Sim ulation F ramew ork
Weha v e implemen ted an initial v ersion of the STRESS
metho d in the Net w ork Sim ulator NS NS is
an ev en tdriv en pac k etlev el sim ulator con trolled and
congured via Tcl and Ob jectTcl or
OTcl
T o supp ort our metho d w eha v e added
mo dules to pro vide LAN supp ort con trolled selec
tiv e loss proto col tracing proling capabilities and
a detailed implemen tation of PIMSM
This imple
men tation serv es as the sim ulation en vironmen t for
F or information ab out the sim ulator see
h ttpcatarinausceduvin t
Our detailed PIMSM sim ulation mimics the unix
pimd implemen tation mo del and hence is able to capture
our case study In addition the building blo c ks w ere
designed to b e reused within the same framew ork to
apply this metho d to other m ultipart y proto cols
Figure giv es a high lev el description of the net
w ork and proto col sim ulation mo dules describ ed b e lo w
Net w ork sim ulation The net w ork mo del consists
essen tially of links and no des The building blo c ks
eg the queues and the forw arding mac hineries
are implemen ted in C as part of NS core mec ha
nisms to impro v e p er pac k et pro cessing p erformance
These building blo c ks are connected using OTcl ob
jects suc h as links LANs and net w ork no des
Link A link consists of the follo wing building
blo c ks
Queue structure pro vides enqueuing dequeuing
and drop primitiv es The queue drop p olicy is
selected from v arious options eg droptail or
RED Delay mo dule sim ulates the transmission and
propagation dela ys on a link
L oss mo dule con trols the loss patterns on the
link The loss ma y be based on the t yp e of
message according to a losstrace le
T r ac er mo dules tap v arious comp onen ts of the
link to capture pac k et and loss trace informa
tion These mo dules pro vide full observ abilit y
of the link ev en ts
LAN A LAN is built of m ultiple links in ad
dition to a r eplic ator When the replicator receiv es
apac k et it copies and deliv ers it to eac h of the con
nected queues and subsequen tly to neigh b oring no des
This mo del facilitates the sim ulation of selectiv e loss
as the loss mo dules for the connected no des are con
trolled indep enden tly Net w ork no de The net w ork no de pro vides the
m ulticast and unicast forw arding mec hanisms inde
p enden t of routing proto cols This mo del resem bles
man y implemen tation asp ects W eplan todev elop an in terface
bet w een the sim ulator and an op erational net w ork running the
pimd co de Ho w ev er the analyses presen ted in this study are
based strictly on the proto col sp ecication indep enden tof the
implemen tation
.
.
.
multicast
forwarding
unicast
forwarding
packet
classifier
network
interface
LAN
replicator
queue delay
loss
network
interface
network
interface
network
interface
loss
tracer
loss
delay
delay queue
queue
Node
Node
Node
tracer
tracer
message agent
Send / Rcv
Send handler 1
Send handler 2
Send handler n
Rcv handler 1
Rcv handler 2
Rcv handler n
timer handlers
state storage
Node
Protocol
network
interface
Figure Net w ork and Proto col sim ulation building blo c ks
that of a unix k ernel forw arding mec hanism F ol
lo wing are the basic building blo c ks for the net w ork
no de
Packet classier receiv es pac k ets from the net
w ork in terfaces or the lo cal proto cols and passes
them to the unicast or m ulticast forw arding mec h
anism as appropriate
Multic ast and unic ast forwar ding pro vide the
lo okup tables used to forw ard m ulticast and
unicast pac k ets The forw arding tables at a
no de are installed b y the routing proto cols run
ning at that no de
Proto col sim ulation The proto col w as implemen ted
using the dynamic ob jectorien ted language OTcl whic h
pro vides exibilit y and rapid protot yping Proto
col mo dules w ere designed to facilitate subsetting of
functions
Arc hitectural mo dules A proto col instance is
attac hed to an NS net w ork no de and performs the
proto col functions sp ecied The proto col consists of
the follo wing arc hitectural building blo c ks
Send and r e c eive primitives also called mes
sage agents handle incoming messages dispatc h
ing to the appropriate pro cessing mo dule and
sending messages to the unicast or m ulticast
forw arding mac hinery within a no de or directly
to the net w ork in terface One ma y think of
these primitiv es as a so c k et la y er
Send hand lers enco de the outgoing messages
in to the prop er pac k et format and pass the mes
sages to the send primitiv es Eac h send handler
handles a sp ecic proto col message t yp e for ex
ample Prune
R e c eive hand lers pro cess the incoming proto
col con trol messages Eac h handler pro cesses a
sp ecic proto col message t yp e
Pr oto c ol state stor age represen ts the state of the
proto col in terms of en tries ags and timer v al
ues An example of state storage is the m ulti
cast routing tables
Timer hand lers pro cess the proto col timed ac
tions up on expiration of timer v alues k ept in
the proto col state
Sim ulating PIMSM
Tov erify that our PIMSM implemen tation conforms
to the proto col sp ecication w e ran sev eral confor
mance testsuites using the sim ulator
Obtaining fault y scenarios T o obtain the faulty
sc enarios ie those that con tain errors w e execute
the metho d stages in order ie scenario generation
sim ulation and tracing and output analysis resp ec
tiv ely and then rev erse the order from the output
duplicates
loss
Sent by S1
Rcvd by R1
Rcvd by R2
time
seq. No.
7
8
9
10
11
12
13
200 250 300 350 400
J2 L1
S1 Node RP Send 12 t 300
L1 Node A Leave G t 300
PIMS Node A Send Prune{NH=C} t 300
PIMR Node C Rcv Prune{NH=C} t 310
PIMS Node C Send Prune{NH=RP} t 310
PIMR Node B Rcv Prune{NH=C} t 310
PIMS Node B Send Join{NH=C} t 310
PIMR Node D Rcv Prune{NH=C} t 310
R2 Node B Rcv 12 t 321
PIMR Node C Rcv Join{NH=C} t 321
PIMS Node C Send Join{NH=RP} t 321
PIMR Node RP Rcv Prune{NH=RP} t 321
PIMR Node A Rcv Join{NH=C} t 321
PIMR Node D Rcv Join{NH=C} t 321
S Node RP Send 13 t 325
PIMR Node RP Rcv Join{NH=RP} t 332
S Node RP Send 14 t 350
R2 Node B Rcv 14 t 371
S Node RP Send 15 t 375
R2 Node B Rcv 15 t 396
x 10
-3
S1 Node RP Send 7 t 175
R1 Node A Rcv 7 t 190
S1 Node RP Send 8 t 200
J2 Node B Join G t 200
PIMS Node B Send Join{NH=D} t 200
PIMR Node A Rcv Join{NH=D} t 210
PIMR Node D Rcv Join{NH=D} t 210
PIMS Node D Send Join{NH=RP} t 210
PIMR Node C Rcv Join{NH=D} t 210
R1 Node A Rcv 8 t 221
R2 Node B Rcv 8 t 221
PIMR Node RP Rcv Join{NH=RP} t 221
S1 Node RP Send 9 t 225
R1 Node A Rcv 9 t 246
R2 Node B Rcv 9 t 246
PIMS Node D Send Assert t 246
PIMS Node C Send Assert t 246
R2 Node B Rcv 9 t 247
R1 Node A Rcv 9 t 247
S1 Node RP Send 10 t 250
PIMS: sent by the PIM component
PIMR: received by the PIM component
NH: next hop
Figure Simple pac k et trace graph sho wing pac k et loss and duplication
to the traces to iden tify the fault y scenarios These
phases are automated b y the to ols pro vided and are
transparen t to the user once the scenario setup is
complete
The pro cess of attributing endp oin t errors to pro
to col actions ma y be automated only if the error
conditions are giv en in terms of suc h proto col ac
tions In practice ho w ev er these proto col error con
ditions are often not kno wn a priori b y the designer
and are usually dened in terms of endp oin t errors
suc haspac k et loss or duplication The supp orting
to ols iden tify endp oin t errors and pro vide a history
of proto col traces The designer then examines the
traces and iden ties the proto col errors
This pro
cess ma y suggest xes to the problem as w ewill sho w
in the results section
Example In our simple example an error condition
is anypac k et loss or duplication exp erienced b ythe
endp oin ts A fault y scenario without pac k et loss
that leads to t w o error conditions is iden tied and
explained Then the proto col actions leading to the
errors are analyzed
The represen tativ e scenario explained here is
JJLL using top ology This scenario w as
iden tied automatically as a fault y scenario T races
in gure giv e the history of the errors found The
rst error ie the pac k et duplication has the host
ev en t J as the closest join or lea v e host ev en t in
In our limited exp erience w eha veiden tied proto col er
rors in the recen t history of the endp oin t errors
its history at time ms The error is a join tran
sien t caused b y parallel paths to the RP The error
is resolv ed using the Assert messages exc hanged dur
ing the duplication at time ms The second error
ie pac k et loss is a lea v e transien t it has a host
ev entLin its recen t history The loss is due to the
prune sentbynode A at ms and is resolv ed bya
pruneoverride sentbynode B at ms
Although the proto col actions leading to the end
poin t errors sp ecied as an ypac k et loss or duplica
tion in this sp ecic example are considered transien t
errors they are not considered proto col design errors
Wedo ho w ev er address proto col design errors in sec
tion Scenario and proto col co v erage
While the fact that wew ere able to disco v er design
errors pro vides some evidence of the metho ds utilit y w e w ould lik e to quan tify the co v erage of proto col
states and p ossible scenarios
The o v erall proto col co v erage has t w o dimensions
The rst is the proto col state co v erage and w e at
tempt to co v er this dimension using the r epr esenta
tive scenarios reac hable states In v estigation of the
loss scenarios do es not aect proto col co v erage sig
nican tly The second dimension is the space of p ossible in
teraction scenarios b et w een these state mac hines in
dieren t routers within the top ology This dimen
sion is explored byin v estigating the sele ctive loss sce
narios
Scenarios co v ered The initial n um ber of sim u
lated scenarios without proto col message loss w as
X
topolog ies
No rep scenarios Where No rep scenarios is the n um ber of r epr e
sentative scenarios equal to in our case discussed
in section and the top ologies are the t w o dis
cussed in section Hence w e sim ulated scenar
ios without proto col message loss
After feeding bac k the link traces for the messages
under study the loss patterns w ere assigned to the
corresp onding links The scenario generator then set
up the sim ulations for the new scenarios with loss
The total n um b er of scenarios with proto col mes
sage loss sim ulated is giv en b y the follo wing form ula
X
T opos
X
Reps
X
Msgs
X
Link s
Link M sg s Link Rtr s
where the terms used are describ ed in the follo w
ing table
T erm Meaning
T op os T op ologies
Reps Represen tativ e Scenarios
Msgs Messages under study
LinkMsgs No messages tra v ersing the link
LinkR trs No routers connected to the link
F or eac h top ology this form ula giv es the n um ber
of scenarios automatically generated after the rst
sim ulation run during whic h the n um b er of messages
and links tra v ersed b y these messages is coun ted
F or example for the rst top ology the messages
under study w ere joins prunes and asserts The rep
resen tativ e scenarios triggered joins prunes and asserts on the LAN and joins and prunes on p oin ttop oin t links Hence the total n um
b er of scenarios with loss b ecame
scenarios F or the second top ology the messages under study
w ere joins and prunes The represen tativ e scenarios
triggered joinsand prunes on the LAN and joins and prunes on p oin ttop oin t links Hence
the total n um b er of scenarios with loss b ecame
scenarios Proto col co de co v erage A large p ortion of the
m ulticast supp ort co de in NS w as annotated auto
matically to pro vide co de tracing Out of pro ce
dures pro cs follo wing is a summary of the pro ce
dures co v ered b y the represen tativ e scenarios
T op ology Pro cs co v ered !ge
T op ology !
T op ology !
T otal !
Pro cedures that w ere not in v ok ed dealt mainly
with sourcesp ecic state whic h w as abstracted in
our test suites or with the mo dularit y of the ob ject
orien ted nature of the co de
Results
This section describ es the proto col design errors un
co v ered for PIMSM under STRESS
Unlik e our simple example ab o v e w e are only in
terested in design ie not transien t errors F or
this w e mo died the error conditions to a v oid join
and lea v e transien ts The new error conditions do
not consider single duplication or loss
F ollo wing is a summary of the ma jor fault y sce
narios encoun tered and ho w they relate to STRESS
F or a more detailed discussion of the proto col errors
and xes see section Summary of Results
W e describ e a partial list of faulty sc enarios captured
b y STRESS W e obtained this list after sim ulating
only a few of the represen tativ e scenarios The traces
pro duced pro vided guidance to disco v er the proto
col errors Design errors disco v ered include Assert JoinPrune and R e gister mec hanisms
Asserts F or the rst top ology gure top ology
a blac k hole w as observ ed for one receiv er
The fault y scenario in this case in v olv ed another
receiv er joining in the recen t history of the blac k hole
By analyzing the proto col trace history after rolling
bac k w e noticed that an Assert pro cess to ok place
righ t b efore the loss
In addition the fault y scenario included the loss of
a join message whic hprev en ted the establishmentof
the branc h of the shared tree from the Assert winner
to the RP Hence the proto col design error is allo wing
a router on a branc h of the tree that is not completely
established to participate in Asserts Joins and Prunes Ov er the same top ology ie
gure top ology sev eral other fault y scenarios
lead to blac k holes The host scenarios in v olv ed one
receiv er lea ving just b efore blac k holes w ere exp eri
enced bythe other receiv er In these cases join and
prune messages o ccurred the recen t history of the
endp oin t error
F urthermore all suc h scenarios included either
i loss of a join message prev en ting a pruned branc h
from being reestablished or ii selectiv e loss of a
prune message prev en ting a join ie pruneoverride from b eing triggered The proto col design error in
this case w as not allo wing a second c hance for routers
with do wnstream mem b ers to o v erride prunes Registers In the second top ology gure top ol
ogy fault y scenarios w ere captured that cause pac k et
duplicates at the endp oin ts
In this case the observ ed fault y scenarios did not
follo w a regular pattern and w ere dev elop ed itera
tiv ely ie when one fault y scenario led to a suggested
x in the proto col the x w as implemen ted and the
metho d rerun to observ e further fault y scenarios
The rst scenario in v olv ed a single host receiving
duplicates merely b y joining the group The pac k ets
w ere b eing deliv ered at least t wice once directly from
the source b y virtue of being on the same LAN
and the second deliv ery from the shared tree after the
r e gister reac hed the RP and w as sentdo wn the shared
tree When the n um b er of pac k et duplicates exceeded
t w o this suggested a lo op The lo op o ccurred when a
pac k et receiv ed o v er the shared tree on the LAN w as
a pic k ed up b y the lo cal router b reregistered
to the RP and c forw arded do wn the shared tree
again The proto col error w as allo wing the pac k ets
to o wdo wn from the shared tree to the originating
LAN and b e reregistered The x w as to prune suc h
sources from the shared tree
The second scenario in v olv ed another receiv er join
ing b efore the duplicates w ere observ ed The pruned
branc h of the shared tree w as reestablished b y the
joining receiv er allo wing the pac k ets to o w do wn
the shared tree to the originating LAN and subse
quen tly causing the lo op
The third scenario in v olv ed a prune message loss
again allo wing the pac k ets to o w do wn the shared
tree to the originating LAN and led to lo oping
Rules w ere added to prev en t pac k ets from b eing
forw arded bac k on their original LANs in the ab o v e
scenarios
Detailed Results
The rest of this section describ es the ab o v e fault y
scenarios in more detail and illustrates ho w the solu
tions w ere dev elop ed with the aid of STRESS After
the solutions w ere in tegrated in to the proto col sim
ulator w e applied regression testing to v erify that
the xes did not in tro duce an y new errors
Assert analysis
F ollo wing is a discussion of the pathological cases
found in the Assert pr o c ess An exhaustiv e list of the
results is not included in this do cumen t for brevit y A
few errors in the PIMSM sp ecication w ere un v eiled
during this pro cess w e fo cus on errors that created
the p ossibilityof pac k et loss ie blac k holes
RP
S1
R2 R1
AB
CD
1
2
RP
S1
R2 R1
AB
CD
5
RP
S1
R2 R1
AB
CD
3
4
1) R1 joins the group. B sends joins towards RP.
2) S1 sends packets to the group. Packets flow
3) R2 joins the group. A sends joins towards RP.
4) The join from C to RP is lost.
RP
S1
R2 R1
AB
CD
6
7
5) Packets forwarded by D onto the LAN are
received by C on an outgoing interface.
6) C Asserts with a winning metric onto the LAN.
7) D removes the LAN from its entry and sends
prunes towards the RP.
down distribution tree and are multicast on the LAN.
(I)
(II)
(III)
(IV)
Figure The Assert scenario under study
The scenario In this scenario the top ology in g
ure w as setup suc h that As nexthop to w ards the
RP is Cand Bs nexthop to w ards the RP is D Consider the sequence of ev en ts sho wn in gure whic h used the represen tativ e scenario JJLL
with the loss of a join message on the link b et w een C
and RP During the last t woev en ts of the scenario steps
and D loses the Assert pro cess to C with lo w er
metric or higher address Subsequen tly D remo v es
the LAN from its en trys in terface list and R stops
receiving pac k ets from S This problem p ersists un
tilunless the branc h of the tree from C to RP is
established
Discussion and x The curren t rules of the PIM
sp ecication aim to guaran tee atmost one forw arder
on a m ultiaccess net w ork Ho w ev er to ensure prop er
deliv ery of pac k ets without pac k et loss the righ t se
man tics should b e exactly one forw arder
The problem arises more sp ecically b ecause the
PIM sp ecication do es not distinguish bet w een an
activ e en try ie an en try created due to arriv al of
data pac k ets eg am ulticast forw arding cac he and
an en try on a branc h of a tree that is not y et estab
lished or an inactive en try An inactiv e en try
ma y win an Assert pro cess resulting in blac k holes
Tosolv e this problem w e mo died the sp ecica
tion to ensure exactly one forw arder seman tics using
the follo wing rule A router receiving a data pac k et
or Assert on an outgoing in terface of a matc hing en
try do es not participate in the Assert pro cess unless
the en try is activ e Figure illustrates the Ac
tiv eState added to the transition diagram to realize
the solution
NoState
State for G
ActiveState
OifDeleted
Rcv join for G; create state, trigger join upstream
Rcv join for G
Rcv pkts for G; activate state, forward pkts
Rcv pkts for G; forward pkts
or
Rcv assert on oif & win; send assert
Rcv assert on oif & lose; delete oif from entry
All oifs deleted
oif: outgoing interface
& entry removed
G: multicast group
Figure T ransition diagram for joins and asserts
JoinPrune analysis
In this analysis w e address the eect of selectiv e loss
of JoinPrune messages Although this problem has
b een addressed in recen t releases of the PIMSM sp ec
ication w e pro vide a more ecien t solution
W e use the top ology giv en in gure I The
represen tativ e scenario used is JJLL with the
second join from no de A lost on the LAN
W e assume that S sends pac k ets to group G
throughout the sim ulation Consider the sequence of
ev en ts giv en in gure I After the last ev en t step
R stops receiving Ss pac k ets This problem
p ersists un til A sends the next p erio dic join to C
and reestablishes the pruned branc hofthe tree A
similar problem is encoun tered in gure II when
the prune sen t from B is selectiv ely lost on the LAN
b y A and receiv ed b y C Discussion and x The solution suggested bythe
PIM sp ecication in tro duces a deletion timer This
ho w ev er increases the lea v e latency and incurs un
necessary data o v erhead
A more ecien t solution w ould be to ha v e the
upstream router C announce a prunealert b efore
remo ving the LAN from its outgoing list b y resending
the prune message previously receiv ed from B Register analysis
F ollo wing is a description of the scenarios that exhibit
pac k et duplication due to r e gister messages and the
suggested xes to eliminate suc h duplication The
xes w ere applied iterativ elyun til the error w as elim
inated
First scenario single source single receiv er
In this scenario w e consider S and R in gure I Consider the sequence of ev en ts in the gure
P ac k et duplication and r e gister lo oping o ccur in
the ab o v e scenario A similar scenario o ccurs when
R joins rst then S starts sending to the group
Suggested xes The required b eha vior is to send
a triggered and p erio dic sourcesp ecic prune o of
the shared tree if a router has sourcesp ecic state for
registering and shared tree state for the same group
regardless of the incoming in terface settings
Second scenario single sender t w o receiv ers
W e assume the implemen tation of the ab o vexesto
the sim ulator then consider the sequence of ev en ts in
gure I I This scenario exhibits pac k et duplica
tion and r e gister lo oping
Suggested x The problem arises b ecause the pac k
ets are forw arded bac k on the originating LAN and
treated as if they w ere new pac k ets originated b y the
directly connected source The follo wing rule solv es
this problem for the giv en scenario
A router receiving join message m ust NOT add
an in terface on the same subnet as a source S
for an y source sp ecic en try for S asso ciated
with same group
Third scenario single source single receiv er
with message loss Considering the scenario in
gure I I I The source sp ecic prune sentfrom A to C when
A ha ving a shared tree state creates the source sp e
cic en try for registering is lost
P ac k et duplication and r e gister lo oping problems
are exp erienced in this scenario The problem p ersists
un til a periodic JoinPrune message is successfully
sen t upstream
Suggested x T o b e robust to at least one mes
sage loss w e suggest the follo wing rule for pac k et
forw arding
A router m ust NOT forw ard a pac k et on to the
subnet from whic h the pac k et w as originated
This is ac hiev ed b y p erforming a c heckonthe
source and the outgoing in terface b efore build
ing a source sp ecic state or b efore forw arding
apac k et
Most implemen tations create a cac he for forw arding pac k
ets This c hec k can b e done only once when creating the cac he
and is not done p er pac k et
RP
R2 R1
AB
C
S1
1
RP
R2 R1
AB
C
RP
R2
R1
A
B
C
2
S1
S1
3
3
4
5
5
4
1) R1 joins the group. B sends joins towards RP.
2) R2 joins the group. A sends joins towards RP.
3) R1 leaves the group. B multicasts prunes onto the LAN.
4) A gets the prune and sends a join to override. The join is lost.
5) C gets the prune and sends it towards RP.
(I)
(II)
3) R1 leaves the group. B multicasts prunes onto the LAN.
4) A does not receive the prune, and so does not override.
5) C gets the prune and sends it towards RP.
Figure The JoinPrune scenario under study
Related w ork
The related w ork falls mainly in the eld of proto
col v erication Most of the literature on m ulticast
proto col design addresses arc hitecture sp ecication
and comparisons b et w een dieren t proto cols Weare
not a w are of an y other w ork to dev elop systematic
metho ds for testing m ultipart y proto col robustness
In addition some concepts of STRESS w ere inspired
byVLSI c hip testing
There is a large body of literature dealing with
v erication of comm unication proto cols Proto col v er
ication t ypically addresses safety liveness and r e
sp onsiveness prop erties Safet y prop erties in
clude freedom from deadlo c ks assertion violations
improp er terminations and unsp ecied receptions
Liv eness prop erties include detection of acceptance
cycles and absence of nonprogress cycles while re
sp onsiv eness prop erties include timeliness and fault
This is dieren t than the incoming in terface c hec k stated
b y the curren t sp ecication In the sp ecic case discussed here
the lo oping m ulticast pac k ets will matc h on the incoming in
terface the LAN for the sourcesp ecic en try tolerance Most proto col v erication systems includ
ing STRESS aim to detect violations of part of these proto col prop erties
In general the t w o main approac hes for proto
col v erication are theorem pro ving and reac habil
it y analysis or mo del c hec king Theorem
pro ving systems dene a set of axioms and construct
relations on these axioms Desirable prop erties of
the proto col are then pro v en mathematically Theo
rem pro ving includes mo delb ase d formalisms suc has
Z and Vienna Dev elopmen t Metho d VDM and lo gicb ase d formalisms including rst order logic
suc h as Nqthm and higher order logic suc has
Protot yp e V erication System PVS F ormal
v erication has b een applied to TCP and TTCP but for m ultipart y proto cols theorem pro ving sys
tems are lik ely to b e ev en more complex and p erhaps
in tractable
Reac habilit y analysis algorithms try to gen
erate and insp ect all the proto col states that are reac h
able from giv en initial states Suc h algorithms suf
fer from the state space explosion problem esp e
RP
S2,R2
A
C
1
2
3
4
1) R2 joins group(G).
RP
S2,R2 R1
AB
C
1
2
3
4
5
1) R2 joins G& S2 sends to G.
2) R1 joins G. B sends join to RP,
A builds routing state& sends join
to G& prune for S2 towards RP.
RP
S2,R2
A
C
1
2
3
4
1) A sends prune to RP; R2 is member
of G& S2 is source. prune is lost.
2) A registers packets from S2 to RP.
3) RP decapsulated &forwards
packets down shared tree.
4) Shared tree packets accepted from
LAN&re-registered to RP forming loop.
2) S2 sends packets to G. A builds source
A sends joins towards RP.
(I) (II) (III)
eliminating prune state for S2 in C.
3) S2 sends to G; A registers to RP.
4) RP decapsulates & forwards
packets down shared tree.
5) Packets forwarded onto LAN are
re-registered by A, causing a loop.
state & registers to RP. Incoming interface
for the state points towards LAN.
3) RP gets registers, decapsulates &
forwards packets down shared tree.
4) Packets down shared tree are accepted
from LAN&re-registered forming a loop.
Figure The r e gister scenarios under study
cially in complex systems as are m ultipart y proto
cols T o circum v en t this problem state reduction
and con trolled partial searc h tec hniques could
be used These tec hniques fo cus only on parts of
the state space and ma y use probabilistic ran
dom or guided searc hes The SPIN to ol uses the sup ertr ac e algorithm that is actually a ran
dom con trolled partial searc h STRESS has similari
ties with guided con trolled partial searc hes Ho w ev er
STRESS explores proto col states based on the repre
sen tativ e scenarios and do es not use a cost function
as do es guided searc h
There is an analogy b et w een STRESS and VLSI
systematic design for testabilit y using BuiltIn
SelfT est BIST BIST pro vides a systematic
tec hnique for c hip testing syn thesis A generic BIST
sc heme is sho wn in gure This tec hnique can be used to detect faults due
to singlestuc kline while STRESS can be used to
detect errors due to single pac k et loss BIST uses a
test pattern generator TPG to pro duce the input
patterns applied to the circuit under test Conceptu
ally this resem bles our use of the scenario generator
The test patterns are c hosen to maximize fault co v er
Test
Circuit
Response
Test
Test
Inputs
Outputs
Error
Test
1
0
Multiplexer
generator
circuit
patterns
signal
under
test
signal
responses
monitor
circuit
Figure Generic BIST sc heme
age with a minim um n um b er of inputs Similarly the
scenario lter c ho oses the represen tativ e scenarios to
maximize proto col co v erage with a minim um n um ber
of scenarios Moreo v er BIST uses a resp onse moni
tor circuit to monitor and detect error signals This
is analogous to our use of tracing and error detection
mo dules The exp ected output for VLSI c hip test
ing is fault co v erage vs test length curv e whic his
similar to proto col co v erage vs scenarios statistics
Summary and F uture W ork
The goals of our metho d are to simplify and sys
tematize robustness analysis of m ultipart y proto cols
Our approac h is not ab out formal or abstract pro ofs
On the con traryw e attempt to pro vide practical meth
o ds to study robustness of real In ternet m ulticast pro
to cols with the aid of systematic sim ulation
This
pap er presen ted our initial attempts to ac hievethese
goals in the con text of one m ulticast routing proto col
W e used scenario generation sim ulation tracing and
output analysis to obtain a set of errorprone scenar
ios In particular w e describ ed sev eral tec hniques
T o circum v en t the state explosion problem w e
in tro duced the notion of r epr esentative sc enar
ios W e obtained these scenarios for the m ul
ticast host mo del using a sc enario lter whic h
excluded redundan t and irrelev an t scenarios
based on practical assumptions
Weiden tied t w o r epr esentative top olo gies based
on the e quivalenc e relationship established for
the proto col under study PIMSM The equiv
alence denition suggested that extending the
sim ulated top ologies w ould not rev eal additional
errors in the proto col
T o capture robustness c haracteristics w estud ied the proto col b eha vior in the presence of sin
gle pac k et loss A LAN mo dule accoun ted for
the sele ctive loss cases exp erienced bym ulticast
messages W e hop e to use a similar lo gic al LAN
mo dule to mo del loss and dela y parameters of
the underlying m ulticast distribution trees and
thereb y extend our metho d to higherlev el m ul
tipart y proto cols
T o reduce the complexit y of our analyses w e
used subsetting of proto col functions states and
messages This allo w ed abstracting out some
proto col details while retaining and fo cusing on
others
W e do not claim that the metho d presen ted here based
on heuristic scenario generation is formal or general enough
y et and w e are addressing these issues in our ongoing and
future w ork
The denition of err or c onditions in terms of
endp oin t errors suc h as data loss or duplica
tion enabled the output analyzer to capture
faulty sc enarios and isolate proto col traces in
the recen t history of the errors
Finally wesho w ed that a large p ortion of the
proto col state space could b e co v ered bysim u
lating a few r epr esentative sc enarios With the
aid of STRESS w ew ere able to disco v er sev eral
proto col design errors in PIMSM and suggest
solutions to these errors
This pap er w as the rst attempt to dev elop and
apply these metho ds W e are encouraged b y our suc
cess in iden tifying proto col errors using these meth
o ds and hop e that w eha v e similar results as wemo v e
on to in v estigate other m ulticast routing and endto
end m ultipart y proto cols
F uture directions for this researc h include
Dev eloping algorithms for automatic scenario
generation that replace the heuristics used in
this study W e are curren tly in v estigating semi
formal approac hes based on w ellestablished
VLSI testing tec hniques and extending them
to syn thesize test top ologies In classical VLSI
test generation is p erformed on a giv en circuit
By con trast net w ork proto col robustness m ust
b e established o v er arbitrary and timev arying
top ologies
F urther applying STRESS to m ulticast routing
to in v estigate a wider range of proto col func
tions
timers and timed actions to complemen t
the triggered actions in v estigated in this
study heterogeneous top ologies including asym
metric and unidirectional links suc h as
satellite links
other m ulticast routing proto cols suc has
D VMRP PIMDM and hierarc hical
PIM in terop erabilitybet w een routing proto cols
Examples include in teraction b et w een uni
cast and m ulticast routing and the in ter
op erabilitybet w een m ulticast routing pro
to cols
Generalizing the metho d and extending it to
apply to endtoend m ultipart y proto cols Ex
amples of suc h proto cols include reliable and
realtime m ulticast transp ort and session man
agemen t proto cols suc h as SRM R TCP and
sdr Toac hiev e this the m ulticast distribution
tree maybe view ed as a lo gic al LANwith v ar
ious selectiv e loss and dela y mo dels
Sensitivit y analysis and p erformance ev aluation
ma y be needed for these proto cols F or ex
ample retransmission and congestion con trol
mec hanisms emplo y ed b y endtoend transp ort
proto cols usually use timer parameters Meth
ods may beneededfor in v estigating the param
eter space of the proto col eg timer v alues
and the net w ork eg delayv alues and study
the eect of the c hange on p erformance in a
systematic and ecien t fashion
Applying to real implemen tation conformance
testing through an em ulation in terface This
facilitates driving the test using represen tativ e
scenarios
Deriving b eha vioral assertion c hec ks based
on the STRESS metho d Assertions can be
used in net w ork managemen t and selfdiagnosing
proto cols
Ac kno wledgemen ts
Wew ould lik e to thank Joseph Bannister ISI George
Eddy ISI Sally Flo yd LBNL Sandeep Gupta USC
John Heideman ISI Kanna Kumar USCISI and
P a vlin Radosla v o v USCISI for their useful com
men ts and insigh tful feedbac k on the pap er
References
F Lin P Ch u and M Liu Proto col V erication us
ing Reac habilit y Analysis Computer Communic ation R e
view V ol No
F Lin P Ch u and M Liu Proto col V erication using
Reac habilit y Analysis the state explosion problem and
relief strategies Pr o c e e dings of the A CM SIGCOMM
D W aitzman S Deering C P artridge Distance V ector
Multicast Routing Proto col No v em b er RF C
J Mo y Multicast Extension to OSPF Internet Dr aft Septem ber D Estrin D F arinacci A Helm y V Jacobson and
L W ei Proto col Indep enden t Multicast Dense Mo de
PIMDM Proto col Sp ecication Pr op ose d Exp erimen
tal RF C Septem ber A J Ballardie PF F rancis and J Cro w croft Core
Based T rees In Pr o c e e dings of the A CM SIGCOMM San F rancisco D Estrin D F arinacci A Helm y D Thaler S Deer
ing M Handley V Jacobson C Liu P Sharma and
L W ei Proto col Indep enden t Multicast Sparse Mo de
PIMSM Motiv ation and Arc hitecture Pr op ose d Ex
p erimental RF C Octob er S Flo yd V Jacobson C Liu S McCanne and L Zhang
A Reliable Multicast F ramew ork for Ligh tw eigh t Sessions
and Application Lev el F raming IEEEA CM T r ansac
tions on NetworkingNo v em b er
H Sc h ulzrinne S Casner R F rederic k and V Jacobson
R TP A T ransp ort Proto col for RealTime Applications
RF C Jan uary S McCanne A Distributed Whiteb oard for Net w ork Con
ferencing UC Berkeley Computer Scienc epr oje ctMa y
V Jacobson and S McCanne v at LBNL Audio Confer
encing T ool URL wwwnr ge elblgovvat S McCanne and V Jacobson vic A Flexible F ramew ork
for P ac k et Video A CM Multime dia No v em b er
M Handley NTE The UCL Net w ork T ext Editor
M Handley The sdr Session Directory An Mb one Con
ference Sc heduling and Bo oking System
D Estrin D F arinacci A Helm y D Thaler S Deer
ing M Handley V Jacobson C Liu P Sharma and
L W ei Proto col Indep enden t Multicast Sparse Mo de
PIMSM Proto col Sp ecication RF C Marc h
D Estrin M Handley A Helm y P Huang and
D Thaler A Dynamic Bo otstrap Mec hanism for
Rendezv ousbased Multicast Routing Submitte d to
IEEEA CM T r ansactions on Networking Ma y E W Dijkstra A note on t w o problems in connection
with graphs Numerische Mathematik V ol
S McCanne and S Flo yd NS Net w ork Sim ulator URL
wwwnr ge elblgovns J Ousterhout Tcl and the Tk T o olkit A ddison Wesley D W etherall and C Lindblad Extending Tcl for Dy
namic Ob jectOrien ted Programming Pr o c e e dings of the
TclTk Workshop T or onto Ontario July
A Helm y Proto col Indep enden t MulticastSparse Mo de
PIMSM Implemen tation Do cumen t Internet Dr aft Jan uary S Flo yd and V Jacobson Random Early Detection Gate
w a ys for Congestion Av oidance IEEEA CM T r ansac
tions on Networking V ol No pages Au
gust K Saleh I Ahmed K AlSaqabi and A Agarw al A
reco v ery approac h to the design of stabilizing comm uni
cation proto cols Journal of Computer Communic ation
V ol No pages April E Clark e and J Wing F ormal Metho ds State of the
Art and F uture Directions A CM Workshop on Str ate gic
Dir e ctions in Computing R ese ar ch V ol No pages
Decem b er A Helm y A Surv ey on Kernel Sp ecication and V erica
tion TR ofCSat USC J Spiv ey Understanding Z a Sp ecication Language and
its F ormal Seman tics Cambridge University Pr ess
C Jones Systematic Soft w are Dev elopmen t using VDM
Pr entic eHal l Intl R Bo y er and J Mo ore A Computational Logic Hand
book A c ademic Pr ess Boston S Owre J Rush b y N Shank er and F Henk e F ormal
v erication for faulttoleran tarc hitectures Prolegomena
to the design of PVS IEEE T r ansactions on Softwar e
Engine ering pages F ebruary M Smith F ormal V erication of Comm unication Proto
cols F OR TEPSTV Confer enc e Octob er D Probst Using partialorder seman tics to a v oid the
state explosion problem in async hronous systems Pr o c
nd Workshop on ComputerA idedV eric ation Springer
V erlag New Y ork
P Go defroid Using partial orders to impro v e automatic
v erication metho ds Pr o c nd Workshop on Computer
A idedV eric ation Springer V erlag New Y ork N Maxemc h uc k and K Sabnani Probabilistic v erication
of comm unication proto cols Pr o c th IFIP WG Int
Workshop on Pr oto c ol Sp e cic ation T esting and V eri
c ation NorthHol land Publ A mster dam C W est Proto col V alidation b y Random State Explo
ration Pr o c th IFIP WG Int Workshop on Pr oto c ol
Sp e cic ation T esting and V eric ation NorthHol land
Publ A mster dam J P ageot and C Jard Exp erience in guiding sim ulation
Pr o c VIIIth Workshop on Pr oto c ol Sp e cic ation T est
ing and V eric ation A tlantic City NorthHol land
Publ A mster dam G Holzmann Design and V alidation of Computer Pro
to cols A TT Bel l L abs Pr entic e Hal l B Murra y and J Ha y es T esting ICs Getting to the Core
of the Problem IEEE Computer Magazine pages
No v em b er B Konemann B Bennetts N Jarw ala and B Nadeau
Dostie BuiltIn SelfT est Assuring System In tegrit y IEEE Computer Magazine pages No v em b er
M Abramo vici M Breuer and A F riedman Digital
Systems T esting and T estable Design A T T L abs
S Deering B F enner D Estrin A Helm yD F arinacci
L W ei M Handley V Jacobson and D Thaler Hierar
c hical PIMSM Arc hitecture for In terDomain Multicast
Routing Internet Dr aft Decem b er D Ra yner OSI conformance testing Computer Networks
and ISDN Systems Sp e cial issue on ConformanceT est
ing V ol No pages S P erl P erformance Assertion Chec king PhD Thesis
MIT Septem ber
Abstract (if available)
Linked assets
Computer Science Technical Report Archive
Conceptually similar
PDF
USC Computer Science Technical Reports, no. 690 (1998)
PDF
USC Computer Science Technical Reports, no. 657 (1997)
PDF
USC Computer Science Technical Reports, no. 727 (2000)
PDF
USC Computer Science Technical Reports, no. 673 (1998)
PDF
USC Computer Science Technical Reports, no. 644 (1997)
PDF
USC Computer Science Technical Reports, no. 663 (1998)
PDF
USC Computer Science Technical Reports, no. 667 (1998)
PDF
USC Computer Science Technical Reports, no. 801 (2003)
PDF
USC Computer Science Technical Reports, no. 726 (2000)
PDF
USC Computer Science Technical Reports, no. 672 (1998)
PDF
USC Computer Science Technical Reports, no. 613 (1995)
PDF
USC Computer Science Technical Reports, no. 696 (1999)
PDF
USC Computer Science Technical Reports, no. 734 (2000)
PDF
USC Computer Science Technical Reports, no. 753 (2002)
PDF
USC Computer Science Technical Reports, no. 730 (2000)
PDF
USC Computer Science Technical Reports, no. 649 (1997)
PDF
USC Computer Science Technical Reports, no. 757 (2002)
PDF
USC Computer Science Technical Reports, no. 775 (2002)
PDF
USC Computer Science Technical Reports, no. 655 (1997)
PDF
USC Computer Science Technical Reports, no. 743 (2001)
Description
Ahmed Helmy, Deborah Estrin. "Simulation-based `STRESS' testing case study: A multicast routing protocol." Computer Science Technical Reports (Los Angeles, California, USA: University of Southern California. Department of Computer Science) no. 674 (1998).
Asset Metadata
Creator
Estrin, Deborah
(author),
Helmy, Ahmed
(author)
Core Title
USC Computer Science Technical Reports, no. 674 (1998)
Alternative Title
Simulation-based `STRESS' testing case study: A multicast routing protocol (
title
)
Publisher
Department of Computer Science,USC Viterbi School of Engineering, University of Southern California, 3650 McClintock Avenue, Los Angeles, California, 90089, USA
(publisher)
Tag
OAI-PMH Harvest
Format
20 pages
(extent),
technical reports
(aat)
Language
English
Unique identifier
UC16270131
Identifier
98-674 Simulation-based `STRESS_ Testing Case Study A Multicast Routing Protocol (filename)
Legacy Identifier
usc-cstr-98-674
Format
20 pages (extent),technical reports (aat)
Rights
Department of Computer Science (University of Southern California) and the author(s).
Internet Media Type
application/pdf
Copyright
In copyright - Non-commercial use permitted (https://rightsstatements.org/vocab/InC-NC/1.0/
Source
20180426-rozan-cstechreports-shoaf
(batch),
Computer Science Technical Report Archive
(collection),
University of Southern California. Department of Computer Science. Technical Reports
(series)
Access Conditions
The author(s) retain rights to their work according to U.S. copyright law. Electronic access is being provided by the USC Libraries, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.
Repository Name
USC Viterbi School of Engineering Department of Computer Science
Repository Location
Department of Computer Science. USC Viterbi School of Engineering. Los Angeles\, CA\, 90089
Repository Email
csdept@usc.edu
Inherited Values
Title
Computer Science Technical Report Archive
Coverage Temporal
1991/2017
Repository Email
csdept@usc.edu
Repository Name
USC Viterbi School of Engineering Department of Computer Science
Repository Location
Department of Computer Science. USC Viterbi School of Engineering. Los Angeles\, CA\, 90089
Publisher
Department of Computer Science,USC Viterbi School of Engineering, University of Southern California, 3650 McClintock Avenue, Los Angeles, California, 90089, USA
(publisher)
Copyright
In copyright - Non-commercial use permitted (https://rightsstatements.org/vocab/InC-NC/1.0/