Computer Science Technical Report Archive

USC Computer Science Technical Reports, no. 657 (1997)

(USC DC Other)

USC Computer Science Technical Reports, no. 657 (1997)

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content STRESS T esting Applied to a Multicast Routing Proto col
Ahmed Helm y Deb orah Estrin
Computer Science Departmen tISI
Univ ersit y of Southern California
Los Angeles CA
email fahelm y estrin guscedu
July
Abstract
Multipart y proto cols supp ort an imp ortan t class of applications ranging from m ultimedia telecon
ferencing to net w ork games Designing widearea m ultipart y proto cols is b ecoming more complex with
the gro wth of the In ternet and the in tro duction of new service mo dels Unexp ected com binations of
ev en ts can driv e proto cols in to undesirable states and ma y lead to errors An ticipating all suc h cases is
often imp ossible and at b est ma y require extensivesim ulation and testing In large systems the cost
of testing all p ossible scenarios exhaustiv ely is prohibitiv e and man y unexp ected cases are not observ ed
un til deplo ymen t
Protot yping in testb eds or individual sim ulations t ypically fo cuses on p erformance under a limited
set of randomized proto col transitions F ormal and analytical mo dels represen ting suc h proto cols tend
to b e complex sometimes rendering the mo del in tractable
In this w ork w e prop ose a metho d for analyzing the robustness of m ultipart y m ulticastbased
proto cols in a systematic fashion W e call our metho d Systematic T esting of R obustness by Examination of
Sele ctedSc enarios STRESS STRESS aims to cut the time and eort needed to explore the pathological
cases of a proto col during its design This pap er has t w o goals to describ e the metho d and to
serv e as a case study of robustness analysis of m ulticast routing proto cols W e do not pro v e correctness
but aim to oer design metho d to ols similar to those used in CAD and VLSI design
In tro duction
In this pap er w e describ e a metho d for Systematic Testing of Robustness by Examination of Sele cte d
Sc enarios STRESS It is based on a sim ulation framew ork supp orted b y a set of to ols and is designed for
studying proto col b eha vior in the con text of pathological cases and scenarios Some of the general concepts
for STRESS dra w from proto col v erication tec hniques and reac habilit y analysis W e apply these
tec hniques to supp ort the design analysis and testing of multip arty pr oto c ols In particular w e in tro duce tec hniques for state and top ology reduction and in v estigate v arious pac k et
loss scenarios to capture robustness c haracteristics The denition of err or c onditions enables us to capture
the faulty err orpr one cases automatically Multipart y proto cols ma y in v olvem ultiple receiv ers and one or more senders These proto cols include
m ulticast routing proto cols eg D VMRP MOSPF PIMDM CBT and PIMSM m ulticast
transp ort proto cols eg SRM R TP and R TCP and m ultipart y applications eg WB v at vic n te and sdr This pap er fo cuses on m ulticast routing proto cols whic h deliv er pac k ets
ecien tly to group mem bers b y establishing distribution trees as sho wn in gure As a case studyw e apply
our metho d to the m ulticast routing proto col Proto col Indep enden t MulticastSparse Mo de PIMSM
This material is based up on w ork supp orted b y the Defense Adv anced Researc h Pro jects Agency D ARP A under Con tract

S
R1
R2
R3
R4 R5
S: sender to the group
Ri: receiver i of the group
Figure Establishing m ulticast deliv ery tree
W e unco v ered sev eral pathological errors in PIMSM through the use of STRESS to ols and ev aluated
solutions to eliminate these errors The suggested solutions ha v e since b een added to the PIMSM sp ecica
tion
The rest of the pap er is organized as follo ws Section pro vides an o v erview of the STRESS metho d
The case study for PIMSM is presen ted in section Results are giv en in section Sections and address
related w ork summary and future w ork resp ectiv ely The Approac hMetho d Ov erview
The r obustness of a proto col is its abilit y to resp ond correctly in the face of net w ork failures and pac k et loss
The goal of the STRESS metho d is to pro vide a framew ork for systematic testing of proto col robustness
through the examination of selected scenarios
F or a giv en proto col w e rst capture a set of errorprone scenarios This is ac hiev ed b y a in v estigating
a r epr esentative subset of the proto col state space and b dening error conditions W e use these scenarios
to ev aluate design tradeos analyze b eha vior and as a test suite to examine v arious implemen tations of
the proto col
Our basic approac h consists of three stages sc enario gener ation prepro cessing tr acing sim ulation
and output analysis p ostpro cessing see gure Scenario Generation
Scenarios are comp osed of routed top ologies and sequences of ev en ts input stim uli and state transitions
and describ e the sim ulation con text that ma y cause proto col transitions Scenario parameters include the
r outedtop olo gy host sc enarios and loss sc enarios Routed top ology
The routed top ology is the net w ork infrastructure up on whic h the proto col op erates eg net w ork no des and
links and the unicast routing that determines howpac k ets are forw arded
No D ABTC An y opinions ndings and conclusions or recommendations expressed in this material are those of
the authors and do not necessarily reect the views of the D ARP A

Scenario
Generation
Host
Scenarios
Routed
Topology
Loss &
Failures
End Point
Tracing
Protocol
Tracing
Code
Annotation
Simulation & Tracing
Output Analysis
Identifying
End Point
Errors
Relating Errors
to Protocol
Code
Profiling
Simulation
Set-up
Link
Tracing
Simulation
Engine
Figure STRESS metho d blo c k diagram
W e try to iden tify simple top ologies that capture a large p ercen tage of the proto cols state space and
to whic h other more complex top ologies ma y b e reduced
F or m uc h of our studyw ec ho ose a LAN with
four connected routers as the basic top ology Weshowho w other top ologies are reducible to the fourr outer
LAN top ology and discuss the limitations of suc h a top ology in section W e further extend the top ology
to capture particular c haracteristics of the proto col under study PIMSM
As a comp onen t of the routed top ology unicast route inconsistencies ma y b e a common source of error
Unicast routing ma y exist in one of the follo wing three states a consisten t routing b transien t inconsisten t
routing and c long liv ed inconsistency Case a requires no c hanges The study of case b is con v ergence
analysis whic h has been addressed elsewhere
W e are particularly in terested in case c
W e add an
inconsisten t unicast routing comp onen t to force the m ulticast routing proto col in to states encoun tered in
suc h pathology and analyze those states
Host scenarios
Host scenarios are com binations of p ossible host actions In our case study these are dened b y the m ulticast
service mo del Host actions include joining or lea ving groups or sending pac k ets to groups F or large
n um b ers of hosts and groups it is prohibitiv ely costly to explore all p ossible com binations exhaustiv ely The simplest m ulticast host scenario has a single source S and t w o receiv ers R and R for the same
group W e shall address this simple scenario in this section and sho w in section ho w it can b e utilized
and extended for our case study W e estimate all the p ossible com binations of our host mo del and try to reduce the n um ber to those
scenarios that ma y aect the proto col state transitions W e call suc h scenarios r epr esentative sc enarios T o
obtain the represen tativ e scenarios w e apply the sc enario lter sho wn in gure F or one source and t w o receiv ers the v e p ossible host ev en ts are source S sending to a group or S
for short receiv er joining a group or J and J for receiv ers R and R resp ectiv ely and receiv er
lea ving a group or L and L for receiv ers R and R resp ectiv ely
F or all p ossible perm utations there exists scenarios considering that eachhost ev en t o ccurs
once Applying proto col constrain ts suc h as a r e c eiver c annot le ave b efor e joining the gr oup reduces the
n um b er of p ossible com binations to scenarios F urther assuming for practicalit y without
Tw o top ologies are said to b e reducible or equiv alen t if they driv e the proto col according to the host scenarios applied
in to the same states exp eriencing the same set of state transitions
F or con v ergence analysis of PIMSM mec hanisms refer to This ma y b e caused byam ulticast region spanning more than one unicast routing AS

Host Events
Protocol Constraints
Practical Input
Symmetry &
Equivalence
Rep.
Scenarios
Figure The Scenario Filter to obtain represen tativ e scenarios
loss of generalit y that the sour c e sends p ackets thr oughout the simulation reduces the n um b er of p ossible
scenarios to scenarios These six scenarios are
JJLL JJLL JLJL
JJLL JJLL JLJL
The n um b er of represen tativ e scenarios can b e ev en reduced further if the host distribution is symmetric
with resp ect to the top ology since the follo wing scenarios will be equiv alen t i equiv alen t to ii equiv alentto and iii equiv alentto ie w e need only in v estigate dieren t host scenarios for the
giv en top ology These scenarios ma y be generated automatically bythe metho d Ho w ev er generalizing the pro cess of
obtaining represen tativ e scenarios for v arious m ultipart y proto cols is curren tly under study Loss and F ailures
The loss and failure scenarios considered include the loss and corruption of pac k ets during transp ort routing
or forw arding within the net w ork or loss of state in router no des due to proto col daemon failures mac hine
crashes or insucien t resources eg memory
Loss of pac k ets P ac k et loss may occurinv arious parts of the net w ork due to congestion or link no de or
in terface failures W e classify these ev en ts as simply pac k et loss regardless of cause and create exhaustiv e
loss scenarios to capture all the p ossible proto col transitions and pathologies due to pac k et loss
F or most m ulticast proto cols when routers are connected via a m ultiaccess net w ork or LAN
hop
b yhop messages are m ulticast on the LAN and ma y exp erience selectiv e loss ie ma y be receiv ed b y
some no des but not others The lik eliho o d of selectiv e loss is increased b y the fact that LANs often con tain
h ubs bridges switc hes and other net w ork devices Selectivelossma y aect proto col robustness Similarly m ultipart y proto cols and applications m ust deal with situations of selectiv e loss This dieren tiates these
applications most clearly from their unicast coun terparts and raises in teresting robustness questions
Our case study illustrates wh y selectiv e loss should b e considered when ev aluating proto col robustness
This lesson is lik ely to extend to the design of higher la y er proto cols that op erate on top of m ulticast and
can ha v e similar selectiv e loss
W e use the term LAN to designate a connected net w ork with resp ect to IPm ulticast This includes shared media suc has
Ethernet or FDDI h ubs switc hes etc

The input to the loss failures substage sho wn in gure is obtained from initial traces of sim ulations
without proto col message loss These traces guide further sim ulations to co v er all p ossible proto col message
loss scenarios
Loss of state State loss in no des ma y o ccur due to crashes loss of m ulticast unicast or all forw arding
en tries W e in v estigate ho w suc h loss aects the proto cols o v erall correctness esp ecially from the end
systems p ersp ectiv e
Sim ulation and T racing
During this stage the proto col mec hanisms are sim ulated and traces are collected
Sim ulation
One desirable approac h for sim ulating complex proto cols is to include detailed mec hanisms of parts of the
proto col while abstracting out others w e call this approac h subsetting Subsetting refers to selecting subsets of the proto col functions while abstracting or remo ving others This
allo ws us to fo cus on sp ecic parts of the proto col state space Subsetting can b e based on
Pr oto c ol functions Subsetting proto col functions or mec hanisms refers to the abstraction of these
functions This maybe ac hiev ed b y replacing a complex mec hanism b y a simpler one exhibiting similar
external b eha vior under relaxed assumptions F or example one ma y use static conguration instead
of sim ulating a detailed b o otstrap algorithm This w a y one ma y study other proto col mec hanisms
assuming correctness of the b o otstrap mec hanism
Pr oto c ol states A study ma y fo cus on sp ecic proto col states This allo ws for example the study of
m ulticast group state without dealing with sourcesp ecic state
Messages typ es This allo ws the examination of sp ecic proto col message t yp es in the absence of others
Note that subsetting do es not p ermit all com binations of the ab o veitems Tomain tain proto col correct
ness an abstracted part has to be replaced b y its equiv alen t that exhibits similar external b eha vior F or
example one ma y not simply remo v e the b o otstrap mec hanism and run the sim ulations
T racing
T racing is the pro cess of logging information ab out ev en ts or pac k ets during the sim ulation run Logged
information is analyzed during the p ostpro cessing ie the output analysis stage In addition some traces
are used as feedbac k to the scenario generator to guide further sim ulations W e consider sev eral kinds of
tracing
Endp oin t tracing T racing endp oin ts includes logging information p ertaining to hosts sending or receiv
ing pac k ets and joining or lea ving m ulticast groups A detailed description of the traces used in the case
study is giv en in section T o iden tify errors and pathologies in the proto col itself w e fo cus on the eect of the m ulticast routing
proto col transitions on the endp oin t pac k et deliv ery as explained in section Proto col state transition tracing A proto col can b e represen ted b y a nite state mac hine automaton
consisting of states transitions and stim uli inputs outputs and timer actions Based on kno wledge of
initial proto col states w e obtain the sequence of proto col transitions b y tracing all stim uli
W e use proto col traces to diagnose and v erify proto col b eha vior and to analyze errors

Link tracing Wek eep trackof pac k ets tra v ersing links or LANs bet w een no des as w ell as the ev en ts
of pac k et loss on links or LANs This information is used in sev eral w a ys in output visualization output
analysis or as feedbac k for scenario generation Links carrying message t yp es of in terest are targeted for
in ten tional pac k et loss in further sim ulations This reduces the n um b er of loss scenarios examined to those
directly aecting the proto col b eha vior under in v estigation
Co de annotation When placed in k ey p oin ts suc h as b eginning of proto col pro cedures or co de mo difying
the state of the proto col co de annotations capture in ternal execution of the proto col mac hinery Weuse
co de annotation to estimate what part of the co de and subsequen tly the proto col has b een executed and
stressed co de co v erage
Output Analysis
One ma jor concern of STRESS is to iden tify pathological cases and indicate when and if an error o ccurred
and wh y This is ac hiev ed in the output analysis stage whic h consists of
Iden tifying endp oin t errors Err or c onditions ma y b e sp ecied with resp ect to endp oin t traces Ex
amples of endp oin t error conditions are blac k holes
and pac k et duplication where more than one cop yof
the same pac k et is receiv ed b y a group mem b er
If the factors during one sim ulation run are relativ ely static ie static unicast routing static top ology
and con trolled loss the error ma y b e attributed to an error in the m ulticast routing proto col
Once the sp ecied error is iden tied b y the output analyzer the trace log is rolled bac k in time to
in v estigate the proto col traces as explained next
Relating errors to proto col After detecting an endp oin t error the output analyzer isolates the p ossible
causes of suc h errors in the form of proto col traces The output analyzer in this case is similar to a logic
analyzer allo wing the designer to na vigate bac kw ard in time and in v estigate the causes of the error
As will be sho wn in section the pro cess of iden tifying a proto col error ma y suggest xes to the
problem
Co de proling The proler captures information ab out the annotated co de suc h as whic h pro cedures
w ere or w ere not in v ok ed and the order and frequency of in v oking proto col pro cedures This information
indicates the p ortion of the proto col stressed b y the examined scenarios
Case Study of PIMSM
Toev aluate the utilit y of STRESS w e applied it to a complex m ulticast routing proto col PIMSM Before
going in to details of the case studyw e rst giv eano v erview of m ulticast routing and PIMSM
Multicast Routing Ov erview
Multicast distribution trees ma y b e established b y either broadcastandprune or explicit join proto cols In
the former suc has D VMRP or PIMDM a m ulticast pac k et is broadcast to all leaf subnet w orks Subnet
w orks with no lo cal mem b ers for the group send prune messages to w ards the sources of the pac k ets to stop
further broadcasts Link state proto cols suc h as MOSPF broadcast mem b ership information to all no des
In con trast in explicit join proto cols suc h as CBT or PIMSM routers send hopb yhop join messages for
the groups and sources for whic h they ha v e lo cal mem bers When receiv ed these messages build routing
state in routers and cause further messages to b e sen t upstream un til the distribution tree is established
Up on arriving at a router a m ulticast pac k et is forw arded according to the routing state
Ablac k hole is an observ able amoun t of consecutivepac k et loss b et w een p erio ds of pac k et deliv ery
AB C
D
1. Receiver sends a PIM join toward the RP
RP
Sender
Receiver
2. Sender sends a PIM register to the RP
3. RP sends data packets
down the established path
establishing a path from RP back to the receiver.
Figure Ho w senders rendezv ous with receiv ers simplie d In this pap er w e study PIMSMs mec hanisms for building shared m ulticast trees F or simplicit yw edo
not address sourcesp ecic trees in this description
As sho wn in gure when a receiv ers lo cal router Adisco v ers it has lo cal receiv ers it starts sending
periodic join messages to w ard a groupsp ecic Rendezv ousP oin t RP The join messages are m ulticast hop
b yhop Eac h router along the path to w ard the RP builds a wildcard an ysource r oute entry for the group
and sends the join messages on to w ard the RPA r oute entry is the state held in a router to main tain the
distribution tree T ypically it includes the source address group address the in terface from whichpac k ets
are accepted inc oming interfac e and the list of in terfaces to whic h pac k ets are sent outgoing list This
state forms a shared RPro oted distribution tree that reac hes all group mem bers When a source rst sends to a group its lo cal router D unicasts r e gister messages to the RP with the
sources data pac k ets encapsulated within Data pac k ets reac hing the RP are forw arded nativ ely do wn the
shared tree to w ard group mem b ers
Similarly when a mem ber lea v es the group a prune message is sen t b y the lo cal router to stop the
m ulticast trac from o wing do wn the branc h leading to the pruned mem ber Being robust to at least a single message loss w as a design goal for PIMSM
Wepoin t out t w o PIM mec hanisms relev anttothisstudy Assert and pruneoverride The PIM Assert
mec hanism is the pro cess bywhic h at most one forw arder for a LAN is selected to a v oid duplicates in case
of m ultiple p oten tial forw arders due to parallel paths to the source or RP The pruneoverride enables a
do wnstream router ie with do wnstream mem bers to retain its established branc h of the tree in case
another router on the same LAN tries to prune that branc h
The rest of this section is outlined as follo ws Section establishes the equiv alence relationship for the
top ology used for the case study Section describ es the sim ulation test suites An example of applying
the metho d is explained in section and the scenario and proto col co v erage ac hiev ed for the case study is
giv en in section T op ology Equiv alence
Tw o top ologies are equiv alen t if they driv e the proto col transitions in to the same states under the same set
of ev en t sequences A top ology is reducible to another top ology with few er connections and routers if the
t w o top ologies are equiv alen t
Toac hiev e this a do wnstream router receiving a prune on its incoming in terface triggers a join upstream

A
B
C
A B
C
Topology I
A B
C D
D
[3-router LAN]
Topology II
[4-router LAN;
Topology III
[4-router LAN;
downstream addition]
upstream addition]
downstream
upstream
Figure The equiv alen t top ologies
Wesho w in this section that for single message loss scenarios the fourr outer LAN top ology adopted in
this study exp eriences the same proto col errors that an Nr outer LAN top ology exp eriences where N
and hence they are equiv alen t Wesho w this relationship for the messages under study for PIMSM namely joins prunes and asserts F or brevit y w e only pro v e equiv alence in the case of prune messages and hin t
to the pro of approachin the other cases W e also iden tify assumptions and limitations under whic h this
equiv alence relationship holds
Prunes
First w e consider Nr outer LAN top ologies where N and resp ectiv ely It is trivial to pro vethat
these top ologies are not equiv alen t for hopb yhop messages
Assumption Nr outer LAN top olo gy wher e N is r e ducible to the thr e er outer LAN top olo gy for
prunes wrt single message loss sc enarios
T o justify our assumption werstpro vethat a fourr outer LAN top ology is reducible to a thr e er outer
LAN top ology Correctness condition If a r outer on the LAN has the LAN as its inc oming interfac e ther e must b e
one other r outer with the LAN in its outgoing list Once this condition is satised violating it is considered
a proto col error
Next w e examine the thr e er outer LAN top ology In gure top ology I assume that A and B are
do wnstream routers and C is an upstream router
In gure top ology I router C has the LAN in its outgoing list router A has the LAN as its incoming
in terface and router B is lea ving the group and so sends a prune to w ards CThe prune is m ulticast
on the LAN
The only case where the correctness condition ma y b e violated is when C receiv es the prune while A
do es not In the other cases either the prune is not receiv ed b y C or is receiv ed b y A whic h triggers a
pruneoverride to reestablish the LAN in Cs outgoing list This is illustrated b y the follo wing selectiv e
loss pattern table for the prune message sentb y B This is to dieren tiate b et w een join latency whic h is not considered a proto col error and a blac k hole whic h is a proto col
error

A C
error
where a indicates noloss and indicates loss The error o ccurs where the upstream router C receiv ed the prune but the router with do wnstream mem bers A did not receiv e it
In gure top ology I I w e add another do wnstream router D The selectiv e loss pattern table follo ws
A D C
error
The only error o ccurs when the upstream router C receiv es the prune but neither of the do wnstream
routers receiv es it If the prune is receiv ed byan y of the do wnstream routers a pruneoverride w ould
reestablish the LAN in Cs outgoing list
F rom the symmetry of the loss patterns and top ology w e see that all errors are triggered b y the same
transitions exp erienced b y router A in top ology I Hence the extended top ology I I do es not in tro duce an y
new errors and exhibits the same external b eha viorasdoestopology I W e conclude that top ology I and
top ology I I are equiv alen t for prunes Wenowsho w that the Nr outer LAN top ology is reducible to the N case where N
With the addition of an upstream router gure top ology I I I no added error cases are encoun tered
The addition of a do wnstream router ho w ev er ma yin tro duce new error scenarios Similar to the fourr outer
LAN case w e establish the follo wing assertion the only err or c ase o c curs when al l downstr e am r outers lose
the prune and the upstr e am r outer r e c eives it If the prune w as receiv ed byanyof the do wnstream routers
the correctness condition w ould b e retained using pruneoverrides The assertion holds in b oth top ologies Hence w e conclude that the Nr outer top ology exp eriences the
same errors as the Nr outer top ology F rom the ab o vew e see that bysim ulating the thr e er outer LAN top ology w e capture all the errors with
resp ect to selectiv e loss for the prune mec hanism that ma y b e exp erienced byan y Nr outer LAN top ology
where N
Joins
F or lossfree scenarios the equiv alence pro of is straigh tforw ard F or lossy scenarios the loss of a join message
sen t to an upstream router ma y lead to join latency but do es not cause blac kholes
Joins leading to pac k et
duplication lead to asserts that are discussed next
Note that if join suppression as describ ed b y the PIMSM sp ecication is implemen ted the equiv alence relation can b e
established similar to the prunes case ab o v e

RP
S1
S2, R2 R1
AB
CD
RP
S1
R2 R1
AB
CD
RP
S2, R2 R1
AB
CD
unicast route
to RP
Topology 1 Topology 2 Overall topology
Figure The top ology used for the case study
Asserts
In most cases pro ofs similar to those presen ted can b e applied to Asserts Ho w ev er since asserts ma ybe
triggered due to parallel paths the base case is established for the fourr outer LAN top ology Figure
top ology I I I represen ts the fourr outer LAN top ology where A and B are do wnstream routers and C and
D are upstream routers
F or our case studyw e use a fourr outer LAN top ology with an added Rendezv ousP oin t RP to capture
shared tree c haracteristics The o v erall ph ysical top ology consists of v e routers four of whic h are connected
via a LAN as sho wn in gure T est suites
In this section w e elab orate on the routed top ology host scenarios and loss pattern generation used for our
case study W e also describ e the simplications and subsettings applied
Ph ysical and routed top ologies The o v erall top ology used is that sho wn in gure F or the unicast
routing proto col weuse acen tralized v ersion of Dijkstras Shortest P ath First SPF algorithm PIMSM uses the underlying unicast routing tables for building m ulticast trees Therefore unicast
routing inconsistencies aect the op eration of PIMSM T oin v estigate suchin teraction w e add a comp onen t
to force inconsistentm ulticast routes b et w een PIM routers as sho wn in gure top ology Inconsisten t
unicast routing cases arise for example when a m ulticast region spans m ultiple unicast domains or ASs

Host scenarios Since proto col states for dieren t groups do not in teract w e consider only one group
Also since proto col states for dieren t sources do not in teract it suces to consider only one source S p er
sim ulation run
The source is mo deled as a constan tbit rate CBR stream with xed pac k et size The
A limitation to the fourr outer LAN top ology is giv en for the esoteric case of three upstream routers and three do wnstream
routers with inconsisten t unicast routing tables This case creates one extra transition that can only b e captured b y at least
a sixr outer LAN top ology W e do not consider this a practically signican t scenario and w e consider its analysis as a sp ecial
case not captured b ythe fourr outer LAN top ology Ho w ev er aside from this exception the Nr outer LAN top ology where N is equiv alentto a fourr outer LAN top ology
wrt asserts
A similar in teraction with unicast routing ma y b e created easily when switc hing to the shortest paths in PIMSM

W e do not consider aggregated source or group en tries in this study
source mo del do es not aect the correctness of the metho d Ho w ev er to assure full con trollabilityo v er the
selectiv e loss mo del w e set the data rate to ensure that no loss o ccurs due to queue o v ero w
While w e consider only a single source w e consider t w o receiv ers R and R for the same group to
accoun t for shared tree state in teractions W e use the host scenarios describ ed in section Loss patterns Wein v estigate all p ossible selectiv e loss scenarios for m ulticast hopb yhop PIMSM mes
sages in this represen tativ e top ology Loss mo dels are applied exhaustiv ely to those links that carry the proto col messages under in v estigation
The tracing stage iden ties these links during the rst sim ulation run without pac k et loss and feeds bac k
the link information to the loss generation mo dule as sho wn in gure As wewillsho w in section the
n um b er of represen tativ e scenarios is quite small and hence the n umberofo v erall lossy scenarios explored
is manageable
W e do not address state loss or no de crashes in this do cumen t Ho w ev er crash scenarios ma y b e imple
mentedina w a y similar to loss scenarios
T racing T race information includes the ev entt yp e send or receiv e the no de exp eriencing the ev en t the
t yp e of message sen t or receiv ed and the time at whic h the ev en t o ccurred Ev ery data pac k et is assigned a
unique sequence n um ber F or example a trace maytakethe follo wing format R No de A Rcvd time meaning that
receiv er R in no de A receiv ed a data pac k et with sequence n um ber at time ms from the b eginning
of the sim ulation run
Subsetting F or brevit y w e do not consider sourcesp ecic trees and switc hing to the shortest paths in
this pap er This is an example of state subsetting since w e consider shared group states while disregarding
sourcesp ecic states
The messages considered in the study are join prune assert and r e gister messages T o study joins prunes and asserts without the eect of r e gistersw e consider a top ology where the source and the RP are
colo cated see S in gure top ology This is an example of message subsetting When studying r e gisters joins and prunes w e consider top ology in gure where a S is the source
hence no de A sends registers to the RP and b the routed top ology has consisten t unicast routing to
eliminate the eect of the assert mec hanism This represen ts function or me chanism subsetting Only triggered actions are in v estigated for simplicit y Timer action analysis is not considered in this
study and is part of our w orkinprogress
Applying the Metho d
This section pro vides an illustrativ e example sho wing ho w STRESS maybe used to iden tify and analyze
errors encoun tered during the sim ulation of the represen tativ e scenarios
Weha v e implemen ted an initial v ersion of the STRESS metho d in the Net w ork Sim ulator NS NS is
an ev en tdriv en pac k etlev el sim ulator con trolled and congured via Tcl and Ob jectTcl or OTcl
T o supp ort our metho d w eha v e added Mo dules to pro vide LAN supp ort con trolled selectiv e loss proto col
tracing proling capabilities and a detailed implemen tation of PIMSM

This implemen tation serv es as
the sim ulation en vironmen t for our case study In addition the building blo c ks w ere designed to b e reused
within the same framew ork to apply this metho d to other m ultipart y proto cols

F or this w e use pac k et size of b ytes and a send in terv al of ms ie source rate of kbs this ensures no queue
drops on the Mbs links used with pac k et queue limit

F or information ab out the sim ulator see h ttpcatarinausceduvin t

Our detailed PIMSM sim ulation mimics the unix pimd implemen tation mo del and hence is able to capture man y
implemen tation asp ects W e plan to dev elop an in terface bet w een the sim ulator and an op erational net w ork running the
pimd co de Ho w ev er the analyses presen ted in this study are based strictly on the proto col sp ecication indep enden tof the
implemen tation

S1 Node RP Sending 7 time 175
R1 Node A Rcvd 7 time 190
S1 Node RP Sending 8 time 200
J2 Node B Join G time 200
PIMS Node B Sending Join{G,NH=D} time 200
PIMR Node A Rcvd Join{G,NH=D} time 210
PIMR Node D Rcvd Join{G,NH=D} time 210
PIMS Node D Sending Join{G,NH=RP} time 210
PIMR Node C Rcvd Join{G,NH=D} time 210
R1 Node A Rcvd 8 time 221
R2 Node B Rcvd 8 time 221
PIMR Node RP Rcvd Join{G,NH=RP} time 221
S1 Node RP Sending 9 time 225
R1 Node A Rcvd 9 time 246
R2 Node B Rcvd 9 time 246
PIMS Node D Sending Assert{G,S} time 246
PIMS Node C Sending Assert{G,S} time 246
R2 Node B Rcvd 9 time 247
R1 Node A Rcvd 9 time 247
S1 Node RP Sending 10 time 250
PIMS: sent by the PIM component
PIMR: received by the PIM component
NH: next hop
duplicates
loss
Sent by S1
Rcvd by R1
Rcvd by R2
time (ms)
sequence No.
7
8
9
10
11
12
13
200
250 300 350
J2
L1
400
14
S1 Node RP Sending 12 time 300
L1 Node A Leave G time 300
PIMS Node A Sending Prune{G,S,NH=C} time 300
PIMR Node C Rcvd Prune{G,S,NH=C} time 310
PIMS Node C Sending Prune{G,S,NH=RP} time 310
PIMR Node B Rcvd Prune{G,S,NH=C} time 310
PIMS Node B Sending Join{G,NH=C} time 310
PIMR Node D Rcvd Prune{G,S,NH=C} time 310
R2 Node B Rcvd 12 time 321
PIMR Node C Rcvd Join{G,NH=C} time 321
PIMS Node C Sending Join{G,NH=RP} time 321
PIMR Node RP Rcvd Prune{G,S,NH=RP} time 321
PIMR Node A Rcvd Join{G,NH=C} time 321
PIMR Node D Rcvd Join{G,NH=C} time 321
S Node RP Sending 13 time 325
PIMR Node RP Rcvd Join{G,NH=RP} time 332
S Node RP Sending 14 time 350
R2 Node B Rcvd 14 time 371
S Node RP Sending 15 time 375
R2 Node B Rcvd 15 time 396
Figure Simple pac k et trace graph sho wing pac k et loss and duplication
Obtaining fault y scenarios T o obtain the faulty sc enarios ie those that con tain errors w e execute the
metho d stages in order ie scenario generation sim ulation and tracing and output analysis resp ectiv ely
and then rev erse the order from the output to the traces to iden tify the fault y scenarios These phases are
automated b y the to ols pro vided and are transparen t to the user once the scenario setup is complete
The pro cess of attributing endp oin t errors to proto col actions ma y be automated only if the error
conditions are giv en in terms of suc h proto col actions In practice ho w ev er these proto col error conditions
are often not kno wn a priori b y the designer and are usually dened in terms of endp oin t errors suc has
pac k et loss or duplication The supp orting to ols iden tify endp oin t errors and pro vide a history of proto col
traces The designer then examines the traces and iden ties the proto col errors
This pro cess ma y suggest
xes to the problem as w e will sho w in the results section
Example In our simple example an err or c ondition is anypac k et loss or duplication exp erienced b y the
endp oin ts A fault y scenario without pac k et loss that leads to t w o error conditions is iden tied and
explained Then the proto col actions leading to the errors are analyzed
The represen tativ e scenario explained here is JJLL using top ology This scenario w as iden tied
automatically as a fault y scenario T races in gure givethe history of the errors found The rst error
ie the pac k et duplication has the host ev en t J as the closest join or lea v e host ev en t in its history at
time ms The error is a join transien t caused b y parallel paths to the RP The error is resolv ed using the
Assert messages exc hanged during the duplication at time ms The second error ie pac k et loss is a
lea v e transien t it has a host ev en t L in its recen thistory The loss is due to the prune sentbynode A at
ms and is resolv ed bya pruneoverride sentbynode B at ms
Although the proto col actions leading to the endp oin t errors sp ecied as an y pac k et loss or duplication
in this sp ecic example are considered transien t errors they are not considered proto col design errors W e
do ho w ev er address proto col design errors in section Scenario and proto col co v erage
While the fact that wew ere able to disco v er design errors pro vides some evidence of the metho ds utilit yw e
w ould lik e to quan tify the co v erage of proto col states and p ossible scenarios
The o v erall proto col co v erage has t w o dimensions The rst is the proto col state co v erage and w e attempt
to co v er this dimension using the r epr esentative scenarios reac hable states In v estigation of the loss scenarios

In our limited exp erience w eha veiden tied proto col errors in the recen t history of the endp oin t errors

do es not aect proto col co v erage signican tly The second dimension is the space of p ossible in teraction scenarios bet w een these state mac hines in
dieren t routers within the top ology This dimension is explored byin v estigating the sele ctive loss scenarios
Scenarios co v ered The initial n um ber of sim ulated scenarios without proto col message loss w as
X
topolog ies
No rep scenarios Where No rep scenarios is the n um ber of r epr esentative scenarios equal to in our case discussed
in section and the top ologies are the t w o discussed in section Hence wesim ulated scenarios
without proto col message loss
After feeding bac k the link traces for the messages under studythe loss patterns w ere assigned to the
corresp onding links The scenario generator then setup the sim ulations for the new scenarios with loss
The total n um b er of scenarios with proto col message loss sim ulated is giv en b y the follo wing form ula
X
T opos
X
Reps
X
Msgs
X
Link s
Link M sg s Link Rtr s A
A
where
T erm Meaning
T op os T op ologies
Reps Represen tativ e Scenarios
Msgs Messages under study
LinkMsgs Num b er of messages tra v ersing the link
LinkR trs Num b er of routers connected to the link
F or example for the rst top ology the messages under study w ere joins prunes and asserts The
represen tativ e scenarios triggered joins prunes and asserts on the LAN and joins and prunes on p oin ttop oin t links F or the second top ology the messages under study w ere joins and prunes The represen tativ e scenarios triggered joins and prunes on the LAN and joins and prunes
on poin ttop oin t links Hence the total n um ber of scenarios with loss b ecame and scenarios
resp ectiv ely Proto col co de co v erage A large p ortion of the m ulticast supp ort co de in ns w as annotated automatically
to pro vide co de tracing The represen tativ e scenarios without loss in v ok ed pro cedures out of o v erall
annotated pro cedures The pro cedures that w ere not in v ok ed dealt mainly with sourcesp ecic state whic h
w as abstracted in our test suites or with the mo dularit y of the ob jectorien ted nature of the co de
Results
This section describ es the proto col design errors unco v ered for PIMSM under STRESS
Unlik e our simple example ab o v e w e are only in terested in design ie not transien t errors F or this w e
mo died the err or c onditions to a v oid join and lea v e transien ts The new err or c onditions do not consider
single duplication or loss
F ollo wing is a summary of the ma jor fault y scenarios encoun tered and ho w they relate to STRESS F or
a more detailed discussion of the proto col errors and xes see section
Summary of Results
W e describ e a partial list of faulty sc enarios captured b y STRESS W e obtained this list after sim ulating
only a few of the represen tativ e scenarios The traces pro duced pro vided guidance to disco v er the proto col
errors Design errors disco v ered include Assert JoinPrune and R e gister mec hanisms
Asserts F or the rst top ology gure top ology a blac kholew as observ ed for one receiv er
The fault y scenario in this case in v olv ed another receiv er joining in the recen t history of the blac k hole
By analyzing the proto col trace history after rolling bac k w e noticed that an Assert pro cess to ok place righ t
b efore the loss
In addition the fault y scenario included the loss of a join message whic hprev en ted the establishmentof
the branc h of the shared tree from the Assert winner to the RP Hence the proto col design error is allo wing
a router on a branc h of the tree that is not completely established to participate in Asserts Joins and Prunes Ov er the same top ology ie gure top ology sev eral other fault y scenarios lead
to blac k holes The host scenarios in v olv ed one receiv er lea ving just b efore blac k holes w ere exp erienced b y
the other receiv er In these cases join and prune messages o ccurred the recen t history of the endp oin t error
F urthermore all suc h scenarios included either i loss of a join message prev en ting a pruned branc h
from b eing reestablished or ii selectiv e loss of a prune message prev en ting a join ie pruneoverride from b eing triggered The proto col design error in this case w as not allo wing a second c hance for routers
with do wnstream mem b ers to o v erride prunes Registers In the second top ology gure top ology fault y scenarios w ere captured that cause pac k et
duplicates at the endp oin ts
In this case the observ ed fault y scenarios did not follo w a regular pattern and w ere dev elop ed iterativ ely
ie when one fault y scenario led to a suggested x in the proto col the x w as implemen ted and the metho d
rerun to observ e further fault y scenarios
Therstscenario in v olv ed a single host receiving duplicates merely b y joining the group The pac k ets
w ere b eing deliv ered at least t wice once directly from the source b yvirtue of b eing on the same LAN
and the second deliv ery from the shared tree after the r e gister reac hed the RP and w as sentdo wn the shared
tree When the n umberofpac k et duplicates exceeded t w o this suggested a lo op The lo op o ccurred when a
pac k et receiv ed o v er the shared tree on the LAN w as a pic k ed up b y the lo cal router b reregistered to
the RP and c forw arded do wn the shared tree again The proto col error w as allo wing the pac k ets to o w
do wn from the shared tree to the originating LAN and b e reregistered Thexw as to prune suc h sources
from the shared tree
The second scenario in v olv ed another receiv er joining b efore the duplicates w ere observ ed The pruned
branc h of the shared tree w as reestablished b y the joining receiv er allo wing the pac k ets to o wdo wn the
shared tree to the originating LAN and subsequen tly causing the lo op
The third scenario in v olv ed a prune message loss again allo wing the pac k ets to o wdo wn the shared
tree to the originating LAN and led to lo oping
Rules w ere added to prev en t pac k ets from being forw arded bac k on their original LANs in the ab o v e
scenarios
Detailed Results
The rest of this section describ es the ab o vefault y scenarios in more detail and illustrates ho w the solutions
w ere dev elop ed with the aid of STRESS

RP
S1
R2 R1
AB
CD
1
2
RP
S1
R2 R1
AB
CD
5
RP
S1
R2 R1
AB
CD
3
4
1) R1 joins the group. B sends joins towards RP.
2) S1 sends packets to the group. Packets flow
3) R2 joins the group. A sends joins towards RP.
4) The join from C to RP is lost.
RP
S1
R2 R1
AB
CD
6
7
5) Packets forwarded by D onto the LAN are
received by C on an outgoing interface.
6) C Asserts with a winning metric onto the LAN.
7) D removes the LAN from its entry and sends
prunes towards the RP.
down distribution tree and are multicast on the LAN.
(I)
(II)
(III)
(IV)
Figure The Assert scenario under study
Assert analysis
F ollo wing is a discussion of the pathological cases found in the Assert pr o c ess An exhaustiv e list of the
results is not included in this do cumen t for brevit y A few errors in the PIMSM sp ecication w ere un v eiled
during this pro cess w e fo cus on errors that created the p ossibilityof pac k et loss ie blac k holes
The scenario In this scenario the top ology in gure w as setup suc hthat As nexthop to w ards the RP
is Cand Bs nexthop to w ards the RP is D Consider the sequence of ev en ts sho wn in gure whic h used the represen tativ e scenario JJLL
with the loss of a join message on the link b et w een C and RP During the last t woev en ts of the scenario steps and D lost the Assert pro cess to C with higher
metric or address Subsequen tly D remo v es the LAN from its en trys in terface list and R stops receiving
pac k ets from S This problem p ersists un tilunless the branc h of the tree from C to RP is established
Discussion and x The curren t rules of the PIM sp ecication aim to guaran tee atmost one forw arder
on a m ultiaccess net w ork Ho w ev er to ensure prop er deliv ery of pac k ets without pac k et loss the righ t
seman tics should b e exactly one forw arder
The problem arises more sp ecically b ecause the PIM sp ecication do es not distinguish bet w een an
activ e en try ie an en try created due to arriv al of data pac k ets eg am ulticast forw arding cac he and
an en try on a branc h of a tree that is not y et established or an inactive en try An inactiv e en try ma y
win an Assert pro cess resulting in blac k holes
Tosolv e this problem w e mo died the sp ecication to ensure exactly one forw arder seman tics using
the follo wing rule

NoState
State for G
ActiveState
OifDeleted
Rcv join for G; create state, trigger join upstream
Rcv join for G
Rcv pkts for G; activate state, forward pkts
Rcv pkts for G; forward pkts
or
Rcv assert on oif & win; send assert
Rcv assert on oif & lose; delete oif from entry
All oifs deleted
oif: outgoing interface
& entry removed
G: multicast group
Figure A simplied state transition diagram for the join and assert pro cessing
A router receiving a data pac k et or Assert on an outgoing in terface of a matc hing en try do es not
participate in the Assert pro cess unless the en try is activ e
Figure illustrates the Activ eState added to the transition diagram to realize the solution
JoinPrune analysis
In this analysis w e address the eect of selectiv eloss of JoinPrune messages Although this problem has
b een addressed in recen t releases of the PIMSM sp ecication w epro vide a more ecien t solution
W e use the top ology giv en in gure I The represen tativ e scenario used is JJLL with the
second join from no de A lost on the LAN
W e assume that S sends pac k ets to group G throughout the sim ulation Consider the sequence of ev en ts
giv en in gure I After the last ev en t step R stops receiving Ss pac k ets This problem p ersists
un til A sends the next p erio dic join to C and reestablishes the pruned branc h of the tree A similar problem
is encoun tered in gure II when the prune sentfrom B is selectiv ely lost on the LAN b y A and receiv ed
b y C Discussion and x The solution suggested b y the PIM sp ecication in tro duces a deletion timer This
ho w ev er increases the lea v e latency and incurs unnecessary data o v erhead
A more ecien t solution w ould be to ha v e the upstream router C announce a prunealert b efore
remo ving the LAN from its outgoing list b y resending the prune message previously receiv ed from B Register analysis
F ollo wing is a description of the scenarios that exhibit pac k et duplication due to r e gister messages and
the suggested xes to eliminate suc h duplication The xes w ere applied iterativ ely un til the error w as
eliminated
First scenario single source single receiv er In this scenario w e consider S and R in gure Consider the sequence of ev en ts in the gure
P ac k et duplication and r e gister lo oping o ccur in the ab o v e scenario A similar scenario o ccurs when R
joins rst then S starts sending to the group
Suggested xes The required b eha vior is to send a triggered and p erio dic sourcesp ecic prune o of
the shared tree if a router has sourcesp ecic state for registering and shared tree state for the same group
regardless of the incoming in terface settings

RP
R2 R1
AB
C
S1
1
RP
R2 R1
AB
C
RP
R2
R1
A
B
C
2
S1
S1
3
3
4
5
5
4
1) R1 joins the group. B sends joins towards RP.
2) R2 joins the group. A sends joins towards RP.
3) R1 leaves the group. B multicasts prunes onto the LAN.
4) A gets the prune and sends a join to override. The join is lost.
5) C gets the prune and sends it towards RP.
(I)
(II)
3) R1 leaves the group. B multicasts prunes onto the LAN.
4) A does not receive the prune, and so does not override.
5) C gets the prune and sends it towards RP.
Figure The JoinPrune scenario under study
Second scenario single sender t w o receiv ers W e assume the implemen tation of the ab o vexes to
the sim ulator then consider the sequence of ev en ts in gure This scenario exhibits pac k et duplication
and r e gister lo oping
Suggested x The problem arises b ecause the pac k ets are forw arded bac k on the originating LAN and
treated as if they w ere new pac k ets originated b y the directly connected source The follo wing rule solv es
this problem for the giv en scenario
A router receiving join message m ust NOT add an in terface on the same subnet as a source S for an y
source sp ecic en try for S asso ciated with same group
Third scenario single source single receiv er with message loss Considering the scenario in
gure The source sp ecic prune sen t from A to C when A ha ving a shared tree state creates the source sp ecic
en try for registering is lost
P ac k et duplication and r e gister lo oping problems are exp erienced in this scenario The problem p ersists
un til a p erio dic JoinPrune message is successfully sen t upstream
Suggested x T o be robust to at least one message loss w e suggest the follo wing rule for pac k et
forw arding
A router m ust NOT forw ard a pac k et on to the subnet from whic h the pac k et w as originated This
is ac hiev ed b y p erforming a c heckonthe source and the outgoing in terface b efore building asource

RP
S2,R2
A
C
1
2
3
4
1) R2 joins the group. A sends joins towards RP.
2) S2 sends packets to the group. A builds source-specific state,
and registers to the RP. The incoming interface for the entry points
towards the LAN.
3) RP gets the registers, decapsulates and forwards the packets
down the shared tree.
4) Packets down the shared tree are accepted from the LAN,
and re-registered to the RP. This forms a register loop.
Figure The rst r e gister scenario under study
sp ecic state or b efore forw arding a pac k et

Related w ork
The related w ork falls mainly in the eld of proto col v erication Most of the literature on m ulticast proto col
design addresses arc hitecture sp ecication and comparisons b et w een dieren t proto cols W e are not a w are
of an y other w ork to dev elop systematic metho ds for testing m ultipart y proto col robustness In addition
some concepts of STRESS w ere inspired b y VLSI c hip testing
There is a large b o dy of literature dealing with v erication of comm unication proto cols Proto col v er
ication t ypically addresses safety liveness and r esp onsiveness prop erties Safet y prop erties include
freedom from deadlo c ks assertion violations improp er terminations and unsp ecied receptions Liv eness
prop erties include detection of acceptance cycles and absence of nonprogress cycles While resp onsiv eness
prop erties include timeliness and fault tolerance Most proto col v erication systems including STRESS
aim to detect violations of part of these proto col prop erties
In general the t w o main approac hes for proto col v erication are theorem pro ving and reac habilit y analysis
or mo del c hec king Theorem pro ving systems dene a set of axioms and construct relations
on these axioms Desirable prop erties of the proto col are then pro v en mathematically Theorem pro ving
includes mo delb ase d formalisms suc h as Z and Vienna Dev elopmen t Metho d VDM and lo gic
b ase d formalisms including rst order logic suc h as Nqthm and higher order logic suc h as Protot yp e
V erication System PVS F ormal v erication has been applied to TCP and TTCP but for
m ultipart y proto cols theorem pro ving systems are lik ely to b e ev en more complex and p erhaps in tractable
Reac habilit y analysis algorithms try to generate and insp ect all the proto col states that are reac hable
from giv en initial states Suc h algorithms suer from the state space explosion problem esp ecially in
complex systems as are m ultipart y proto cols T o circum v entthis problem state reduction and con trolled
partial searc h tec hniques could be used These tec hniques fo cus only on parts of the state space
and ma y use probabilistic random or guided searc hes The SPIN to ol uses the sup ertr ac e
algorithm that is actually a random con trolled partial searc h STRESS has similarities with guided con trolled
partial searc hes Ho w ev er STRESS explores proto col states based on the r epr esentative sc enarios and do es
not use a cost function as do es guided searc h

Most implemen tations create a cac he for forw arding pac k ets This c hec k can b e done only once when creating the cac he
and is not done p er pac k et
This is dieren t than the incoming in terface c hec k stated b y the curren t sp ecication In the sp ecic case discussed here
the lo oping m ulticast pac k ets will matc h on the incoming in terface the LAN for the sourcesp ecic en try
RP
S2,R2 R1
AB
C
1
2
3
4
5
1) R2 joins the group and S2 sends to the group. A establishes
2) R1 joins the group. B sends join towards RP.
3) S2 sends packets to the group; A sends registers to RP.
4) RP decapsulates and forwards data packets down the shared tree.
5) The packets forwarded onto the LAN are re-registered by A,
causing a register loop.
routing state and sends a join to the group, and a prune
for S2 towards RP.
This eliminates the prune state for S2 in C.
Figure The second r e gister scenario used for the study
There is an analogy bet w een STRESS and VLSI systematic design for testabilit y using BuiltIn
SelfT est BIST BIST pro vides a systematic tec hnique for c hip testing syn thesis This tec hnique can
b e used to detect faults due to singlestuc kline while STRESS can b e used to detect errors due to single
pac k et loss BIST uses a test pattern generator TPG to pro duce the input patterns applied to the circuit
under test Conceptually this resem bles our use of the scenario generator The test patterns are c hosen
to maximize fault co v erage with a minim um n um ber of inputs Similarly the scenario lter c ho oses the
represen tativ e scenarios to maximize proto col co v erage with a minim um n um ber of scenarios Moreo v er
BIST uses a resp onse monitor circuit to monitor and detect error signals This is analogous to our use of
tracing and error detection mo dules
The exp ected output for VLSI c hip testing is fault co v erage vs test length curv e Although this is
similar to proto col co v erage vs scenarios statistics w e add the co v erage of p ossible in teraction scenarios
as another output dimension
Summary and F uture W ork
The goals of our metho d are to simplify and systematize robustness analysis of m ultipart y proto cols This
pap er presen ted our initial attempts to ac hiev e these goals in the con text of one m ulticast routing proto col
W e used scenario generation sim ulation tracing and output analysis to obtain a set of errorprone scenarios
In particular w e describ ed sev eral tec hniques
T o circum v en t the state explosion problem w ein tro duced the notion of r epr esentative sc enarios W e
obtained these scenarios for the m ulticast host mo del using a sc enario lter whic h excluded redundan t
and irrelev an t scenarios based on practical assumptions
W e iden tied t w o r epr esentative top olo gies based on the e quivalenc e relationship established for the
proto col under study PIMSM The equiv alence denition suggested that extending the sim ulated
top ologies w ould not rev eal additional errors in the proto col
T o capture robustness c haracteristics w e studied the proto col b eha vior in the presence of single pac k et
loss A LAN mo dule accoun ted for the sele ctive loss cases exp erienced bym ulticast messages Wehope
to use a similar lo gic al LAN mo dule to mo del loss and dela y parameters of the underlying m ulticast
distribution trees and thereb y extend our metho d to higherlev el m ultipart y proto cols
T o reduce the complexit y of our analyses w e used subsetting of proto col functions states and messages
This allo w ed abstracting out some proto col details while retaining and fo cusing on others

RP
S2,R2
A
C
1
2
3
4
1) A sends prune towards the RP; since R2 is a member
of the group and S2 is a source. The prune is lost.
2) A registers packets from S2 to RP.
3) RP decapsulated and forwards packets down the
shared tree.
4) Packets from the shared tree are accepted from the LAN
and re-registered to the RP. This creates a register loop.
Figure The third r e gister scenario used for the study
The denition of err or c onditions in terms of endp oin t errors suc h as data loss or duplication enabled
the output analyzer to capture faulty sc enarios and isolate proto col traces in the recen t history of the
errors
Finallyw esho w ed that a large p ortion of the proto col state space could b e co v ered bysim ulating a
few r epr esentative sc enarios With the aid of STRESS w ew ere able to disco v er sev eral proto col design
errors in PIMSM and suggest solutions to these errors
This pap er w as the rst attempt to dev elop and apply these metho ds W e are encouraged b y our success
in iden tifying proto col errors using these metho ds and hop e that w eha v e similar results as wemo veonto
in v estigate other m ulticast routing and endtoend m ultipart y proto cols
F uture directions for this researc h include
F urther applying STRESS to m ulticast routing to in v estigate a wider range of proto col functions
timers and timed actions to complemen t the triggered actions in v estigated in this study heterogeneous top ologies including asymmetric and unidirectional links suc h as satellite links
other m ulticast routing proto cols suc hasD VMRP PIMDM and hierarc hical PIM
in terop erabilit y bet w een routing proto cols Examples include in teraction bet w een unicast and
m ulticast routing and the in terop erabilitybet w een m ulticast routing proto cols
Generalizing the metho d and extending it to apply to m ultipart y proto cols Examples of suc h proto cols
include reliable and realtime m ulticast transp ort and session managemen t proto cols suc h as SRM
R TCP and sdr Toac hiev e this the m ulticast distribution tree ma y b e view ed as a lo gic al LAN with
v arious selectiv e loss and dela y mo dels
Applying to real implemen tation conformance testing through an em ulation in terface This facili
tates driving the test using represen tativ e scenarios
Deriving beha vioral assertion c hec ks based on the STRESS metho d Assertions can be used in
net w ork managemen t and selfdiagnosing proto cols
Ac kno wledgemen ts
Wew ould lik e to thank our colleagues at USCISI George Eddy John Hiedeman Kanna Kumar and P a vlin
Radosla v o v for their useful commen ts on early v ersions of the pap er

References
F Lin P Ch u and M Liu Proto col V erication using Reac habilit y Analysis Computer Communic ation R eview
V ol No F Lin P Ch u and M Liu Proto col V erication using Reac habilit y Analysis the state explosion problem and
relief strategies Pr o c e e dings of the A CM SIGCOMM
D W aitzman S Deering C P artridge Distance V ector Multicast Routing Proto col No v em b er RF C
J Mo y Multicast Extension to OSPF Internet Dr aftSeptem b er D Estrin D F arinacci A Helm y V Jacobson and L W ei Proto col Indep enden t Multicast Dense Mo de
PIMDM Proto col Sp ecication Pr op osedExp erimental RF C URL httpnetwebusce dupimpimdmPIM
DM ftxtps ggz Septem b er A J Ballardie P F F rancis and J Cro w croft Core Based T rees In Pr o c e e dings of the A CM SIGCOMM San
F rancisco D Estrin D F arinacci A Helm y D Thaler S Deering M Handley V Jacobson C Liu P Sharma and
L W ei Proto col Indep enden t Multicast Sparse Mo de PIMSM Motiv ation and Arc hitecture Pr op ose d
Exp erimental RF C URL httpnetwebusce dupimpimsmPIMA r ch ftxtps ggz Octob er S Flo yd V Jacobson C Liu S McCanne and L Zhang A Reliable Multicast F ramew ork for Ligh tw eigh t
Sessions and Application Lev el F raming IEEEA CM T r ansactions on NetworkingNo v em b er
H Sc h ulzrinne S Casner R F rederic k and V Jacobson R TP A T ransp ort Proto col for RealTime Applica
tions RF C Jan uary S McCanne A Distributed Whiteb oard for Net w ork Conferencing UC Berkeley Computer Sciencepr oje ct Ma y V Jacobson and S McCanne v at LBNL Audio Conferencing T o ol URL httpwwwnr ge elblgovvat
S McCanne and V Jacobson vic A Flexible F ramew ork for P ac k et Video A CM Multime dia No v em ber
M Handley NTE The UCL Net w ork T ext Editor URL httpwwwmic ensccsuclacukmic enscto olsnt
helpab outhtml M Handley The sdr Session Directory An Mb one Conference Sc heduling and Bo oking System URL
httpugwwwe dacukmic ear chivesdrhtml D Estrin D F arinacci A Helm y D Thaler S Deering M Handley V Jacobson C Liu P Sharma and
L W ei Proto col Indep enden t Multicast Sparse Mo de PIMSM Proto col Sp ecication RF C URL
httpnetwebusce dupimpimsmPIMSMvExpRF C ftxtps ggz Marc h D Estrin M Handley A Helm y P Huang and D Thaler A Dynamic Bo otstrap Mec hanism
for Rendezv ousbased Multicast Routing Submitte d to IEEEA CM T r ansactions on Networking URL
httpwwwusce dudeptcste chnic al r ep ortshtml Ma y E W Dijkstra Anote ont w o problems in connection with graphs Numerische Mathematik V ol
S McCanne and S Flo yd NS Net w ork Sim ulator URL httpwwwnr ge elblgovns J Ousterhout Tcl and the Tk T o olkit A ddison Wesley D W etherall and C Lindblad Extending Tcl for Dynamic Ob jectOrien ted Programming Pr o c e e dings of the
TclTk Workshop T or onto Ontario July A Helm y Proto col Indep enden t MulticastSparse Mo de PIMSM Implemen tation Do cumen t Internet Dr aft
URL httpwwwusce dudeptcste chnic al r ep ortshtml Jan uary K Saleh I Ahmed K AlSaqabi and A Agarw al Areco v ery approac h to the design of stabilizing comm uni
cation proto cols Journal of Computer Communic ation V ol No pages April E Clark e and J Wing F ormal Metho ds State of the Art and F uture Directions A CM Workshop on Str ate gic
Dir e ctions in Computing R ese ar ch V ol No pages Decem b er A Helm y A Surv ey on Kernel Sp ecication and V erication T e chnic al R ep ort of the Computer Scienc e
Dep artment University of Southern California URL httpwwwusce dudeptcste chnic al r ep ortshtml
J Spiv ey Understanding Z a Sp ecication Language and its F ormal Seman tics Cambridge University Pr ess C Jones Systematic Soft w are Dev elopmen t using VDM Pr entic eHal l Intl R Bo y er and J Mo ore A Computational Logic Handb o ok A c ademic Pr ess Boston S Owre J Rush b y N Shank er and F Henk e F ormal v erication for faulttoleran tarc hitectures Prolegomena
to the design of PVS IEEE T r ansactions on Softwar e Engine ering pages F ebruary M Smith F ormal V erication of Comm unication Proto cols F OR TEPSTV Confer enc e Octob er D Probst Using partialorder seman tics to a v oid the state explosion problem in async hronous systems Pr o c
nd Workshop on ComputerA idedV eric ation Springer V erlag New Y ork
P Go defroid Using partial orders to impro v e automatic v erication metho ds Pr o c nd Workshop on Computer
A idedV eric ation Springer V erlag New Y ork N Maxemc h uc k and K Sabnani Probabilistic v erication of comm unication proto cols Pr o c th IFIP WG Int Workshop on Pr oto c ol Sp e cic ation T esting and V eric ation NorthHol land Publ A mster dam C W est Proto col V alidation b y Random State Exploration Pr o c th IFIP WG Int Workshop on Pr oto c ol
Sp e cic ation T esting and V eric ation NorthHol land Publ A mster dam
J P ageot and C Jard Exp erience in guiding sim ulation Pr o c VIIIth Workshop on Pr oto c ol Sp e cic ation
T esting and V eric ation A tlantic City NorthHol land Publ A mster dam G Holzmann Design and V alidation of Computer Proto cols A TT Bel l L abs Pr entic e Hal l B Murra y and J Ha y es T esting ICs Getting to the Core of the Problem IEEE Computer Magazine pages
No v em b er B Konemann B Bennetts N Jarw ala and B NadeauDostie BuiltIn SelfT est Assuring System In tegrit y
IEEE Computer Magazine pages No v em b er
S Deering B F enner D Estrin A Helm y D F arinacci L W ei M Handley V Jacobson and
D Thaler Hierarc hical PIMSM Arc hitecture for In terDomain Multicast Routing Internet Dr aft URL
httpnetwebusce duahelmyinter domainmultic astint er do m ps Decem b er D Ra yner OSI conformance testing Computer Networks and ISDN Systems Sp e cial issue on Conformanc e
T esting V ol No pages S P erl P erformance Assertion Chec king PhD Thesis MIT Septem b er

Abstract (if available)

Linked assets

Computer Science Technical Report Archive

Conceptually similar

PDF

USC Computer Science Technical Reports, no. 674 (1998)

PDF

USC Computer Science Technical Reports, no. 690 (1998)

PDF

USC Computer Science Technical Reports, no. 727 (2000)

PDF

USC Computer Science Technical Reports, no. 673 (1998)

PDF

USC Computer Science Technical Reports, no. 696 (1999)

PDF

USC Computer Science Technical Reports, no. 644 (1997)

PDF

USC Computer Science Technical Reports, no. 649 (1997)

PDF

USC Computer Science Technical Reports, no. 663 (1998)

PDF

USC Computer Science Technical Reports, no. 655 (1997)

PDF

USC Computer Science Technical Reports, no. 726 (2000)

PDF

USC Computer Science Technical Reports, no. 755 (2002)

PDF

USC Computer Science Technical Reports, no. 757 (2002)

PDF

USC Computer Science Technical Reports, no. 608 (1995)

PDF

USC Computer Science Technical Reports, no. 599 (1995)

PDF

USC Computer Science Technical Reports, no. 801 (2003)

PDF

USC Computer Science Technical Reports, no. 667 (1998)

PDF

USC Computer Science Technical Reports, no. 730 (2000)

PDF

USC Computer Science Technical Reports, no. 716 (1999)

PDF

USC Computer Science Technical Reports, no. 672 (1998)

PDF

USC Computer Science Technical Reports, no. 775 (2002)

Description

Ahmed Helmy and Deborah Estrin. "STRESS' testing applied to multicast routing protocol." Computer Science Technical Reports (Los Angeles, California, USA: University of Southern California. Department of Computer Science) no. 657 (1997).

Asset Metadata

Creator Estrin, Deborah (author), Helmy, Ahmed (author)

Core Title USC Computer Science Technical Reports, no. 657 (1997)

Alternative Title STRESS' testing applied to multicast routing protocol (title)

Publisher Department of Computer Science,USC Viterbi School of Engineering, University of Southern California, 3650 McClintock Avenue, Los Angeles, California, 90089, USA (publisher)

Tag OAI-PMH Harvest

Format 22 pages (extent), technical reports (aat)

Language English

Unique identifier UC16269692

Identifier 97-657 `STRESS_ Testing Applied to a Multicast Routing Protocol (filename)

Legacy Identifier usc-cstr-97-657

Format 22 pages (extent),technical reports (aat)

Rights Department of Computer Science (University of Southern California) and the author(s).

Internet Media Type application/pdf

Source 20180426-rozan-cstechreports-shoaf (batch), Computer Science Technical Report Archive (collection), University of Southern California. Department of Computer Science. Technical Reports (series)

Access Conditions The author(s) retain rights to their work according to U.S. copyright law. Electronic access is being provided by the USC Libraries, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.

Repository Name USC Viterbi School of Engineering Department of Computer Science

Repository Location Department of Computer Science. USC Viterbi School of Engineering. Los Angeles\, CA\, 90089

Repository Email csdept@usc.edu

Inherited Values

Title Computer Science Technical Report Archive

Coverage Temporal 1991/2017

Repository Email csdept@usc.edu

Repository Name USC Viterbi School of Engineering Department of Computer Science

Repository Location Department of Computer Science. USC Viterbi School of Engineering. Los Angeles\, CA\, 90089

Publisher Department of Computer Science,USC Viterbi School of Engineering, University of Southern California, 3650 McClintock Avenue, Los Angeles, California, 90089, USA (publisher)